harbor: --harness module:Class for your own agents, and loopback for host-side agents

.claude/docs/REPO_WALKTHROUGH.md

@@ -211,7 +211,7 @@ rfcs/
 │   └── tester.md                 # Write high-signal, failing tests
 │
 └── hooks/                    # Automation scripts
-    ├── lint.sh                   # Run ruff format check
+    ├── lint.sh                   # Check CI import sort, format, and lint (src/ tests/)
     ├── test.sh                   # Run pytest
     ├── check-debug.sh            # Find debug code
     ├── post-push-pr.sh           # Validate PR after push (freshness, CI, conflicts)

.claude/hooks/lint.sh

@@ -1,13 +1,6 @@
 #!/bin/bash
-# Lint check for OpenEnv
-# Replicates the exact arc f pipeline from fbsource:
-#   1. usort format — sort imports (matches arc f's usort pass)
-#   2. ruff format  — code formatting, line-length 88 (matches arc f's ruff-api pass)
-#   3. ruff check   — lint rules (E, F, W)
-#
-# usort is scoped to src/ and tests/ only. envs/ uses ruff format only
-# because standalone usort and pyfmt's usort disagree on import ordering
-# inside try/except blocks in some env files.
+# Match CI: usort format, then ruff format, then ruff check on src/ and tests/.
+# Format disposable copies so uncommitted edits are never overwritten.
 
 set -e
 
@@ -19,18 +12,30 @@ if ! command -v uv &> /dev/null; then
 fi
 
 echo "=== Running import sort + format check ==="
-# Ask the same tools as arc f whether the tree is formatted, without writing to
-# it. Formatting in place and then running `git checkout --` to undo it also
-# reverts the author's uncommitted edits in those files, and leaves reformatted
-# Markdown behind, because the restore only ever covered *.py.
+# CI checks the combined result: ruff can undo an intermediate usort change.
+FORMAT_DIR=$(mktemp -d)
+trap 'rm -rf "$FORMAT_DIR"' EXIT
+cp -RL src tests pyproject.toml "$FORMAT_DIR/"
+if [ -f .gitignore ]; then
+    cp .gitignore "$FORMAT_DIR/"
+fi
+# Ruff only respects .gitignore inside a Git repository.
+git -C "$FORMAT_DIR" init -q
+uv run usort format "$FORMAT_DIR/src/" "$FORMAT_DIR/tests/"
+uv run ruff format "$FORMAT_DIR/src/" "$FORMAT_DIR/tests/"
+
 FORMAT_FAILED=0
-uv run usort check src/ tests/ || FORMAT_FAILED=1
-uv run ruff format --check src/ tests/ envs/ || FORMAT_FAILED=1
+while IFS= read -r -d '' file; do
+    if ! cmp -s "$file" "$FORMAT_DIR/$file"; then
+        echo "$file"
+        FORMAT_FAILED=1
+    fi
+done < <(find -L src tests -type f -name '*.py' -print0)
 
 if [ "$FORMAT_FAILED" -ne 0 ]; then
     echo ""
     echo "ERROR: the files listed above need formatting."
-    echo "Run: uv run usort format src/ tests/ && uv run ruff format src/ tests/ envs/"
+    echo "Run: uv run usort format src/ tests/ && uv run ruff format src/ tests/"
     exit 1
 fi
 echo "Import sort + format check passed!"

CLAUDE.md

@@ -292,7 +292,7 @@ docker build -t echo-env:latest -f envs/echo_env/server/Dockerfile .
 Scripts in `.claude/hooks/` are used by skills and can be run directly:
 
 ```bash
-bash .claude/hooks/lint.sh          # Run ruff format check
+bash .claude/hooks/lint.sh          # Check CI import sort, format, and lint (src/ tests/)
 bash .claude/hooks/test.sh          # Run pytest (excludes special envs)
 bash .claude/hooks/check-debug.sh   # Find debug code (print, breakpoint, TODO)
 bash .claude/hooks/post-push-pr.sh  # Validate PR after push (freshness, CI, conflicts)

docs/source/environments/harbor.md

@@ -384,27 +384,31 @@ Exit codes:
 | code | meaning |
 |---|---|
 | `0` | success |
-| `1` | ran, but failed. `rollout` returns this if **any** rollout in the batch was unusable |
+| `1` | ran, but failed. `rollout` returns this if **any** rollout in the batch was unusable; `info --harness` returns it if construction or wiring checks fail |
 | `2` | usage error: a missing or invalid flag. Nothing ran |
 
 The `1` and `2` split matters if you are scripting this: `2` means the command never started, so
 retrying it unchanged will fail the same way.
 
 ### `openenv harbor info`
 
-Report what this machine can run. Read-only: boots no sandbox, starts no server, makes no rollout.
+Report what this machine can run. Boots no sandbox, starts no server, makes no rollout.
+With `--harness`, imports and constructs that agent locally using dummy capture credentials and
+temporary logs. Only check code you trust; its constructor runs, but its setup and run methods do not.
 
 | flag | type | default | meaning |
 |---|---|---|---|
 | `--llm-url` | str | `""` | OpenAI-spec endpoint. Optional here; without it the LLM section is skipped and the rest still reports |
 | `--model` | str | `""` | Served model id. Auto-detected when the endpoint serves exactly one |
 | `--dataset` | str | none | Dataset spec. Repeatable, or comma-separated |
 | `--env-file` | path | `""` | dotenv with provider credentials, loaded before the checks |
+| `--harness` | str | `""` | Check a seam name or trusted `module:Class`; show its capture wiring and try constructing it |
 | `--verbose` | flag | off | List all 30 harnesses, not only the 16 validated |
 | `--json` | flag | off | Emit machine-readable JSON instead of the text report |
 
 ```bash
 openenv harbor info --llm-url $LLM --dataset org/tasks --json
+openenv harbor info --harness my_package.agent:MyAgent --model org/model --json
 ```
 
 The JSON has four top-level keys: `llm`, `sandboxes`, `datasets`, `harnesses`. Each sandbox entry is
@@ -415,6 +419,12 @@ openenv harbor info --llm-url $LLM --json \
   | jq -r '.sandboxes[] | select(.available) | .name'
 ```
 
+`--harness` adds a `harness` object with `ok` and either the dummy wiring or an error in `detail`.
+It catches import errors, incorrect subclasses, constructor failures, and host-side agents that
+declare neither `api_base` nor `api_key` as constructor parameters or options fields. A successful
+check confirms construction only; it does not validate model calls, capture or sandbox installation.
+Without `--model` or an endpoint's detected model, the check uses `preflight-model`.
+
 ### `openenv harbor rollout`
 
 Run rollouts with no env server involved. This is the debugging path: if `rollout` works and `serve`
@@ -552,11 +562,29 @@ ids and logprobs, and where the agent emits a trajectory, its own record agreed
 
 Supporting four dialects rather than chat-completions alone is what makes the last four rows work.
 
-`terminus-2` runs in the server process rather than inside the sandbox, so it reaches the proxy on
-localhost and needs no public URL.
+`terminus-2` and host-side agents of your own run in the server process rather than inside the
+sandbox, so they are handed the proxy's localhost URL and need no public one.
 
 The other 14 known agents have a seam but are untested; run `openenv harbor info --verbose` to list
-them. Anything Harbor supports can be reached with `--harness module:Class`.
+them.
+
+### Your own agent
+
+Name any Harbor agent class by import path, `--harness my_package.agent:MyAgent`, in `rollout`, in
+`serve` rollouts and from a trainer. Naming a class imports its module, so `serve` accepts only the
+import paths its operator lists with `--allow-harness my_package.agent:MyAgent`. How the class is
+pointed at the proxy depends on what it is:
+
+| the class | what it gets |
+|---|---|
+| a subclass of an agent above (its `name()` is a known harness) | that agent's wiring. If OpenEnv runs that agent through its own subclass, such as `openenv.harbor.install_fixes:InterceptPi` for pi, subclass that one instead: the error names it |
+| a host-side agent (a `BaseAgent` that calls the model from the server process) | `api_base` (the proxy's `/v1` URL, on localhost) and `api_key` (the rollout's session id) as constructor kwargs |
+| any other agent installed into the sandbox | refused: add a seam for it in `openenv/harbor/seams.py` |
+
+A host-side agent declares both kwargs in its options and sends every model call to `api_base` with
+`api_key` as the bearer token; capture, reward and training traces then work as for any other
+harness. Nothing reaches the sandbox or the verifier. Options the server has no way to pass, such as
+provider extras, are best read from the agent's own environment variables.
 
 ## Supported sandboxes
 

docs/source/tutorials/harbor-harness.md

@@ -20,6 +20,17 @@ openenv harbor rollout \
 
 Against a hosted provider you get an evaluation rollout: the reward and the full trace. Against vLLM or SGLang you also get the token ids and logprobs that training needs. `openenv harbor info` checks first which agents, sandboxes and capture level this machine can use.
 
+Before running an agent of your own, check its wiring and constructor without starting a sandbox:
+
+```bash
+openenv harbor info --harness my_package.agent:MyAgent --model org/model --json
+```
+
+This imports and constructs the named class with dummy capture credentials; only name code you
+trust. It reports errors such as a missing constructor setting or undeclared `api_base`/`api_key`,
+and returns exit code 1 on failure. Success confirms construction, while model calls and capture
+remain untested. See [Your own agent](../environments/harbor#your-own-agent) for the contract.
+
 ## Train on the Captures
 
 Each rollout comes back as a `TrainingTrace`: the token ids, logprobs and loss masks of every model call, plus the task's reward. It doesn't depend on a trainer, so any framework can train on it. TRL has a worked example: `AsyncGRPOTrainer` with a `HarnessRolloutWorker` runs each task to completion, reads back the trace, and syncs the new weights into the same vLLM server.

envs/harbor_env/README.md

@@ -396,27 +396,31 @@ Exit codes:
 | code | meaning |
 |---|---|
 | `0` | success |
-| `1` | ran, but failed. `rollout` returns this if **any** rollout in the batch was unusable |
+| `1` | ran, but failed. `rollout` returns this if **any** rollout in the batch was unusable; `info --harness` returns it if construction or wiring checks fail |
 | `2` | usage error: a missing or invalid flag. Nothing ran |
 
 The `1` and `2` split matters if you are scripting this: `2` means the command never started, so
 retrying it unchanged will fail the same way.
 
 ### `openenv harbor info`
 
-Report what this machine can run. Read-only: boots no sandbox, starts no server, makes no rollout.
+Report what this machine can run. Boots no sandbox, starts no server, makes no rollout.
+With `--harness`, imports and constructs that agent locally using dummy capture credentials and
+temporary logs. Only check code you trust; its constructor runs, but its setup and run methods do not.
 
 | flag | type | default | meaning |
 |---|---|---|---|
 | `--llm-url` | str | `""` | OpenAI-spec endpoint. Optional here; without it the LLM section is skipped and the rest still reports |
 | `--model` | str | `""` | Served model id. Auto-detected when the endpoint serves exactly one |
 | `--dataset` | str | none | Dataset spec. Repeatable, or comma-separated |
 | `--env-file` | path | `""` | dotenv with provider credentials, loaded before the checks |
+| `--harness` | str | `""` | Check a seam name or trusted `module:Class`; show its capture wiring and try constructing it |
 | `--verbose` | flag | off | List all 30 harnesses, not only the 16 validated |
 | `--json` | flag | off | Emit machine-readable JSON instead of the text report |
 
 ```bash
 openenv harbor info --llm-url $LLM --dataset org/tasks --json
+openenv harbor info --harness my_package.agent:MyAgent --model org/model --json
 ```
 
 The JSON has four top-level keys: `llm`, `sandboxes`, `datasets`, `harnesses`. Each sandbox entry is
@@ -427,6 +431,12 @@ openenv harbor info --llm-url $LLM --json \
   | jq -r '.sandboxes[] | select(.available) | .name'
 ```
 
+`--harness` adds a `harness` object with `ok` and either the dummy wiring or an error in `detail`.
+It catches import errors, incorrect subclasses, constructor failures, and host-side agents that
+declare neither `api_base` nor `api_key` as constructor parameters or options fields. A successful
+check confirms construction only; it does not validate model calls, capture or sandbox installation.
+Without `--model` or an endpoint's detected model, the check uses `preflight-model`.
+
 ### `openenv harbor rollout`
 
 Run rollouts with no env server involved. This is the debugging path: if `rollout` works and `serve`
@@ -564,11 +574,29 @@ ids and logprobs, and where the agent emits a trajectory, its own record agreed
 
 Supporting four dialects rather than chat-completions alone is what makes the last four rows work.
 
-`terminus-2` runs in the server process rather than inside the sandbox, so it reaches the proxy on
-localhost and needs no public URL.
+`terminus-2` and host-side agents of your own run in the server process rather than inside the
+sandbox, so they are handed the proxy's localhost URL and need no public one.
 
 The other 14 known agents have a seam but are untested; run `openenv harbor info --verbose` to list
-them. Anything Harbor supports can be reached with `--harness module:Class`.
+them.
+
+### Your own agent
+
+Name any Harbor agent class by import path, `--harness my_package.agent:MyAgent`, in `rollout`, in
+`serve` rollouts and from a trainer. Naming a class imports its module, so `serve` accepts only the
+import paths its operator lists with `--allow-harness my_package.agent:MyAgent`. How the class is
+pointed at the proxy depends on what it is:
+
+| the class | what it gets |
+|---|---|
+| a subclass of an agent above (its `name()` is a known harness) | that agent's wiring. If OpenEnv runs that agent through its own subclass, such as `openenv.harbor.install_fixes:InterceptPi` for pi, subclass that one instead: the error names it |
+| a host-side agent (a `BaseAgent` that calls the model from the server process) | `api_base` (the proxy's `/v1` URL, on localhost) and `api_key` (the rollout's session id) as constructor kwargs |
+| any other agent installed into the sandbox | refused: add a seam for it in `openenv/harbor/seams.py` |
+
+A host-side agent declares both kwargs in its options and sends every model call to `api_base` with
+`api_key` as the bearer token; capture, reward and training traces then work as for any other
+harness. Nothing reaches the sandbox or the verifier. Options the server has no way to pass, such as
+provider extras, are best read from the agent's own environment variables.
 
 ## Supported sandboxes
 

src/openenv/cli/commands/harbor.py

@@ -187,6 +187,67 @@ def _hub_not_found(exc: Exception) -> bool:
     return getattr(response, "status_code", None) == 404
 
 
+def _check_harness(harness: str, model: str) -> dict[str, Any]:
+    """Construct the rollout's agent with dummy capture credentials, without setup or run."""
+    import inspect
+    import io
+
+    result: dict[str, Any] = {"name": harness, "ok": False}
+    try:
+        # Imports, constructors and the process-env warning may print. Keep --json readable.
+        with (
+            contextlib.redirect_stdout(io.StringIO()),
+            tempfile.TemporaryDirectory() as tmp,
+        ):
+            from harbor.agents.factory import AgentFactory
+            from openenv.harbor import seams
+            from openenv.harbor.rollout import build_trial_config, process_env
+
+            seam = seams.get(harness)
+            url, session = "http://127.0.0.1:9", "openenv-harness-preflight"
+            *_, proc_env = seam.resolve(base_url=url, session=session, model=model)
+            config = build_trial_config(
+                task_dir=tmp,
+                harness=harness,
+                sandbox="docker",
+                intercept_url=url,
+                session_id=session,
+                model=model,
+                trial_name="preflight",
+                trials_dir=tmp,
+            )
+            with process_env(harness, proc_env):
+                agent = AgentFactory.create_agent_from_config(
+                    config.agent, logs_dir=Path(tmp)
+                )
+            if seam.host_side and agent.name() not in seams.SEAMS:
+                accepted = set(inspect.signature(type(agent).__init__).parameters)
+                # Harbor 0.22 predates options models; constructor parameters still work.
+                options = getattr(agent, "options", None)
+                if options is not None:
+                    accepted.update(type(options).model_fields)
+                missing = {"api_base", "api_key"} - accepted
+                if missing:
+                    raise ValueError(
+                        f"{harness} must declare {', '.join(sorted(missing))} as constructor "
+                        "parameters or options fields; Harbor can silently ignore these kwargs."
+                    )
+        result.update(
+            ok=True,
+            import_path=seam.import_path,
+            dialect=seam.dialect,
+            host_side=seam.host_side,
+            status=seam.status,
+            model=config.agent.model_name,
+            kwargs=config.agent.kwargs,
+            agent_env=config.agent.env,
+            process_env=proc_env,
+        )
+    except (Exception, SystemExit) as exc:  # noqa: BLE001 - a failed check is report data
+        result["detail"] = f"{type(exc).__name__}: {exc}"
+    return result
+
+
 @app.command("info")
 def info(
     llm_url: Annotated[str, typer.Option("--llm-url", help=_LLM_HELP_OPTIONAL)] = "",
@@ -207,6 +268,14 @@ def info(
     env_file: Annotated[
         str, typer.Option("--env-file", help="dotenv file with provider credentials.")
     ] = "",
+    harness: Annotated[
+        str,
+        typer.Option(
+            "--harness",
+            help="Check a seam name or trusted `module:Class` by constructing it with dummy "
+            "capture credentials. Runs no agent or sandbox.",
+        ),
+    ] = "",
     verbose: Annotated[
         bool,
         typer.Option("--verbose", help="List every harness, not only validated ones."),
@@ -228,11 +297,23 @@ def info(
         api_key=api_key or None,
         auth_header=auth_header,
     )
-    print(
-        json.dumps(caps.to_dict(), indent=2)
-        if json_output
-        else caps.render(verbose=verbose)
+    checked = (
+        _check_harness(harness, model or caps.llm.get("model") or "preflight-model")
+        if harness
+        else None
     )
+    if json_output:
+        data = caps.to_dict()
+        if checked is not None:
+            data["harness"] = checked
+        print(json.dumps(data, indent=2))
+    else:
+        print(caps.render(verbose=verbose))
+        if checked is not None:
+            print("\nharness preflight (construction only; capture is untested)")
+            print(json.dumps(checked, indent=2))
+    if checked is not None and not checked["ok"]:
+        raise typer.Exit(1)
 
 
 @app.command("rollout")
@@ -399,6 +480,16 @@ def serve(
     reward_key: Annotated[
         str, typer.Option("--reward-key", help=_REWARD_KEY_HELP)
     ] = "",
+    allow_harness: Annotated[
+        Optional[list[str]],
+        typer.Option(
+            "--allow-harness",
+            help=(
+                "Accept this `module:Class` harness in clients' rollouts. Naming a class imports "
+                "its module on this server, so other import paths are refused. Repeatable."
+            ),
+        ),
+    ] = None,
 ) -> None:
     """Serve Harbor tasks over the OpenEnv Task API and MCP.
 
@@ -422,6 +513,8 @@ def serve(
     )
     # The UI's laptop defaults (this machine's token, private URLs) are for a loopback-only server.
     os.environ["OPENENV_HARBOR_UI_HOST"] = host
+    if allow_harness:
+        os.environ["OPENENV_HARBOR_ALLOW_HARNESS"] = ",".join(allow_harness)
     serve_harbor(
         llm_url=llm_url,
         model=model or None,