feat(agents): add the UiPath Delegate SDK agent as a built-in harness [PILOT-7267]

README.md

@@ -12,7 +12,7 @@
 **evaluating and benchmarking AI coding agents and their skills** — built for CLI
 and skill builders — with sandboxing, reproducibility, and data-driven analysis.
 It runs a real agent (**Claude Code**, **Codex**, **Google Antigravity /
-Gemini**, or **OpenCode**) in a sandbox against declarative YAML tasks, then scores the files and
+Gemini**, **OpenCode**, or **UiPath Delegate**) in a sandbox against declarative YAML tasks, then scores the files and
 commands it actually produced. Not an "agentic coding" benchmark: it measures how
 effective your CLI and skills are when used by coding agents.
 
@@ -33,7 +33,7 @@ telemetry. See [How it compares](https://coder-eval.com/docs/comparison).
 - **Sandboxed execution** in isolated environments with resource limits
 - **Weighted, continuous scoring** (0.0–1.0) with fractional credit and thresholds
 - **Many criterion types** — from file checks to code similarity and LLM-graded rubrics
-- **Agent abstraction** — Claude Code, Codex, Antigravity (Gemini), and OpenCode today, extensible via a plugin SPI
+- **Agent abstraction** — Claude Code, Codex, Antigravity (Gemini), OpenCode, and UiPath Delegate today, extensible via a plugin SPI
 - **Experiment layer** — A/B agent configs (models, tools, prompts) side-by-side
 - **Full telemetry** — every tool call, token counts, and cost, with real-time streaming
 
@@ -216,6 +216,7 @@ The step's exit code is coder-eval's own: non-zero on any failed task.
 | [Codex](docs/agents/CODEX.md) | Running the OpenAI Codex agent |
 | [Antigravity (Gemini)](docs/agents/ANTIGRAVITY.md) | Running the Google Antigravity / Gemini agent |
 | [OpenCode](docs/agents/OPENCODE.md) | Running the OpenCode agent on open-weight models |
+| [UiPath Delegate SDK](docs/agents/DELEGATE_SDK.md) | Running UiPath Autopilot's Delegate agent via the @uipath/delegate-stdio host |
 | [Run-Limit Parity](docs/agents/HARNESS_PARITY.md) | What each run_limits field means on every harness |
 | [A/B Experiments](docs/AB_EXPERIMENTS.md) | Compare models / tools / prompts across the same tasks |
 | [Bring Your Own Dataset](docs/DATASETS.md) | Fan a single task out over a dataset |

docker/Dockerfile

@@ -64,6 +64,10 @@ COPY experiments/default.yaml ./experiments/default.yaml
 # `--driver docker` does not support it. Adding it means a pinned version that
 # travels with the release tag (as CLAUDE_CODE_VERSION does) plus an
 # env_passthrough block; see docs/agents/OPENCODE.md "Running in Docker".
+# The same holds for `delegate-sdk`: its `@uipath/delegate-stdio` Node host is
+# not baked in, and its UiPath auth env is not in the default allowlist -- a
+# docker run needs an overlay image + `-D sandbox.docker.env_passthrough_extra`;
+# see docs/agents/DELEGATE_SDK.md "Running in Docker".
 #
 # CODER_EVAL_UV_EXTRAS carries ADDITIONAL opt-in extras on top of those; it
 # defaults to none. `make docker-image-full` passes `--extra uipath`, which

docs/EXTENDING.md

@@ -285,8 +285,9 @@ The base package ships **no** plugin rates; only the built-in table.
 ## See also
 
 - [Claude Code](agents/CLAUDE_CODE.md) · [Codex](agents/CODEX.md) ·
-  [Antigravity](agents/ANTIGRAVITY.md) · [OpenCode](agents/OPENCODE.md) — the
-  built-in agents, each registered via this same SPI
+  [Antigravity](agents/ANTIGRAVITY.md) · [OpenCode](agents/OPENCODE.md) ·
+  [UiPath Delegate SDK](agents/DELEGATE_SDK.md) — the built-in agents, each
+  registered via this same SPI
 - [Task Definition Guide](TASK_DEFINITION_GUIDE.md) — the criterion catalogue
 - [CLAUDE.md](https://github.com/UiPath/coder_eval/blob/main/CLAUDE.md) — architecture
   and extension points in depth

docs/USER_GUIDE.md

@@ -39,7 +39,7 @@ coder-eval run tasks/hello_date.yaml --stream full  # live LLM output
 | `-D path=value` / `--set` | Override any resolved task-config field (`agent`/`run_limits`/`sandbox` roots), e.g. `-D run_limits.max_turns=30 -D agent.permission_mode=plan -D agent.sdk_options.effort=high`. Repeatable; schema-validated. This is the way to set permission mode, turn/timeout limits, token/USD budget caps, tools, plugins, and SDK options. |
 | `--model, -m` | Shorthand alias for `-D agent.model=…` (e.g., `claude-sonnet-5`) |
 | `--driver` | Shorthand alias for `-D sandbox.driver=…` (`tempdir` or `docker`) |
-| `--type, -T` | Override agent type for all tasks (`claude-code`, `codex`, `antigravity`, `opencode`, or a plugin kind). |
+| `--type, -T` | Override agent type for all tasks (`claude-code`, `codex`, `antigravity`, `opencode`, `delegate-sdk`, or a plugin kind). |
 | `--repeats` | Run each `(task, variant)` N times (≥1); overrides experiment/variant `repeats:`. See [Replicates](#replicates). |
 | `--resume` | Resume an interrupted run: skip tasks already finalized in `--run-dir` and run the rest, folding prior results into `run.json`. Requires `--run-dir`. A task with *any* final status (incl. FAILED/ERROR) counts as finalized, so resume does **not** retry failures — delete a task's `task.json` to force a re-run. A config mismatch is warned, not refused. |
 | `--sample N` | For dataset-backed tasks, run a fixed-seed random N-row sample (reproducible; cheap smoke test). See [Bring Your Own Dataset](DATASETS.md). |