Cost & token accounting for every run: per-role, per-model ledger surfaced on completion

[khaliqgant]

Motivation

We can turn issues into reviewed PRs, but we cannot answer "what did this PR cost?" Cursor's agent-swarm model-economics work makes dollar-per-task the optimization target: the same job ran $10,565 (frontier model in every role) vs $1,339 (smart planner + cheap workers), and workers burn 69–90% of tokens while being a minority of cost. NightCTO already has a metered cost ladder (packages/openrouter-client), but the Factory — the actual code-writing swarm — flies blind. This gives Factory a first-class cost/usage ledger so every run reports its own spend, per role and per model. It is the foundation for budget enforcement and cost-aware model routing (filed as follow-ups, blocked on this).

Goal

Record token + USD usage for every dispatched run, attributed by role (implementer / reviewer / babysitter / triage) and model, and surface a bounded per-run cost summary on run completion — riding the existing observability event pipeline, not a new side channel.

What to build

• A CostLedger accounting seam (src/cost/ledger.ts) that accumulates { runId, role, model, inputTokens, outputTokens, usd } records and exposes getRunTotal(runId) + getRunByRole(runId).
• A pricing table (src/cost/pricing.ts) mirroring the shape of nightcto's packages/openrouter-client/src/models.ts (costPer1MInput / costPer1MOutput keyed by model id) with estimateCostUsd(model, inTok, outTok). An unknown model yields usd: null and a bounded cost.model.unpriced event — never a throw.
• A usage-reporting seam on the fleet/spawn path: when the fleet/agent runtime reports token usage for a spawned agent, record it against the run + role; where the runtime reports nothing, record tokens: null (do not fabricate).
• Emit a bounded run.cost.v1 event on run completion via src/observability/events.ts (respect the closed attribute allowlist — model id, role, token counts, usd only; never task text / paths / prompts). Attach the aggregate to the run record so cloud-reporter.ts and callers can read per-run + per-role spend.

Constraints

• Preserve the bounded-allowlist and non-fatal posture of the observability layer (src/observability/cloud-reporter.ts:63-67) — cost accounting must never alter control flow or become a content side channel.
• Deterministic: no wall-clock / random in ledger identity; keep the per-run traceId correlation from events.ts.

Acceptance

• CostLedger records per-role, per-model token + USD; getRunTotal / getRunByRole return the breakdown.
• src/cost/pricing.ts prices known models; unknown → null + bounded event, no throw.
• A single run.cost.v1 bounded event is emitted on completion and the aggregate is on the run record.
• Manifest updated: add the new feature(s) to .agentworkforce/features/manifest.yaml (id, name, cli/api, description, location → the real new files, verify_tier) and bump catalog.feature_count + the matching tier_counts so featuremap:check stays green.
• Public surface exported from src/index.ts.

✅ End-to-end verification — Factory success criteria (REQUIRED)

"Compiles" is not done. Drive a full run through the in-process fake fleet, feed synthetic per-agent usage, and observe a correct per-role / per-model cost summary emitted — then check the harness in.

Run it

npm run build
npm run featuremap:check
npm test
npm run verify:e2e

The E2E (test/e2e/run-cost-accounting.test.ts, runnable via npm test / verify:e2e) must:

1. Run a dispatch → spawn → writeback lifecycle on the in-process fake fleet (src/fleet/internal-fleet-client.ts) with two roles on two different models, each reporting synthetic token usage.
2. Assert the CostLedger has per-role, per-model records and getRunTotal equals the sum of the priced parts.
3. Assert exactly one bounded run.cost.v1 event is emitted and it carries only allowlisted attributes — a raw task string / path must not be reachable (assert the negative).
4. Assert an unpriced model yields usd: null + the bounded cost.model.unpriced event and does not throw.

Deliverable

• test/e2e/run-cost-accounting.test.ts proving per-run / per-role cost is captured and emitted, one command.
• Fails loudly if usage isn't attributed, if the event carries non-allowlisted content, or if an unpriced model throws (no false green).

Anchor files

• src/orchestrator/factory.ts (dispatch / spawn / writeback / reconcile seams) → new src/cost/{ledger,pricing}.ts
• src/observability/events.ts (bounded event contract; add run.cost.v1), src/observability/cloud-reporter.ts:63-67 (non-fatal posture)
• src/ports/fleet.ts / src/fleet/internal-fleet-client.ts (usage-reporting seam; fake fleet for the E2E)
• src/config/schema.ts (any config toggle), src/index.ts (public barrel)
• .agentworkforce/features/manifest.yaml + .agentworkforce/features/verify/procedures.md
• Pricing-table precedent: nightcto packages/openrouter-client/src/models.ts

Out of scope

• Budget enforcement and cost-aware model routing (separate follow-up issues, blocked on this one).
• Capturing usage from CLIs that report none — record null, don't estimate token counts you don't have.

Related

• Observability follow-up Export OpenTelemetry spans over OTLP (observability v1 follow-up) #120 (OTLP spans around dispatch/spawn/writeback) — same seams.
• Motivation: Cursor "agent swarm model economics".

[kjgbot]

Factory human input request

Agent: ar-185-babysit-factory
Issue: 185
Question: Should PR #190 remain scoped to the Factory producer contract in issue #185 with Cloud consumer/dashboard support delivered in a separate Cloud PR, or must #190 wait until that cross-repository dashboard support is implemented and deployable?

[kjgbot]

Factory observed the linked pull request merge and completed this issue.