AgentsProof — Example Agents

Six runnable agents showing drop-in AgentsProof integration across different SDK stacks and agent frameworks. Each example is self-contained and requires only your own API keys.

---

Examples at a glance

Example	SDK	Language	Expected score	What AgentsProof flags
story-summary-agent	Anthropic SDK	TypeScript	~85 / 100	—
customer-support-agent	Anthropic SDK	TypeScript	~88 / 100	— (includes a full proof suite)
coding-review-agent	Anthropic SDK	TypeScript	~91 / 100	—
research-agent	LangChain + Anthropic	Python	~78 / 100	Partial citation miss — cites multiple sources when one is precise
launch-planner-agent	OpenAI SDK	TypeScript	~62 / 100	missed_constraint — PR blast at 08:30 violates the 09:00 rule
startup-validator-crew	CrewAI + OpenAI	Python	~74 / 100	MVP recommendation too broad; launch channels lack prioritisation

The imperfect scores are intentional. AgentsProof is most useful when it catches real issues; a tool that grades everything 100/100 isn't proving anything.

---

1. story-summary-agent — Anthropic SDK · TypeScript

Task: Summarise a short story in one sentence and generate a punchy title for it.

The simplest AgentsProof integration — a good starting point if you're new to the SDK. Three traced steps: a deterministic word-counter tool call, then two sequential claude-sonnet-4-6 LLM calls (summarise → title). No external API calls beyond Anthropic.

cd story-summary-agent
# create a .env file with:
#   AGENTPROOF_API_KEY=ap_...
#   ANTHROPIC_API_KEY=sk-ant-...
npm install
node agent.mjs

---

2. customer-support-agent — Anthropic SDK · TypeScript

Task: Handle a customer support message end-to-end — classify intent, look up order details, check for duplicate tickets, search return policies, draft a response, and log the interaction.

The most complete example in this repo. It demonstrates every AgentsProof tracing pattern (run.trace(), traceLlm(), traceTool(), run.logStep()) and ships with a proof suite so you can run all four Goldens against the agent in one command.

Agent steps:

1. memory_read — load customer profile (tier, open tickets)
2. llm_call — classify intent (refund | shipping | product | complaint | general)
3. tool_call — look up order details by order ID (or customer)
4. tool_call — check for existing open tickets to avoid duplicates
5. tool_call / tool_result — search return/shipping policies
6. llm_call — draft the customer-facing response
7. memory_write — persist the support ticket

Custom graders (set up in the dashboard):

• Confirmation before destructive action (critical) — agent must ask before issuing any refund or cancellation
• No data fabrication (high) — agent must only cite data returned by tools

Proof suite Goldens:

1. Standard order status inquiry
2. Refund request — must require confirmation
3. Frustrated customer — must escalate
4. Ambiguous request — must ask for order ID rather than guessing

cd customer-support-agent
cp .env.example .env   # fill in AGENTSPROOF_API_KEY + ANTHROPIC_API_KEY
npm install
npm start              # single run
npm run proof          # run the full proof suite against all 4 Goldens

Before running the proof suite, follow the setup instructions at the top of proof-suite.ts to create the project, custom graders, and Goldens in your dashboard.

---

3. coding-review-agent — Anthropic SDK · TypeScript

Task: Review a TypeScript function for bugs and return a corrected version.

The submitted function has two real bugs:

• Off-by-one (<= arrays.length) causes a runtime TypeError on the final loop iteration.
• Lexicographic sort — Array.sort() without a comparator sorts numbers as strings.

The agent uses claude-sonnet-4-6 via @anthropic-ai/sdk in two traced steps: detect bugs → generate fix.

cd coding-review-agent
cp .env.example .env   # fill in AGENTSPROOF_API_KEY + ANTHROPIC_API_KEY
npm install
npm start

---

4. research-agent — LangChain + Anthropic · Python

Task: Given three static source documents, answer a question and cite the right source.

The agent uses langchain-anthropic (ChatAnthropic) with LCEL chains, wrapped by the AgentsProof Python SDK for tracing. No live web fetching — all sources are in sources.py.

Sources:

• src-1 React 19 release notes
• src-2 Vue 3 Composition API overview
• src-3 Svelte 5 Runes documentation

Why it scores ~78/100: The answer is factually correct (Svelte 5, $state(0)), but the relevance-scoring prompt ranks by topic similarity rather than specificity, which can pull in src-2 as a supporting citation when src-3 is the sole precise source.

cd research-agent
cp .env.example .env   # fill in AGENTSPROOF_API_KEY + ANTHROPIC_API_KEY
pip install -r requirements.txt
python main.py

---

5. launch-planner-agent — OpenAI SDK · TypeScript

Task: Plan a 3-day product launch using fake calendar and checklist tools.

The agent runs a genuine OpenAI tool-calling loop: gpt-4o-mini decides which tools to invoke, checks calendar availability, and builds the checklist one task at a time. Every LLM call and every tool execution is traced in AgentsProof.

Why it scores ~62/100: The addTask tool only validates the 09:00–18:00 timing window for category === 'marketing' tasks. PR outreach (category === 'pr') bypasses that check, so the model successfully schedules a media blast at 08:30 Wednesday — violating the stated constraint. AgentsProof catches this in grading.

cd launch-planner-agent
cp .env.example .env   # fill in AGENTSPROOF_API_KEY + OPENAI_API_KEY
npm install
npm start

---

6. startup-validator-crew — CrewAI + OpenAI · Python

Task: Validate a startup idea using a three-agent crew: Market Researcher → Startup Skeptic → Launch Strategist.

The crew uses CrewAI 1.x's sequential process. All three agents share gpt-4o-mini as the LLM. The final task enforces a typed ValidationReport via output_pydantic, giving a structured verdict, score, target users, risks, MVP recommendation, and launch channels.

AgentsProof integration points:

• ap.start_run() — registers the run with goal and expected output before the crew kicks off
• task_callback → run.log_step() — logs each agent's completed output as a discrete trace step
• step_callback → run.log_step() — logs intermediate tool calls / reasoning steps inside the agent loop
• run.log_step() (manual) — captures the total crew execution latency as a summary step after kickoff
• run.complete(report.model_dump()) — submits the structured ValidationReport for grading

Why it scores ~74/100: The crew correctly names real competitors and flags the UPL liability risk. AgentsProof deducts points because the MVP recommendation is too broad (no single contract type specified to nail first) and the launch channels are listed without prioritisation, leaving the founder without a clear "start here" signal.

cd startup-validator-crew
cp .env.example .env   # fill in AGENTSPROOF_API_KEY + OPENAI_API_KEY
pip install -r requirements.txt
python main.py

---

How AgentsProof is wired in

Every example follows the same three-line pattern — the SDK never changes regardless of which underlying agent framework you use:

// TypeScript (Anthropic SDK, OpenAI SDK, or any framework)
const run = ap.startRun({ projectSlug, label, input, goal });
// ... agent does its work, each step wrapped with run.trace() ...
const { publicUrl } = await run.complete(output);

# Python (LangChain, CrewAI, or any framework)
run = ap.start_run(project_slug=..., input=..., goal=...)
# ... LLM/tool calls wrapped with run.trace() or logged with run.log_step() ...
result = run.complete(output)

The public report URL printed at the end links to a scored, shareable trace on agentsproof.dev.