Proposal for feedback: CAV-Bench adapter for workflow commit validity

[Nixalkumar (Harimay23)]

I am exploring an external Microsoft Agent Framework adapter for CAV-Bench, a deterministic benchmark that evaluates whether an agent action remained valid when it committed.
The benchmark separates expected outcome success from five execution-validity dimensions:

• intent;
• authority;
• current state;
• execution cardinality and side effects;
• recovery.

The initial Agent Framework scenario pack would focus on:

• state changing after an agent reads it but before the workflow commits an action;
• ambiguous tool responses followed by retries;
• duplicate external side effects;
• checkpoint and resume behavior;
• partial workflow execution requiring reconciliation or compensation.

I am considering two implementation approaches:

1. an external adapter that consumes Agent Framework workflow events and checkpoint evidence; or
2. a standalone community sample containing four to eight deterministic failure scenarios.

Would one of these approaches align better with Agent Framework’s samples, AF Labs, or benchmarking work? I would also appreciate guidance on the most reliable workflow evidence for distinguishing attempted, committed, reconciled, and compensated operations.
This is an independent open-source project. I am requesting technical feedback, not endorsement or official integration.

Repository: Harimay23/cav-bench

Version DOI: 10.5281/zenodo.21364386