Add conformance/negative/: vectors, failure codes, and runner (closes the suite gap for the enforcement layer)

conformance/negative/README.md

@@ -0,0 +1,95 @@
+# Negative conformance vectors for the enforcement layer
+
+Implements the proposal of issue #29 and the conformance-suite gap of issue #19.
+
+The security value of an enforcement point is defined by its behaviour on inputs that are
+*almost* right. This directory holds test vectors for those inputs: actions that are correctly
+signed but unauthorized, replayed, stale, bound to the wrong principal, carried by malformed
+evidence, or stripped of their observability references.
+
+## Design rules
+
+**1. Two rejection verdicts, never one.** A rejection because evidence was measured and found
+wrong (`REJECT`) is a different verdict from a rejection because evidence could not be measured
+at all (`UNMEASURABLE`). Both must refuse the action. Collapsing them either lets unmeasurable
+inputs fail open, or sends operators hunting for frauds that never happened. Every vector
+declares which of the two it expects.
+
+**2. Every category carries a positive control.** Each category includes at least one
+almost-identical input that MUST pass. A suite made only of must-reject inputs cannot
+distinguish an enforcement layer that works from one that rejects everything. The runner fails
+the whole suite if any category lacks its positive control.
+
+**3. Vectors are framework-agnostic.** A vector describes an action, its authorization
+evidence, and the state the enforcement point can consult. It never references a specific
+implementation. Each implementation provides one adapter (see `runner.py`).
+
+**4. Declared gaps are part of the suite.** Category 3 (expired or revoked mandates) is
+covered here for expiry only. Revocation vectors are absent because the contributing
+implementation has no revocation mechanism, by documented decision, and vectors for a path
+never exercised in production would be design fiction. The gap is stated rather than filled.
+Contributions from implementations that exercise revocation are the way to close it.
+
+## Failure codes (proposed enumeration)
+
+| Code | Verdict | Meaning |
+|---|---|---|
+| `CONTEXT_NOT_SUPPORTED` | REJECT | Signature valid, but the consulted state does not carry this action (wrong target, closed auction, exceeded bound) |
+| `REPLAY_CONSUMED` | REJECT | The authorization was already used once; one authorization covers one act |
+| `STALE_DECISION` | REJECT | The evidence is older than the declared freshness bound |
+| `PRINCIPAL_MISMATCH` | REJECT | Approval bound to a different agent, tool, or signing device than the one acting |
+| `REFERENCE_MISSING` | REJECT | A required observability reference is absent or zeroed |
+| `REFERENCE_MISMATCH` | REJECT | The named reference diverges from the one already bound to this action's context |
+| `MALFORMED_EVIDENCE` | UNMEASURABLE | Evidence present but not decodable to the declared shape; consumers must fail closed |
+| `ORACLE_UNAVAILABLE` | UNMEASURABLE | The state needed to judge could not be read; absence of measurement is never a pass |
+
+## Vector schema
+
+All eighteen vectors live as a single array in `vectors/negative_vectors.json`; the schema
+below describes one entry.
+
+```json
+{
+  "id": "neg-cat1-001",
+  "category": 1,
+  "description": "what makes this input almost right, and why it must fail",
+  "input": {
+    "action": { },
+    "authorization": { },
+    "state": { }
+  },
+  "expected": {
+    "verdict": "REJECT | UNMEASURABLE | PASS",
+    "code": "one of the enumeration above, absent for PASS",
+    "reason_must_mention": ["substring the refusal message must contain"]
+  },
+  "positive_control": false
+}
+```
+
+`reason_must_mention` exists because a correct verdict with a wrong reason is a latent bug:
+an enforcement point can reject for the wrong cause and pass the suite while hunting the
+wrong class of attack.
+
+## Running
+
+```
+python conformance/negative/runner.py --adapter your_adapter.py
+```
+
+The adapter exposes `evaluate(vector) -> {"verdict": ..., "code": ..., "reason": ...,
+"entry_point": ...}`, where `entry_point` names the production entry point the adapter
+dispatched through for that vector. The runner exits non-zero on: any verdict mismatch,
+any code mismatch (positive controls included), any missing `reason_must_mention`
+substring, any category lacking a positive control (a control counts only if it expects
+`PASS`), any answer naming no entry point, or any category whose positive controls
+resolved through a different entry point than its negative vectors. The last two rules
+exist because a suite whose positive controls certify a test double certifies nothing:
+an adapter that reads the answer key passes every content check and fails only there.
+
+## Provenance
+
+Vectors are generalized from an enforcement layer running in production since June 2026
+(hash-chained decision registries, out-of-band human confirmation, on-chain context checks).
+Each rejection code in the enumeration was produced by a real refusal before it was named
+here. The field notes behind them are public: https://github.com/avp9-nexus/nexus-art

conformance/negative/reference_adapter.py

@@ -0,0 +1,67 @@
+"""Reference adapter: a minimal enforcement layer that satisfies the negative suite.
+
+Ships WITH the suite for one reason: a runner nobody can execute end-to-end is a promise.
+This adapter is the suite's own positive control. It is deliberately small and readable;
+it is not a product.
+"""
+
+
+def evaluate(vector):
+    inp = vector["input"]
+    action, auth, state = inp.get("action", {}), inp.get("authorization", {}), inp.get("state", {})
+
+    def reject(code, reason):
+        return {"verdict": "REJECT", "code": code, "entry_point": "reference_adapter.decide", "reason": reason}
+
+    def unmeasurable(code, reason):
+        return {"verdict": "UNMEASURABLE", "code": code, "entry_point": "reference_adapter.decide", "reason": reason}
+
+    # Evidence shape first: fail closed on what cannot be decoded, with its own code.
+    oracle = state.get("oracle_response", "ABSENT_KEY")
+    if oracle is None:
+        return unmeasurable("ORACLE_UNAVAILABLE", "state could not be measured; absence is never a pass")
+    if isinstance(oracle, dict):
+        if oracle.get("shape") == "address" and not oracle.get("bytes_hex", "").startswith("0" * 24):
+            return unmeasurable("MALFORMED_EVIDENCE", "leading bytes do not encode the declared address type")
+        if oracle.get("shape", "").endswith("_6_fields") and oracle.get("fields_served") != 6:
+            return unmeasurable("MALFORMED_EVIDENCE", f"expected 6 fields, served {oracle.get('fields_served')}")
+
+    # Observability references: the omission itself is refused, before anything else.
+    if "registry_ref" in action:
+        ref = action["registry_ref"]
+        if set(ref.replace("0x", "")) == {"0"}:
+            return reject("REFERENCE_MISSING", "required registry reference is zeroed")
+        target = state.get(action.get("target"), {})
+        bound = target.get("bound_registry_ref")
+        if bound and bound != ref:
+            return reject("REFERENCE_MISMATCH", f"reference diverges from the one bound to this context ({bound})")
+
+    # Principal binding.
+    if auth.get("decision_for") and auth["decision_for"] != action.get("actor"):
+        return reject("PRINCIPAL_MISMATCH", f"decision names agent {auth['decision_for']}, actor differs")
+    if "required_signer" in action and auth.get("connected_device_derives") not in (None, action["required_signer"]):
+        return reject("PRINCIPAL_MISMATCH", "connected device derives a different key; wrong device")
+
+    # Replay.
+    if auth.get("consumed"):
+        return reject("REPLAY_CONSUMED", "this one-time authorization was already used")
+    if auth.get("session_secret") and auth.get("current_session"):
+        if auth["session_secret"] != f"SECRET_OF_SESSION_{auth['current_session']}":
+            return reject("REPLAY_CONSUMED", "secret belongs to a previous session")
+
+    # Freshness.
+    if "decision_written_at" in auth:
+        from datetime import datetime
+        age = (datetime.fromisoformat(state["now"].replace("Z", "+00:00"))
+               - datetime.fromisoformat(auth["decision_written_at"].replace("Z", "+00:00"))).total_seconds()
+        if age > auth["freshness_bound_seconds"]:
+            return reject("STALE_DECISION", f"decision is {int(age)}s old, freshness bound exceeded")
+
+    # Context: signature validity alone never suffices; the state must carry the action.
+    target = state.get(action.get("target"), {})
+    if target.get("settled"):
+        return reject("CONTEXT_NOT_SUPPORTED", "target auction is settled; this action replays a closed context")
+    if "ceiling" in target and int(action.get("amount", 0)) > int(target["ceiling"]):
+        return reject("CONTEXT_NOT_SUPPORTED", f"amount exceeds the ceiling the state carries ({target['ceiling']})")
+
+    return {"verdict": "PASS", "code": None, "entry_point": "reference_adapter.decide", "reason": "context supports the action"}

conformance/negative/runner.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python3
+"""Negative conformance runner (issue #29, gap #19).
+
+Loads the vector suite, calls one implementation adapter per vector, and fails on:
+  - any verdict mismatch,
+  - any failure-code mismatch (positive controls included),
+  - any missing reason substring (a right verdict for a wrong reason is a latent bug),
+  - any category lacking its POSITIVE CONTROL (a control counts only if it expects PASS),
+  - any adapter answer that names no entry_point, and any category whose positive
+    controls dispatched through a different entry point than its negative vectors
+    (a suite that certifies a test double certifies nothing - review 4996153628).
+
+The last rule is structural, not cosmetic: a suite made only of must-reject inputs cannot
+distinguish an enforcement layer that works from one that rejects everything.
+
+Adapter contract: a Python file exposing
+    evaluate(vector: dict) -> {"verdict": "PASS|REJECT|UNMEASURABLE", "code": str|None,
+                               "reason": str, "entry_point": str}
+where entry_point names the production entry point the adapter dispatched through.
+
+Usage:
+    python runner.py --adapter path/to/adapter.py [--vectors vectors/negative_vectors.json]
+"""
+import argparse
+import importlib.util
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+HERE = Path(__file__).parent
+
+
+def load_adapter(path: Path):
+    # An adapter spanning several files must be able to import its siblings (review 4996153628).
+    sys.path.insert(0, str(path.resolve().parent))
+    spec = importlib.util.spec_from_file_location("acs_adapter", path)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    if not hasattr(mod, "evaluate"):
+        sys.exit(f"FATAL: adapter {path} exposes no evaluate(vector)")
+    return mod
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--adapter", required=True, type=Path)
+    ap.add_argument("--vectors", type=Path, default=HERE / "vectors" / "negative_vectors.json")
+    args = ap.parse_args()
+
+    suite = json.loads(args.vectors.read_text(encoding="utf-8"))
+    vectors = suite["vectors"]
+    adapter = load_adapter(args.adapter)
+
+    # Structural gate first: every category present must carry a positive control.
+    cats = defaultdict(lambda: {"neg": 0, "pos": 0})
+    for v in vectors:
+        # A declared flag is not a positive control; only an expected PASS is (review 4996153628:
+        # a must-reject vector flagged positive_control satisfied the gate with zero must-pass inputs).
+        is_pos = v.get("positive_control") and v["expected"]["verdict"] == "PASS"
+        cats[v["category"]]["pos" if is_pos else "neg"] += 1
+    missing = [c for c, k in sorted(cats.items()) if k["pos"] == 0]
+    if missing:
+        print(f"SUITE INVALID: categories without a positive control: {missing}")
+        print("A suite of pure rejections cannot tell a working layer from one that rejects everything.")
+        return 2
+
+    failures = []
+    entry_points = defaultdict(lambda: {"pos": set(), "neg": set()})
+    for v in vectors:
+        try:
+            out = adapter.evaluate(v) or {}
+        except Exception as e:  # an adapter crash is a failure, never a skip
+            failures.append((v["id"], f"adapter raised {type(e).__name__}: {e}"))
+            continue
+        exp = v["expected"]
+        # The suite certifies an enforcement layer, not a test double (review 4996153628):
+        # every adapter answer must name the production entry point it dispatched through.
+        ep = out.get("entry_point")
+        if not ep:
+            failures.append((v["id"], "adapter reported no entry_point - cannot tell the enforcement layer from a test double"))
+            continue
+        entry_points[v["category"]]["pos" if v.get("positive_control") else "neg"].add(ep)
+        if out.get("verdict") != exp["verdict"]:
+            failures.append((v["id"], f"verdict {out.get('verdict')!r} != expected {exp['verdict']!r}"))
+            continue
+        # code and reason are compared on PASS vectors too (review 4996153628: a positive
+        # control's code field previously went uncompared).
+        if True:
+            if out.get("code") != exp.get("code"):
+                failures.append((v["id"], f"code {out.get('code')!r} != expected {exp.get('code')!r}"))
+                continue
+            reason = (out.get("reason") or "").lower()
+            for needle in exp.get("reason_must_mention", []):
+                if needle.lower() not in reason:
+                    failures.append((v["id"], f"reason does not mention {needle!r}: {reason[:120]!r}"))
+                    break
+
+    for c, k in sorted(entry_points.items()):
+        if k["pos"] and k["neg"] and k["pos"] != k["neg"]:
+            failures.append((f"cat{c}", f"positive controls resolved through {sorted(k['pos'])} but negatives through {sorted(k['neg'])} - the gate certified a different code path than the one negatively tested"))
+
+    print(f"{len(vectors) - len(failures)}/{len(vectors)} vectors conform "
+          f"({sum(k['pos'] for k in cats.values())} positive controls across {len(cats)} categories)")
+    for vid, why in failures:
+        print(f"  FAIL {vid}: {why}")
+    return 1 if failures else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

conformance/negative/vector.schema.json

@@ -0,0 +1,41 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "ACS negative conformance vector suite",
+  "type": "object",
+  "required": ["suite", "vectors"],
+  "properties": {
+    "suite": { "type": "string" },
+    "vectors": {
+      "type": "array",
+      "minItems": 1,
+      "items": {
+        "type": "object",
+        "required": ["id", "category", "description", "input", "expected", "positive_control"],
+        "properties": {
+          "id": { "type": "string", "pattern": "^(neg|pos)-cat[0-9]+-[0-9]{3}$" },
+          "category": { "type": "integer", "minimum": 1, "maximum": 6 },
+          "description": { "type": "string", "minLength": 20 },
+          "input": {
+            "type": "object",
+            "required": ["action", "authorization", "state"],
+            "properties": {
+              "action": { "type": "object" },
+              "authorization": { "type": "object" },
+              "state": { "type": "object" }
+            }
+          },
+          "expected": {
+            "type": "object",
+            "required": ["verdict"],
+            "properties": {
+              "verdict": { "enum": ["PASS", "REJECT", "UNMEASURABLE"] },
+              "code": { "enum": ["CONTEXT_NOT_SUPPORTED", "REPLAY_CONSUMED", "STALE_DECISION", "PRINCIPAL_MISMATCH", "REFERENCE_MISSING", "REFERENCE_MISMATCH", "MALFORMED_EVIDENCE", "ORACLE_UNAVAILABLE"] },
+              "reason_must_mention": { "type": "array", "items": { "type": "string" } }
+            }
+          },
+          "positive_control": { "type": "boolean" }
+        }
+      }
+    }
+  }
+}