Skip to content

docs: worked example for an alternative rule format under the §5.4 clarification#41

Open
eeee2345 wants to merge 1 commit into
CiscoDevNet:mainfrom
eeee2345:docs/worked-example-alternative-rule-format
Open

docs: worked example for an alternative rule format under the §5.4 clarification#41
eeee2345 wants to merge 1 commit into
CiscoDevNet:mainfrom
eeee2345:docs/worked-example-alternative-rule-format

Conversation

@eeee2345

Copy link
Copy Markdown

The §5.4 rule-corpus clarification (spec.md line 450) invites adopters to use the seed's CodeGuard worked example, fork it, or substitute another format that satisfies FR-037 and FR-041. The current docs ship one worked example (example-detection-rule.md, CodeGuard, source-function units). Adopters whose targets are agentic-system artifacts — MCP servers, skill files, tool manifests, IO traces — lack a sibling worked example for the case where the unit being checked is not a source function.

This PR adds one such sibling. It walks an alternative open rule format (Agent Threat Rules, MIT-licensed, github.com/Agent-Threat-Rule/agent-threat-rules) and maps a sample rule to the same Detector FRs the CodeGuard worked example covers (FR-037, FR-041, FR-042, FR-043, FR-044, FR-045, FR-090, US-14). The framing is non-exclusive throughout: ATR is one example of a format that satisfies the §5.4 contract, not a recommendation against CodeGuard, and the docs note mixed corpora are valid.

Why this fits the docs conventions:

  • Cites the spec; restates nothing normatively (per docs/README.md).
  • Marks audience at top (operator answering the §5.4 clarification).
  • Provides options and decision criteria, not a mandate.
  • Adds no new requirements; the requirement is FR-041, which is format-independent.

Files:

  • docs/worked-examples/example-alternative-rule-format.md (new, 92 lines)
  • docs/README.md (5-line audience-map entry pointing at both worked examples)

Context: I authored the ATR rule corpus. ATR has been merged into Cisco AI Defense's skill-scanner via PRs #79 and #99. This PR is independent of that work and improves the spec for any adopter picking a format, not the ATR project specifically; I would be glad to revise framing or drop ATR-specific references if the maintainers prefer a generic "non-CodeGuard worked example" with a different reference format.

The §5.4 rule-corpus clarification invites adopters to use the seed's
CodeGuard worked example, fork it, or substitute another format that
satisfies FR-037 and FR-041. Adopters with agentic-system targets
(MCP servers, skill files, tool manifests) need a worked example for
artifacts that don't map onto source-function units.

This adds docs/worked-examples/example-alternative-rule-format.md
walking one such substitution (ATR, MIT-licensed) and maps it to the
same Detector FRs the CodeGuard example covers. The new file is a
sibling of example-detection-rule.md, not a replacement.

docs/README.md gains a small audience-map entry pointing readers
picking a rule format at both worked examples.
@santosomar santosomar requested review from jallbrit, santosomar and thomas-bartlett and removed request for santosomar and thomas-bartlett June 10, 2026 23:55

@thomas-bartlett thomas-bartlett left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, and I appreciate the disclosure. I agree there is a useful idea here, but I do not think this belongs in docs/ as written. It expands what Foundry evaluates, not just how a rule is formatted, and docs/README.md says new requirements belong in spec.md.

Foundry’s Target is source code plus an optional testbed (spec.md:100). The spec does not define MCP manifests, skill files, tool manifests, or agent IO traces as first-class target units. That scope change then reinterprets FR-037 (spec.md:423, per-function rule sweep), FR-043 (spec.md:440, file/function location), and FR-090 (spec.md:654, fixed fingerprint tuple). In particular, “the spec is unit-agnostic” contradicts FR-090’s MUST.

You offered to drop the ATR-specific framing for a generic non-CodeGuard example. I would take that path, after a vendor-neutral spec.md proposal lands.

@santosomar

Copy link
Copy Markdown
Collaborator

I agree with Thomas here. I think this is the right call.
The key distinction is format vs. target. §5.4 lets adopters swap the rule format as long as FR-037 and FR-041 are still satisfied. What this PR actually does is change the unit under evaluation — from source functions to MCP manifests, skill files, tool manifests, and IO traces. That’s a Target scope change, not a format substitution, and per docs/README.md new requirements/scope belong in spec.md, not docs/.
The FR-090 point is the clearest blocker: the fingerprint tuple is a normative MUST built around a specific unit model, so “the spec is unit-agnostic” can’t be asserted in docs/ without first changing the spec. FR-037 (per-function sweep) and FR-043 (file/function location) carry the same source-function assumption.
None of this is a knock on the underlying idea, agentic artifact targets are a real gap, and the COI disclosure is appreciated. I’d just sequence it the way Thomas suggested.

@santosomar santosomar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated in my comment above

@eeee2345

eeee2345 commented Jul 7, 2026

Copy link
Copy Markdown
Author

You're right, and thanks for the precise framing — format-vs-target is the distinction I collapsed. Changing the unit from source functions to MCP manifests / skill files / tool manifests / IO traces is a Target scope change, and asserting the spec is unit-agnostic in docs/ without first changing the spec (FR-090's fingerprint tuple, FR-037's per-function sweep, FR-043's location) was putting the claim in the wrong place.

I've sequenced it the way you and Thomas suggested and opened #44 to do this at the spec level first: a MINOR widening of Principle VIII from "path + symbol" to a structural identity that also covers agentic artifacts, with FR-090a/043a/037a as opt-in siblings behind [NEEDS CLARIFICATION], and mixed-target scans / the Evidence-gate analog explicitly left out to keep the blast radius small. The failure it's grounded in is the SC-005 one Principle VIII already names — artifact findings can't form a fingerprint today, so dedup collapses.

I'll hold this PR pending the outcome of #44; once the unit model lands I'll rework the worked example to ride on it (and I'm happy for it to be a generic non-CodeGuard example rather than an ATR-specific one, per my earlier note). Closing or converting this to draft — whichever you prefer for tracking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants