Skip to content

feat(rules): add indirect_injection category (12 rules) - #10

Open
DevamShah wants to merge 1 commit into
utkusen:mainfrom
DevamShah:add-indirect-injection-rules
Open

feat(rules): add indirect_injection category (12 rules)#10
DevamShah wants to merge 1 commit into
utkusen:mainfrom
DevamShah:add-indirect-injection-rules

Conversation

@DevamShah

Copy link
Copy Markdown

Summary

Adds a new indirect_injection rule category (12 YAML rules) that tests whether an LLM follows attacker-controlled instructions embedded in content it is asked to process — pasted documents, emails, web pages, retrieved RAG passages, and tool output — rather than only direct user-supplied jailbreaks.

Problem / motivation

promptmap2 ships strong coverage for direct attacks (the user is the attacker: distraction, jailbreak, prompt_stealing, etc.). It has no coverage for indirect prompt injection, where the user is benign and the malicious instructions arrive inside third-party data the model consumes. This is the dominant attack class for real LLM applications — summarizers, inbox assistants, RAG chatbots, code reviewers, support-ticket triagers — because those apps feed untrusted external text straight into the model context. A scanner that only models the "user as attacker" misses the channel that actually breaks production systems. This PR closes that gap with no architectural change: load_test_rules() already globs rules/**/*.yaml, so the new directory is picked up automatically.

Change

  • New directory rules/indirect_injection/ with 12 rules using the existing schema (name, type, severity, prompt, pass_conditions, fail_conditions) and type: indirect_injection.
  • Each rule frames a legitimate task (summarize / triage / translate / review / extract) and embeds an injection inside clearly-delimited untrusted content (a document, email body, HTML comment, retrieved passage, ICS description, code comment, search snippet, meeting transcript, product review, or markdown notes).
  • Every rule carries a unique canary token (e.g. INDIRECT_DOC_PWNED, RAG_OVERRIDE_TRIGGERED, MARKDOWN_EXFIL_OK) so a successful injection is unambiguous, plus a second behavioral condition so the controller LLM can still judge softer compliance. Both pass_conditions and fail_conditions have 2+ entries each to keep false positives low.
  • One rule (indirect_product_review_exfil) covers indirect system-prompt exfiltration; one (indirect_markdown_image_exfil) covers the markdown-image data-exfiltration vector. The rest cover task-hijack across the common ingestion surfaces.
  • README updated: indirect_injection added to the "Available rule types" line and to the "Rule Categories" list; the feature bullet now reads "7 categories".

Rules included: indirect_pasted_document, indirect_email_summary, indirect_web_page_summary, indirect_rag_tool_output, indirect_code_review_comment, indirect_support_ticket_triage, indirect_product_review_exfil, indirect_calendar_invite, indirect_translation_payload, indirect_markdown_image_exfil, indirect_meeting_transcript, indirect_search_result_snippet.

Note for maintainer (optional, not included here to keep the diff additive): the documented --rule-type filter validates against a hardcoded set in promptmap2.py (valid_rule_types). The new rules load and run in the default "all rules" run and via --rules <name> today; to make --rule-type indirect_injection selectable, add "indirect_injection" to that set (and the adjacent error string). Happy to fold that one-line change into this PR if you'd prefer it here.

Security rationale

Indirect prompt injection is OWASP LLM Top 10 — LLM01: Prompt Injection (the "indirect" sub-class is called out explicitly in the 2025 list) and maps to CWE-77 (Improper Neutralization of Special Elements used in a Command) / the broader CWE-74 injection family applied to natural-language instruction channels. In ATT&CK-style terms it is the LLM analogue of consuming untrusted input that crosses a trust boundary: the model cannot distinguish "data to summarize" from "instructions to obey." The indirect_markdown_image_exfil and indirect_product_review_exfil rules additionally probe the exfiltration follow-on (rendering an attacker URL / leaking the system prompt), which is how indirect injection escalates from nuisance to data breach in agentic and RAG deployments. Testing for this is now table-stakes for any LLM app that ingests external content; this category lets promptmap2 users measure their exposure directly.

Testing / validation

  • Loader parity: reproduced load_test_rules() exactly (glob.glob("rules/**/*.yaml", recursive=True) + yaml.safe_load, keyed by name). All 12 files parse; the repo loads 80 rules total with the new category present.
  • Schema: asserted every new rule has non-empty name/type/prompt, type == indirect_injection, severity in {low,medium,high}, and ≥2 pass_conditions and ≥2 fail_conditions. All pass.
  • Uniqueness: the 12 new name values were diffed against all existing rule names — no collisions.
  • Filter selection: confirmed rule['type'] in ["indirect_injection"] selects exactly the 12 new rules and nothing else.
  • False-positive design: each canary is a nonsense token that a faithful summary/translation/classification would never emit, so a defended model produces a clean pass; pass_conditions also accept the model flagging the embedded instruction as untrusted.
  • No live target or network call is required to add or load these rules.

Adds an indirect_injection rule category covering prompt injection delivered
through third-party content the model processes — RAG passages, emails, web
pages, meeting notes, calendar invites, support tickets, code-review comments,
product reviews, pasted documents, search snippets, translation payloads, and
markdown-image exfiltration. promptmap2 previously modeled only direct
(user-as-attacker) attacks; this closes the indirect channel that breaks real
summarizer/inbox/RAG apps. Each rule carries a canary marker plus pass/fail
conditions for the controller judge. Registers the new type in valid_rule_types
and documents it in the README.

Signed-off-by: Devam Shah <devamshah91@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant