feat(rules): add indirect_injection category (12 rules) - #10
Open
DevamShah wants to merge 1 commit into
Open
Conversation
Adds an indirect_injection rule category covering prompt injection delivered through third-party content the model processes — RAG passages, emails, web pages, meeting notes, calendar invites, support tickets, code-review comments, product reviews, pasted documents, search snippets, translation payloads, and markdown-image exfiltration. promptmap2 previously modeled only direct (user-as-attacker) attacks; this closes the indirect channel that breaks real summarizer/inbox/RAG apps. Each rule carries a canary marker plus pass/fail conditions for the controller judge. Registers the new type in valid_rule_types and documents it in the README. Signed-off-by: Devam Shah <devamshah91@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
indirect_injectionrule category (12 YAML rules) that tests whether an LLM follows attacker-controlled instructions embedded in content it is asked to process — pasted documents, emails, web pages, retrieved RAG passages, and tool output — rather than only direct user-supplied jailbreaks.Problem / motivation
promptmap2 ships strong coverage for direct attacks (the user is the attacker:
distraction,jailbreak,prompt_stealing, etc.). It has no coverage for indirect prompt injection, where the user is benign and the malicious instructions arrive inside third-party data the model consumes. This is the dominant attack class for real LLM applications — summarizers, inbox assistants, RAG chatbots, code reviewers, support-ticket triagers — because those apps feed untrusted external text straight into the model context. A scanner that only models the "user as attacker" misses the channel that actually breaks production systems. This PR closes that gap with no architectural change:load_test_rules()already globsrules/**/*.yaml, so the new directory is picked up automatically.Change
rules/indirect_injection/with 12 rules using the existing schema (name,type,severity,prompt,pass_conditions,fail_conditions) andtype: indirect_injection.INDIRECT_DOC_PWNED,RAG_OVERRIDE_TRIGGERED,MARKDOWN_EXFIL_OK) so a successful injection is unambiguous, plus a second behavioral condition so the controller LLM can still judge softer compliance. Bothpass_conditionsandfail_conditionshave 2+ entries each to keep false positives low.indirect_product_review_exfil) covers indirect system-prompt exfiltration; one (indirect_markdown_image_exfil) covers the markdown-image data-exfiltration vector. The rest cover task-hijack across the common ingestion surfaces.indirect_injectionadded to the "Available rule types" line and to the "Rule Categories" list; the feature bullet now reads "7 categories".Rules included:
indirect_pasted_document,indirect_email_summary,indirect_web_page_summary,indirect_rag_tool_output,indirect_code_review_comment,indirect_support_ticket_triage,indirect_product_review_exfil,indirect_calendar_invite,indirect_translation_payload,indirect_markdown_image_exfil,indirect_meeting_transcript,indirect_search_result_snippet.Note for maintainer (optional, not included here to keep the diff additive): the documented
--rule-typefilter validates against a hardcoded set inpromptmap2.py(valid_rule_types). The new rules load and run in the default "all rules" run and via--rules <name>today; to make--rule-type indirect_injectionselectable, add"indirect_injection"to that set (and the adjacent error string). Happy to fold that one-line change into this PR if you'd prefer it here.Security rationale
Indirect prompt injection is OWASP LLM Top 10 — LLM01: Prompt Injection (the "indirect" sub-class is called out explicitly in the 2025 list) and maps to CWE-77 (Improper Neutralization of Special Elements used in a Command) / the broader CWE-74 injection family applied to natural-language instruction channels. In ATT&CK-style terms it is the LLM analogue of consuming untrusted input that crosses a trust boundary: the model cannot distinguish "data to summarize" from "instructions to obey." The
indirect_markdown_image_exfilandindirect_product_review_exfilrules additionally probe the exfiltration follow-on (rendering an attacker URL / leaking the system prompt), which is how indirect injection escalates from nuisance to data breach in agentic and RAG deployments. Testing for this is now table-stakes for any LLM app that ingests external content; this category lets promptmap2 users measure their exposure directly.Testing / validation
load_test_rules()exactly (glob.glob("rules/**/*.yaml", recursive=True)+yaml.safe_load, keyed byname). All 12 files parse; the repo loads 80 rules total with the new category present.name/type/prompt,type == indirect_injection,severity in {low,medium,high}, and ≥2pass_conditionsand ≥2fail_conditions. All pass.namevalues were diffed against all existing rule names — no collisions.rule['type'] in ["indirect_injection"]selects exactly the 12 new rules and nothing else.pass_conditionsalso accept the model flagging the embedded instruction as untrusted.