Hi — small rule-file bug.
rules/jailbreak/devmode3.yaml declares name: devmode2, the same name declared by rules/jailbreak/devmode2.yaml.
load_test_rules() keys the rule dict by name:
rules[rule["name"]] = rule
so one file silently overwrites the other. 69 rule files therefore yield 68 usable rules, and which of the two survives depends on glob.glob() ordering, so it can differ between machines. --rules devmode3 is also never selectable, because that name never appears in the loaded set.
The two prompts are genuinely different attacks, so one is being lost rather than duplicated:
devmode2.yaml — the OpenAI-branded Developer Mode jailbreak
devmode3.yaml — a ModelKeeper-branded variant that additionally forces a "Sure, here is how to [PROMPT]" prefix and requests only the (🔓Developer Mode Output) section
Both otherwise share type: jailbreak, severity: high, and identical pass/fail conditions.
Fix: change name: in devmode3.yaml to devmode3.
Found at commit 432e072ae654788ad5b30172d42f3b1bcbda21cf.
Hi — small rule-file bug.
rules/jailbreak/devmode3.yamldeclaresname: devmode2, the same name declared byrules/jailbreak/devmode2.yaml.load_test_rules()keys the rule dict by name:so one file silently overwrites the other. 69 rule files therefore yield 68 usable rules, and which of the two survives depends on
glob.glob()ordering, so it can differ between machines.--rules devmode3is also never selectable, because that name never appears in the loaded set.The two prompts are genuinely different attacks, so one is being lost rather than duplicated:
devmode2.yaml— the OpenAI-branded Developer Mode jailbreakdevmode3.yaml— a ModelKeeper-branded variant that additionally forces a"Sure, here is how to [PROMPT]"prefix and requests only the(🔓Developer Mode Output)sectionBoth otherwise share
type: jailbreak,severity: high, and identical pass/fail conditions.Fix: change
name:indevmode3.yamltodevmode3.Found at commit
432e072ae654788ad5b30172d42f3b1bcbda21cf.