Repository navigation
Commit 9972543
Feat/content safety decision (#1196)
* feat(content-safety): add opt-in decision API auditing
Add an opt-in content safety plugin that audits content at the model and
tool lifecycle checkpoints through the System One decision API. The default
provider remains none and nothing is mounted automatically.
- strict tool.content_safety configuration with Ark connection defaults
- strict decision response arbitration and on_error policy
- streaming partial suppression with final aggregated audit
- audited SkillToolset load/resource/script/list/search tools
- LLM Shield adapter and real-ADK E2E coverage
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(content-safety): add opt-in live System One integration test
Exercise the real Ark System One endpoint through the production plugin
path when VEADK_CONTENT_SAFETY_LIVE=1 and a key are supplied. Normal test
runs skip it so no CI job calls the production service by accident.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(content-safety): avoid duplicate pytest test basenames
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(content-safety): rename decision plugin and drop skill adapter
Rename ContentSafetyPlugin to DecisionPlugin to match the reviewed
architecture, remove the skill pre-audit adapter from this change, and
use StateBuilder.build_after_model for model-output auditing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(content-safety): reject unsafe non-Ark decision fallback
Only derive the System One endpoint from a known Ark HTTPS base. Require
an explicit decision.api_key for custom endpoints so Ark credentials are
never silently sent to another provider.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(content-safety): ignore hidden thought parts in model audits
ADK may attach hidden reasoning as Part(thought=True). Auditing that text
caused benign final answers to be blocked after the model had already
produced safe visible output. Filter hidden thought parts from after_model
state and preserve them when replacing visible event text.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(content-safety): apply llm_shield app id before legacy import
The legacy LLM Shield module constructs a plugin at import time, so a
config-only llm_shield_app_id was read too late and startup failed unless
TOOL_LLM_SHIELD_APP_ID was already exported. Set the env inside the lock
before importing, and restore it even when the import raises.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(decision): extract reusable Decision API client
Move the Decision API protocol into veadk.decision with typed request and
response models. content_safety now adapts its question configuration at the
boundary and consumes the typed response, keeping the dependency one-way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(content-safety): harden decision and llm shield paths
Prevent non-Ark credentials from being reused for an explicit Ark decision
endpoint. Audit visible text in mixed text/function-call events, run legacy
LLM Shield callbacks outside the event loop, load .env before applying
decision overrides, reject plaintext remote endpoints, and store API keys as
SecretStr. Add regression coverage for each fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(content-safety): auto-mount from config.yaml
Mount the content-safety plugin from VeADK Runner and AgentKit when
`tool.content_safety` is present, so callers no longer have to load the
config, build the plugin, and wire it into App manually.
- Agent gains an optional `content_safety` field (None/False/True/dict/config)
and a `content_safety_plugins` property for manually built ADK Apps.
- `load_content_safety_config_if_present()` treats missing files or blocks as
disabled while still failing fast on invalid configuration.
- Plugin resolution deduplicates by name and keeps explicitly supplied plugins.
- Runner now supports app-only construction while injecting plugins.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(content-safety): enable built-in risk questions by default
- merge five built-in questions for provider=decision
- allow per-name overrides and appends, add use_default_questions
- fix question type to noul and reject user-supplied type
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(content-safety): add stable policy prefix to audit state
- prepend fixed audit policy outside the untrusted payload
- shorten built-in question instructions
- keep truncation limited to dynamic content
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(content-safety): expand built-in risk questions to 18 categories
- merge the documented 12 categories with existing defaults
- add self_harm, child_safety, and fraud_social_engineering
- keep existing names and the 0.5 default threshold
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(content-safety): tier default question thresholds by risk
Lower thresholds for safety-critical categories (child safety and
self-harm) and raise them for broad, high-false-positive categories such
as compute abuse and public order. Export the per-question threshold
mapping and update tests to assert the tiering.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>1 parent f511e04 commit 9972543
27 files changed
Lines changed: 5550 additions & 2 deletions
File tree
- tests
- content_safety
- decision
- e2e
- veadk
- content_safety
- adapters
- decision
- integrations/agentkit
- utils
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
0 commit comments