Skip to content

fix(tier3): prevent false positives in network exfiltration checks - #141

Merged
rng1995 merged 6 commits into
NVIDIA:mainfrom
kweinmeister:fix/security-network-exfiltration-flag-matching
Sep 17, 2026
Merged

rng1995 merged 6 commits into
NVIDIA:mainfrom
kweinmeister:fix/security-network-exfiltration-flag-matching

Conversation

@kweinmeister

Copy link
Copy Markdown
Contributor

Summary

Fixes false positives in the network exfiltration security check.

The previous check relied on broad substring matching across the entire command string. As a result, benign commands were frequently flagged as security risks. For example:

  • Using standard curl flags like curl -f -sS or curl -sSf (which lowercase-matched -F form uploads).
  • Piped or chained utilities like grep -F, awk -F, or unzip -d /tmp following a download.
  • URLs or package names containing words like post, put, or --format=json (e.g. /blog/posts/123, cloud-storage-fuse).

This PR tightens the check so it only triggers on actual exfiltration attempts:

  • Anchors commands to avoid matching package names or URLs as the network binary.
  • Scopes arguments to their specific command segment so piped or chained commands don't trigger flags.
  • Distinguishes case-sensitive upload flags (-F, -d, -T / --upload-file) from safe options (-f, -Follow).
  • Reliably detects secrets passed in query strings or headers (e.g. Authorization: Bearer $TOKEN).
  • Keeps the standalone Harbor evaluation template (eval.py) in sync and adds regression tests covering both safe and unsafe command patterns.

Verification

  • I am familiar with the Contributing Guidelines
  • Added or updated focused tests
  • Updated documentation for user-visible changes
  • Ran make lint
  • Ran make test
  • Ran make build
  • Did not add credentials, private datasets, or proprietary benchmark content

Release Impact

  • No user-visible release note needed
  • Updated CHANGELOG.md

Fix unanchored substring matching in network exfiltration security checks
where flags like -f, --form, and verbs like post/put matched inside ordinary
hyphenated words, arguments, or URLs (e.g. -for- in iam-helper-for-policy-management,
-fuse in cloud-storage-fuse, or --format=json).

Enforce CLI flag word boundaries, quote-aware segments, and command position anchoring:
- Anchor network client binary invocations to command positions and subshells (bash -c).
- Support disjoint quote-aware argument segmentation to prevent ReDoS and avoid slicing into quoted flags (-A, -H).
- Enforce explicit HTTP method flags (-X, --request, --method) and verbs (POST, PUT, PATCH).
- Support single and double quoted HTTP method arguments, attached data payloads (-dsecret), and file uploads (-T, --upload-file).
- Support secret environment variables embedded in complex quoted strings with spaces (e.g. Authorization headers).
- Distinguish uppercase multipart -F and clustered short flags (-sSF, -FsS) from lowercase -f (--fail).
- Synchronize constants byte-for-byte between eval_core checks and Harbor template via _SHARED_SECURITY_CONSTANTS.
- Add grouped and sorted safe and unsafe command regression test suites in tests/tier3/test_checks_secret_patterns.py.

Signed-off-by: Karl Weinmeister <kweinmeister@google.com>
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
kweinmeister and others added 3 commits September 14, 2026 16:36
- Identify command executable positions and unwrap shell wrappers
- Respect shell quote semantics and redact literal secrets from evidence
- Cover HTTPie implicit request bodies and safe query parameters
- Replace quadratic regex scanning with bounded single-pass tokenization
- Maintain parity between eval_core and standalone Harbor template

Signed-off-by: Karl Weinmeister <kweinmeister@google.com>
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
Comment thread src/skillevaluator/tier3/eval_core/checks.py
Comment thread src/skillevaluator/tier3/eval_core/checks.py Outdated
Comment thread src/skillevaluator/tier3/eval_core/checks.py
…nwrapping

Address PR NVIDIA#141 review feedback on network exfiltration checks:
- Support attached option arguments (--data=val, --upload-file=val, --post-data=val, --raw=val) and curl short option bundles (-d, -T, -F, -X).
- Unwrap standard execution wrappers (sudo, nohup, doas, stdbuf, setsid, time, builtin, xargs), recursively evaluate eval, exempt inert formatters (echo, printf), and fail closed on unrecognized wrappers with network executables.
- Prevent HTTPie format switches (--json, -j, --form, -f, --multipart) from triggering without body evidence, and support field@file form syntax.
- Tokenize adjacent/unspaced redirections (<, <<, <<<, >, >>).
- Maintain 100% parity between eval_core/checks.py and Harbor templates/eval.py.

Signed-off-by: Karl Weinmeister <kweinmeister@google.com>
Signed-off-by: Karl Weinmeister <kweinmeister@google.com>

# Conflicts:
#	CHANGELOG.md
@rng1995
rng1995 merged commit 0818709 into NVIDIA:main Sep 17, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants