Skip to content

docs: document missing-value semantics and override handling in environment policies #386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dangrondahl wants to merge 2 commits into
base: main
Choose a base branch
from
Open

docs: document missing-value semantics and override handling in environment policies #386

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
edf8171
docs: document missing-value semantics and override handling in envir…
dangrondahl Sep 7, 2026
ba9464b
docs: correct missing-value semantics and pin the override rule
dangrondahl Sep 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion getting_started/policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ artifacts:

### Exceptions

You can add exceptions to policy rules using [policy expressions](/policy-reference/environment_policy#policy-expressions).
You can add exceptions to policy rules using [policy expressions](/policy-reference/environment_policy#policy-expressions). Note how expressions behave when a value is [missing](/policy-reference/environment_policy#missing-values), which is common for artifacts reported without provenance.

```yaml
_schema: https://docs.kosli.com/schemas/policy/v1
Expand Down
13 changes: 13 additions & 0 deletions policy-reference/environment_policy.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ An environment policy is a YAML file that declares compliance requirements for a
<ParamField path="artifacts.attestations" type="array" default="[]">
List of attestations every artifact must have. Each element is a required-attestation rule.

A rule is evaluated against the newest attestation whose `name` and `type` match the rule, including override attestations — an override supersedes the status of the attestation it overrides. Overriding a non-compliant attestation can therefore bring an environment back to compliant, and overriding a compliant one down to non-compliant makes the environment non-compliant.

<Expandable title="attestation rule properties">
<ParamField path="artifacts.attestations[].type" type="string" required>
The [attestation type](#attestation-types) to require. Cannot be `*` when `name` is also `*`.
Expand Down Expand Up @@ -152,6 +154,17 @@ Parentheses control precedence: `${{ flow.name == 'prod' and (flow.tags.team ==
| `exists(arg)` | Returns `true` if `arg` is not null. | `${{ exists(flow) }}` |
| `matches(input, regex)` | Returns `true` if `input` matches the regular expression. | `${{ matches(artifact.name, "^datadog:.*") }}` |

### Missing values

A value is missing when the context does not supply it. This is common rather than exceptional:

- `flow` is missing for any artifact reported without provenance.
- `flow.tags.<key>` is missing for a tag the flow does not have.

The comparison operators (`==`, `!=`, `<`, `>`, `<=`, `>=`), the membership operator `in`, and the functions `exists()` and `matches()` all evaluate to `false` when either operand is missing. A missing operand is not an error — the comparison is simply not satisfied.

@claude claude Bot Sep 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — == and != are in this list, but the generated schema doesn't put them there, and != is the one that inverts a policy.

schemas/policy/v1.json:72 is generated from the API's Pydantic models (per CLAUDE.md's Schemas section, kosli-dev/server is the source of truth) and states the rule as:

A missing (None) context value never matches: matches() and the ordering comparisons (<, >, <=, >=) return false if either operand is missing, and in is a list-membership test that returns false unless its right-hand side is a list containing the left-hand value.

That enumeration is matches() + the four ordering comparisons, with in described separately. This line adds == and !=. == is harmless either way, but != is not: under ordinary inequality semantics flow.tags.risk-level != "high" on a flow with no risk-level tag evaluates to true, not false. Written as an exception —

trail-compliance:
  required: true
  exceptions:
    - if: ${{ flow.tags.risk-level != "high" }}

— that waives trail compliance for every untagged flow, which is the opposite of what this page tells the author to expect. Since the section is meant to become the canonical statement, it shouldn't be broader than the source it's transcribed from.

Recommendation: mirror the schema's enumeration (matches(), the ordering comparisons, and in), and add ==/!= back only once someone confirms them against kosli-dev/server. If != really does return false for a missing operand, that's the single most surprising fact in this section and deserves its own sentence rather than a slot in a list.

Two smaller things on the same line:

  • in — the schema's rule is stricter than "either operand is missing": in returns false unless the right-hand side is a list containing the value. Worth stating, since flow.name in flow.tags.teams fails for a reason this page doesn't cover.
  • exists() — returning false for a missing operand is exists()'s contract, not an instance of the missing-value rule, and grouping it under "the comparison is simply not satisfied" reads oddly two lines above "test it with exists()". Pulling it out of the list would make line 166 land harder.

Fix this →


To branch on whether a value is present at all, test it with `exists()` rather than relying on a comparison against it.

## Constraints

- `_schema` is the only required field. All other fields are optional and use server defaults when omitted.
Expand Down