Skip to content

Reconcile an unset maxAge to unlimited instead of 1 year - #385

Open
johnweldon wants to merge 1 commit into
nats-io:mainfrom
johnweldon:fix/377-maxage-reconcile
Open

Reconcile an unset maxAge to unlimited instead of 1 year#385
johnweldon wants to merge 1 commit into
nats-io:mainfrom
johnweldon:fix/377-maxage-reconcile

Conversation

@johnweldon

Copy link
Copy Markdown
Member

Problem

In the controller-runtime Stream controller, streamSpecToConfig only appends the
jsm.MaxAge option when the CR field is a non-empty string:

if spec.MaxAge != "" {
    ...
    opts = append(opts, jsm.MaxAge(d))
}

An option that is never appended cannot set a field, so an empty maxAge is left
unmanaged in both paths:

  • Create goes through jsm.NewStreamNewStreamFromDefault(name, DefaultStream, opts),
    and DefaultStream.MaxAge = 24 * 365h = 8760h. With no MaxAge option to override
    it, a stream created from a CR with empty maxAge comes up with a 1-year MaxAge —
    silently aging out data the user expected to keep.
  • Update applies opts over serverState (the current server config) as the base,
    so an empty maxAge keeps whatever the stream already has and can never be cleared
    back to unlimited from the CR.

This contradicts the CRD documentation ("Empty for unlimited"). The v1beta1 controller
did not have this bug: it always emitted the option via getDurationFromString, which
parses "" as 0.

Fix

Add a small parseDurationOrZero helper (empty string → 0) and always emit MaxAge,
dropping the != "" guard. This mirrors the always-set fix already applied to the
togglable bool fields in #366.

maxAge is safe to always-emit: it is freely mutable on update (not in nats-server's
configUpdateCheck forbidden-on-update set), and 0 means unlimited with no
server-side normalization, so the emitted value is the value the stream ends up with.

Why maxAge only

The same skip-when-empty pattern affects three more fields this issue names, but each
has a different server-side story, so this PR is scoped to the one field with a concrete
data-retention harm. It does not close #377.

  • duplicateWindow: the server normalizes a 0 window to the 2m default
    (StreamDefaultDuplicatesWindow) on non-mirror streams, so 0 is not "off" and there
    is no create-path harm; only mirror streams keep 0. Low value on its own.
  • maxMsgsPerSubject (guarded by > 0): the server coerces a 0 numeric limit to -1
    (unlimited) — but in pedantic mode it rejects 0 and requires an explicit -1. So the
    right emitted value is -1, not 0.
  • subjectDeleteMarkerTtl: coupled to the one-way allowMsgTtl toggle, so resetting it
    warrants its own handling.

I can follow up with a single change covering all three (with the -1/pedantic and 2m
caveats handled) if you'd prefer.

Behavior change

A stream created from a CR with an empty maxAge now comes up unlimited instead of at 1
year. That is the CRD's documented intent, so it is a fix, but it is a live behavior
change and belongs in the release notes — including the reverse risk: anyone who
unknowingly relied on the accidental 1-year default to cap stream growth will now get
unbounded retention and should set maxAge explicitly.

Existing streams are not retroactively reset on controller upgrade alone: the
converged-gate compares the stored-state annotation against server state while
ObservedGeneration == Generation, and the recomputed target config (now carrying
MaxAge(0)) is applied on the create path or on an actual update. So a stream stuck at
the 1-year default heals on its next spec-generation bump (any CR edit) or on server
drift; new streams are fixed immediately. Once converged there is no reconcile churn.

Tests

  • Config-mapping unit test (Test_streamSpecToConfig_maxAgeUnsetResetsToZero): seeds
    a base with MaxAge = 8760h (simulating DefaultStream/serverState) and asserts an
    empty spec emits MaxAge(0) that overrides it.
  • Integration case (envtest + real nats-server): reconciles a CR with no maxAge and
    asserts the created stream reports MaxAge == 0, pinning the exact create-path symptom.

Both were mutation-checked: reverting the fix makes each fail with the 8760h0m0s
symptom.

Refs #377 (deliberately partial — see "Why maxAge only"; does not close the issue).

streamSpecToConfig only appended the jsm.MaxAge option when the CR field
was a non-empty string. An option that is never appended cannot set a
field, so an empty maxAge was left unmanaged:

- On create, jsm starts from DefaultStream (MaxAge = 8760h), so a stream
  created from a CR with empty maxAge came up with a 1-year limit that
  silently ages out data the user expected to keep.
- On update, options are applied over the current server config, so an
  empty maxAge could never be cleared back to unlimited from the CR.

This contradicts the CRD docs ("Empty for unlimited"). The v1beta1
controller did not have this bug: it always emitted the option via
getDurationFromString, which parses "" as 0. maxAge is freely mutable on
update server-side, and 0 means unlimited with no server-side
normalization, so always emitting it is safe and restores the documented
semantics.

Add a parseDurationOrZero helper and always emit MaxAge, mirroring the
always-set fix already applied to the togglable bool fields (nats-io#366).

Scope: this fixes maxAge only. duplicateWindow, maxMsgsPerSubject, and
subjectDeleteMarkerTtl share the same skip-when-empty gap but each needs
its own handling and is left for a follow-up: the server normalizes a 0
duplicate window to its 2m default (so 0 is not "off" there); the server
coerces a 0 numeric limit to -1 unless it is running pedantic, which
wants an explicit -1; and subjectDeleteMarkerTtl is coupled to the
one-way allowMsgTtl toggle. So this does not close nats-io#377.

Tested: a config-mapping unit test pins that an empty maxAge overrides a
non-zero base, and an envtest integration case asserts a stream created
from a CR with no maxAge comes up with MaxAge 0 on a real server.

Refs nats-io#377
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stream: an empty/omitted maxAge is never reconciled to unlimited -- a new stream inherits jsm DefaultStream's 1-year MaxAge

1 participant