Skip to content

Add --infer-true/--infer-false for user-defined boolean strings - #2188

Draft
johnkerl wants to merge 1 commit into
mainfrom
issue-1651-user-defined-booleans
Draft

Add --infer-true/--infer-false for user-defined boolean strings#2188
johnkerl wants to merge 1 commit into
mainfrom
issue-1651-user-defined-booleans

Conversation

@johnkerl

Copy link
Copy Markdown
Owner

Addresses #1651.

What this does

Adds opt-in flags letting users specify exactly which field values should be inferred as boolean when read from data files:

$ printf 'id,active,verified\n1,True,yes\n2,False,no\n3,true,maybe\n' |   mlr --infer-true True,yes --infer-false False,no --c2p put '$ta = typeof($active); $tv = typeof($verified)'
id active verified ta     tv
1  True   yes      bool   bool
2  False  no       bool   bool
3  true   maybe    string string

This follows option 1 from the issue discussion (inference-level, not a verb), so it works uniformly across CSV/TSV/DKVP/NIDX/etc. input, and also from .mlrrc (infer-true True etc.) per @Poshi's comment.

Approach

  • New --infer-true {a,b,c} / --infer-false {a,b,c} flags in the Miscellaneous flag section, alongside the existing inference flags (-S/-A/-O). Each takes a comma-separated list of values.
  • pkg/mlrval/mlrval_infer.go: a package-level map[string]bool (nil unless the flags are used, so the default hot path only pays a nil check) is consulted before scan-based inference in inferNormally and inferWithOctalAsInt. -A composes via inferNormally; -S bypasses all inference including this, regardless of flag order.
  • Matching is exact and case-sensitive; unlisted values are unaffected. Only inference from data files is affected — JSON string values stay strings (per the issue: JSON booleans are already well-defined), and DSL string literals are unaffected.
  • As with ints (0xff) and floats, the original string representation is retained, so CSV/TSV output round-trips unchanged (yes stays yes). JSON output emits canonical true/false — this required fixing marshalJSONBool to use the underlying boolean value rather than mv.String(), which would otherwise emit invalid JSON like bare yes.

Testing

  • New unit test TestInferUserDefinedBooleans in pkg/mlrval/mlrval_infer_test.go (go test ./pkg/... passes).
  • New regression cases test/cases/io-infer-flags/user-defined-booleans{,-json} (full regression suite passes).
  • make lint clean; docs regenerated (reference-main-data-types, flag list, man pages).

Open design questions

  • Flag naming/syntax: --infer-true/--infer-false with comma-separated lists mirrors the issue discussion (--true True,on,yes --false False,off,no). Alternatives: a single --infer-booleans True=true,False=false-style mapping flag, or repeatable flags. Also: commas are the separator, so values containing commas can't be specified.
  • Case-sensitivity: @aragilar suggested an optional case-insensitivity flag (e.g. --infer-booleans-fold-case). Deferred for now — exact matching keeps the semantics predictable; case variants can be listed explicitly.
  • Special values (the "(and special values)" in the issue title, e.g. mapping strings to inf/NaN): deferred, since opt-in inference of the standard true/infinity literals is covered by [feature request] Opt-in type-inferencers for "true", "infinity", etc in file data #965 and could share this mechanism (a user-defined string-to-Mlrval table) if desired.
  • DSL exposure: the discussion also floated using the mapping in the boolean() DSL function; currently boolean() is unchanged and only file-input inference is affected.

🤖 Generated with Claude Code

By default Miller does not infer booleans from data files: "true" and
"false" (let alone "True", "yes", "on", etc.) are read as strings. This
adds opt-in flags letting users specify exactly which field values
should be inferred as boolean true/false when read from data files:

  mlr --infer-true True,yes,on --infer-false False,no,off ...

Details:

* Matching is exact and case-sensitive; unlisted values are unaffected.
* As with ints and floats, the original string representation is
  retained for non-JSON output, so CSV/TSV/etc. round-trip unchanged.
* JSON output emits canonical true/false (fixed marshalJSONBool to use
  the underlying boolean rather than the retained string, which would
  otherwise produce invalid JSON for e.g. "yes").
* Works with the -O and -A inference variants; -S (no inference)
  bypasses it, as it bypasses all other type inference.
* Works from .mlrrc as infer-true/infer-false, like any other flag.

Includes unit tests, regression-test cases under
test/cases/io-infer-flags/, and doc updates (reference-main-data-types,
regenerated flag list and man pages).

Addresses #1651.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@johnkerl johnkerl changed the title Add --infer-true/--infer-false for user-defined boolean strings Add --infer-true/--infer-false for user-defined boolean strings Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant