Add --infer-true/--infer-false for user-defined boolean strings - #2188
Draft
johnkerl wants to merge 1 commit into
Draft
Add --infer-true/--infer-false for user-defined boolean strings#2188johnkerl wants to merge 1 commit into
--infer-true/--infer-false for user-defined boolean strings#2188johnkerl wants to merge 1 commit into
Conversation
By default Miller does not infer booleans from data files: "true" and "false" (let alone "True", "yes", "on", etc.) are read as strings. This adds opt-in flags letting users specify exactly which field values should be inferred as boolean true/false when read from data files: mlr --infer-true True,yes,on --infer-false False,no,off ... Details: * Matching is exact and case-sensitive; unlisted values are unaffected. * As with ints and floats, the original string representation is retained for non-JSON output, so CSV/TSV/etc. round-trip unchanged. * JSON output emits canonical true/false (fixed marshalJSONBool to use the underlying boolean rather than the retained string, which would otherwise produce invalid JSON for e.g. "yes"). * Works with the -O and -A inference variants; -S (no inference) bypasses it, as it bypasses all other type inference. * Works from .mlrrc as infer-true/infer-false, like any other flag. Includes unit tests, regression-test cases under test/cases/io-infer-flags/, and doc updates (reference-main-data-types, regenerated flag list and man pages). Addresses #1651. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--infer-true/--infer-false for user-defined boolean strings
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #1651.
What this does
Adds opt-in flags letting users specify exactly which field values should be inferred as boolean when read from data files:
This follows option 1 from the issue discussion (inference-level, not a verb), so it works uniformly across CSV/TSV/DKVP/NIDX/etc. input, and also from
.mlrrc(infer-true Trueetc.) per @Poshi's comment.Approach
--infer-true {a,b,c}/--infer-false {a,b,c}flags in the Miscellaneous flag section, alongside the existing inference flags (-S/-A/-O). Each takes a comma-separated list of values.pkg/mlrval/mlrval_infer.go: a package-levelmap[string]bool(nil unless the flags are used, so the default hot path only pays a nil check) is consulted before scan-based inference ininferNormallyandinferWithOctalAsInt.-Acomposes viainferNormally;-Sbypasses all inference including this, regardless of flag order.0xff) and floats, the original string representation is retained, so CSV/TSV output round-trips unchanged (yesstaysyes). JSON output emits canonicaltrue/false— this required fixingmarshalJSONBoolto use the underlying boolean value rather thanmv.String(), which would otherwise emit invalid JSON like bareyes.Testing
TestInferUserDefinedBooleansinpkg/mlrval/mlrval_infer_test.go(go test ./pkg/...passes).test/cases/io-infer-flags/user-defined-booleans{,-json}(full regression suite passes).make lintclean; docs regenerated (reference-main-data-types, flag list, man pages).Open design questions
--infer-true/--infer-falsewith comma-separated lists mirrors the issue discussion (--true True,on,yes --false False,off,no). Alternatives: a single--infer-booleans True=true,False=false-style mapping flag, or repeatable flags. Also: commas are the separator, so values containing commas can't be specified.--infer-booleans-fold-case). Deferred for now — exact matching keeps the semantics predictable; case variants can be listed explicitly.inf/NaN): deferred, since opt-in inference of the standardtrue/infinityliterals is covered by [feature request] Opt-in type-inferencers for "true", "infinity", etc in file data #965 and could share this mechanism (a user-defined string-to-Mlrval table) if desired.boolean()DSL function; currentlyboolean()is unchanged and only file-input inference is affected.🤖 Generated with Claude Code