fix(stats): stabilize truncated-normal tails and narrow intervals - #260
Open
AminMohamed-3 wants to merge 6 commits into
Open
fix(stats): stabilize truncated-normal tails and narrow intervals#260AminMohamed-3 wants to merge 6 commits into
AminMohamed-3 wants to merge 6 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Valid tail intervals such as
[9, 10]maketruncnorm.pdfandppfreturn infinities and can make the scalar CDF raiseZeroDivisionError: subtracting two saturated normal CDFs gives a zero normalization mass. This implements a fix for #259.Keep interval probabilities in logarithmic form, use a factored eight-node quadrature for nearly coincident bounds, and invert log probabilities with four fixed Newton steps. The pure-NumPy kernels preserve the existing public wrapper and server dispatch path. Invalid probabilities, bounds and nonpositive scales now produce SciPy-style NaNs; broadcast, dtype-promotion, source-refusal and budget checks remain covered.
The numerical work is more expensive, so this proposal replaces the old base costs
28/51/81with 315/844/1392 for PDF/CDF/PPF per broadcast output element. These are conservative analytical numerical bounds at weight 1, before configured dtype pricing, not hardware calibrations or measured averages. The complete branch derivation is indocs/reference/truncnorm-cost.md; source docstrings, generated-documentation formulas and regeneratedops.jsonagree. Eight quadrature nodes and four inverse iterations are part of that derivation.Validation:
pyright src/flopscope testspassed with zero errors and four existing export warnings. Repository Ruff lint/format, diff check, generated-doc verification and theops.jsonsynchronization check passed. The full Linux CI matrix has not run locally.Numerical limits: this addresses the reported 9- and 40-sigma cases and tested narrow intervals; it is not an arbitrary-finite-double accuracy guarantee. At extremely large standardized bounds (for example near
1e10), cancellation or overflow can still produce failures, and quantile spacing limits inversion on very narrow intervals.Developed, independently reviewed and tested with AI assistance. The fdlibm-derived coefficients retain their original license notice.