Fix metric() showing an extra significant digit on rounding carry - #359
Fix metric() showing an extra significant digit on rounding carry#359vidigoat wants to merge 1 commit into
Conversation
metric() derives the number of decimal places from the mantissa's exponent, assuming the mantissa keeps its digit count. When rounding carries it up a power of ten (9.999 -> 10.0, 99.99 -> 100) it gains an integer digit and shows one significant figure too many, e.g. metric(9999) returned '10.00 k' instead of '10.0 k'. The existing guard only handled the mantissa reaching 1000 (a full SI-bucket crossing). Detect the carry against the next power of ten and bump the exponent by one, which recomputes the decimal places and, when the mantissa reaches 1000, still crosses into the next bucket exactly as before.
|
Ran this on Windows 11 / CPython 3.14.7 against main Fail-before on unmodified main, reproducing the report: The fix also repairs the same carry defect in cases the PR body doesn't claim: the sub-unity regime ( Full suite: main 715 passed / 74 skipped; branch 721 passed / 69 skipped, no failures either side. The +6 is this PR's six new parametrized rows; the skip delta 74→69 is a rebase artifact (the branch predates #360's si_LK locale, which added 5 skipped i18n params on main), not a regression. Probed ~28 edge inputs side by side (bucket crossings The six new test rows: five fail on main's Comment only, not an approval. |
Summary
metric()can show one significant figure too many when rounding carries the mantissa up a power of ten.The neighbours prove the inconsistency — same magnitude, correct 3 significant figures:
Cause
The number of decimal places is derived from the mantissa's position in its SI bucket (
digits = precision - exponent % 3 - 1), which assumes the mantissa keeps its integer-digit count. When rounding todigitsplaces carries the mantissa up a power of ten (9.999 → 10.0,99.99 → 100), it gains an integer digit and therefore shows an extra significant figure.The existing guard only handled the mantissa rounding all the way to
1000(a full bucket crossing, e.g.999.9 → 1.00 k); it never handled the→ 10and→ 100crossings inside a bucket.Fix
Compare the rounded mantissa against the next power of ten (
10 ** (exponent % 3 + 1)) rather than the hard-coded1000, and bump the exponent by one to absorb the carry. When the mantissa reaches1000this still crosses into the next SI bucket exactly as before, so all existing outputs (includingmetric(999.9, "V") == "1.00 kV") are unchanged.This matches the documented contract that the prefix is chosen "so that non-significant zero digits are required" and that
precisionis "the number of digits the output should contain."Test cases for the within-bucket carry are added to
test_metric; the full suite passes.