Skip to content

Copy normalized path segments in time proportional to the segment - #86

Merged
quinnj merged 1 commit into
masterfrom
claude/normpath-linear
Oct 22, 2026
Merged

quinnj merged 1 commit into
masterfrom
claude/normpath-linear

Conversation

@quinnj

@quinnj quinnj commented Oct 22, 2026

Copy link
Copy Markdown
Member

Follow-up to #80. normpath still took time quadratic in the path length when a path had many segments and dot segments. The cause is append!(output, codeunits(segment)): append! checks the destination and source for aliasing via objectid, which hashes the segment's whole parent string. So copying each kept segment cost the length of the entire path. The allocation-growth test from #80 could not see this, because no extra memory is allocated. resolvereference and joinpath both run their result through normpath.

Segments are now copied byte by byte (_append_codeunits!).

normpath input (Julia 1.12) before after
"/a/b/c/./../../g" 496 ns 329 ns
"/a/b/../c/./d/../e/f/g/../h" 917 ns 597 ns
"name.ext/./"^512 (5.6 KB) 586 µs 53 µs
"/" * "a/"^2_500 * "../"^2_500 (12.5 KB) 6.5 ms 0.23 ms
"/" * "a/"^25_000 * "../"^25_000 (125 KB) 1.82 s 2.3 ms
"/" * "a/"^250_000 * "../"^250_000 (1.25 MB) 46.7 s 23 ms

Paths without dot segments take the existing no-copy fast path and are unchanged (18 → 15 ns, within noise). Julia 1.10 shows the same per-append! cost (measured: a 1-byte append! from a SubString of a 10 MB string takes ~4 ms).

Verification. New test in "Path normalization boundaries and allocation growth": a 500 KB path of segments and dot segments normalizes to "/" in under 1 s. With this change it takes milliseconds. Without it, the same test took 30.2 s and failed. Full suite passes on Julia 1.12 and 1.10.

Review.

  • normpath(::AbstractString) is the only caller of the new helper. Its two former append! sites both receive SubString{String} views of str.
  • UTF-8 is copied byte for byte, so invalid sequences pass through unchanged; the existing 0xff/0xfe case covers that.
  • push! after sizehint! doesn't reallocate.
  • The method is concrete, so the --trim=safe native job's path stays resolvable.

🤖 Generated with Claude Code

`normpath` appended each kept segment with `append!(output, codeunits(segment))`.
`append!` checks the two arrays for aliasing through `objectid`, which hashes the
segment's whole parent string, so every segment cost the length of the whole
path: "/" * "a/"^25_000 * "../"^25_000 (125 KB) took 1.8 s and a 1.25 MB path
47 s. `resolvereference` and `joinpath` run every path through `normpath`.
Segments are now copied byte by byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.89%. Comparing base (d9689ac) to head (74304b3).

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
+ Coverage   93.75%   93.89%   +0.14%     
==========================================
  Files           5        5              
  Lines         416      426      +10     
==========================================
+ Hits          390      400      +10     
  Misses         26       26              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@quinnj
quinnj merged commit a8e1823 into master Oct 22, 2026
12 checks passed
@quinnj
quinnj deleted the claude/normpath-linear branch October 22, 2026 09:11
@quinnj quinnj mentioned this pull request Oct 22, 2026
quinnj referenced this pull request Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant