Repository navigation
perf(arrow-buffer): use a static address for buffers without an allocation - #12141
Open
azecevic000 wants to merge 1 commit into
Open
azecevic000 wants to merge 1 commit into
azecevic000 wants to merge 1 commit into
Conversation
Contributor
|
Thanks @azecevic000, this looks nice. Could you please open an issue for this and link this PR and #12140 to it? 🙏 |
Contributor
|
Does this issue also exist in the rust standard lib when comparing empty vectors or strings? |
Contributor
Author
Yes, exactly the same issue.
Thank you, I've opened it and linked it. |
…ation Buffers without an allocation used a dangling pointer in the unmapped first page: `ALIGNMENT` from `MutableBuffer::new(0)`, or the pointer of an empty `Vec` (`0x1` for `Vec<u8>`). Rust compares empty slices with `memcmp` and `bcmp` of length 0, and on x86-64 CPUs with AVX512VL and AVX512BW, glibc 2.35 and later make these calls slow at an unmapped address: up to about 100 ns instead of 2 ns. So each compare of values of an all-empty string or binary array was slow, for example in Parquet statistics and in the comparison and sort kernels. All such buffers now point at one static, aligned area, as Arrow C++ does (`zero_size_area`): `dangling_ptr()`, a `Vec` without an allocation in `From<Vec<T>> for MutableBuffer`, and `MutableBuffer` after shrinking to size 0, which kept the pointer to the freed memory.
azecevic000
force-pushed
the
arrow-buffer-zero-size-area
branch
from
November 16, 2026 16:54
e1f831a to
6239496
Compare
azecevic000
marked this pull request as ready for review
November 16, 2026 17:07
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
#12269
Rationale for this change
Buffers without an allocation have a dangling pointer in the unmapped first page. On x86-64 CPUs with AVX512 and glibc >=2.35 , the
memcmpandbcmpof length 0 that Rust uses to compare empty slices take up to about 100 ns at such an address, instead of 2 ns. So each compare of values of an all-empty string or binary array is slow. The CPU walks the page tables on every call, because the TLB does not cache unmapped pages. Then it cancels the page fault, because the mask covers it.What changes are included in this PR?
All buffers without an allocation now point at
ZERO_SIZE_AREA, a static aligned to 256 bytes (at leastALIGNMENTon all targets), as Arrow C++ does withzero_size_area:dangling_ptr()returns its address. Thecfg(miri)variant is no longer necessary, because the address now comes from a real static.From<Vec<T>> for MutableBuffer(so alsoBuffer::from_vec) uses it for aVecwithout an allocation.MutableBufferuses it after it shrinks to size 0. Before, it kept the pointer to the freed memory.Empty buffers already shared addresses before (
0x80, or0x1forVec<u8>), so code cannot rely on a unique address for them.Are these changes tested?
test_buffers_without_allocation_point_to_zero_size_areachecks the three places. The existing tests pass.Intel Xeon Platinum 8375C (AVX512VL and AVX512BW), glibc 2.35 or later, one core, 1,048,576 empty strings.
arrow_writerbenchmark,string_emptyfrom #12140, criterion median per iteration:mainstring_empty/defaultstring_empty/bloom_filterstring_empty/parquet_2string_empty/zstdstring_empty/zstd_parquet_2string_empty/cdcstring_empty/number_distinct_valuesWith this PR,
string_empty/defaultis as fast as the same values in an allocated buffer (15.04 ms and 15.05 ms).No regressions: the existing
string,string_non_nullandstring_dictionarycases (21 benchmarks) show no change (-3.8% to +3.5%, all within criterion's noise). Inbuffer_create, the changes are -1.7% to +2.8%; a control run of the unchanged code moved the same three bool cases by +1.2% to +1.8%.Are there any significant user-facing changes?
No. Only the address of empty buffers changes.
AI use
An AI coding agent wrote the code. I have reviewed it and verified the benchmark results.