Skip to content

Add all-empty string case to the parquet arrow_writer benchmark - #12140

Merged
Rich-T-kid merged 4 commits into
apache:mainfrom
azecevic000:parquet-bench-string-empty
Nov 16, 2026
Merged

Rich-T-kid merged 4 commits into
apache:mainfrom
azecevic000:parquet-bench-string-empty

Conversation

@azecevic000

@azecevic000 azecevic000 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

#12269

Rationale for this change

Benchmark for the case of a large empty buffer. There is a real regression in glibc >=2.35 on x86 AVX512.

What changes are included in this PR?

Just a benchmark case as recommended in CONTRIBUTING.md.

Are these changes tested?

Are there any significant user-facing changes?

No.

AI use

An AI coding agent wrote the code. I have reviewed it and verified the benchmark results.

`string_empty` writes one non-null Utf8 column with 1M empty values.
Like `concat`, `take` and `interleave`, `from_iter_values` gives an
all-empty array a values buffer without an allocation, so its pointer
is dangling.

@Rich-T-kid Rich-T-kid left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code makes sense, im a little puzzled as to what this benchmark would be used for though

Comment thread parquet/benches/arrow_writer.rs Resolved
@Rich-T-kid
Rich-T-kid self-requested a review November 14, 2026 03:07
Co-authored-by: RIchard Baah <137434454+Rich-T-kid@users.noreply.github.com>
@Rich-T-kid

Copy link
Copy Markdown
Contributor

@azecevic000 there is a CI failure in your fork

@Rich-T-kid Rich-T-kid added the development-process Internal changes; PRs with this label are excluded from changelog label Nov 16, 2026
@Rich-T-kid
Rich-T-kid enabled auto-merge November 16, 2026 14:16
@Rich-T-kid

Copy link
Copy Markdown
Contributor

thanks @azecevic000

@Rich-T-kid
Rich-T-kid added this pull request to the merge queue Nov 16, 2026
Merged via the queue into apache:main with commit d65329d Nov 16, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Internal changes; PRs with this label are excluded from changelog parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants