Skip to content

[58_maintenance] fix(arrow-schema): validate UnionFields when deserializing - #11760

Merged
alamb merged 3 commits into
apache:58_maintenancefrom
Jefffrey:backport_11638_58
Oct 22, 2026
Merged

alamb merged 3 commits into
apache:58_maintenancefrom
Jefffrey:backport_11638_58

Conversation

@Jefffrey

Copy link
Copy Markdown
Contributor

…11638)

- Closes apache#11576.

`UnionFields` derived `Deserialize`, so deserializing could build a
value with duplicate or negative type ids, skipping the checks that

`Serialize` is still derived (transparent), but `Deserialize` is now
implemented by hand: it reads the same sequence of `(type_id, field)`
pairs and builds the value with `UnionFields::try_new`, returning a
serde error if validation fails. The wire format doesn't change, so
anything serialized before still deserializes as long as it was valid.

I kept serde rather than removing it, since `DataType` (and therefore
`Field` and `Schema`) derive serde and need `UnionFields` to support it.

Yes, two new tests in `fields.rs` (behind the `serde` feature, using
`postcard` like the existing `Field` serde tests): a round trip of a
valid `UnionFields`, and deserializing input with a duplicate type id
and with a negative type id, both of which now fail. The second test
fails without this change.

Deserializing an invalid `UnionFields` (or a `DataType::Union`
containing one) now returns an error instead of succeeding.

AI disclosure: I used an AI coding assistant to write the change, the
tests and this description, and I reviewed all of it.

---------

Co-authored-by: ump45nose <ump45nose@users.noreply.github.com>
@Jefffrey Jefffrey added the bug label Oct 21, 2026
@github-actions github-actions Bot added the arrow-core Core crates related to representing arrow data label Oct 21, 2026
@Jefffrey
Jefffrey marked this pull request as ready for review October 21, 2026 12:09
@alamb

alamb commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Merging as MSRV failure is unrelated. I will fix in this PR

Audit was fixed by a prior PR

@alamb

alamb commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

(actually I take it back, we need to resolve a conflict. Iw ill do so)

@alamb

alamb commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Audit failure should be fixed in

so merging this one

@alamb
alamb merged commit 561c176 into apache:58_maintenance Oct 22, 2026
27 of 28 checks passed
@Jefffrey
Jefffrey deleted the backport_11638_58 branch October 22, 2026 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow-core Core crates related to representing arrow data bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants