Skip to content

Expose bit-field position and width in variable reflection - #1106

Open
ipburbank wants to merge 2 commits into
compiler-research:mainfrom
ipburbank:feature/bitfield-reflection
Open

Expose bit-field position and width in variable reflection#1106
ipburbank wants to merge 2 commits into
compiler-research:mainfrom
ipburbank:feature/bitfield-reflection

Conversation

@ipburbank

@ipburbank ipburbank commented Sep 2, 2026

Copy link
Copy Markdown

A bit-field data member is indistinguishable from an ordinary one through the reflection API: nothing reports whether a variable is a bit-field or how many bits it declares, and GetVariableOffset converts the field offset with toCharUnitsFromBits, which truncates to whole bytes and discards the sub-byte position. A binding layer is therefore forced to treat a bit-field as a whole-width member and read or write the entire storage unit containing it — returning the neighbouring fields on a read and clobbering them on a write.

IsBitFieldVariable and GetVariableBitWidth expose FieldDecl::isBitField() and getBitWidthValue(). Both return a false / -1 sentinel for a non-bit-field or a non-FieldDecl, so any variable can be queried without first checking its kind.

GetVariableBitOffset recovers the discarded position without duplicating the record-layout walk:

bit_offset = 8 * GetVariableOffset(var, parent) + getFieldOffset(FD) % 8

This is exact. GetVariableOffset already accumulates anonymous struct/union nesting and non-virtual base-class offsets, and every level of that accumulation apart from the innermost field is a record-typed member or a base subobject, both of which are byte-aligned; only the innermost FieldDecl can sit at a sub-byte offset. The residue is in fact frame-independent: getFieldOffset is relative to FD->getParent() while the accumulated byte offset is relative to parent, but the two frames differ by a sum of exact multiples of 8 bits, and residues mod 8 are invariant under adding those.

Delegating rather than walking the layout a second time also makes floor(GetVariableBitOffset(v, p) / 8) equal GetVariableOffset(v, p) by construction, so a caller's byte offset and its shift can never disagree. An independent walk would not have that property, and a disagreement between the two is precisely the silent corruption these APIs exist to prevent — so the delegation is deliberate rather than an economy.

A field in a virtual base subobject is not supported, and the limitation is documented on the API. The byte offset ultimately comes from ASTRecordLayout::getBaseClassOffset, which consults BaseOffsets — non-virtual bases only — so an assertions-off build reads a default-inserted CharUnits(0) and also pollutes the cached layout. That is pre-existing in GetVariableOffset (the byte offset is wrong for the same field today) and is left alone here to keep this change atomic; it deserves its own issue and fix, presumably via getVBaseClassOffset.

Tested in VariableReflectionTest for the true and false cases, declared widths, offsets both at and away from a byte boundary, a bit-field reached through multiple inheritance (where the base offset is non-zero, so the base-class traversal genuinely runs), and two-level anonymous struct-in-union nesting. The suite also asserts the floor(bit_offset / 8) == GetVariableOffset invariant over every bit-field in every fixture, which is ABI-independent and therefore the strongest available check. The few ABI-dependent absolute offsets live in a separate VariableReflection_BitFieldOffsetItaniumABI test, skipped early on Windows, because the MS ABI starts a fresh allocation unit for a bit-field that follows a non-bit-field.

This is the CppJIT-side equivalent of wlav/cppyy#57. The fixes for that issue against the cppyy stack were wlav/CPyCppyy#69, wlav/cppyy#332 and wlav/cppyy-backend#39; rather than merge them, the issue's maintainer directed the work upstream to compiler-research, noting a new backend would need a different placement. The four-hop cling chain those PRs added (TClingDataMemberInfoTClingTDataMemberCppyy:: → C API) collapses into the three APIs here.

The downstream consumer is compiler-research/cppjit#73, which uses these three APIs to fix Python-level bit-field reads and writes; it cannot build until this lands and its CPPINTEROP_GIT_TAG is bumped.

A bit-field data member is indistinguishable from an ordinary one through
the reflection API: there is no way to ask whether a variable is a
bit-field, nor how many bits it declares. A downstream binding layer
therefore treats a bit-field as a whole-width member and reads or writes
the entire storage unit that contains it.

Expose clang's FieldDecl::isBitField() and getBitWidthValue() so callers
can identify bit-fields and size them. Both return a false / -1 sentinel
for a non-bit-field or a non-FieldDecl, so any variable can be queried
without checking its kind first.
GetVariableOffset converts a field's offset with toCharUnitsFromBits,
which truncates to whole bytes and so discards the sub-byte position a
bit-field needs. Recover it without duplicating the record-layout walk:

    bit_offset = 8 * GetVariableOffset(var, parent) + getFieldOffset(FD) % 8

This is exact. GetVariableOffset already accumulates anonymous
struct/union nesting and non-virtual base-class offsets, and every level
of that accumulation apart from the innermost field is a record-typed
member or a base subobject, both of which are byte-aligned; only the
innermost FieldDecl can sit at a sub-byte offset. The residue is in fact
frame-independent: the two frames differ by a sum of exact multiples of
8 bits, and residues mod 8 are invariant under adding those.

Delegating also makes floor(GetVariableBitOffset(v, p) / 8) equal to
GetVariableOffset(v, p) by construction, so a caller's byte offset and
its shift can never disagree. An independent second walk would not have
that property, and a disagreement between the two would corrupt reads
silently.

A field in a virtual base subobject is not supported: the underlying
byte offset comes from ASTRecordLayout::getBaseClassOffset, which
consults non-virtual bases only. The limitation is documented on the API.
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.65217% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.77%. Comparing base (9bbdbb2) to head (af40725).

Files with missing lines Patch % Lines
lib/CppInterOp/CppInterOp.cpp 95.65% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1106      +/-   ##
==========================================
+ Coverage   87.74%   87.77%   +0.02%     
==========================================
  Files          23       23              
  Lines        6429     6452      +23     
==========================================
+ Hits         5641     5663      +22     
- Misses        788      789       +1     
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOp.cpp 90.58% <95.65%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOp.cpp 90.58% <95.65%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant