fix(validation): enforce equal arity for relationship column arrays - #375
christianeu-db wants to merge 1 commit into
Conversation
4206786 to
457aae0
Compare
The spec (core-spec/spec.md, "Relationships") requires from_columns and to_columns to correspond positionally and to have the same number of columns, but nothing enforced it. JSON Schema cannot express a cross-array equality constraint, so a document with mismatched arities (for example from_columns: [a, b] with to_columns: [c]) validated successfully despite describing an incomplete tuple join. Add validate_relationship_column_arity to validation/validate.py as a new endpoint-validation rule, alongside validate_references. It emits an [Arity] error per mismatched relationship and skips documents that already fail schema validation. Covered by tests in validation/tests/test_validate.py. Addresses Problem 6 (Relationship Column Array Arity) from apache#374 Signed-off-by: Chris Eubank <108756251+christianeu-db@users.noreply.github.com>
457aae0 to
9d5fec3
Compare
kayemkim
left a comment
There was a problem hiding this comment.
The change itself looks right to me. The rule is already in the spec text (spec.md, "Important Notes": "Both arrays must have the same number of columns", with the same note on to_columns in spec.yaml), the schema has minItems: 1 on both arrays but nothing that ties their lengths together, and I confirmed on main that a relationship with from_columns: [customer_id, region_id] and to_columns: [region] validates with exit 0. With this patch it fails with the [Arity] error, and a document where from_columns is missing altogether is left to the schema error as intended. The full validation-ci matrix (3.11 to 3.14) passes on a merge with current main, and all 11 semantic-model YAML files in the repo (examples and converter fixtures) keep their current pass/fail status.
One thing worth flagging for whoever merges: this is the third open PR adding the same check. #307 (August 5) puts it inside validate_references with a [Relationship] tag, and #170 (June 26) bundles it with a field-existence check. Both are still waiting for review. I don't have a preference on which one lands, but it would be good to pick one and close the others so the authors aren't left waiting, and so the message tag and placement stay consistent with whichever is chosen.
@jbonofre would you mind taking a look? I don't have a strong preference on which one merges (although #170 has been open for a while and covers other validation which might be tackled in other PRs as well). But, it would be nice to close this long-standing issue (especially, since it is showing up on recent field reports)? |
Summary
Enforce the spec's already-documented requirement that a relationship's
from_columnsandto_columnshave the same number of columns.$defs/Relationshipconstrains each array withminItems: 1but not their relative length, so a document with mismatched arity — describing an incomplete tuple join (e.g.from_columns: [a, b]withto_columns: [c]) — validated successfully.core-spec/spec.md("Relationships") already states the two arrays must correspond positionally and have the same number of columns; JSON Schema cannot express this cross-array equality, so the rule is enforced invalidation/validate.py:validate_relationship_column_aritycheck, alongside the existing endpoint-validation rules (validate_references), emitting an[Arity]error per mismatched relationship. It skips documents that already fail schema validation.Covered by tests in
validation/tests/test_validate.py(equal-length simple and composite pass; both mismatch directions rejected; non-listto_columnsskipped without crashing).No spec or JSON Schema change is needed — this only closes the enforcement gap.
Related Issues
Addresses Problem 6 (Relationship Column Array Arity) from #374
Checklist
Validation
validation/are updatedTests
Compliance
Documentation and examples: no change — the spec already documents the constraint and the existing examples already conform.