Add bidirectional Apache Ossie <-> Cube converter

[MikeNitsenko]

Summary

Adds a bidirectional converter between Apache Ossie semantic models and Cube data models, under converters/cube/. Pure offline YAML transform — no Cube deployment, API token, or network access required, matching the other Python converters in this repo.

• Import (ossie-cube import): Cube files → Ossie. Cube-only constructs (segments, pre-aggregations, hierarchies, folders, view curation, formats, access policies, …) are preserved in custom_extensions[CUBE], so Cube → Ossie → Cube is lossless.
• Export (ossie-cube export): Ossie → Cube files. Cube has a meta field at every level, so Ossie constructs Cube has no slot for (unique_keys, foreign-vendor custom_extensions, the structured form of ai_context) are parked under meta.ossie rather than dropped — making Ossie → Cube → Ossie lossless too.

The Ossie semantic_model maps to a Cube view, not a cube. Cube users are view-first, and Cube's own AI agent reads meta.ai_context only from views and individual members — cube-level AI context is explicitly not consumed — so the view is the natural model boundary.

Fan-out semantics

Cube corrects for join row-multiplication at query time: when a cube sits on the multiplied side of a join it builds SELECT DISTINCT <primary key> FROM <join>, joins that key set back to the measure's own cube, and aggregates there, so each source row is counted once. A static Ossie expression has no way to inherit that. Cube also refuses outright when the measures themselves span cubes that fan out.

So the converter emits the fan-out-safe form wherever one exists, and reports the cases where none exists:

Cube measure	Ossie expression	Safe under fan-out?
bare count	COUNT(DISTINCT <pk>)	Yes, exactly — Cube renders count(pk) normally and count(distinct pk) when multiplied; COUNT(DISTINCT pk) equals both
count_distinct / count_distinct_approx	COUNT(DISTINCT x) / APPROX_COUNT_DISTINCT(x)	Yes, inherently
min / max	MIN(x) / MAX(x)	Yes — idempotent under duplication
sum, avg, count + sql	SUM(x), AVG(x), COUNT(x)	No

Only the last row is at risk, and only when its cube is the to (one) side of a relationship in the model. That is computable from the Ossie graph, and the converter records a structured FANOUT_UNSAFE_METRIC issue naming the metric, the dataset, and the responsible relationship. Refusing the model outright would leave the spoke on the other side with nothing, and these are the metrics most worth converting; --strict-fanout restores the refusal, mirroring Cube's own.

Going the other way, an Ossie metric combining several aggregates is decomposed into one public: false measure per aggregate, each declared on the cube its own operand reads, plus a type: number measure referencing them. Cube's correction then applies per aggregate instead of once for the whole expression. The parts carry meta.ossie.part_of, so import skips them and inlines their SQL back through the references, recovering the original expression exactly.

This points at a spec gap. Ossie has no additivity or grain declaration to record non-additivity properly. dbt's non_additive_dimension is the nearest precedent, and this repo's dbt converter already loses the same information (osi_to_msi.py hard-codes non_additive_dimension=None, with a named CUMULATIVE_SEMANTICS_LOSS issue type). Raised separately as #290.

Other design notes

• Member references follow Cube's semantics rather than being uniform: {CUBE.member} when the dataset declares a field of that name (reuses the member's SQL, compile-time checked), {CUBE}.column for a raw physical column, and {other_cube.member} across cubes — which is also what gives a cross-dataset Ossie metric its implicit join. The cube's own name is never emitted, so models survive extends.
• Calculated measures inline their {other_measure} references, because that is what Cube itself does; Ossie has no metric-to-metric reference. Cycles are rejected. Locating the aggregates inside a composite expression uses sqlglot, already a runtime dependency of the dbt and NVIDIA GSF converters for the same purpose.
• Measure filters fold into CASE WHEN … END inside the aggregate, matching Cube's own applyMeasureFilters rendering and the filtered-aggregation idiom the Ossie expression language endorses.
• Dimension type: number omits Ossie datatype rather than asserting a precision the model does not carry — Cube collapses Integer/Decimal/Float into one type, and the spec says to omit when unknown. The original type is stashed.
• type: geo dimensions split into <name>_latitude / <name>_longitude, since an Ossie field holds one expression and a geo dimension has two.

Unsupported constructs

• extends — resolving it means reproducing Cube's definition-merge semantics exactly, so it is refused rather than half-applied.
• Jinja-templated YAML and .js/.ts models — preserved verbatim and never half-converted. Jinja is detected per file, the same rule Cube's own CubeSchemaConverter uses for the Rollup Designer.

Losses the converter can absorb are returned as structured ConverterIssues (following the osi-dbt converter) rather than printed to stderr and forgotten, so a pipeline can gate on them.

Testing

226 tests, 96% line coverage:

• example-based unit tests per direction, plus CLI behavior tests;
• fixture round-trip tests in both directions, including a TPC-DS model derived from examples/tpcds_semantic_model.yaml as the converter guide asks;
• every emitted Ossie document validated against core-spec/osi-schema.json;
• Hypothesis property-based round-trip tests over generated Cube models, with a seeded fallback so the properties still run where hypothesis is unavailable.

Also verified by hand against a real production Cube model: round trip content-identical with original filenames preserved, output passing validation/validate.py, and the fan-out guard correctly flagging the two measures on the joined cube's one side.

Related Issues

Related to #290 (spec has no way to declare a non-additive metric).

Checklist

Specification

• N/A — no core-spec/ changes

Ontology

• N/A — no ontology/ changes

Converters

• New converters include tests under the converter's test directory
• CUBE registered in the supported-vendors table in converters/README.md

Validation

• N/A — no validation/ changes; emitted models are validated by the existing validation/validate.py

Documentation

• converters/cube/README.md documents the full mapping, the fan-out behavior, requirements, and limitations

Examples

• N/A — no new spec constructs; the existing TPC-DS example is used as the test baseline

Tests

• All existing tests pass (226 tests, 96% coverage; CI workflow added)
• New functionality is covered by tests

Compliance

• ASF license headers are present on all new source files
• No new runtime dependencies (PyYAML only, as with the other Python converters). The jsonschema dev dependency already ships in converters/orionbelt/

AI assistance

This contribution was developed with AI assistance (Claude). I have reviewed the code and tests and take responsibility for them, per the ASF Generative Tooling Guidance.

[Copilot]

Pull request overview

Adds a new converters/cube/ Python converter that round-trips between Apache Ossie semantic models and Cube YAML data models, including a CLI and extensive tests/fixtures, and registers CUBE in the supported-vendors list.

Changes:

• Introduces bidirectional conversion logic (convert_cube_to_ossie / convert_ossie_to_cube) with stash/parking mechanisms to preserve unmapped constructs and enable lossless round-trips.
• Adds a full test suite (fixtures, property-based tests with Hypothesis fallback, CLI tests) plus Cube converter documentation and packaging (pyproject.toml).
• Adds a dedicated GitHub Actions workflow to run Cube converter tests in CI and updates converters/README.md to list CUBE.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
converters/README.md	Registers CUBE as a supported vendor extension.
converters/cube/src/ossie_cube/_common.py	Shared conversion utilities (YAML handling, stash protocol, expression translation, mappings).
converters/cube/src/ossie_cube/cube_to_osi.py	Cube → Ossie conversion implementation (datasets/fields/relationships/metrics + preservation).
converters/cube/src/ossie_cube/osi_to_cube.py	Ossie → Cube export implementation (layout, meta parking, joins/measures/view generation).
converters/cube/src/ossie_cube/converter_issues.py	Structured issue types + issue log for lossy/unsafe conversions.
converters/cube/src/ossie_cube/cli.py	ossie-cube CLI: import/export behavior, IO, error reporting.
converters/cube/src/ossie_cube/init.py	Public API surface for the converter package.
converters/cube/README.md	Converter documentation: mapping table, fan-out semantics, usage, limitations.
converters/cube/pyproject.toml	Packaging + dev dependencies/test config for the converter.
converters/cube/tests/**	Comprehensive unit, fixture round-trip, property-based, edge-case, and CLI tests.
converters/cube/tests/fixtures/**	Cube model fixtures used for round-trip and baseline tests (including TPC-DS).
.github/workflows/converter-cube-ci.yml	CI workflow for the Cube converter test suite.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[MikeNitsenko]

Both addressed in d362c9e.

members_by_cube collisions. Correct, and worse than described. The two sanitization passes could disagree: {"Order Status", "order status"} collapsed to a single order_status with no error in _convert_model, and was only rejected later in _build_dimensions — after measures had already been placed against the short member set.

Names are now resolved once in _resolve_dimension_names and reused by every stage, so sanitization and collision detection happen in exactly one place. Added a test that a collision is rejected with a metric present, which is the ordering that was fragile.

This also fixed something not mentioned: the old set included both halves of a split geo dimension (location_latitude, location_longitude). Those never exist as Cube dimensions — they merge back into location — so a metric referencing one emitted an unresolvable {CUBE.location_latitude}. The halves now resolve to the dimension they merge into.

_HypothesisRnd.chance() ignoring p. Also correct. Now weighted, and drawn so the minimal value means False, which shrinks toward the smallest model rather than the largest. Verified the realised probability matches p exactly for every value the builders use.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

converters/cube/src/ossie_cube/osi_to_cube.py:721

• Using queue.pop(0) makes this BFS O(n²) due to repeated list shifting; on larger relationship graphs this can become unnecessarily slow. Use an index cursor (or a deque) to avoid O(n) pops from the front.

    queue = [base]
    while queue:
        current = queue.pop(0)
        for neighbor in adjacency.get(current, []):
            if neighbor in paths:
                continue
            paths[neighbor] = f"{paths[current]}.{neighbor}"
            entries.append({"join_path": paths[neighbor], "includes": "*"})
            queue.append(neighbor)

converters/cube/tests/test_roundtrip.py:42

• Typo: "licence" is misspelled here (the rest of the repo uses "license").

    licence headers on the fixtures) are not part of the data model, and key order

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

converters/cube/src/ossie_cube/osi_to_cube.py:685

• When the model has foreign-vendor custom_extensions but the imported Cube model had multiple views and none was selected (mapped_view is missing), export currently drops those extensions while logging PARKED_IN_META. This causes avoidable data loss and the issue type/message is inconsistent ("parked" vs "dropped"). Prefer parking the extensions on a deterministic view (or failing fast) so Ossie -> Cube stays lossless even in the "no mapped view" case.

        if foreign and mapped is None:
            issues.add(IssueType.PARKED_IN_META, "model",
                       "no mapped view to park foreign-vendor custom_extensions on; "
                       "they have no Cube home and are dropped")

converters/cube/README.md:279

• The README hard-codes an exact test count ("234 tests"), but the PR description claims a different number. Since this value will drift over time, it’s better to avoid a specific count (or generate it automatically) to prevent documentation from becoming stale.

234 tests at 96% line coverage: example-based unit tests per direction, CLI
behavior tests, fixture round-trip tests (including the

[MikeNitsenko]

Both worth addressing, and the first one was a real bug. Fixed in 31d49d2.

Dropped foreign-vendor extensions. Reproduced it: import a Cube model with two views (so no view is mapped), add a SNOWFLAKE extension to the Ossie model, export — and the extension is gone, reported under an issue type that says "parked".

Now refused, with the fix in the message (--view <name> on the import so a view is mapped). I considered parking on a deterministic view as suggested, but rejected it: import only reads parked extensions back from the mapped view, so that would have looked lossless while still losing them. Refusing is the honest option and matches the converter's stated contract of never dropping a field silently.

The type/message inconsistency was in two places, not one. PARKED_IN_META is documented as preserved-but-invisible-to-Cube, so reporting a genuine drop under it would let a caller gating on issue types conclude the data survived. Added DROPPED_NO_CUBE_EQUIVALENT and used it for relationship ai_context — a Cube join entry takes only name/sql/relationship, with no meta, which makes it the one construct here with nowhere to go.

Hard-coded test count. Removed from the README. It had already drifted out of sync with the PR description, which is exactly the point — the number carries no information a reader needs, while the description of what the suite covers does.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.

[MikeNitsenko]

Confirmed and fixed in d92711e. Reproduced it first — with longitude first and a city dimension between the halves, city disappeared from the output entirely:

emitted dimensions:
   {'name': 'home', 'type': 'geo', 'latitude': {...}, 'longitude': {...}}
'city' present? False

I took the second of your two suggestions rather than the first. Reserving the placeholder earlier would have worked, but the index arithmetic was the fragile part, so it's gone: dimensions are built into a dict keyed by target name, with order taken from each name's first appearance. That's well defined however the halves are arranged — either order, adjacent or not.

Probing around it turned up two more order-dependent paths in the same code:

• A geo base colliding with an ordinary field of the same name. With the ordinary field first, export emitted two dimensions called home (invalid Cube); with it second, sanitize_name correctly rejected it. Order was deciding whether an invalid model was produced. Now checked during name resolution.
• Two fields both claiming the same half silently discarded one. Now rejected.

Also validated the geo part and of values, and moved the missing-half check into name resolution so every geo problem is caught in one place before anything is built.

Six cases pinned by tests: either order, collision in both orders, duplicate half, missing half, unknown part.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.

[MikeNitsenko]

Correct, and it was the third instance of this — so rather than patch the flagged line I audited all seven export-side uses. Exactly one was a genuine park. Fixed in c22e6d2.

site	was	now
unique_keys	parked	parked (correct)
extra semantic_model entries	parked	dropped
dimension.is_time role	parked	dropped
dimension.is_time opt-out	parked	dropped
synthesized primary-key dimension	parked	approximated
no datatype → Cube type: string	parked	approximated
cross-dataset metric placement	parked	approximated

The import-direction uses were all genuine parks and are unchanged.

Added APPROXIMATED for the middle group, which neither existing type described: nothing is lost and nothing is hidden, but Cube requires a value Ossie does not carry, so the converter chose one and the output asserts slightly more than the input did. Labelling that "parked" was wrong in the same way as labelling a drop "parked" — nothing was parked in either case.

Three types kept apart on purpose, since that is what a caller gating on them needs: preserved-but-unreadable-by-Cube, actually lost, and emitted-with-a-guess. Two of the three were indistinguishable before.