add Hex exporter

[WilliamKelley]

Summary

Adds an export converter for Hex's semantic layer format. To preface, let me say I appreciate your time reviewing this rather large contribution. Please let me know if you have questions about any of what follows, I'd be happy to answer and help simplify.

We're still working on the import side of things. It's taken longer because we're shifting utilities out from internal libraries into a public library so that we can transform our expression languages (Hex calculation language; SQL with Hex placeholders) into SQL with Ossie's expression language.

---

For the foundation of Hex's semantic layer, we've open-sourced several pieces in our own public library: hex-sl-utils. The relevant parts for this exporter are

• pydantic models with validators
• load_* functions that process file data into these pydantic models

I'm glad to see the Ossie definitions are taking on a similar format.

Approach

I've taken a pretty structured approach to this converter that diverges from other ones. At Hex we've written and maintained converters for a few external semantic layer formats, and we've come to a particular way of writing them. I've carried that over here and have tried to communicate these norms in the README and CONTRIBUTING documents. I'll briefly restate here.

Conversion happens in distinct phases: load, convert, dump. Roughly, that translates to reading files into memory, constructing valid models, converting between formats, and writing them back to files.

During conversion, issues can obviously arise. These are abstracted to a "Problem".

• Problems have severity levels: fatal, error, warning info. Fatal problems block execution. Errors mean things have to be dropped. Warnings mean processable with caveats. Informational is just that.
• Problems have cause paths. These are a breadcrumb trail back to the source definition in the original document where the issue arose. They're useful for debugging at runtime or on the command line, and they're really useful for enabling tight IDE integration. (For example, in Hex we integrate this with a monaco editor to highlight problems directly in the editor.)

To fluently construct problems, each phase has a waterfall-style of execution that goes through the document and processes each field with successive functions calls. A Python contextmanager is a clever way to manage this concern.

The end goal to traverse and process as much of the input as possible until we can proceed no further. That should return the most useful set of output for the consumer.

Complexities

I've tried to note complexities, notes, and assumptions in the source-code as they arose.

The main challenge was that Ossie expresses Metrics as a top-level concept, and not tied to a Dataset. Hex takes the opposite approach, placing Measures underneath a Model. During export, we have to "pushdown" these global metrics to a respective dataset. We end up analyzing the expression for all the datasets referenced, and then assign it to essentially our best guess. Ideally this puts them on "fact" models and not "dimension" models, but it's an ambiguous task. We do this at the same time that we "pushdown" Ossie Relationships to Hex Relations which are underneath a Model. We also have to transform the expression, because we have a specific syntax for referencing a Field/Dimension that exists on another Dataset/Model.

Ossie prescribes that expressions can be written in a variety of Dialects. Where these are SQL Dialects, these make sense. But I do not understand why expression languages (like MDX, MAQL, Thoughtspot, Sigma) are allowed to be written in an interchange format. These are fundamentally not interchangeable. They're proprietary syntaxes that dont execute on the underlying warehouse sql dialect engine and need to be transformed by the vendor. I would appreciate your guidance here to help understand why this is part of the spec and how are we expected to handle it?

I found Ossie's Expression Language spec helpful to understand syntax and construction of logical references (i.e. FieldExpr), but I found the logical spec lacking with respect to when and how this can be used. References look like normal identifiers (dataset.field looks like table.column); there's no delimiters that distinguish a logical or a physical column. When it comes to an Ossie Field expression, can it contain only physical references, or logical ones too? If it can contain both, how do you reliably tell them apart? If it can contain logical columns, can these be cross-dataset references, e.g. other-dataset.field? (In this exporter, I assumed Ossie Fields could only contain physical column which simplifies things, but it poses a challenge for our importer as Hex allows both in Dimension expressions.)

---

The uninteresting part of this contribution is all of the setup/scaffolding. I've tried to isolate this to the first commit, but some things were tacked on to the end.

• a uv project with lastest Astral tooling
• a justfile with command recipes
• a continuous-integration workflow

Checklist

Specification

• Spec changes are included in core-spec/ and follow the existing structure
• Spec changes have been discussed on the mailing list or in a linked issue
• Breaking changes to the spec are clearly called out in the summary

Ontology

• Ontology changes in ontology/ are consistent with spec changes
• New or modified terms are defined and documented

Converters

• Converter logic in converters/ is updated to reflect spec or ontology changes
• New converters include tests under the converter's test directory

Validation

• Validation rules in validation/ are updated if the spec changed
• New validation cases are covered by tests

Documentation

• docs/ is updated to reflect any user-facing changes
• New features or behaviors are documented with examples where appropriate
• CONTRIBUTING.md is updated if the contribution process changed

Examples

• examples/ are added or updated for any new spec constructs or converter support

Tests

• All existing tests pass (pytest / CI green)
• New functionality is covered by tests

Compliance

• ASF license headers are present on all new source files
• No third-party dependencies are added without PMC/IPMC approval

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Adds a new Ossie → Hex exporter (library + CLI) and wires it into the repo’s spec metadata and CI so Hex becomes a first-class “vendor” target.

Changes:

• Adds a new ossie-hex Python package implementing a phased load/convert/dump pipeline with structured problem reporting.
• Updates the Ossie spec/docs/schema to include HEX as a well-known vendor example.
• Adds CLI entrypoint, test suite (including snapshots), and a dedicated CI workflow for the converter.

Reviewed changes

Copilot reviewed 88 out of 89 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
python/src/ossie/models.py	Adds HEX vendor and improves pydantic alias handling for from serialization/validation.
core-spec/spec.yaml	Adds HEX to vendor name examples.
core-spec/spec.md	Documents HEX as a well-known vendor example.
core-spec/ossie-schema.json	Adds HEX to schema examples for vendor field.
converters/hex/tests/init.py	Adds package init (license header).
converters/hex/tests/utils.py	Adds test snapshot helpers for problems and Hex project YAML output.
converters/hex/tests/test_smoke.py	Adds smoke tests for public API surface and CLI --help.
converters/hex/tests/cli/test_cli_export.py	Adds end-to-end CLI export tests and report formatting assertions.
converters/hex/tests/ossie_to_hex/utils.py	Adds quick Ossie model builders for tests.
converters/hex/tests/ossie_to_hex/test_load_ossie_document.py	Tests Ossie YAML loading/validation (file errors, YAML errors, schema errors).
converters/hex/tests/ossie_to_hex/test_load_ossie_semantic_model.py	Tests semantic model member validation/removal behavior.
converters/hex/tests/ossie_to_hex/test_load_ossie_dataset.py	Tests dataset loading (invalid field removal).
converters/hex/tests/ossie_to_hex/test_load_ossie_field.py	Tests field loading (expression validation, datatype defaults).
converters/hex/tests/ossie_to_hex/test_load_ossie_metric.py	Tests metric loading (parsing, reference validation, datatype defaults).
converters/hex/tests/ossie_to_hex/test_load_ossie_relationship.py	Tests relationship validation (dataset resolution, column length matching).
converters/hex/tests/ossie_to_hex/test_load_ossie_expression.py	Tests expression loading behavior for fields vs metrics.
converters/hex/tests/ossie_to_hex/test_load_ossie_dialect_expression.py	Tests parsing/validation for dialect expressions.
converters/hex/tests/ossie_to_hex/test_load_ossie_dialect.py	Tests dialect parsing/defaulting behavior.
converters/hex/tests/ossie_to_hex/test_dump_hex_resource.py	Tests Hex resource YAML writing + path traversal protection.
converters/hex/tests/ossie_to_hex/test_dump_hex_project.py	Tests project dir creation/resolution + traversal protection.
converters/hex/tests/ossie_to_hex/test_convert_ossie_name.py	Tests identifier normalization rules.
converters/hex/tests/ossie_to_hex/test_convert_ossie_datatype.py	Tests datatype mapping behavior.
converters/hex/tests/ossie_to_hex/test_convert_ossie_dialect.py	Tests Ossie→Hex dialect mapping + expression-language fallback.
converters/hex/tests/ossie_to_hex/test_convert_ossie_dialect_expression.py	Tests SQL placeholder rewriting to Hex semantic references.
converters/hex/tests/ossie_to_hex/test_convert_ossie_field.py	Tests field conversion to Hex dimension, including dropped features warnings.
converters/hex/tests/ossie_to_hex/test_convert_ossie_dataset.py	Tests dataset conversion to Hex model and unique key behaviors.
converters/hex/tests/ossie_to_hex/test_convert_ossie_relationship.py	Tests relationship analysis/assignment and relation conversion.
converters/hex/tests/ossie_to_hex/test_convert_ossie_metric.py	Tests metric analysis/assignment and measure conversion.
converters/hex/src/ossie_hex/util/problem.py	Introduces structured Problem model and severity/phase typing.
converters/hex/src/ossie_hex/util/context.py	Adds base conversion Context with phase/path scoping and problem reporting.
converters/hex/src/ossie_hex/util/yaml.py	Adds YAML 1.2 boolean semantics loader/dumper helpers for converter I/O.
converters/hex/src/ossie_hex/util/parse_sql.py	Adds SQL parsing wrapper around vendored sqlglot.
converters/hex/src/ossie_hex/util/database_table.py	Adds heuristics to distinguish table references vs SQL queries.
converters/hex/src/ossie_hex/ossie_to_hex/problem_code.py	Adds stable problem codes and summary strings for CLI grouping.
converters/hex/src/ossie_hex/ossie_to_hex/context/analysis.py	Adds analysis structures (metric + relationship) needed for assignment decisions.
converters/hex/src/ossie_hex/ossie_to_hex/context/assignment.py	Adds assignment structures for anchoring metrics/relationships to Hex models.
converters/hex/src/ossie_hex/ossie_to_hex/context/hex_ids.py	Adds mapping from Ossie names → Hex ids across entity kinds.
converters/hex/src/ossie_hex/ossie_to_hex/context/context.py	Implements ExportContext scoping and shared state for export pipeline.
converters/hex/src/ossie_hex/ossie_to_hex/context/init.py	Exposes export context types for internal use/tests.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_document.py	Implements load phase document reading/parsing/validation.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_semantic_model.py	Implements load phase semantic model filtering of invalid members.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_dataset.py	Implements load phase dataset sanitization (fields + unique_keys).
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_field.py	Implements load phase field datatype/expression validation.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_metric.py	Implements load phase metric datatype/expression validation.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_relationship.py	Implements load phase relationship validation.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_datatype.py	Implements load phase datatype defaulting with warnings.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_dialect.py	Implements load phase dialect parsing/defaulting.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_expression.py	Implements load phase expression filtering by dialect validity.
converters/hex/src/ossie_hex/ossie_to_hex/load_ossie_dialect_expression.py	Implements sqlglot parsing and reference validation for expressions.
converters/hex/src/ossie_hex/ossie_to_hex/build_assignments.py	Implements assignment planning for ambiguous metric anchoring/relationships.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_document.py	Converts loaded Ossie docs into Hex projects.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_semantic_model.py	Converts a semantic model into a Hex project (with analyze/assign steps).
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_dataset.py	Converts datasets to Hex models (source/table/query heuristics, keys→unique).
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_field.py	Converts fields to Hex dimensions (datatype + expr_sql + uniqueness).
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_metric.py	Converts metrics to Hex measures (analysis + resolver-based SQL rewrite).
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_relationship.py	Converts relationships to Hex relations and attaches them to models.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_name.py	Normalizes Ossie names into valid Hex entity ids.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_datatype.py	Maps Ossie datatypes to Hex types.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_dialect.py	Maps Ossie dialects to Hex SQL dialects (fallback handling).
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_expression.py	Chooses an expression dialect and converts it to Hex SQL.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_dialect_expression.py	Rewrites qualified refs like dataset.field into ${...} references.
converters/hex/src/ossie_hex/ossie_to_hex/dump_hex_resource.py	Serializes/writes Hex resources with path traversal protection.
converters/hex/src/ossie_hex/ossie_to_hex/dump_hex_project.py	Writes Hex project directory and resources.
converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_to_hex.py	Public API orchestration for load/convert/dump phases.
converters/hex/src/ossie_hex/ossie_to_hex/init.py	Exposes convert_ossie_to_hex as module public API.
converters/hex/src/ossie_hex/ossie/ossie_dialect_name.py	Adds a small typing helper for dialect name literals.
converters/hex/src/ossie_hex/ossie/init.py	Exposes Ossie dialect name typing helper.
converters/hex/src/ossie_hex/hex/_brand.py	Imports Hex domain types from hex-sl-utils and re-exports them.
converters/hex/src/ossie_hex/hex/hex_sql.py	Defines HexSql annotated type and examples.
converters/hex/src/ossie_hex/hex/utils.py	Adds utility for temporal datatype checks.
converters/hex/src/ossie_hex/hex/init.py	Exposes Hex domain types and helpers for converter.
converters/hex/src/ossie_hex/cli/main.py	Adds ossie-hex export CLI command and argument parsing.
converters/hex/src/ossie_hex/cli/report.py	Adds human-readable conversion report formatting.
converters/hex/src/ossie_hex/cli/main.py	Adds module entrypoint for CLI execution.
converters/hex/src/ossie_hex/cli/init.py	Exposes CLI main.
converters/hex/src/ossie_hex/init.py	Exposes top-level convert_ossie_to_hex.
converters/hex/scripts/test-python-matrix.sh	Adds local version-matrix runner mirroring CI.
converters/hex/pyproject.toml	Adds packaging metadata and tool (uv/ruff/pytest/ty) configuration.
converters/hex/justfile	Adds dev workflows (setup/check/test/build/snapshots).
converters/hex/README.md	Adds converter user docs (CLI + Python API + feature mapping).
converters/hex/CONTRIBUTING.md	Adds converter-specific dev guide and conventions.
converters/hex/.vscode/settings.json	Adds editor defaults for local development.
converters/hex/.vscode/extensions.json	Recommends VS Code extensions for formatting/linting.
converters/hex/.python-version	Pins a Python version for local tooling.
converters/README.md	Adds Hex entry to converter vendor list.
.github/workflows/converter-hex-ci.yml	Adds CI workflow for checks/tests/build for the new converter package.

Suppressed comments (4)

converters/hex/src/ossie_hex/util/context.py:1

• phase_scope always resets current_phase_name to None on exit, which breaks correct phase attribution if phase_scope is ever nested (or called while another phase is set). Store the previous phase value and restore it in finally to make phase scoping properly stack-safe.
  converters/hex/src/ossie_hex/util/database_table.py:1
• The unquoted identifier branch [A-Za-z0-9_$-]* allows - in table name parts. In most SQL dialects, a dash requires quoting (otherwise it’s parsed as a minus operator), so this can incorrectly classify strings like foo-bar as a table ref instead of a query, affecting whether base_sql_table vs base_sql_query is emitted. Consider removing - from the unquoted pattern (keeping it only in quoted forms) to make is_table_name() more conservative.
  converters/hex/tests/utils.py:1
• Tests are importing a private helper (_reorder_fields) from dump_hex_resource. This couples tests to an internal implementation detail and makes refactors harder. Prefer testing the public behavior by calling dump_hex_resource() (writing to a temp dir) or exposing a small public utility (e.g., reorder_hex_fields(...)) if snapshotting in-memory structures is important.
  converters/hex/src/ossie_hex/ossie_to_hex/convert_ossie_expression.py:1
• The warning interpolates enum objects (ctx.ossie_dialect, fallback.dialect) which can produce less user-friendly strings (e.g., OssieDialect.ANSI_SQL) rather than the expected canonical dialect names. Use .value for both so CLI output is stable and consistent with user input (ansi_sql, snowflake, etc.).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.