Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
2d42730
feat(thoughtspot): package skeleton, CI workflow, and ASF header gate
djwaldo Sep 1, 2026
dedd394
fix(thoughtspot): add ASF header to pyproject.toml, pin CI actions to…
djwaldo Sep 1, 2026
b64cab6
feat(thoughtspot): vocabulary constants with dialect registration flag
djwaldo Sep 1, 2026
aa65d28
feat(thoughtspot): YAML 1.2 codec so on/off/yes/no survive a round trip
djwaldo Sep 1, 2026
0d04ee3
test(thoughtspot): pin loader/dumper fixes independently with plain-P…
djwaldo Sep 1, 2026
96cff37
feat(thoughtspot): structured never-silent issue reporting
djwaldo Sep 1, 2026
e5b516f
feat(thoughtspot): custom_extensions stash with staleness-guarded res…
djwaldo Sep 1, 2026
6f70186
feat(thoughtspot): identifier normalisation with case-folded collisio…
djwaldo Sep 1, 2026
c9f3431
fix(thoughtspot): reject ambiguous column refs; document normalise AS…
djwaldo Sep 1, 2026
881fc4b
feat(thoughtspot): key derivation that refuses to manufacture false keys
djwaldo Sep 1, 2026
134606c
fix(thoughtspot): report empty-to_columns relationships, document KD3…
djwaldo Sep 1, 2026
7098e4d
docs(thoughtspot): README with direction statement and coverage matrix
djwaldo Sep 1, 2026
e235138
docs(thoughtspot): add identifier-derivation limitations, tighten L2 …
djwaldo Sep 1, 2026
57b3fbd
build(thoughtspot): match sibling converters' packaging and CI shape
djwaldo Sep 1, 2026
71a2b5a
fix(thoughtspot): correct KD2's upstream-warning claim; harden _yaml …
djwaldo Sep 1, 2026
6b495f8
fix(thoughtspot): fold diacritics in normalise(); close a column-ref …
djwaldo Sep 1, 2026
589dbab
docs(thoughtspot): future-tense unimplemented claims; name the extern…
djwaldo Sep 1, 2026
afa275d
docs(thoughtspot): fix README dev commands for the hatchling/uv packa…
djwaldo Sep 2, 2026
1d79c8a
fix(thoughtspot): flip DIALECT_IS_REGISTERED now that ossie#351 merged
djwaldo Sep 2, 2026
262cd87
feat(thoughtspot): expression catalog vocabulary and spec-coverage gate
djwaldo Sep 2, 2026
386e3f4
fix(thoughtspot): reconcile 137 vs 146 with an explicit divergence set
djwaldo Sep 2, 2026
a4d25b3
fix(thoughtspot): correct Spelling audit, drop dead code, gate empty …
djwaldo Sep 2, 2026
317cc40
feat(thoughtspot): expression emitters for direct, passthrough and un…
djwaldo Sep 2, 2026
e6ec98f
fix(thoughtspot): enforce the E8 wrapper instead of relying on caller…
djwaldo Sep 2, 2026
128654d
fix(thoughtspot): make the E8 PARTITION BY guard whitespace-tolerant
djwaldo Sep 2, 2026
976cef9
feat(thoughtspot): catalog entries for aggregate functions and type c…
djwaldo Sep 2, 2026
0f9ff63
feat(thoughtspot): catalog entries for date/time functions
djwaldo Sep 2, 2026
35a265a
feat(thoughtspot): catalog entries for string functions
djwaldo Sep 2, 2026
1771f55
fix(thoughtspot): reject a passthrough template that wraps itself
djwaldo Sep 2, 2026
fc25f2a
feat(thoughtspot): catalog entries for mathematical and conditional f…
djwaldo Sep 2, 2026
9dffb12
feat(thoughtspot): catalog entries for operators and constructs
djwaldo Sep 2, 2026
0fde55b
fix(thoughtspot): escape IN/NOT IN braces; force external dispatch fo…
djwaldo Sep 2, 2026
35e4212
feat(thoughtspot): catalog entries for window functions
djwaldo Sep 2, 2026
022ee18
feat(thoughtspot): reverse-direction inventory (ThoughtSpot -> Ossie)
djwaldo Sep 2, 2026
4e8b5a5
fix(ci): trigger thoughtspot CI when the upstream spec file changes
djwaldo Sep 2, 2026
a9392a4
fix(thoughtspot): custom_extensions_fragment must be keyed by column,…
djwaldo Sep 2, 2026
30ac574
fix(thoughtspot): correct the template contract and two catalog defec…
djwaldo Sep 2, 2026
05708f2
docs(thoughtspot): remove citations to a private, deleted workspace f…
djwaldo Sep 2, 2026
161c127
fix(thoughtspot): remove remaining internal-process leakage; add pass…
djwaldo Sep 2, 2026
3fa38b1
feat(thoughtspot): shallow formula tokenizer for reference rewriting …
djwaldo Sep 2, 2026
2391c05
fix(thoughtspot): stop quote state leaking into bracket bodies, deleg…
djwaldo Sep 2, 2026
650a1aa
fix(thoughtspot): loosen column-ref table grammar to match its own am…
djwaldo Sep 2, 2026
f536d0d
docs(thoughtspot): drop review-process wording from a test comment
djwaldo Sep 2, 2026
a25a7db
feat(thoughtspot): bidirectional datatype map with declared losses named
djwaldo Sep 2, 2026
66e45b3
docs(thoughtspot): remove citations to an internal revision id
djwaldo Sep 2, 2026
55d260c
feat(thoughtspot): TML document set with the guid/brace/ordering/YAML…
djwaldo Sep 2, 2026
8326bd0
fix(thoughtspot): sanitise dump_document_set filenames and strip nest…
djwaldo Sep 2, 2026
3466a79
fix(thoughtspot): reserve minted filenames by their final form, and c…
djwaldo Sep 2, 2026
2764b35
test(thoughtspot): add a self-policing guard against unresolvable ref…
djwaldo Sep 2, 2026
dc89534
feat(thoughtspot): convert TML columns to Ossie fields with verbatim …
djwaldo Sep 2, 2026
4cf68eb
fix(thoughtspot): give convert_field a formulas map instead of an inl…
djwaldo Sep 3, 2026
25b01eb
fix(thoughtspot): log the genuine silent losses review round 2 found
djwaldo Sep 3, 2026
8b9475b
feat(thoughtspot): convert TML measures to Ossie metrics, composing c…
djwaldo Sep 3, 2026
3e9d4a2
fix(thoughtspot): stash the source metric shape for round-trip fidelity
djwaldo Sep 3, 2026
0f88a8c
fix(thoughtspot): stop silent double aggregation on group_aggregate a…
djwaldo Sep 3, 2026
babf764
fix(thoughtspot): only warn when an aggregate is nested, not when it …
djwaldo Sep 3, 2026
f66af0c
feat(thoughtspot): complete the TML -> Ossie direction with keys, sta…
djwaldo Sep 3, 2026
1d17e7a
fix(thoughtspot): stash unconsumed column properties instead of dropp…
djwaldo Sep 3, 2026
279971d
fix(thoughtspot): validate relationship targets, centralize the ident…
djwaldo Sep 3, 2026
23c317c
fix(thoughtspot): read SQL View columns from sql_view_columns[], not …
djwaldo Sep 3, 2026
11925f3
feat(thoughtspot): build Table TML documents from Ossie datasets
djwaldo Sep 3, 2026
f241a13
fix(thoughtspot): a bare field's portable expression names the wareho…
djwaldo Sep 3, 2026
b4cc599
fix(thoughtspot): read the stashed warehouse column name back, and th…
djwaldo Sep 3, 2026
51a7d22
refactor(thoughtspot): centralize every custom_extensions[THOUGHTSPOT…
djwaldo Sep 3, 2026
82b74d2
feat(thoughtspot): build the Model TML document under the R3-R9 impor…
djwaldo Sep 3, 2026
fe24955
fix(thoughtspot): rewrite formula cross-references to match normalise…
djwaldo Sep 3, 2026
998a5b1
fix(thoughtspot): rename FULL_OUTER to OUTER in every emitted join type
djwaldo Sep 3, 2026
9d2b49b
fix(thoughtspot): drop is_hidden/was_auto_generated from emitted columns
djwaldo Sep 3, 2026
f0ccf6b
feat(thoughtspot): complete the Ossie -> TML direction with joins and…
djwaldo Sep 3, 2026
109b44a
fix(thoughtspot): a formula cross-reference is not a runtime parameter
djwaldo Sep 3, 2026
0f2bd90
fix(thoughtspot): audit and enforce X5's stash-currency rule across e…
djwaldo Sep 3, 2026
36aa2da
fix(thoughtspot): unsurfaced-column membership is a second, orthogona…
djwaldo Sep 3, 2026
074455a
test(thoughtspot): TPC-DS and minimal fixture pairs
djwaldo Sep 3, 2026
e6ae6be
feat(thoughtspot): CLI entry point for both directions
djwaldo Sep 3, 2026
6bcb445
test(thoughtspot): round-trip both directions, asserting preservation…
djwaldo Sep 3, 2026
1a57ab3
fix(thoughtspot): restore a referencing join's own name and stop dupl…
djwaldo Sep 3, 2026
30b0833
test(thoughtspot): property-based round-trip over adversarial identif…
djwaldo Sep 3, 2026
42c6519
docs(thoughtspot): README, packaging and CI for the bidirectional con…
djwaldo Sep 3, 2026
17d52fe
docs(thoughtspot): generate expression, reverse, datatype and payload…
djwaldo Sep 7, 2026
c8de6f2
fix(thoughtspot): state the substance of internal rule citations, dro…
djwaldo Sep 7, 2026
e232647
fix(thoughtspot): close the last mapping-doc rule-id family (I)
djwaldo Sep 7, 2026
5fcc34f
fix(thoughtspot): swap ONE_TO_MANY relationship endpoints per spec
djwaldo Sep 7, 2026
b25e9e2
fix(thoughtspot): degrade non-Latin field/metric names instead of dro…
djwaldo Sep 8, 2026
d905a3f
test(thoughtspot): realign the TPC-DS fixture with the reference model
djwaldo Sep 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/converter-thoughtspot-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

name: Converters ThoughtSpot CI
on:
push:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
# catalog.py's spec_construct_names() parses this file directly (it is the
# upstream-spec coverage oracle, not vendored into converters/thoughtspot/),
# so a change here must trigger this workflow even though it touches no
# converter file. No sibling workflow parses the spec, hence this addition
# is not mirrored anywhere else.
- 'core-spec/expression_language.md'
pull_request:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
- 'core-spec/expression_language.md'

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]

steps:
- name: Checkout project
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "${HOME}/.local/bin" >> "${GITHUB_PATH}"

- name: Sync dependencies
working-directory: converters/thoughtspot
run: |
uv sync

- name: Unit Tests
working-directory: converters/thoughtspot
run: |
uv run pytest
1 change: 1 addition & 0 deletions converters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The Ossie specification currently defines extensions for the following vendors:
| `WISDOM` | WisdomAI domain |
| `NVIDIA_GSF` | NVIDIA Generative Semantic Fabric standalone YAML |
| `SIGMA` | Sigma Computing data model |
| `THOUGHTSPOT` | ThoughtSpot TML (Model + Table/SQL View) |

Each vendor may define custom extensions (via the `custom_extensions` field in the Ossie spec) to carry vendor-specific metadata that does not have an equivalent in the core specification.

Expand Down
25 changes: 25 additions & 0 deletions converters/thoughtspot/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

__pycache__/
*.py[cod]
.pytest_cache/
*.egg-info/
dist/
build/
.venv/
venv/
311 changes: 311 additions & 0 deletions converters/thoughtspot/README.md

Large diffs are not rendered by default.

76 changes: 76 additions & 0 deletions converters/thoughtspot/docs/datatype-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<!-- GENERATED FILE -- do not edit by hand.
Produced by `tools/generate_reference_docs.py` from:
- `src/ossie_thoughtspot/datatypes.py`
Regenerate with:
uv run --python 3.13 python tools/generate_reference_docs.py
tests/test_reference_docs_current.py fails the suite if this file
drifts from what the generator currently produces. -->

# Ossie <-> ThoughtSpot Datatype Map

The bidirectional Ossie <-> ThoughtSpot TML datatype map. The map is **not injective** — several Ossie types collapse onto one TML spelling and cannot be told apart on the way back; see "Not injective" below.

## The closed Ossie datatype enum

`Boolean`, `Date`, `DateTime`, `DateTimeTz`, `Decimal`, `Float`, `Integer`, `Opaque`, `String`, `Time`

## Ossie -> TML

| Ossie datatype | TML `data_type` (default) | Notes |
|---|---|---|
| `Boolean` | `BOOLEAN` | connection-dependent spelling — `BOOLEAN` by default, `BOOL` when the connection's own TML spells it that way |
| `Date` | `DATE` | exact, single spelling |
| `DateTime` | `DATE_TIME` | exact, single spelling |
| `DateTimeTz` | `DATE_TIME` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Decimal` | `DOUBLE` | exact, single spelling |
| `Float` | `DOUBLE` | connection-dependent spelling — `DOUBLE` by default, `FLOAT` when the connection's own TML spells it that way; ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Integer` | `INT64` | exact, single spelling |
| `Opaque` | `VARCHAR` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `String` | `VARCHAR` | exact, single spelling |
| `Time` | `VARCHAR` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

A column with no declared `datatype` at all infers `INT64` rather than raising — `datatype` is optional in Ossie, but TML rejects a column with no `db_column_properties` block at all.

## TML -> Ossie

| TML `data_type` | Ossie datatype |
|---|---|
| `BOOL` | `Boolean` |
| `BOOLEAN` | `Boolean` |
| `DATE` | `Date` |
| `DATE_TIME` | `DateTime` |
| `DOUBLE` | `Decimal` |
| `FLOAT` | `Float` |
| `INT64` | `Integer` |
| `VARCHAR` | `String` |

A TML `data_type` outside this map returns no Ossie datatype at all — `datatype` is optional in Ossie, so omitting it is preferred over inventing one.

## Not injective — declared losses

| Ossie datatype | Why the round trip is lossy |
|---|---|
| `DateTimeTz` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Float` | ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Opaque` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `Time` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

Loading