Polyglot

Rust/Wasm-powered SQL transpiler for 32+ dialects, inspired by sqlglot.

Polyglot parses, generates, transpiles, and formats SQL across 32+ database dialects. It ships as:

a Rust crate (polyglot-sql)
a TypeScript/WASM SDK (@polyglot-sql/sdk)
a Python package (polyglot-sql)
a Go SDK (github.com/tobilg/polyglot/packages/go) backed by the native FFI library

There's also a playground where you can try it out in the browser, as well as Rust API Docs, TypeScript API Docs, and Python API Docs.

Release notes are tracked in CHANGELOG.md.

Features

Transpile SQL between any pair of 32 dialects
Parse SQL into a fully-typed AST
Generate SQL back from AST nodes
Format / pretty-print SQL
Fluent builder API for constructing queries programmatically
Validation with syntax, semantic, and schema-aware checks
Column lineage and OpenLineage-compatible payload generation
Compact query analysis facts for projections, relations, CTEs, and set operations
AST visitor utilities for walking, transforming, and analyzing queries
Stack-safety hardening on native targets via default-on stacker
C FFI shared/static library for multi-language bindings (polyglot-sql-ffi)
Python bindings powered by PyO3 (polyglot-sql on PyPI)

Supported Dialects (32)


Athena	BigQuery	ClickHouse	CockroachDB	Databricks
Doris	Dremio	Drill	Druid	DuckDB
Dune	Exasol	Fabric	Hive	Materialize
MySQL	Oracle	PostgreSQL	Presto	Redshift
RisingWave	SingleStore	Snowflake	Solr	Spark
SQLite	StarRocks	Tableau	Teradata	TiDB
Trino	TSQL

Quick Start

Rust

use polyglot_sql::{transpile, DialectType};

// Transpile MySQL to PostgreSQL
let result = transpile(
    "SELECT IFNULL(a, b) FROM t",
    DialectType::MySQL,
    DialectType::Postgres,
).unwrap();
assert_eq!(result[0], "SELECT COALESCE(a, b) FROM t");

use polyglot_sql::builder::*;

// Fluent query builder
let query = select(["id", "name"])
    .from("users")
    .where_(col("age").gt(lit(18)))
    .order_by(["name"])
    .limit(10)
    .build();

See the full Rust crate README for more examples.

TypeScript

npm install @polyglot-sql/sdk

import { transpile, Dialect } from '@polyglot-sql/sdk';

// Transpile MySQL to PostgreSQL
const result = transpile(
  'SELECT IFNULL(a, b) FROM t',
  Dialect.MySQL,
  Dialect.PostgreSQL,
);
console.log(result.sql[0]); // SELECT COALESCE(a, b) FROM t

import { select, col, lit } from '@polyglot-sql/sdk';

// Fluent query builder
const sql = select('id', 'name')
  .from('users')
  .where(col('age').gt(lit(18)))
  .orderBy(col('name').asc())
  .limit(10)
  .toSql('postgresql');

See the full TypeScript SDK README for more examples.

Python

pip install polyglot-sql

import polyglot_sql

result = polyglot_sql.transpile(
    "SELECT IFNULL(a, b) FROM t",
    read="mysql",
    write="postgres",
)
print(result[0])  # SELECT COALESCE(a, b) FROM t

See the full Python bindings README.

Go

go get github.com/tobilg/polyglot/packages/go

The Go module contains the PureGo wrapper only. Runtime API calls require a separate matching polyglot-sql-ffi shared library (.so, .dylib, or .dll) from the same Polyglot release or a local FFI build:

cargo build -p polyglot-sql-ffi --profile ffi_release
export POLYGLOT_SQL_FFI_PATH="$PWD/target/ffi_release/libpolyglot_sql_ffi.so"

import (
    "fmt"

    polyglot "github.com/tobilg/polyglot/packages/go"
)

client, err := polyglot.OpenDefault()
if err != nil {
    panic(err)
}
defer client.Close()

result, err := client.Transpile(
    "SELECT IFNULL(a, b) FROM t",
    "mysql",
    "postgres",
)
if err != nil {
    panic(err)
}
fmt.Println(result[0]) // SELECT COALESCE(a, b) FROM t

The Go SDK uses PureGo over polyglot-sql-ffi; it does not download native libraries or bundle release artifacts in the Go module. Build/download the FFI shared library and set POLYGLOT_SQL_FFI_PATH or pass its path to polyglot.Open. See the full Go SDK README.

Lineage and OpenLineage Output

Polyglot can trace column lineage through SQL queries and can generate OpenLineage-compatible JSON payloads from that analysis. The OpenLineage support currently produces columnLineage dataset facets, optional schema facets, and JobEvent / RunEvent payloads for supported query shapes such as SELECT, INSERT ... SELECT, and CREATE TABLE AS SELECT.

OpenLineage transport and client emission are intentionally out of scope: Polyglot builds payloads for callers to inspect, persist, or send through their own infrastructure.

TypeScript SDK examples: packages/sdk/README.md
Python examples: crates/polyglot-sql-python/README.md
Go examples: packages/go/README.md

Compact Query Analysis

For applications that need summary facts instead of a full AST or full lineage graph, analyze_query / analyzeQuery returns a compact payload with output projections, direct visible relations, transitive physical baseTables, CTE names and top-level cteFacts, original starProjections, set-operation branches, transform kinds, conservative projection nullability, optional type hints, and upstream column references. The API is additive and uses the same optional ValidationSchema shape as schema-aware validation and lineage. Validation uses broad type families, while query analysis preserves detailed schema type strings such as DECIMAL(10,2) for typeHint values when they can be parsed.

Validation schema JSON uses:

{
  "strict": true,
  "tables": [
    {
      "name": "orders",
      "schema": "analytics",
      "aliases": ["o"],
      "primaryKey": ["id"],
      "uniqueKeys": [["external_id"]],
      "foreignKeys": [
        {
          "columns": ["customer_id"],
          "references": { "table": "customers", "columns": ["id"] }
        }
      ],
      "columns": [
        { "name": "id", "type": "INT", "nullable": false, "primaryKey": true },
        { "name": "amount", "type": "DECIMAL(10,2)", "nullable": true }
      ]
    }
  ]
}

Use the type key for column types in JSON. dataType / data_type are not accepted aliases for this schema payload.

Format Guard Rails

SQL formatting runs through guard limits in Rust core to prevent pathological inputs from exhausting memory:

maxInputBytes: 16 MiB (default)
maxTokens: 1_000_000 (default)
maxAstNodes: 1_000_000 (default)
maxSetOpChain: 256 (default)

Guard failures return error codes in the message (E_GUARD_INPUT_TOO_LARGE, E_GUARD_TOKEN_BUDGET_EXCEEDED, E_GUARD_AST_BUDGET_EXCEEDED, E_GUARD_SET_OP_CHAIN_EXCEEDED).

Configuration surface by runtime:

Rust: configurable via format_with_options.
WASM: configurable via format_sql_with_options / format_sql_with_options_value.
TypeScript SDK: configurable via formatWithOptions.
C FFI: configurable via polyglot_format_with_options.
Python: configurable via keyword-only format_sql(..., max_*) overrides.

WASM low-level example (from polyglot-sql-wasm exports):

import init, { format_sql_with_options } from "./polyglot_sql_wasm.js";

await init();
const raw = format_sql_with_options(
  "SELECT a,b FROM t",
  "generic",
  JSON.stringify({
    maxInputBytes: 2 * 1024 * 1024,
    maxTokens: 250000,
    maxAstNodes: 250000,
    maxSetOpChain: 128
  }),
);
const result = JSON.parse(raw);

Stack Safety

On native Rust builds, polyglot-sql enables the optional stacker feature by default. This adds stack-growth protection around the deepest parser / generator / transpile entry points so pathological or heavily nested SQL is less likely to abort the process with a stack overflow.

Important scope notes:

Rust, C FFI, and Python native builds inherit this by default.
WASM does not use stacker; polyglot-sql-wasm depends on the core crate with default-features = false.
A few paths still use explicitly larger thread stacks as defense-in-depth for very deep workloads, especially some test harnesses, the Python worker thread, and the bench_json example.

If you want to disable stacker for a native Rust build, turn off default features and opt back into the ones you need:

[dependencies]
polyglot-sql = { version = "0.4.0", default-features = false, features = ["all-dialects", "transpile"] }

That can reduce overhead slightly on trusted inputs, but you lose the default stack-growth protection for deeply nested SQL.

Project Structure

polyglot/
├── crates/
│   ├── polyglot-sql/           # Core Rust library (parser, generator, builder)
│   ├── polyglot-sql-function-catalogs/ # Optional dialect function catalogs (feature-gated data)
│   ├── polyglot-sql-wasm/      # WASM bindings
│   ├── polyglot-sql-ffi/       # C ABI bindings (.so/.dylib/.dll + .a/.lib + header)
│   └── polyglot-sql-python/    # Python bindings (PyO3 + maturin, published on PyPI)
├── packages/
│   ├── sdk/                    # TypeScript SDK (@polyglot-sql/sdk on npm)
│   ├── go/                     # Go SDK backed by polyglot-sql-ffi
│   ├── documentation/          # TypeScript API documentation site
│   ├── playground/             # Playground for testing the SDK (React 19, Tailwind v4, Vite)
│   └── python-docs/            # Python API documentation site (Cloudflare Pages)
├── examples/
│   ├── rust/                   # Rust example
│   ├── typescript/             # TypeScript SDK example
│   └── c/                      # C FFI example
└── tools/
    ├── sqlglot-compare/        # Test extraction & comparison tool
    └── bench-compare/          # Performance benchmarks

Examples

Standalone example projects are available in the examples/ directory. Each one pulls the latest published package and can be run independently.

Rust

cargo run --manifest-path examples/rust/Cargo.toml

TypeScript

cd examples/typescript
pnpm install --ignore-workspace && pnpm start

Building from Source

# Build Rust core
cargo build -p polyglot-sql

# Build C FFI crate (shared/static libs + generated header)
cargo build -p polyglot-sql-ffi --profile ffi_release

# Build Python extension / wheel
make develop-python
make build-python

# Build WASM + TypeScript SDK
make build-all

# Or step by step:
cd crates/polyglot-sql-wasm && wasm-pack build --target bundler --release
cd packages/sdk && npm run build

C FFI

Polyglot provides a stable C ABI in crates/polyglot-sql-ffi.

Crate README: crates/polyglot-sql-ffi/README.md
Generated header: crates/polyglot-sql-ffi/polyglot_sql.h
Example program: examples/c/main.c
Make targets:
- make build-ffi
- make generate-ffi-header
- make build-ffi-example
- make test-ffi

For tagged releases (v*), CI also attaches prebuilt FFI artifacts and checksums to GitHub Releases.

Python Bindings

Polyglot provides first-party Python bindings in crates/polyglot-sql-python.

Crate README: crates/polyglot-sql-python/README.md
Package name on PyPI: polyglot-sql
Make targets:
- make develop-python
- make test-python
- make typecheck-python
- make build-python

Function Catalogs

Optional dialect function catalogs are provided via crates/polyglot-sql-function-catalogs.

Crate README: crates/polyglot-sql-function-catalogs/README.md
Core feature flags:
- stacker (enabled by default on native polyglot-sql builds)
- function-catalog-clickhouse
- function-catalog-duckdb
- function-catalog-all-dialects
Intended behavior: compile-time inclusion, one-time load in core, auto-use during schema validation type checks.

Testing

Polyglot currently runs 10,220 SQLGlot fixture cases plus additional project-specific suites. All strict pass/fail suites are at 100% in the latest verification run.

Category	Count	Pass Rate
SQLGlot generic identity	956	100%
SQLGlot dialect identity	3,554	100%
SQLGlot transpilation	5,513	100%
SQLGlot transpile (generic)	145	100%
SQLGlot parser	29	100%
SQLGlot pretty-print	23	100%
Lib unit tests	835	100%
Custom dialect identity	276	100%
Custom dialect transpilation	347	100%
ClickHouse parser corpus (non-skipped)	7,047	100%
FFI integration tests	20	100%
Python bindings tests (`make test-python`)	69	100%
Total (strict Rust/FFI pass/fail case count)	18,745	100%

# Setup fixtures (required once)
make setup-fixtures

# Run all tests
make test-rust-all          # All SQLGlot fixture suites
make test-rust-lib          # Lib unit tests (835)
make test-rust-verify       # Full strict verification suite
make test-ffi               # FFI crate integration tests

# Individual test suites
make test-rust-identity     # 956 generic identity cases
make test-rust-dialect      # 3,554 dialect identity cases
make test-rust-transpile    # 5,513 transpilation cases
make test-rust-transpile-generic # 145 generic transpile cases
make test-rust-parser       # 29 parser cases
make test-rust-pretty       # 23 pretty-print cases

# Additional tests
make test-rust-roundtrip    # Organized roundtrip unit tests
make test-rust-matrix       # Dialect matrix transpilation tests
make test-rust-compat       # SQLGlot compatibility tests
make test-rust-errors       # Error handling tests
make test-rust-functions    # Function normalization tests

# TypeScript SDK tests
cd packages/sdk && npm test

# Full comparison against Python SQLGlot
make test-compare

Benchmarks

make bench-compare          # Compare polyglot-sql vs sqlglot performance
make bench-rust             # Rust benchmarks (JSON output)
make bench-python           # Python sqlglot benchmarks (JSON output)
cargo bench -p polyglot-sql  # Criterion benchmarks

Fuzzing

cargo +nightly fuzz run fuzz_parser
cargo +nightly fuzz run fuzz_roundtrip
cargo +nightly fuzz run fuzz_transpile

Makefile Targets

Target	Description
`make help`	Show all available commands
`make build-all`	Build core release + FFI + Python + bindings + WASM/SDK
`make build-wasm`	Build WASM package + TypeScript SDK
`make build-ffi`	Build C FFI crate (`ffi_release` profile)
`make generate-ffi-header`	Generate C header via cbindgen/build.rs
`make build-ffi-example`	Build + run C example against FFI lib
`make develop-python`	Build/install Python extension in uv-managed env
`make build-python`	Build Python wheels with maturin
`make test-ffi`	Run FFI integration tests
`make test-rust`	Run SQLGlot-named Rust tests in `polyglot-sql`
`make test-rust-all`	Run all 10,220 SQLGlot fixture cases
`make test-rust-lib`	Run 835 lib unit tests
`make test-rust-verify`	Full verification suite
`make test-rust-clickhouse-parser`	Run strict ClickHouse parser suite
`make test-rust-clickhouse-coverage`	Run ClickHouse coverage suite (report-only)
`make test-compare`	Compare against Python sqlglot
`make bench-compare`	Performance comparison
`make bench-rust-parsing-report`	Run `rust_parsing` bench and generate Markdown report
`make bench-parse`	Core parse benchmark (polyglot vs sqlglot)
`make bench-parse-quick`	Faster core parse benchmark mode
`make bench-parse-full`	Parse benchmark including optional parsers
`make extract-fixtures`	Regenerate JSON fixtures from Python
`make setup-fixtures`	Create fixture symlink for Rust tests
`make generate-bindings`	Generate TypeScript type bindings
`make test-python`	Run Python bindings tests
`make typecheck-python`	Run Python bindings type-check
`make documentation-build`	Build documentation site
`make documentation-deploy`	Deploy documentation to Cloudflare Pages
`make python-docs-build`	Build Python API docs site
`make python-docs-deploy`	Deploy Python API docs to Cloudflare Pages
`make playground-build`	Build playground
`make playground-deploy`	Deploy playground to Cloudflare Pages
`make clean`	Remove all build artifacts

Rust Parsing Markdown Report

For a release-note friendly summary of the pure Rust parse benchmark (short, long, tpch, crazy), run:

make bench-rust-parsing-report

This will:

run cargo bench -p polyglot-sql --bench rust_parsing
read Criterion results from target/criterion/rust_parse_quick_equivalent/parse_one
generate target/criterion/rust_parsing_report.md with a Markdown table (mean, std dev, 95% CI, baseline delta if available)

Latest generated result snapshot:

Generated: 2026-02-26 13:43:10 UTC
Source: target/criterion/rust_parse_quick_equivalent/parse_one

Query	Mean	Std Dev	95% CI (mean)	Change vs baseline
short	51.28 us	481.03 ns	51.09 us - 51.60 us	-8.40%
long	259.61 us	666.53 ns	259.23 us - 260.01 us	-5.21%
tpch	268.59 us	776.85 ns	268.15 us - 269.07 us	-0.03%
crazy	1.03 ms	66.07 us	992.65 us - 1.07 ms	+6.05%

Licenses

MIT
sqlglot MIT

Name		Name	Last commit message	Last commit date
Latest commit History 140 Commits
.cargo		.cargo
.github/workflows		.github/workflows
crates		crates
docs		docs
examples		examples
licenses		licenses
packages		packages
scripts		scripts
tools		tools
.gitattributes		.gitattributes
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Polyglot

Features

Supported Dialects (32)

Quick Start

Rust

TypeScript

Python

Go

Lineage and OpenLineage Output

Compact Query Analysis

Format Guard Rails

Stack Safety

Project Structure

Examples

Rust

TypeScript

Building from Source

C FFI

Python Bindings

Function Catalogs

Testing

Benchmarks

Fuzzing

Makefile Targets

Rust Parsing Markdown Report

Licenses

About

Uh oh!

Releases 33

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Polyglot

Features

Supported Dialects (32)

Quick Start

Rust

TypeScript

Python

Go

Lineage and OpenLineage Output

Compact Query Analysis

Format Guard Rails

Stack Safety

Project Structure

Examples

Rust

TypeScript

Building from Source

C FFI

Python Bindings

Function Catalogs

Testing

Benchmarks

Fuzzing

Makefile Targets

Rust Parsing Markdown Report

Licenses

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 33

Contributors

Uh oh!

Languages