[simonw]

Adds a Column statistics item to each column's cog menu on the table page (and to the mobile "Column actions" dialog). It opens a modal that summarises the column by running aggregate SQL queries built in JavaScript against /<database>/-/query.json.

The item is only offered when the actor has execute-sql permission on the database. In that case the table page data gains an executeSql key. column-stats.js is not loaded with the page: a small inline loader (_column_stats_loader.html) imports it the first time the dialog is opened, the same way the Export data dialog loads export-dialog.js.

In a work tree build a feature where each column cog action menu has a "Column statistics" item - clicking that item opens a modal dialog which displays statistics about that column - for text columns that's most common values, stats about value length - for numeric columns it's more interesting than that. Let's do a bit of percentile analysis in there too. Under the hood it works by firing off SQL queries using SQLite window functions and suchlike. For the moment let's have that option only available if SQL queries are allowed against the database, and the SQL queries are built in JavaScript.

Then:

The column statistics panel needs to take into account the currently applied filters on the table page

At the top of the dialog add pills for all of the other columns so you can open the stats dialog once and then switch between columns - include the column type

Claude Opus 5.5

What the dialog shows

• Overview: matching rows, not null, null, distinct. If a column mixes SQLite storage types, a breakdown of integer, real, text and blob.
• Numbers:
  • min, max, mean, median, sample standard deviation, sum
  • a histogram with "nice" bucket widths (one bucket per value for small integer ranges)
  • a box plot on the same scale as the histogram
  • percentiles (p1 to p99), computed with row_number() over (order by v) and linear interpolation, matching numpy.percentile and statistics.quantiles(method="inclusive")
  • outlier counts beyond 1.5 × the interquartile range
• Text:
  • min, max, mean and median length, plus length percentiles
  • counts of blank, padded, multi-line, numeric-looking and ISO date values
  • earliest and latest date, and a by-year, month or decade chart for mostly-date columns
• Most common values: the top 10, each linking to the table filtered to that value. This comes first for categorical text columns. Values that appear only once are left out when they'd just be an arbitrary pick from a tie.
• SQL queries: every query used, each with an "Open in SQL editor" link. The list is shown even when a query fails.

Each section shows its own errors, so one timed-out query doesn't hide the rest. Closing the dialog cancels any queries still running.

Respects the current filters

Statistics cover the rows matching the table page's current filters, including _search, _where and _through. _next pagination is excluded, so they cover every matching row, not just the current page.

• The server sends the filter's where clause, its parameters and a readable description as executeSql.filter.
• Parameters are inlined into the SQL as literals rather than sent as query string parameters. /-/query.json binds every parameter as a string, which would break numeric filters such as ?age__gt=40 on columns with no declared type.
• A "Use all rows" / "Use filtered rows" toggle switches between the filtered rows and the whole table.
• Value links add to the current filters instead of replacing them.

Switching columns

Pills at the top of the dialog list every column with its declared type, from the new executeSql.columns list. That includes columns hidden with _nocol and rowid for tables without a primary key, so you can switch columns without reopening the dialog. The pills follow the ARIA tabs pattern: a single Tab stop, arrow keys and Home/End to move, Enter or Space to switch. The filtered/all-rows choice carries over when switching.

SQL query pages

Arbitrary SQL query pages (/<database>/-/query?sql=) and read-only stored queries get the dialog too. Each result column gets a cog menu with a single "Column statistics" item, and at narrow widths, where headers are hidden, a "Column statistics" button appears above the results. These come from the small query-column-actions.js, so column-stats.js itself is still only loaded when the dialog first opens.

• The statistics wrap the query with an explicit column list, with query_rows("id", "score", "id_2") as (<query>) select .... This works for queries with their own with clause, order by, limit or a trailing comment.
• Duplicate result column names (such as two id columns from a join) are made unique on the server.
• Query parameters are inlined as string literals, which is how the query page itself binds them.
• Statistics cover every row the query returns, not just the max_returned_rows shown on the page, with a note when the two differ.
• Most common values link to the SQL editor with the query narrowed to that value.
• Pills show the SQLite storage type seen in the returned rows (mixed if more than one), since query results have no declared types.
• Not offered for queries using magic parameters (their values can't be passed back to /-/query.json), write queries, non-SELECT statements, or queries that returned no rows.

The data comes from a new query_page_data template context variable, exposed to JavaScript as window._datasetteQueryData.

Tests

• tests/test_table_html.py:
  • executeSql is gated on allow_sql
  • the filter data (pagination excluded, typed parameters, description)
  • the column list
• tests/test_playwright.py:
  • exact numeric statistics and percentiles, and text column statistics
  • query error display
  • percentile SQL checked against Python's statistics.quantiles
  • filtered statistics and the toggle
  • typed parameters on an untyped view column
  • parameter inlining edge cases (quoted identifiers, string literals, comments)
  • switching columns with the mouse and keyboard

The full Playwright suite passes in Chromium, and the column statistics tests pass in Firefox and WebKit. In the full pytest suite, a few tests/test_cli_serve_server.py startup tests failed under -n auto (a different subset each run). All six pass when that file is run on its own.

Known limits

• Integers larger than 2^53 used in a filter lose precision on the way through JavaScript.
• Percentiles use a window function sort, which took about 0.8s on 1M rows in local testing. On larger tables that section may hit sql_time_limit_ms.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 0% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (0a643b2) to head (06fa5be).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
datasette/views/database.py	0.00%	20 Missing ⚠️
datasette/views/table.py	0.00%	17 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #3038   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         78      78           
  Lines      13483   13520   +37     
=====================================
- Misses     13483   13520   +37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[simonw]

Make it so column-stats.js isn't loaded until the first time the user selects the Column statistics option to open the modal, similar to how the Export data button works (look at recent commits)