From e9fc37e06d1b2b013f0505ffea5a52d31ebf0457 Mon Sep 17 00:00:00 2001 From: Aaron Sachs <898627+asachs01@users.noreply.github.com> Date: Mon, 17 Aug 2026 12:12:28 -0400 Subject: [PATCH] fix(publish): republish the site when an entry's public state changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A listing page is a static artefact: its `base.loop` over a content table is expanded once, at full-publish time, and baked into the slot. Per-entry publishing rewrites that entry's own artefact and nothing else, so the moment an entry enters or leaves public visibility every index that links to it is wrong, and stays wrong until a human presses Publish. A deleted post keeps a live card pointing at a 404; a post scheduled for 09:00 is missing from the index until someone notices. Publish, scheduled publish, unpublish, and delete now ask `server/publish/autoSitePublish.ts` for a background full-site republish, which is the only thing that re-expands a loop. Four rules make that affordable and safe: - Coalesced: the first request opens a 5s batch window and every request inside it is absorbed, so a backlog of forty posts costs one publish. The window is half a `publishScheduler` tick, so one tick's due rows land in a single batch. - Never re-entrant: at most one run is in flight, because two would race the slot swap. Requests raised during a run collapse into exactly one follow-up window. - Never recursive: guaranteed structurally, with an architecture test that fails the build if a new caller appears. A runtime origin check would lie — plugin publish.* handlers run in the QuickJS worker and their RPCs return on their own event-loop task. - Never a surprise publish: `publishDraftSite` promotes the draft, so a run proceeds only while the draft already matches what is published. It then changes no page and re-expands the loops and nothing else. With unpublished site edits present the run is skipped and logged. The rebuild is background work, so an author's request returns as soon as their entry commits. A failed run is logged and dropped: the entry publish already committed and the bake reaches `swapSlot` only on success, so the live site is untouched. Attribution is the system actor, the convention the scheduled-publish tick already uses — nobody asked for this site publish, which is also why `publishDraftSite` now takes `string | null`. `AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE=0` (also false/off/no) restores the old behaviour for operators who publish on their own cadence. Default is on: the stale listing is a correctness bug, not a preference. --- .env.example | 8 + .env.production.example | 8 + CHANGELOG.md | 4 + compose.prod.yml | 2 + docs/deployment/README.md | 3 +- docs/features/content-storage.md | 2 + docs/features/publisher.md | 42 +++ docs/reference/architecture-tests.md | 1 + docs/server.md | 1 + server/handlers/cms/data/rows.ts | 14 +- server/publish/autoSitePublish.ts | 249 +++++++++++++++++ server/publish/publishScheduler.ts | 12 + server/publish/publishSite.ts | 9 +- server/repositories/publish.ts | 3 +- .../auto-site-publish-callers.test.ts | 93 +++++++ src/__tests__/server/autoSitePublish.test.ts | 256 ++++++++++++++++++ 16 files changed, 702 insertions(+), 5 deletions(-) create mode 100644 server/publish/autoSitePublish.ts create mode 100644 src/__tests__/architecture/auto-site-publish-callers.test.ts create mode 100644 src/__tests__/server/autoSitePublish.test.ts diff --git a/.env.example b/.env.example index 33fe0d113..3ed6e1414 100644 --- a/.env.example +++ b/.env.example @@ -27,6 +27,14 @@ PORT=3001 UPLOADS_DIR=./uploads STATIC_DIR=./dist +# ─── Publishing ────────────────────────────────────────────────────────────── +# Publishing, unpublishing, or deleting a content entry triggers a coalesced +# background site republish, so baked listing pages (a /blog index and the like) +# stop showing the set from before the change. On by default. Set to +# 0 / false / off / no if you publish the site on your own cadence. +# +# AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE=0 + # ─── AI credential encryption ─────────────────────────────────────────────── # Local dev auto-creates .tmp/secret.key. Production deployments must set # INSTATIC_SECRET_KEY to the output of: diff --git a/.env.production.example b/.env.production.example index b9254dbb1..e64062e6f 100644 --- a/.env.production.example +++ b/.env.production.example @@ -38,6 +38,14 @@ HOST_PORT=3001 # networks or a non-Docker reverse proxy. TRUSTED_PROXY_CIDRS= +# ─── Publishing ────────────────────────────────────────────────────────────── +# Publishing, unpublishing, or deleting a content entry triggers a coalesced +# background site republish, so baked listing pages (a /blog index and the like) +# stop showing the set from before the change. On by default — leave this empty +# unless you publish the site on your own cadence, in which case set it to +# 0 / false / off / no. +AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE= + # ─── Database — Postgres mode only ─────────────────────────────────────────── # These three are consumed by the postgres service and embedded in DATABASE_URL. # REQUIRED for Postgres deployments — set POSTGRES_PASSWORD to a real secret. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0bfb7858b..d137fc02d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ This project is pre-1.0. Breaking changes may appear in minor or patch releases ## Unreleased +### Content and publishing + +- Kept published listing pages in step with their entries. A listing expands its loop at full-publish time and bakes the result, so publishing, scheduling, unpublishing, or deleting an entry used to leave every index that links to it showing the previous set — a deleted post kept a live card pointing at a 404, and a scheduled post stayed off the index until someone published by hand. Those four paths now trigger a background site republish, coalesced so a batch of entries costs one publish, and skipped while the site draft has unpublished edits so an entry publish never pushes unfinished design work live. Set `AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE=0` to publish on your own cadence instead. + ## 0.0.16 - 2026-08-11 ### Media and integrations diff --git a/compose.prod.yml b/compose.prod.yml index 17154ab0d..c8bf8ffbc 100644 --- a/compose.prod.yml +++ b/compose.prod.yml @@ -31,6 +31,8 @@ services: STATIC_DIR: /app/dist INSTATIC_SECRET_KEY: ${INSTATIC_SECRET_KEY:-} TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-} + # Empty means unset, which is the default: on. + AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE: ${AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE:-} volumes: - uploads:/app/uploads depends_on: diff --git a/docs/deployment/README.md b/docs/deployment/README.md index 7a50504e7..4009e1b1f 100644 --- a/docs/deployment/README.md +++ b/docs/deployment/README.md @@ -2,7 +2,7 @@ This index maps supported deployment targets to the files, variables, and persistence rules they need. -Instatic is one Bun server packaged by the root `Dockerfile`. The server reads runtime configuration from `server/config.ts`: `PORT`, `DATABASE_URL`, `UPLOADS_DIR`, `STATIC_DIR`, `PUBLIC_ORIGIN`, and `TRUSTED_PROXY_CIDRS`. Reversible server secrets, including AI provider credentials, plugin secret settings, and MFA TOTP seeds, are encrypted with `INSTATIC_SECRET_KEY` when configured. Database migrations run automatically on boot in `server/index.ts`. +Instatic is one Bun server packaged by the root `Dockerfile`. The server reads runtime configuration from `server/config.ts`: `PORT`, `DATABASE_URL`, `UPLOADS_DIR`, `STATIC_DIR`, `PUBLIC_ORIGIN`, and `TRUSTED_PROXY_CIDRS`. Single-feature switches are read by the module that owns the feature rather than by `server/config.ts` — `AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE` in `server/publish/autoSitePublish.ts` is one; the Runtime Contract below lists every variable an operator sets, wherever it is read. Reversible server secrets, including AI provider credentials, plugin secret settings, and MFA TOTP seeds, are encrypted with `INSTATIC_SECRET_KEY` when configured. Database migrations run automatically on boot in `server/index.ts`. --- @@ -32,6 +32,7 @@ STATIC_DIR built admin SPA directory; /app/dist in the Docker image INSTATIC_SECRET_KEY base64 32-byte key for encrypted server secrets PUBLIC_ORIGIN comma-separated public origin(s) the CSRF check trusts; auto-detected from RENDER_EXTERNAL_URL / RAILWAY_PUBLIC_DOMAIN on those platforms TRUSTED_PROXY_CIDRS optional; trusts proxy socket peers for forwarded client-IP attribution only (audit logs, rate-limit keys) — NOT used for CSRF +AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE optional; default on. Set 0/false/off/no to stop publishing, unpublishing, or deleting a content entry from triggering a background site republish. Leave it on unless you publish on your own cadence — the republish is what keeps baked listing pages in step with their entries ``` Generate `INSTATIC_SECRET_KEY` with `bun run scripts/generate-secret-key.ts` before adding Anthropic, OpenAI, or OpenRouter credentials or enabling TOTP MFA in production. Without it, the admin can load but saving reversible secrets fails because there is no stable encryption key. diff --git a/docs/features/content-storage.md b/docs/features/content-storage.md index 6318a22f6..f08497d63 100644 --- a/docs/features/content-storage.md +++ b/docs/features/content-storage.md @@ -243,6 +243,8 @@ For **post-types**, public row routes require an explicitly authored entry templ `status: 'scheduled'` with `scheduled_publish_at: `. The publisher's scheduler tick (`server/publish/publishScheduler.ts`) polls for rows where `scheduled_publish_at <= now()`, fires `publishDataRow(...)`, and flips the row to `published`. On failure, the row drops back to `draft`. +A row publish writes that row's own artefact and nothing else, so any baked listing page that loops over its table would still show the pre-publish set. Every path that changes an entry's public visibility — this tick, and the publish / unpublish / delete routes — therefore asks `server/publish/autoSitePublish.ts` for a coalesced background site republish. See [docs/features/publisher.md](publisher.md) → "Keeping listings honest". + --- ## Cookbook diff --git a/docs/features/publisher.md b/docs/features/publisher.md index 7a729b533..0d4d51b99 100644 --- a/docs/features/publisher.md +++ b/docs/features/publisher.md @@ -58,6 +58,7 @@ server/publish/ ├── mediaPrefetch.ts, loopPrefetch.ts — pre-warm caches needed by the renderer ├── republish.ts — bulk re-publish on site-level changes ├── publishScheduler.ts — scheduled publish jobs +├── autoSitePublish.ts — coalesced site republish after an entry's public state changes ├── runtime/ — per-site bun install workspace serving └── loopRuntime.ts — loop runtime asset ``` @@ -319,6 +320,46 @@ The exclusive namespaces `/_instatic/css/*` (`serveSiteCss`) and `/_instatic/ass publish whose disk write failed. Unknown paths under either prefix 404 rather than falling through. +### Keeping listings honest — automatic republish + +Baking everything to disk has one consequence that has to be handled +explicitly: **a listing is a static artefact too**. A `base.loop` over a content +table is expanded once, at full-publish time, and the resulting cards are baked +into the slot. Per-entry publishing (`publishDataRow`) rewrites that entry's own +artefact and nothing else — it never re-expands anybody else's loop. + +So the moment an entry enters or leaves public visibility, every listing that +links to it is stale, and stays stale until someone presses Publish: a deleted +post keeps a live card pointing at a 404, and a scheduled post that fires at +09:00 is missing from the index until a human notices. + +`server/publish/autoSitePublish.ts` closes that gap. Every path that changes an +entry's public visibility — publish, scheduled publish, unpublish, delete — +calls `requestAutoSitePublish(db, uploadsDir)` next to the `emitContentEntry*` +call it already makes, and the module runs one `publishDraftSite` in the +background. Four rules make that affordable and safe: + +| Rule | How | +|---|---| +| **Coalesced** | The first request opens a 5-second batch window; every request inside it is absorbed. Forty posts going live cost one site publish. The window is half a `publishScheduler` tick, so one tick's worth of due rows lands in a single batch. | +| **Never re-entrant** | At most one run is in flight — two would race the slot swap. Requests raised during a run collapse into exactly one follow-up window (the running publish may have read the database before their entry committed). | +| **Never recursive** | Guaranteed structurally: the publish pipeline never calls the trigger, and `src/__tests__/architecture/auto-site-publish-callers.test.ts` fails the build if a new caller appears. A runtime origin check would lie — plugin `publish.*` handlers run in the QuickJS worker and their RPCs return on their own event-loop task. | +| **Never a surprise publish** | `publishDraftSite` promotes the *draft*. A run therefore only proceeds while the draft already matches what is published, so it changes no page and its only effect is re-expanding the loops. With unpublished site edits present the run is skipped and logged — that operator is about to publish anyway. | + +The rebuild is background work: the author's request returns as soon as their +entry is committed. A failed run is logged under `[publish:auto]` and dropped — +the entry publish already committed, and the bake reaches `swapSlot` only after +it succeeds, so a failure leaves the live site exactly as it was. Attribution is +the system actor (`published_by_user_id = null`), the same convention the +scheduled-publish tick uses: nobody asked for this site publish. + +Operators who publish on their own cadence set +`AUTO_SITE_PUBLISH_ON_ENTRY_CHANGE=0` (also `false` / `off` / `no`) and keep the +old behaviour, where only an explicit Publish rebuilds the site. Default is on. +Note the cost of leaving it on: each automatic run writes a new +`site_snapshots` row and a new `data_row_versions` row per page, exactly as a +manual Publish does. + --- ## `` assembly @@ -377,6 +418,7 @@ Because `serializeCsp` sorts, the same plugins + adapters always emit a **byte-i | `server/publish/moduleJsBundle.ts` | Module-JS channel: `buildSiteModuleJsMap` (fresh), `buildPublishedSiteModuleJsMap` (memoised per publishVersion + site, invalidated by `bumpPublishVersion()`), and `injectModuleScripts` (per-page `