Skip to content

feat: serve Reports from an engine service over packs - #44

Merged
devchaudhary24k merged 25 commits into
mainfrom
architecture/engine-service
Oct 26, 2026
Merged

devchaudhary24k merged 25 commits into
mainfrom
architecture/engine-service

Conversation

@devchaudhary24k

Copy link
Copy Markdown
Member

Rebuilds how a repository's Report is built and served.

What changes

  • Engine: the line pass counts the Mainline first, newest first, within a time budget; quiet Builds count the rest. Bulk Commits, lockfiles and generated files are not counted. commitscape build writes the Postgres export, the pack and the Report's meta from one reading of the history.
  • Engine service (apps/engine, new): Rust (axum, tokio) answering every question the Site used to work out in Node, from immutable packs in R2, kept on disk and in memory. Bearer token, golden tests against the old answers, its own Docker image.
  • Builder: one walk, one load, pack upload, GraphQL login lookups, a rebuild command; Sentry traces continue from the Site's request, PostHog events under a hashed id.
  • Site: answers from the engine service (the in-memory view and the TypeScript copy of the metrics are gone), sign-in only (Cards, Shared Reports, shared Proof of Work and /mcp stay public), Leaderboards, Races, Crews, Seasons and Rivals removed, Sentry no longer sends cookies, headers or IPs, requests named by route, PostHog through /ingest.
  • Database: migrations 0029–0033 drop the daily sums and the crowd tables and add the pack, lines left and Report meta.
  • Landing page (apps/landing, new): static, for Cloudflare Pages.
  • Docs: README with architecture diagrams and eight translations, ADR-0028, CONTEXT, DEPLOY.
  • Domain: defaults move to app.commitscape.com.

Measured on react (35k commits)

Before After
Default branch's lines, one core 61 s (every branch) 5.7 s
Cold Build first Report 12.6 s, done 17.9 s
Universe 14.2 s 98 ms
Memory answering ~300 MB (Node) 27–67 MB (engine service)

Checks

cargo fmt, clippy, layering and 291 Rust tests; every package's typecheck, lint, build and tests; pnpm e2e 61/61.

Deploying

Deploy all three images together on an empty database and bucket, then run node builder.mjs rebuild if anything was built before the engine service existed. DEPLOY.md has every setting.

Comment thread crates/commitscape/src/build.rs Dismissed
Comment thread crates/commitscape/src/build.rs Dismissed
One config file, apps/site/src/lib/analytics-config.ts, lists every kind
of usage data and, per region of countries, what runs before the visitor
chooses, at Only basic and at Allow all. The EU, the EEA, the UK,
Switzerland and India start at basic; everywhere else starts at all.

- The browser runs PostHog cookieless until the visitor allows all, then
  adds autocapture, rage and dead clicks, heatmaps, session replay with
  console, network timings and canvas, and identifies the person.
- A banner in the corner, Settings and the privacy page set the choice.
- /ingest forwards the part of the address the level allows, for GeoIP.
- The server's events follow the same level and join the browser's
  session; Cards, Shared Reports and App installs are counted too.
- Delete my data also deletes the person in PostHog.
- The privacy page is drawn from the config.
share, health, surviving and report each send one count to the Site's
/api/telemetry: the command, how long it took, how it ended, the version
and the platform, under a random install id kept in the cache folder.
The first run says so; COMMITSCAPE_TELEMETRY=0 or DO_NOT_TRACK=1 turns it
off. The Site checks every field, limits each address, and passes it to
PostHog at the region's level. Tests run the binary with it off.
@devchaudhary24k
devchaudhary24k merged commit 4e11f5d into main Oct 26, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants