Skip to content

fix(db): keep the DSN's application_name across pool reuse (blank pg_stat_activity behind pgbouncer) - #3491

Merged
nicoloboschi merged 1 commit into
mainfrom
fix/dsn-application-name
Aug 14, 2026
Merged

fix(db): keep the DSN's application_name across pool reuse (blank pg_stat_activity behind pgbouncer)#3491
nicoloboschi merged 1 commit into
mainfrom
fix/dsn-application-name

Conversation

@nicoloboschi

@nicoloboschi nicoloboschi commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

A DSN like

postgresql://user:pass@host:5432/db?application_name=my-worker

labels the session correctly under psql, and correctly on the service's first connection — then reports an empty application_name in pg_stat_activity for the rest of the process's life once a connection pooler sits in front of PostgreSQL. Everything keyed on the session name loses attribution: pg_stat_activity, operator dashboards, the [DB_WAITS] diagnostics in worker/poller.py that print app=, and any SQL reading current_setting('application_name'). No error — just blank names, and only in the deployed topology.

Root cause

Not a DSN-parsing gap: asyncpg does forward the DSN's application_name in the startup packet (it passes unrecognized DSN query parameters through as server_settings). The loss happens on connection reuse.

asyncpg's pool runs RESET ALL when a connection is released (Connection.get_reset_query). Direct to PostgreSQL that is harmless — RESET ALL restores the value from the startup packet, which carried the name. Behind pgbouncer it is not: the server connection's startup packet is the pooler's own and has no application_name; pgbouncer applies the client's value with a SET when it links client to server. RESET ALL therefore clears it to empty, and pgbouncer — which already believes the value is applied — never re-issues it. Only the first acquire on each server connection is attributed.

Fix

Re-assert the name from the pool's setup hook, which asyncpg runs on every acquire, after the release-time RESET ALL. This is the same mechanism the backend already relies on to keep hnsw.ef_search and the other session GUCs applied across connection reuse (the init=/setup= pair right below it). set_config() rather than SET because the name is operator-supplied and SET does not accept bind parameters.

DSNs without the parameter are untouched — the caller's init_callback is passed through unwrapped, so there is no extra statement per acquire.

Verification

Against pgbouncer 1.25.1 → PostgreSQL 16, reading the real backend's name out of pg_stat_activity:

acquire #1 #2 #3
before hs-worker-7 '' ''
after hs-worker-7 hs-worker-7 hs-worker-7

Also verified: unchanged on a direct connection; unchanged when the DSN carries no application_name; the caller's init_callback still runs on every acquire; and under session pooling with 5 concurrent clients × 6 acquires across 6 server connections, 0/30 mis-attributed — with a name containing a single quote, confirming the bind parameter is injection-safe.

Note on transaction pooling: session-scoped state is inherently not guaranteed to reach the backend serving the next statement, so residual mis-attribution is possible there. That applies equally to the session GUCs this backend already sets (hnsw.ef_search, statement_timeout, pg_trgm.similarity_threshold) and is out of scope here.

Tests

tests/test_dsn_application_name.py — deterministic, no DB (asyncpg.create_pool monkeypatched to capture the hooks):

  • parser: present / absent / other-params / repeated parameter (libpq last-wins) / empty value / unparseable DSN
  • the setup hook issues set_config('application_name', $1, false) and re-applies it on a second acquire
  • the wrapper still invokes the caller's init_callback (session GUCs must not be displaced)
  • no application_name in the DSN → the callback is passed through unwrapped; no callback either → hooks stay None

A DSN like postgresql://...?application_name=my-worker labels the session
correctly under psql and on the first connection, then reports an empty
application_name in pg_stat_activity for the rest of the service's life
once a connection pooler sits in front of PostgreSQL.

asyncpg does forward the DSN's application_name in the startup packet (it
passes unrecognized DSN query parameters through as server_settings), so a
direct connection is attributed correctly. The pool, however, runs RESET ALL
on release. Direct to PostgreSQL that is harmless - RESET ALL restores the
startup-packet value. Behind pgbouncer the server connection's startup packet
is the pooler's own, with no application_name; pgbouncer applies the client's
name with a SET when it links client to server, so RESET ALL resets it to
empty and pgbouncer - which believes the value is already applied - never
re-issues it. Only the first acquire on each server connection is attributed.

Re-assert the name from the pool's setup hook, which asyncpg runs on every
acquire after RESET ALL. This is the same mechanism the backend already
relies on to keep hnsw.ef_search and the other session GUCs applied across
connection reuse. set_config() rather than SET because the name is
operator-supplied and SET does not accept bind parameters.

Verified against pgbouncer 1.25.1 -> PostgreSQL 16: before, acquires 2..n
report ''; after, every acquire reports the configured name, including under
concurrent load across multiple server connections.
@nicoloboschi
nicoloboschi force-pushed the fix/dsn-application-name branch from 3b9a786 to dc1c9af Compare August 14, 2026 13:06
@nicoloboschi nicoloboschi changed the title fix(db): honor the DSN's application_name under asyncpg (empty pg_stat_activity attribution) fix(db): keep the DSN's application_name across pool reuse (blank pg_stat_activity behind pgbouncer) Aug 14, 2026
@nicoloboschi
nicoloboschi merged commit 6956dc0 into main Aug 14, 2026
214 of 216 checks passed
@nicoloboschi
nicoloboschi deleted the fix/dsn-application-name branch August 14, 2026 13:35
nicoloboschi added a commit that referenced this pull request Aug 14, 2026
The pool wires its init callback as `setup=` as well as `init=`, so the
session GUCs are re-applied on every acquire. That is required for a plain
asyncpg pool — releasing a connection runs RESET ALL, which wipes them — but
it is pure waste for deployments that pin the same settings on the role or
database, since RESET ALL restores them to exactly the values we would resend.
Behind a transaction-mode pooler that wasted round trip is also its own
server-side transaction, which is the cost #3499 measured.

HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=false drops the per-acquire hook and
keeps the open-time one, so a connection is still configured when it is
created. application_name is deliberately outside the trade-off: pgbouncer
never re-issues it after RESET ALL (#3491), so it keeps its per-acquire hook
either way.

On the vchord text-search backend the set includes search_path
(bm25_catalog, tokenizer_catalog), where losing the value fails recall
outright rather than degrading it — called out in the docs and the env
template so operators pin it before turning the flag off.

Default is true — unchanged behaviour.

Refs #3499
nicoloboschi added a commit that referenced this pull request Aug 14, 2026
…session setup (#3501)

* perf(worker): one pooled connection per poll cycle, one statement of session setup

The worker's claim fabric acquired a pooled connection *per active schema*.
Every acquire runs the pool's setup callback (the session GUCs, since asyncpg
wipes them with RESET ALL on release) and every release runs
RESET ALL / UNLISTEN * / CLOSE ALL / pg_advisory_unlock_all. Behind a
transaction-mode pooler each of those statements is its own server-side
transaction, so the ceremony multiplied by the number of flagged schemas:
~12 statements per schema-visit for 2 useful queries, ~463 acquire/release
cycles/s at 12 workers x ~22 schemas, and a commit rate an order of magnitude
above the useful work.

Two changes, either of which removes most of the cost:

1. claim_batch acquires once for the whole cycle and runs the active-schema
   scan plus every per-schema claim on that connection. Each schema's claim
   still opens its own transaction, so FOR UPDATE SKIP LOCKED semantics and
   lock hold times are unchanged. The progress logger's scan moves inside the
   connection it already held, for the same reason.

2. The pool's session setup issues one SELECT set_config(...) instead of N
   separate SETs. Extension GUCs (hnsw.ef_search, pg_trgm.similarity_threshold)
   may not exist on the cluster and would fail the batched statement as a
   whole, so it falls back to applying them one at a time, skipping only the
   ones the server rejects — the same tolerance the per-SET try/except had.

Fixes #3499

* feat(db): flag to skip the per-acquire session setup

The pool wires its init callback as `setup=` as well as `init=`, so the
session GUCs are re-applied on every acquire. That is required for a plain
asyncpg pool — releasing a connection runs RESET ALL, which wipes them — but
it is pure waste for deployments that pin the same settings on the role or
database, since RESET ALL restores them to exactly the values we would resend.
Behind a transaction-mode pooler that wasted round trip is also its own
server-side transaction, which is the cost #3499 measured.

HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=false drops the per-acquire hook and
keeps the open-time one, so a connection is still configured when it is
created. application_name is deliberately outside the trade-off: pgbouncer
never re-issues it after RESET ALL (#3491), so it keeps its per-acquire hook
either way.

On the vchord text-search backend the set includes search_path
(bm25_catalog, tokenizer_catalog), where losing the value fails recall
outright rather than degrading it — called out in the docs and the env
template so operators pin it before turning the flag off.

Default is true — unchanged behaviour.

Refs #3499
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant