fix(db): keep the DSN's application_name across pool reuse (blank pg_stat_activity behind pgbouncer) - #3491
Merged
Merged
Conversation
A DSN like postgresql://...?application_name=my-worker labels the session correctly under psql and on the first connection, then reports an empty application_name in pg_stat_activity for the rest of the service's life once a connection pooler sits in front of PostgreSQL. asyncpg does forward the DSN's application_name in the startup packet (it passes unrecognized DSN query parameters through as server_settings), so a direct connection is attributed correctly. The pool, however, runs RESET ALL on release. Direct to PostgreSQL that is harmless - RESET ALL restores the startup-packet value. Behind pgbouncer the server connection's startup packet is the pooler's own, with no application_name; pgbouncer applies the client's name with a SET when it links client to server, so RESET ALL resets it to empty and pgbouncer - which believes the value is already applied - never re-issues it. Only the first acquire on each server connection is attributed. Re-assert the name from the pool's setup hook, which asyncpg runs on every acquire after RESET ALL. This is the same mechanism the backend already relies on to keep hnsw.ef_search and the other session GUCs applied across connection reuse. set_config() rather than SET because the name is operator-supplied and SET does not accept bind parameters. Verified against pgbouncer 1.25.1 -> PostgreSQL 16: before, acquires 2..n report ''; after, every acquire reports the configured name, including under concurrent load across multiple server connections.
nicoloboschi
force-pushed
the
fix/dsn-application-name
branch
from
August 14, 2026 13:06
3b9a786 to
dc1c9af
Compare
nicoloboschi
added a commit
that referenced
this pull request
Aug 14, 2026
The pool wires its init callback as `setup=` as well as `init=`, so the session GUCs are re-applied on every acquire. That is required for a plain asyncpg pool — releasing a connection runs RESET ALL, which wipes them — but it is pure waste for deployments that pin the same settings on the role or database, since RESET ALL restores them to exactly the values we would resend. Behind a transaction-mode pooler that wasted round trip is also its own server-side transaction, which is the cost #3499 measured. HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=false drops the per-acquire hook and keeps the open-time one, so a connection is still configured when it is created. application_name is deliberately outside the trade-off: pgbouncer never re-issues it after RESET ALL (#3491), so it keeps its per-acquire hook either way. On the vchord text-search backend the set includes search_path (bm25_catalog, tokenizer_catalog), where losing the value fails recall outright rather than degrading it — called out in the docs and the env template so operators pin it before turning the flag off. Default is true — unchanged behaviour. Refs #3499
nicoloboschi
added a commit
that referenced
this pull request
Aug 14, 2026
…session setup (#3501) * perf(worker): one pooled connection per poll cycle, one statement of session setup The worker's claim fabric acquired a pooled connection *per active schema*. Every acquire runs the pool's setup callback (the session GUCs, since asyncpg wipes them with RESET ALL on release) and every release runs RESET ALL / UNLISTEN * / CLOSE ALL / pg_advisory_unlock_all. Behind a transaction-mode pooler each of those statements is its own server-side transaction, so the ceremony multiplied by the number of flagged schemas: ~12 statements per schema-visit for 2 useful queries, ~463 acquire/release cycles/s at 12 workers x ~22 schemas, and a commit rate an order of magnitude above the useful work. Two changes, either of which removes most of the cost: 1. claim_batch acquires once for the whole cycle and runs the active-schema scan plus every per-schema claim on that connection. Each schema's claim still opens its own transaction, so FOR UPDATE SKIP LOCKED semantics and lock hold times are unchanged. The progress logger's scan moves inside the connection it already held, for the same reason. 2. The pool's session setup issues one SELECT set_config(...) instead of N separate SETs. Extension GUCs (hnsw.ef_search, pg_trgm.similarity_threshold) may not exist on the cluster and would fail the batched statement as a whole, so it falls back to applying them one at a time, skipping only the ones the server rejects — the same tolerance the per-SET try/except had. Fixes #3499 * feat(db): flag to skip the per-acquire session setup The pool wires its init callback as `setup=` as well as `init=`, so the session GUCs are re-applied on every acquire. That is required for a plain asyncpg pool — releasing a connection runs RESET ALL, which wipes them — but it is pure waste for deployments that pin the same settings on the role or database, since RESET ALL restores them to exactly the values we would resend. Behind a transaction-mode pooler that wasted round trip is also its own server-side transaction, which is the cost #3499 measured. HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=false drops the per-acquire hook and keeps the open-time one, so a connection is still configured when it is created. application_name is deliberately outside the trade-off: pgbouncer never re-issues it after RESET ALL (#3491), so it keeps its per-acquire hook either way. On the vchord text-search backend the set includes search_path (bm25_catalog, tokenizer_catalog), where losing the value fails recall outright rather than degrading it — called out in the docs and the env template so operators pin it before turning the flag off. Default is true — unchanged behaviour. Refs #3499
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A DSN like
labels the session correctly under
psql, and correctly on the service's first connection — then reports an emptyapplication_nameinpg_stat_activityfor the rest of the process's life once a connection pooler sits in front of PostgreSQL. Everything keyed on the session name loses attribution:pg_stat_activity, operator dashboards, the[DB_WAITS]diagnostics inworker/poller.pythat printapp=, and any SQL readingcurrent_setting('application_name'). No error — just blank names, and only in the deployed topology.Root cause
Not a DSN-parsing gap: asyncpg does forward the DSN's
application_namein the startup packet (it passes unrecognized DSN query parameters through asserver_settings). The loss happens on connection reuse.asyncpg's pool runs
RESET ALLwhen a connection is released (Connection.get_reset_query). Direct to PostgreSQL that is harmless —RESET ALLrestores the value from the startup packet, which carried the name. Behind pgbouncer it is not: the server connection's startup packet is the pooler's own and has noapplication_name; pgbouncer applies the client's value with aSETwhen it links client to server.RESET ALLtherefore clears it to empty, and pgbouncer — which already believes the value is applied — never re-issues it. Only the first acquire on each server connection is attributed.Fix
Re-assert the name from the pool's
setuphook, which asyncpg runs on every acquire, after the release-timeRESET ALL. This is the same mechanism the backend already relies on to keephnsw.ef_searchand the other session GUCs applied across connection reuse (theinit=/setup=pair right below it).set_config()rather thanSETbecause the name is operator-supplied andSETdoes not accept bind parameters.DSNs without the parameter are untouched — the caller's
init_callbackis passed through unwrapped, so there is no extra statement per acquire.Verification
Against pgbouncer 1.25.1 → PostgreSQL 16, reading the real backend's name out of
pg_stat_activity:hs-worker-7''''hs-worker-7hs-worker-7hs-worker-7Also verified: unchanged on a direct connection; unchanged when the DSN carries no
application_name; the caller'sinit_callbackstill runs on every acquire; and under session pooling with 5 concurrent clients × 6 acquires across 6 server connections, 0/30 mis-attributed — with a name containing a single quote, confirming the bind parameter is injection-safe.Note on transaction pooling: session-scoped state is inherently not guaranteed to reach the backend serving the next statement, so residual mis-attribution is possible there. That applies equally to the session GUCs this backend already sets (
hnsw.ef_search,statement_timeout,pg_trgm.similarity_threshold) and is out of scope here.Tests
tests/test_dsn_application_name.py— deterministic, no DB (asyncpg.create_poolmonkeypatched to capture the hooks):set_config('application_name', $1, false)and re-applies it on a second acquireinit_callback(session GUCs must not be displaced)application_namein the DSN → the callback is passed through unwrapped; no callback either → hooks stayNone