Problem
Uvicorn executes the FastAPI lifespan independently in every worker process.
When SQLBot runs with multiple workers, each worker can submit the same
terminology, data-training, table, and datasource embedding initialization
jobs during application startup.
These jobs load embedding models and perform CPU-intensive calculations.
Duplicate execution can consume significant CPU and memory, slow down startup,
and affect other services on the same host.
Process-local synchronization cannot coordinate different worker processes.
Steps to reproduce
-
Configure SQLBot with PostgreSQL and embedding enabled.
-
Start SQLBot with multiple Uvicorn workers:
uvicorn main:app --host 127.0.0.1 --port 8000 --workers 2
-
Observe that the FastAPI lifespan runs independently in both workers and
both workers can submit startup embedding jobs.
Expected behavior
Only one worker should submit startup embedding initialization jobs during
application startup.
Other workers should continue starting normally without submitting duplicate
jobs.
Proposed approach
Use a PostgreSQL session-level advisory lock so that workers connected to the
same SQLBot database coordinate startup embedding initialization.
PostgreSQL is already required by the current deployment, so this avoids
introducing Redis only for startup coordination.
Known limitation
PostgreSQL advisory locks are local to a database. Workers connected to
different databases or PostgreSQL instances would not share the same lock.
Those deployments would require a shared external coordinator, such as Redis.
Problem
Uvicorn executes the FastAPI lifespan independently in every worker process.
When SQLBot runs with multiple workers, each worker can submit the same
terminology, data-training, table, and datasource embedding initialization
jobs during application startup.
These jobs load embedding models and perform CPU-intensive calculations.
Duplicate execution can consume significant CPU and memory, slow down startup,
and affect other services on the same host.
Process-local synchronization cannot coordinate different worker processes.
Steps to reproduce
Configure SQLBot with PostgreSQL and embedding enabled.
Start SQLBot with multiple Uvicorn workers:
Observe that the FastAPI lifespan runs independently in both workers and
both workers can submit startup embedding jobs.
Expected behavior
Only one worker should submit startup embedding initialization jobs during
application startup.
Other workers should continue starting normally without submitting duplicate
jobs.
Proposed approach
Use a PostgreSQL session-level advisory lock so that workers connected to the
same SQLBot database coordinate startup embedding initialization.
PostgreSQL is already required by the current deployment, so this avoids
introducing Redis only for startup coordination.
Known limitation
PostgreSQL advisory locks are local to a database. Workers connected to
different databases or PostgreSQL instances would not share the same lock.
Those deployments would require a shared external coordinator, such as Redis.