From 20cf3bcc2ba79a5f72ad20f074a6232b3d06ac5b Mon Sep 17 00:00:00 2001 From: James Hush Date: Thu, 13 Aug 2026 15:37:30 +0800 Subject: [PATCH 1/4] docs(pipecat-cloud): document that min-agents changes apply asynchronously (T-3351) Raising min-agents returns a success response as soon as the change is accepted. It does not mean the new reserved instances exist or are ready. Customers who raise min-agents and then dial a second later race the scale-up and get cold starts. Adds a "Changing min-agents at Runtime" section to the capacity planning guide with the poll-before-traffic recipe (reconciledDeploymentId == desiredDeploymentId, ready, readyReplicas minus activeSessionCount), plus a warning on the scaling page and a note on the CLI --min-agents param. --- api-reference/cli/cloud/deploy.mdx | 5 +++- pipecat-cloud/fundamentals/scaling.mdx | 9 ++++++ pipecat-cloud/guides/capacity-planning.mdx | 35 +++++++++++++++++++++- 3 files changed, 47 insertions(+), 2 deletions(-) diff --git a/api-reference/cli/cloud/deploy.mdx b/api-reference/cli/cloud/deploy.mdx index 01dc1dab..52d21992 100644 --- a/api-reference/cli/cloud/deploy.mdx +++ b/api-reference/cli/cloud/deploy.mdx @@ -61,7 +61,10 @@ pipecat cloud deploy [ARGS] [OPTIONS] Minimum number of agent instances to keep warm at all times. Default is 0, which means the agent will scale down to zero when not in use. Setting this to - 1 or higher avoids cold starts. + 1 or higher avoids cold starts. Raising this value is applied + asynchronously, so new warm instances take roughly 30 seconds to become + available. See [Changing min-agents at + Runtime](/pipecat-cloud/guides/capacity-planning#changing-min-agents-at-runtime). diff --git a/pipecat-cloud/fundamentals/scaling.mdx b/pipecat-cloud/fundamentals/scaling.mdx index cdcf9598..eecc6a83 100644 --- a/pipecat-cloud/fundamentals/scaling.mdx +++ b/pipecat-cloud/fundamentals/scaling.mdx @@ -241,6 +241,15 @@ pipecat cloud deploy [agent-name] [image] --min-agents 1 --max-agents 5 Please note that changing your scaling parameters will not disrupt any active sessions. If you reduce your max instance count below the number of currently active sessions, you will still be billed for the duration of those sessions. + + Scaling changes are applied asynchronously. A successful response means the + change was accepted, not that new warm instances are ready. If you raise + `min-agents` and start sessions seconds later, those sessions can still hit a + [cold start](#cold-starts). See [Changing min-agents at + Runtime](../guides/capacity-planning#changing-min-agents-at-runtime) for how + to check that the new capacity is ready before you send traffic. + + ## Capacity Planning Effective capacity planning is crucial for production deployments to ensure your agents respond immediately. diff --git a/pipecat-cloud/guides/capacity-planning.mdx b/pipecat-cloud/guides/capacity-planning.mdx index cd78d5a0..b5fb3578 100644 --- a/pipecat-cloud/guides/capacity-planning.mdx +++ b/pipecat-cloud/guides/capacity-planning.mdx @@ -71,11 +71,44 @@ To determine the optimal reserved instance count for your deployment, consider: apps, consider a waiting UX or message. +## Changing min-agents at Runtime + +You can raise or lower your reserved agent count at any time, with the CLI, the dashboard, or the REST API. The change does not take effect right away. + +A `min-agents` change is applied **asynchronously**. A successful response means Pipecat Cloud accepted the change, not that the new reserved instances exist or are ready to take a session. New instances still need roughly 30 seconds to become available, and that can take longer for a large image or an agent that does heavy setup work. + +This is a common cause of **unexpected cold starts right after raising `min-agents`**. If you raise the reserved count and then start sessions a second or two later, those sessions race the scale-up. They land on instances that are still starting, so you see long start times even though your configuration looks correct. + +### Wait for the New Instances Before Sending Traffic + +Rather than waiting a fixed number of seconds, poll the agent and check its real state. `GET /v1/agents/{agentName}` returns everything you need: + +```bash +curl https://api.pipecat.daily.co/v1/agents/my-agent \ + -H "Authorization: Bearer YOUR_PRIVATE_API_KEY" +``` + +Wait until all three of these are true: + +1. `reconciledDeploymentId` equals `desiredDeploymentId`. Your change has been picked up. +2. `ready` is `true`. The agent is in a good state. +3. `currentRevision.readyReplicas` minus `activeSessionCount` covers the number of sessions you are about to start. There is enough free warm capacity for your traffic. + +Poll every few seconds until those conditions hold, then start your sessions. + + + For a known busy period, raise `min-agents` **minutes** before your traffic + starts, not seconds. Polling protects you from a race, but giving the pool + time to fill is simpler and safer. See + [Cold-starts](../fundamentals/scaling#cold-starts) for what makes a scale-up + slower. + + ## Cost-Efficient Scaling Strategies - **Development/Testing**: Use `min-agents: 0` to minimize costs during development - **Production Voice AI**: Set `min-agents` to cover your baseline traffic to avoid cold starts -- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods +- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods. Make the change ahead of the traffic and confirm the new capacity is ready, as described in [Changing min-agents at Runtime](#changing-min-agents-at-runtime) - **Cap session duration**: Use [`--max-session-duration`](/api-reference/cli/cloud/deploy#param-max-session-duration) as a safety net against runaway sessions. If your application's calls are meant to be short, setting a tight cap (e.g. 120s for a 60s use case) prevents a buggy bot from racking up hours of charges. - **Monitoring**: Regularly review your warm capacity utilization to optimize your configuration From f088fbaa2b5e9470d8651436f9ed6500fd23f1ba Mon Sep 17 00:00:00 2001 From: James Hush Date: Thu, 13 Aug 2026 15:46:49 +0800 Subject: [PATCH 2/4] docs(pipecat-cloud): trim min-agents runtime note to a single tip Reverts the scaling.mdx and deploy.mdx changes. Keeps only a short tip on the capacity planning page: raise min-agents minutes ahead of traffic, and note that the change applies asynchronously. Refs T-3351 --- api-reference/cli/cloud/deploy.mdx | 5 +--- pipecat-cloud/fundamentals/scaling.mdx | 9 ------ pipecat-cloud/guides/capacity-planning.mdx | 35 ++++------------------ 3 files changed, 6 insertions(+), 43 deletions(-) diff --git a/api-reference/cli/cloud/deploy.mdx b/api-reference/cli/cloud/deploy.mdx index 52d21992..01dc1dab 100644 --- a/api-reference/cli/cloud/deploy.mdx +++ b/api-reference/cli/cloud/deploy.mdx @@ -61,10 +61,7 @@ pipecat cloud deploy [ARGS] [OPTIONS] Minimum number of agent instances to keep warm at all times. Default is 0, which means the agent will scale down to zero when not in use. Setting this to - 1 or higher avoids cold starts. Raising this value is applied - asynchronously, so new warm instances take roughly 30 seconds to become - available. See [Changing min-agents at - Runtime](/pipecat-cloud/guides/capacity-planning#changing-min-agents-at-runtime). + 1 or higher avoids cold starts. diff --git a/pipecat-cloud/fundamentals/scaling.mdx b/pipecat-cloud/fundamentals/scaling.mdx index eecc6a83..cdcf9598 100644 --- a/pipecat-cloud/fundamentals/scaling.mdx +++ b/pipecat-cloud/fundamentals/scaling.mdx @@ -241,15 +241,6 @@ pipecat cloud deploy [agent-name] [image] --min-agents 1 --max-agents 5 Please note that changing your scaling parameters will not disrupt any active sessions. If you reduce your max instance count below the number of currently active sessions, you will still be billed for the duration of those sessions. - - Scaling changes are applied asynchronously. A successful response means the - change was accepted, not that new warm instances are ready. If you raise - `min-agents` and start sessions seconds later, those sessions can still hit a - [cold start](#cold-starts). See [Changing min-agents at - Runtime](../guides/capacity-planning#changing-min-agents-at-runtime) for how - to check that the new capacity is ready before you send traffic. - - ## Capacity Planning Effective capacity planning is crucial for production deployments to ensure your agents respond immediately. diff --git a/pipecat-cloud/guides/capacity-planning.mdx b/pipecat-cloud/guides/capacity-planning.mdx index b5fb3578..b212e07c 100644 --- a/pipecat-cloud/guides/capacity-planning.mdx +++ b/pipecat-cloud/guides/capacity-planning.mdx @@ -71,44 +71,19 @@ To determine the optimal reserved instance count for your deployment, consider: apps, consider a waiting UX or message. -## Changing min-agents at Runtime - -You can raise or lower your reserved agent count at any time, with the CLI, the dashboard, or the REST API. The change does not take effect right away. - -A `min-agents` change is applied **asynchronously**. A successful response means Pipecat Cloud accepted the change, not that the new reserved instances exist or are ready to take a session. New instances still need roughly 30 seconds to become available, and that can take longer for a large image or an agent that does heavy setup work. - -This is a common cause of **unexpected cold starts right after raising `min-agents`**. If you raise the reserved count and then start sessions a second or two later, those sessions race the scale-up. They land on instances that are still starting, so you see long start times even though your configuration looks correct. - -### Wait for the New Instances Before Sending Traffic - -Rather than waiting a fixed number of seconds, poll the agent and check its real state. `GET /v1/agents/{agentName}` returns everything you need: - -```bash -curl https://api.pipecat.daily.co/v1/agents/my-agent \ - -H "Authorization: Bearer YOUR_PRIVATE_API_KEY" -``` - -Wait until all three of these are true: - -1. `reconciledDeploymentId` equals `desiredDeploymentId`. Your change has been picked up. -2. `ready` is `true`. The agent is in a good state. -3. `currentRevision.readyReplicas` minus `activeSessionCount` covers the number of sessions you are about to start. There is enough free warm capacity for your traffic. - -Poll every few seconds until those conditions hold, then start your sessions. - For a known busy period, raise `min-agents` **minutes** before your traffic - starts, not seconds. Polling protects you from a race, but giving the pool - time to fill is simpler and safer. See - [Cold-starts](../fundamentals/scaling#cold-starts) for what makes a scale-up - slower. + starts, not seconds. A `min-agents` change is applied asynchronously, so new + instances still need time to become available after the change is accepted. + See [Cold-starts](../fundamentals/scaling#cold-starts) for what makes a + scale-up slower. ## Cost-Efficient Scaling Strategies - **Development/Testing**: Use `min-agents: 0` to minimize costs during development - **Production Voice AI**: Set `min-agents` to cover your baseline traffic to avoid cold starts -- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods. Make the change ahead of the traffic and confirm the new capacity is ready, as described in [Changing min-agents at Runtime](#changing-min-agents-at-runtime) +- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods. Make the change ahead of the traffic, not as it starts - **Cap session duration**: Use [`--max-session-duration`](/api-reference/cli/cloud/deploy#param-max-session-duration) as a safety net against runaway sessions. If your application's calls are meant to be short, setting a tight cap (e.g. 120s for a 60s use case) prevents a buggy bot from racking up hours of charges. - **Monitoring**: Regularly review your warm capacity utilization to optimize your configuration From 6fdf28f3f41f5d51b53fee2ad9d62ace13d8cbc5 Mon Sep 17 00:00:00 2001 From: James Hush Date: Thu, 13 Aug 2026 16:26:00 +0800 Subject: [PATCH 3/4] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- pipecat-cloud/guides/capacity-planning.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipecat-cloud/guides/capacity-planning.mdx b/pipecat-cloud/guides/capacity-planning.mdx index b212e07c..b14f1adb 100644 --- a/pipecat-cloud/guides/capacity-planning.mdx +++ b/pipecat-cloud/guides/capacity-planning.mdx @@ -83,7 +83,7 @@ To determine the optimal reserved instance count for your deployment, consider: - **Development/Testing**: Use `min-agents: 0` to minimize costs during development - **Production Voice AI**: Set `min-agents` to cover your baseline traffic to avoid cold starts -- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods. Make the change ahead of the traffic, not as it starts +- **Time-Based Scaling**: Consider modifying your reserved count for known high-traffic periods. Make the change ahead of the traffic, not as it starts. - **Cap session duration**: Use [`--max-session-duration`](/api-reference/cli/cloud/deploy#param-max-session-duration) as a safety net against runaway sessions. If your application's calls are meant to be short, setting a tight cap (e.g. 120s for a 60s use case) prevents a buggy bot from racking up hours of charges. - **Monitoring**: Regularly review your warm capacity utilization to optimize your configuration From 8db5ab1551b981490f7a416d4cc4f66b2b66ce0e Mon Sep 17 00:00:00 2001 From: James Hush Date: Thu, 13 Aug 2026 16:28:00 +0800 Subject: [PATCH 4/4] Update min-agents recommendation timing in documentation --- pipecat-cloud/guides/capacity-planning.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipecat-cloud/guides/capacity-planning.mdx b/pipecat-cloud/guides/capacity-planning.mdx index b14f1adb..bbe50e03 100644 --- a/pipecat-cloud/guides/capacity-planning.mdx +++ b/pipecat-cloud/guides/capacity-planning.mdx @@ -72,7 +72,7 @@ To determine the optimal reserved instance count for your deployment, consider: - For a known busy period, raise `min-agents` **minutes** before your traffic + For a known busy period, raise `min-agents` **3-5 minutes** before your traffic starts, not seconds. A `min-agents` change is applied asynchronously, so new instances still need time to become available after the change is accepted. See [Cold-starts](../fundamentals/scaling#cold-starts) for what makes a