Skip to content

Commit 19018b8

Browse files
test(proxy): bound TCP buffers in flush deadline regression (#139)
## Summary Make `TestHandleHTTP_RealSocketFlushDeadline` reach socket backpressure within its existing five-second budget. This is a test-only change; production proxy behavior is unchanged. On Linux with Go 1.25.6, the original paced 1500-byte stream can remain buffered for the entire test budget. The write-deadline control passes, but the flush test fails before establishing that a flush actually blocked. ## Change In `services/nvpair-proxy/zombie_test.go`, request a 4096-byte proxy send buffer and a 1024-byte stalled-client receive buffer. Keep real sockets, small chunks, pacing, the 300 ms idle deadline, and the existing cancelled-workload assertions. No production files, routing, discovery, cluster trust, HTTP/JSON-RPC contracts, or release versions change. Go's `ResponseController.Flush` already selects `FlushError` before `Unwrap`; removing `Unwrap` is not needed. ## Validation Existing local results for head `66866782304b35278527620b27e23497d8ed9d04`, based on `develop` at `937cf6c96d0268f3d9349cea88914e47b4251d77` (Linux, Go 1.25.6): - The original flush test failed after five seconds while the write control passed. With bounded buffers, the flush test passed three repetitions at approximately 0.36 seconds each. - Both real-socket deadline controls passed 25 repetitions under the race detector. - In an audit copy, disabling only deadline arming in `FlushError` made the revised test fail all five repetitions. The copied production file was restored; no production edit is in this PR. - Full proxy normal and race suites passed with explicit `GOFLAGS=-buildvcs=false`. Three optional browser/external-engine test entries were skipped. - `go vet`, formatting, and whitespace checks passed. <details> <summary>Validation commands and additional audit checks</summary> Run these from `services/nvpair-proxy`: ```sh # Baseline: original flush test fails; write control passes. go test ./... -run '^TestHandleHTTP_RealSocket(Flush|Write)Deadline$' -count=1 -v -timeout=45s # Updated test and focused race coverage. go test ./... -run '^TestHandleHTTP_RealSocketFlushDeadline$' -count=3 -v -timeout=45s go test ./... -race -run '^(TestStatusCapture_|TestHandleHTTP_RealSocket|TestHandleHTTP_UpstreamDiesMidStream)' -count=10 -timeout=180s go test ./... -race -run '^TestHandleHTTP_RealSocket(Flush|Write)Deadline$' -count=25 -timeout=120s # Full suites, with VCS stamping explicitly disabled. GOFLAGS=-buildvcs=false go test ./... -count=1 -timeout=180s GOFLAGS=-buildvcs=false go test ./... -race -count=1 -timeout=180s go vet ./... gofmt -l zombie_test.go git diff --check ``` The baseline command's failure is the original regression witness, not a result from the updated test. Separate local audit diagnostics exercised Ollama and LM Studio profiles five times each. Stalled-client runs observed a flush error, handler unwind before client-close cleanup, upstream cessation, and exactly one cancelled terminal event with no completion. Some runs also observed subsequent write errors. Healthy streams with 150 ms token gaps and a 50 ms idle-write deadline delivered their full body and exactly one completion, five times per profile. These diagnostics are audit-only evidence, not tests committed by this PR. The committed regression retains its existing terminal assertions. Skipped entries: `TestCORSBrowserFixture/ollama`, `TestCORSBrowserFixture/lmstudio`, and `TestCORSExternalEngineParity`. </details> ## Limitations and risk - Socket-buffer sizes are requests to the OS, not exact portable capacities. Supported-platform runtime CI remains an acceptance gate. - The ordinary local E2E build fails with `error obtaining VCS status: exit status 128`. The unstamped suite results do not establish that VCS-stamped builds pass. - Windows amd64 and macOS arm64 test binaries cross-compiled with `go test -buildvcs=false -c`; neither was executed on those platforms. - Repository-wide desktop and unrelated service suites were not rerun. No deployment or migration is included. ## Checklist - [x] Read the Contributing Guidelines and signed off every commit. - [x] Existing tests cover the change; the negative control checks sensitivity to a missing flush deadline. - [x] No runtime documentation or contract changes are needed for this test-only change. - [x] Checked the diff and description for credentials, private cluster identities, and generated artifacts. - [x] Recorded validation results and limitations above. - [x] Declared no release bump because production compiled output is unchanged. <details> <summary>Release metadata: test-only change, no version bump</summary> The repository requires the following no-release values for automation. <!-- pair-release-intent:v1 --> ### Changelog title n/a ### Changelog body n/a ### Bumps - services: none - nvpair-cluster-manager: none - nvpair-engine-manager: none - nvpair-errors: none - nvpair-job-scheduler: none - nvpair-manual-nodes: none - nvpair-node-info: none - nvpair-node-scanner: none - nvpair-node-settings: none - nvpair-proxy: none - nvpair-tui: none - nvpair-ui-broker: none - nvpair-workload-manager: none <!-- /pair-release-intent:v1 --> </details> Signed-off-by: Josh Stevenson <j.stevenson.cs@gmail.com>
1 parent 60cb443 commit 19018b8

1 file changed

Lines changed: 12 additions & 1 deletion

File tree

‎services/nvpair-proxy/zombie_test.go‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -413,7 +413,15 @@ func TestHandleHTTP_BoundsRealSocketFlushes(t *testing.T) {
413413

414414
ln, err := net.Listen("tcp", "127.0.0.1:0")
415415
require.NoError(t, err, "listen")
416-
srv := &http.Server{Handler: http.HandlerFunc(p.soleFacade().handleHTTP)}
416+
// Limit the sender as well: default TCP buffers can absorb the entire
417+
// five-second paced stream without ever blocking a flush.
418+
srv := &http.Server{
419+
Handler: http.HandlerFunc(p.soleFacade().handleHTTP),
420+
ConnContext: func(ctx context.Context, c net.Conn) context.Context {
421+
assert.NoError(t, c.(*net.TCPConn).SetWriteBuffer(4096), "set proxy write buffer")
422+
return ctx
423+
},
424+
}
417425
go func() { _ = srv.Serve(ln) }()
418426
defer srv.Close()
419427

@@ -430,6 +438,9 @@ func TestHandleHTTP_BoundsRealSocketFlushes(t *testing.T) {
430438
_, err = conn.Write([]byte(reqText))
431439
require.NoError(t, err, "write request")
432440

441+
// Bound the receive buffer so a paced stream reaches backpressure within
442+
// the test budget rather than fitting in host-dependent TCP buffers.
443+
require.NoError(t, conn.(*net.TCPConn).SetReadBuffer(1024), "set client read buffer")
433444
// Read the status line only, then stop reading so the proxy's send buffer
434445
// backs up and a Flush blocks.
435446
require.NoError(t, conn.SetReadDeadline(time.Now().Add(5*time.Second)))

0 commit comments

Comments
 (0)