feat(middleware): define HTTP response pre-return interface

[pimlock]

Summary

Define the public HTTP_RESPONSE/PRE_RETURN contract without enabling response processing in the proxy. This lets reviewers evaluate the protobuf lifecycle, naming, extensibility, and transport shape independently from the runtime implementation.

The proposed stream begins with a preflight action, uses lockstep pass-through or transform actions for body units, marks the final unit with end_of_stream, and ends with a shared middleware session-end reason plus typed protocol-error detail.

API interaction examples

See HTTP response pre-return middleware API interaction examples for whole-body, streaming, skip, and block-delivery exchanges, failure scenarios, and an illustrative Rust/tonic middleware implementation.

Related Issue

Part of #2691

Changes

• Add the bidirectional HttpResponsePreReturn.Evaluate service with preflight, body unit, body result, and terminal event messages.
• Preflight returns skip, inspect, or block_delivery; body results return pass_through or transform. OpenShell tells each stage which body modes it may select and whether it may defer bytes across units.
• The final body unit carries end_of_stream. Response trailers are deferred from V1 with field numbers reserved.
• Share MiddlewareSessionEnd and its reason enum across streaming middleware APIs, with disconnects split by direction and typed protocol-error detail.
• Accept the HTTP_RESPONSE/PRE_RETURN manifest binding and add in-process and remote gRPC stream interfaces.
• Compile the content-guard example in lint and branch checks.

Breaking changes

WebSocketSessionEndReason and WebSocketSessionEnd are replaced by the shared MiddlewareSessionEndReason and MiddlewareSessionEnd. NORMAL_CLOSE is now NORMAL, UPSTREAM_REJECTED is now UPSTREAM_FAILURE, and PEER_DISCONNECT is split into DOWNSTREAM_DISCONNECT and UPSTREAM_DISCONNECT. Enum numbers are unchanged, so binary wire compatibility is preserved; generated symbols and JSON names change. This lands in the pre-0.1.0 window.

Testing

• mise run pre-commit passes
• mise run test passes
• Unit tests added/updated
• E2E tests added/updated, deferred to feat(network): inspect HTTP responses before return #3074

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated, deferred to docs(middleware): document response pre-return workflow #3075

Stack

1. feat(middleware): broaden HTTP header mutation authority #3072 - shared header mutation authority
2. feat(middleware): define HTTP response pre-return interface #3073 - response protobuf and transport interface
3. feat(network): inspect HTTP responses before return #3074 - response runtime, relay integration, and existing example
4. docs(middleware): document response pre-return workflow #3075 - documentation and middleware guide reorganization

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[pimlock]

gator-agent

PR Review Status

The interface and caller migration are internally consistent, and the pre-0.1.0 symbol changes are not blockers. One enforcement gap must be fixed before this can advance: OpenShell currently accepts the new response binding even though no supported HTTP relay invokes it.

Action required: keep HTTP_RESPONSE/PRE_RETURN manifest registration rejected until the response runner is wired into every supported HTTP relay, or land that wiring with this change.

Blocking findings:

• GATOR-11ef5b93-01: accepted response middleware bindings are silently unenforced

Carried findings:

• None

Gator metadata

• Validation: Project-valid as part of accepted issue #2691 and the documented response-middleware stack.
• Docs: Interface documentation is deferred to #3075; no runtime response behavior is enabled by this PR.
• Checks: Current-head required checks are green.
• E2E: Not requested for this interface-only review; the operator did not authorize applying test:e2e or rerunning workflows.
• Head SHA: 11ef5b932f859cf8f9a14a35270e18eb536a0c49
• Base SHA: 4b3b641b0c5c93ddc9e6d51ac11b45a271aa70b4
• Merge base SHA: 4b3b641b0c5c93ddc9e6d51ac11b45a271aa70b4
• Patch ID: 02da4f41c061dcfb3ea51e92faa8123f75da53be
• Gator payload: 8
• Review mode: initial
• Previous reviewed SHA: none
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:in-review

[pimlock]

gator-agent

PR Review Status

Thanks @pimlock. I checked your September 2 update that rejects HTTP_RESPONSE/PRE_RETURN until relay dispatch exists and adds a regression test. The manifest now fails closed for that binding, so GATOR-11ef5b93-01 is resolved; the independent delta review found no new blocking issues. Branch Checks are running, so Gator will monitor the pipeline next.

Blocking findings:

• No blocking findings remain

Carried findings:

• GATOR-11ef5b93-01: resolved at this head by rejecting the undispatched response binding

Gator metadata

• Validation: Project-valid as part of accepted issue #2691 and the response-middleware stack.
• Docs: The protobuf contract is documented in-tree; runtime integration and the user guide remain scoped to #3074 and #3075.
• Checks: OpenShell / Branch Checks is pending; OpenShell / Helm Lint, DCO, dependency review, and published E2E gate statuses are green.
• E2E: No test:e2e label was applied because this PR leaves response dispatch disabled, and the operator did not authorize E2E dispatch, /ok to test, or workflow reruns.
• Head SHA: 81ac0972414a9bd4cdc484642eaf30f7bcea877d
• Base SHA: 4b3b641b0c5c93ddc9e6d51ac11b45a271aa70b4
• Merge base SHA: 4b3b641b0c5c93ddc9e6d51ac11b45a271aa70b4
• Patch ID: a9784c43c09dea502f2c981aece30cb981ee88a8
• Gator payload: 8
• Review mode: follow_up
• Previous reviewed SHA: 11ef5b932f859cf8f9a14a35270e18eb536a0c49
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:watch-pipeline

[drew]

Review feedback

I do not think the V1 interface is ready to approve yet. I found four contract-level issues:

1. The PR omits response trailers required by accepted issue #2691. The stream exposes only preflight, body, and session-end events, so middleware cannot inspect or safely mutate trailers. This leaves trailer-carried integrity metadata or sensitive data outside response enforcement. Please restore trailer events and declared trailer names, or formally re-scope the accepted issue.

   Event definitions · Accepted issue

2. block_delivery expands middleware authority beyond the accepted design. Issue #2691 explicitly defers successful response denial, but this PR adds it to preflight and body processing regardless of on_error. It also describes only a “platform error,” without specifying its status, body, HEAD behavior, or stable error code. Please remove it from V1 or update and re-accept the design with complete client-visible semantics.

   Preflight blocking · Body blocking

3. A shared body-mode list conflicts with stage-local payload limits. max_payload_bytes is the effective limit for the current stage, while every stage is documented as receiving the same permitted_body_modes. For a 2 MiB response and stages limited to 1 MiB and 4 MiB, WHOLE_BODY_BYTES must either be incorrectly offered to the first stage or incorrectly withheld from the second. Please compute response-shape eligibility once, then intersect it with each stage effective limit.

   Preflight limits and modes

4. The disconnect enum change is parse-compatible but not semantically wire-compatible. Previously, value 2 meant generic PEER_DISCONNECT; it now means specifically DOWNSTREAM_DISCONNECT. A new middleware connected to an older supervisor will classify an old upstream disconnect as downstream. Please preserve value 2 as direction-neutral and allocate new values for directional reasons, or version the contract.

   New session-end enum

The overall stream and transport shape is otherwise coherent. I also verified that the earlier Gator finding is resolved: HTTP_RESPONSE/PRE_RETURN registration is now rejected until relay dispatch lands.

[drew]

Additional review feedback

A follow-up contract pass found three additional items beyond my earlier review:

1. The PR description and protobuf disagree about reserved trailer fields. The description says response trailers are deferred from V1 “with field numbers reserved,” but proto/supervisor_middleware.proto contains no reserved declaration. The current event/result oneofs immediately assign session-end field 3 and expose no reservation for the trailer messages described in #2691. Please either add the intended reservations or correct the description and document how trailers will be added compatibly.

   Current event/result envelopes · Accepted trailer design

2. Several other V1 contract changes have not been reconciled with the accepted plan. In addition to trailers and blocking, this interface replaces the planned body_end event with end_of_stream, adds skip_remaining and cross-unit deferral, reduces stream inputs from the full effective limit to half the limit, adds a whole-body accumulation timeout, and moves diagnostics into the decision envelopes. Some may be good changes, but they alter the accepted public contract. Please update #2691 and obtain maintainer confirmation for the revised V1 surface before freezing its field numbers and semantics.

   end_of_stream · skip_remaining · deferral contract

3. The new upstream-disconnect path lacks behavioral coverage. The mapping unit test verifies that WebSocketTerminationCause::UpstreamDisconnect maps to the enum, but no relay test verifies that an upstream EOF actually delivers UPSTREAM_DISCONNECT to an opened middleware session. Since the directional split changes externally observed lifecycle semantics, please add a relay-level test that closes the upstream and asserts the terminal reason.

   Upstream relay EOF handling · Mapping-only test