Skip to content

Neither route nor gateway-api ingress mode provisions routing for OpenShell's per-sandbox ExposeService HTTP virtual-hosting #234

Description

@jameswnl

Summary

Neither route nor gateway-api ingress mode provisions any routing for OpenShell's per-sandbox ExposeService HTTP virtual-hosting feature — only a single, fixed hostname pointing at the gateway's gRPC port ever gets created. Any client that follows OpenShell's own documented mechanism for reaching a sandbox's exposed HTTP service gets a bare 404 from the platform router itself, before the request ever reaches the OpenShell gateway pod.

This was traced from downstream failures in jameswnl/lightspeed-cloud-agents (issues #209, #218, #221, #249) chasing a "bare HTTP/2 404, no body/content-type" on Host-header-routed sandbox HTTP calls, while the sandbox itself is confirmed healthy (openshell sandbox exec -n <name> -- curl http://127.0.0.1:8080/health returns 200 OK) and gRPC control-plane calls to the same gateway keep working fine.

Why this is a HyperShell gap, not an OpenShell client bug

OpenShell's ExposeService feature hands back a genuine, distinct HTTPS hostname per sandbox — {workspace}--{sandbox}.{base_domain} (openshell-server/src/service_routing.rs::endpoint_host), on the same port as the gateway's main listener. The gateway then demuxes internally by parsing the Host header (service_routing.rs::parse_host / proxy_sandbox_service_request) — but only once the raw TLS bytes actually reach the gateway process. A normal HTTPS client hitting that per-sandbox URL presents that hostname as the TLS SNI, not the gateway's own registered hostname.

Checking both of HyperShell's ingress-provisioning paths in components/control-plane/internal/gateway/reconciler.go:

  • route modereconcilePassthroughRoute (~line 472-520) creates exactly one OpenShift Route with termination: passthrough and a single fixed spec.host. OpenShift's router matches passthrough Routes by TLS SNI against that one registered hostname, and does not support wildcard hostnames on passthrough termination. A client connecting with SNI = a per-sandbox vhost has no matching Route, so the router returns its own generic "no route matched" response — a bare 404 with no body — before the gateway pod ever sees the connection. This matches the observed symptom exactly.
  • gateway-api modereconcileGatewayAPIResources (~line 821+) creates a single GRPCRoute with one fixed hostnames: [...] entry pointed at the gateway's gRPC port. No HTTPRoute is created at all, so there's no routing path for arbitrary per-sandbox hostnames here either.

So in both modes, HyperShell provisions routing for exactly one hostname per gateway — sufficient for the gRPC control plane, but structurally incompatible with OpenShell's per-sandbox virtual-hosting design for HTTP service exposure.

(Related but distinct: #220, closed, proposed switching route mode from passthrough to reencrypt termination — that fixes a TLS trust-chain problem [UnknownIssuer] on the existing single hostname, and wouldn't by itself add wildcard/multi-hostname matching. Flagging it here only for code-region cross-reference, not as a fix for this issue.)

Ask

Is per-sandbox HTTP service exposure (ExposeService) an intended, supported use case for HyperShell-provisioned gateways? If so, this likely needs either:

  • a wildcard-hostname HTTPRoute/Route (*.<base_domain>) pointed at the gateway's HTTP-serving port (requires a wildcard-capable router/cert, which may itself be restricted on managed clusters like ROSA/OSD — worth scoping early), or
  • some other routing mechanism entirely.

If it's out of scope by design, it'd be worth documenting that explicitly so downstream consumers (e.g. lightspeed-cloud-agents) don't keep re-diagnosing what looks like a transient bug. That downstream project is currently evaluating an OpenShell-gRPC-tunnel workaround (ForwardTcp) specifically to route around this gap — useful signal that it's blocking real usage, not a theoretical concern.

Environment

Observed on a HyperShell-provisioned staging gateway (route ingress mode). Not reproduced on local Kind or a separately/manually-provisioned real OCP cluster, consistent with this being specific to how HyperShell provisions ingress rather than an OpenShell core issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions