Skip to content

Merge e2e kubernetes openshift - #1

Closed
jgarciao wants to merge 1 commit into
mainfrom
merge-e2e-kubernetes-openshift
Closed

Merge e2e kubernetes openshift#1
jgarciao wants to merge 1 commit into
mainfrom
merge-e2e-kubernetes-openshift

Conversation

@jgarciao

Copy link
Copy Markdown
Owner

Running mise run e2e:kubernetes on OpenShift required manual namespace creation, SCC grants, Helm value overrides, and cleanup. A separate e2e:openshift task existed but only checked pod readiness without running the Rust e2e test suite, and even with the suite wired up the SSH-relay sandbox connect path stalled to the ready timeout because kubectl port-forward cannot carry round-trip-heavy SSH over the internet.

The harness now auto-detects OpenShift via the route.openshift.io API group and, on OpenShift, both configures the cluster and switches the gateway transport automatically:

  • Drives the gateway through a passthrough OpenShift Route secured with mandatory mTLS instead of port-forward, so the connect suites (live_policy_update, port_forward, sync, connect-based sandbox_lifecycle, settings_management) actually pass. Computes the Route host from the cluster ingress domain, extracts client mTLS material from the openshell-client-tls secret, waits for the Route to serve mTLS, asserts a certless caller is rejected at the TLS handshake, and registers an mTLS CLI gateway pointing at the Route.
  • Applies an SCC-compatible Helm values overlay that removes hardcoded runAsUser/fsGroup, letting OpenShift assign UIDs from the namespace range.
  • Grants the privileged SCC to openshell-sandbox before Helm install and removes it during cleanup.
  • Grants the anyuid SCC to the PostgreSQL fixture service account in DB scenarios and removes it during cleanup.
  • All oc commands use --context to target the correct cluster.

The OpenShift e2e overlay (ci/values-openshift-e2e.yaml) turns TLS back on, enables the Route, promotes the cert-verified caller to a dev principal, and forces image.pullPolicy/supervisor.image.pullPolicy to Always so runs against the latest upstream image use it instead of a stale copy cached on the cluster nodes. Every OpenShift branch is gated on OPENSHIFT_DETECTED, so the vanilla-Kubernetes port-forward path is unchanged.

The Helm template for podSecurityContext is wrapped with {{- with }} so null values omit the block instead of rendering invalid YAML.

The separate e2e:openshift task and e2e-openshift.sh script are removed since e2e:kubernetes now covers OpenShift.

TESTING.md is updated with Kubernetes e2e documentation including OpenShift auto-detection, dropping the e2e-host-gateway feature on remote clusters, pinning IMAGE_TAG when the CLI and image versions differ, task variants, and environment variables.

The debug-openshell-cluster skill gains an OpenShift platform row and two SCC failure patterns (gateway rejected over hardcoded runAsUser, sandbox missing the privileged SCC) covering the SCC handling and podSecurityContext behavior this change introduces.

Running `mise run e2e:kubernetes` on OpenShift required manual namespace
creation, SCC grants, Helm value overrides, and cleanup. A separate
`e2e:openshift` task existed but only checked pod readiness without
running the Rust e2e test suite, and even with the suite wired up the
SSH-relay `sandbox connect` path stalled to the ready timeout because
`kubectl port-forward` cannot carry round-trip-heavy SSH over the
internet.

The harness now auto-detects OpenShift via the `route.openshift.io` API
group and, on OpenShift, both configures the cluster and switches the
gateway transport automatically:

- Drives the gateway through a passthrough OpenShift Route secured with
  mandatory mTLS instead of port-forward, so the connect suites
  (live_policy_update, port_forward, sync, connect-based
  sandbox_lifecycle, settings_management) actually pass. Computes the
  Route host from the cluster ingress domain, extracts client mTLS
  material from the openshell-client-tls secret, waits for the Route to
  serve mTLS, asserts a certless caller is rejected at the TLS
  handshake, and registers an mTLS CLI gateway pointing at the Route.
- Applies an SCC-compatible Helm values overlay that removes hardcoded
  runAsUser/fsGroup, letting OpenShift assign UIDs from the namespace
  range.
- Grants the privileged SCC to openshell-sandbox before Helm install
  and removes it during cleanup.
- Grants the anyuid SCC to the PostgreSQL fixture service account in
  DB scenarios and removes it during cleanup.
- All oc commands use --context to target the correct cluster.

The OpenShift e2e overlay (ci/values-openshift-e2e.yaml) turns TLS back
on, enables the Route, promotes the cert-verified caller to a dev
principal, and forces `image.pullPolicy`/`supervisor.image.pullPolicy`
to Always so runs against the `latest` upstream image use it instead of
a stale copy cached on the cluster nodes. Every OpenShift branch is
gated on OPENSHIFT_DETECTED, so the vanilla-Kubernetes port-forward path
is unchanged.

The Helm template for podSecurityContext is wrapped with {{- with }} so
null values omit the block instead of rendering invalid YAML.

The separate e2e:openshift task and e2e-openshift.sh script are removed
since e2e:kubernetes now covers OpenShift.

TESTING.md is updated with Kubernetes e2e documentation including
OpenShift auto-detection, dropping the e2e-host-gateway feature on
remote clusters, pinning IMAGE_TAG when the CLI and image versions
differ, task variants, and environment variables.

The debug-openshell-cluster skill gains an OpenShift platform row and
two SCC failure patterns (gateway rejected over hardcoded runAsUser,
sandbox missing the privileged SCC) covering the SCC handling and
podSecurityContext behavior this change introduces.

Signed-off-by: Jorge Garcia Oncins <jgarciao@redhat.com>
@jgarciao
jgarciao changed the base branch from main to fix/3033-helm-podsecuritycontext-null August 31, 2026 14:49
@jgarciao
jgarciao changed the base branch from fix/3033-helm-podsecuritycontext-null to main August 31, 2026 14:49
@jgarciao

Copy link
Copy Markdown
Owner Author

Sent opendatahub-io#30 instead to get CodeRabbit reviews

@jgarciao jgarciao closed this Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant