HMR binding does not work in kubernetes

[jplimack]

Describe the bug

I'm running a local kind cluster on my macbook and trying to deploy my Sveltekit app. Its working great outside of k8s, but I need to setup a bunch of ancillary things and its time to upgrade the stack.

In order to fully test my stack, I require TLS to work and use mkcert to generate local certificates, and also have cert-manager configured to provide self-signed certificates.

I cannot figure out the magical incantation to get both node+HMR to bind to IPv4 AND also provide a client configuration pointing at the dns-name, (not 0.0.0.0).

So if in my vite config I set things to bind to 0.0.0.0, I get IPv4, but then my client sees

"hmr" was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint This request has been blocked; this endpoint must be available over WSS.

or

       GET https://0.0.0.0/hmr/ net::ERR_CERT_AUTHORITY_INVALID

if i set vite to use my dns name, it will bind to IPv6, and then nothing works. Wish there was a way to provide both a bind/listen parameter as well as the hostname to provide to the client so that it would understand that my Ingress controller is terminating TLS. (if I enable passthru TLS, then its L4 and can't do the ws connection upgrade which is L7).

Reproduction

ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frontend
  namespace: myapp
  annotations:
    cert-manager.io/cluster-issuer: local-issuer-dev
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" # Backend uses HTTPS
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/proxy-set-headers: "Upgrade $http_upgrade"
    nginx.ingress.kubernetes.io/proxy-set-connection: "Connection upgrade"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/websocket-services: "frontend-ws"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - myapp.com
      secretName: myapp-tls
  rules:
    - host: myapp.com
      http:
        paths:
          - path: /hmr/
            pathType: Prefix
            backend:
              service:
                name: frontend-ws
                port:
                  number: 24678
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 3030

service

# Service for the WebSocket
apiVersion: v1
kind: Service
metadata:
  name: frontend-ws
  namespace: {{ .Values.global.namespace }}
  labels:
    app: frontend
    type: websocket
spec:
  selector:
    app: frontend
  ports:
    - protocol: TCP
      port: 24678 # WebSocket service port
      targetPort: 24678 # WebSocket container port
  type: ClusterIP

---
# Service for the main frontend
apiVersion: v1
kind: Service
metadata:
  name: frontend-service
  namespace: {{ .Values.global.namespace }}
  labels:
    app: frontend
    type: main
spec:
  selector:
    app: frontend
  ports:
    - protocol: TCP
      port: 3030 # Main frontend service port
      targetPort: 3030 # Main frontend container port
  type: ClusterIP

deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: {{ .Values.global.namespace }}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      securityContext:
        sysctls:
          - name: net.ipv6.conf.all.disable_ipv6
            value: "1"
          - name: net.ipv6.conf.default.disable_ipv6
            value: "1"
      initContainers:
        - name: disable-ipv6
          image: docker.io/busybox
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh", "-c"]
          args: 
            - sysctl -w net.ipv6.conf.all.disable_ipv6=1; # these seem to have zero effect
              sysctl -w net.ipv6.conf.default.disable_ipv6=1 # these seem to have zero effect
          securityContext:
            privileged: true
      containers:
        - name: frontend
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          ports:
            - containerPort: 3030
            - containerPort: 24678 # Expose the WebSocket port
          volumeMounts:
            - name: tls-volume
              mountPath: "/app/certs"
              readOnly: true
      volumes:
        - name: tls-volume
          secret:
            secretName: myapp-tls

Logs

No response

System Info

Kubernetes (kind)  v1.27.3

# laptop

  System:
    OS: macOS 14.1
    CPU: (12) arm64 Apple M3 Pro
    Memory: 96.73 MB / 36.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 23.3.0 - /opt/homebrew/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.9.1 - /opt/homebrew/bin/npm
  Browsers:
    Brave Browser: 131.1.73.91
    Chrome: 131.0.6778.86
    Safari: 17.1
  npmPackages:
    @sveltejs/adapter-auto: ^3.2.5 => 3.3.1
    @sveltejs/adapter-static: ^3.0.5 => 3.0.6
    @sveltejs/kit: ^2.0.0 => 2.8.2
    @sveltejs/vite-plugin-svelte: ^3.0.0 => 3.1.2
    svelte: ^4.2.7 => 4.2.19
    vite: ^5.0.3 => 5.4.11

inside container

 Binaries:
    Node: 23.3.0 - /usr/local/bin/node
    Yarn: 1.22.22 - /usr/local/bin/yarn
    npm: 10.9.0 - /usr/local/bin/npm
  npmPackages:
    @sveltejs/adapter-auto: ^3.2.5 => 3.3.1
    @sveltejs/adapter-static: ^3.0.5 => 3.0.6
    @sveltejs/kit: ^2.0.0 => 2.8.2
    @sveltejs/vite-plugin-svelte: ^3.0.0 => 3.1.2
    svelte: ^4.2.7 => 4.2.19
    vite: ^5.0.3 => 5.4.11

Severity

blocking all usage of SvelteKit

Additional Information

vite.config.ts

import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vitest/config';

// import fs from 'fs';
// import path from 'path';
// let key = fs.readFileSync(path.resolve(__dirname, 'certs/tls.key'))
// let cert = fs.readFileSync(path.resolve(__dirname, 'certs/tls.crt'))

export default defineConfig({
	plugins: [sveltekit(),


	],
	server: {
		host: '0.0.0.0',
		port: 3030,
		strictPort: true,
		hmr: {
			protocol: 'wss', // Use WebSocket protocol / use wss for tls, but not behind an ingress.
			// host: 'myapp.com', // This will mess up the binding, as it will bind to IPv6 addresses
			host: '0.0.0.0', // this will mess up the client as the client wants a DNS name
			port: 24678, // Match the expected HMR port
			clientPort: 443, // Adjust as needed
			overlay: true,    // Show HMR errors on the browser overlay
			path: "/hmr/",
		},
		// https: {
		// 	key: key,
		// 	cert: cert,
		// },
	},
	test: {
		include: ['src/**/*.{test,spec}.{js,ts}']
	}
});

GET https://0.0.0.0/hmr/ net::ERR_CERT_AUTHORITY_INVALID

If i flip to use the dns name, it binds wrong

/app # lsof -Pni | grep LISTEN
node     21 root 21u  IPv4 2802096      0t0  TCP 127.0.0.1:24678 (LISTEN)
node     21 root 34u  IPv4 2802097      0t0  TCP *:3030 (LISTEN)

/app # grep -A2 hmr vite.config.ts
		hmr: {
			protocol: 'wss', // Use WebSocket protocol / use wss for tls, but not behind an ingress.
			host: 'myapp.com', // This will fuck up the binding, as it will bind to IPv6 addresses

[jplimack]

seems like maybe theres a way to pass a localAddress to bind to, and this would need to be done through the plumbing of Vite? https://github.com/nodejs/node/blob/main/doc/api/http.md

I'm not a JS expert by any means, any and all help/pointers/tips are appreciated. im also having a hard time believing I'm the first to pioneer this path, so I'm assuming that I'm doing many things wrong.

[ekusiadadus]

You can separate the browser's public hostname from the address Vite binds to by sharing the main HTTP server with HMR. In the Vite 5.4.11 version you listed, omitting hmr.port makes the WebSocket handler attach to the existing server, which already listens on 0.0.0.0:3030.

For TLS termination at ingress, try this server configuration:

server: {
  host: '0.0.0.0',
  port: 3030,
  strictPort: true,
  hmr: {
    protocol: 'wss',
    host: 'myapp.com',
    clientPort: 443,
    path: '/hmr/'
  }
}

Then route both / and /hmr/ to frontend-service:3030, preserving the path and WebSocket upgrade. The separate frontend-ws:24678 route is no longer needed in this arrangement.

There is also a TLS mismatch in the configuration shown: the ingress annotation says the upstream is HTTPS, but server.https is commented out in Vite. With ingress terminating TLS and Vite serving plain HTTP inside the cluster, remove that annotation or set nginx.ingress.kubernetes.io/backend-protocol: "HTTP". The browser still uses HTTPS/WSS. Its certificate must be trusted and valid for myapp.com; using 0.0.0.0 as the browser hostname does not solve that.

To verify, check that the browser opens wss://myapp.com/hmr/ and gets a 101 Switching Protocols, then edit a component and check that the change arrives. This addresses the connection configuration; if the socket connects but changes do not arrive, check the separate file-sync/watch path into the container.

I checked this against Vite 5.4.11's WebSocket implementation and Vite's HMR documentation, but have not reproduced your kind/ingress setup.