Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,12 @@ FROM golang:1.26.5-bookworm@sha256:18aedc16aa19b3fd7ded7245fc14b109e054d65d22ed5
ENV DEBIAN_FRONTEND=noninteractive
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# shellcheck is here for actionlint, which shells out to it to check every `run:` block. Without it
# actionlint silently skips that half of its job and still exits 0, so the local lint passes on a
# workflow CI rejects. Found exactly that way: `task lint` green, CI red, same commit.
RUN apt-get update \
&& apt-get -y install --no-install-recommends \
git curl ca-certificates vim less jq sudo acl gnupg unzip openssh-client \
git curl ca-certificates vim less jq sudo acl gnupg unzip openssh-client shellcheck \
&& install -m 0755 -d /etc/apt/keyrings \
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
-o /etc/apt/keyrings/githubcli-archive-keyring.gpg \
Expand All @@ -40,13 +43,19 @@ RUN apt-get update \
# here behaves identically to the one the gateway will really run against.
# NODE_MAJOR -> the current Node LTS line. 22 runs `.ts` under `node --test`
# natively (type stripping), which is why this repo's client suite needs ZERO test deps.
# ACTIONLINT_VERSION -> https://github.com/rhysd/actionlint/releases
# HADOLINT_VERSION -> https://github.com/hadolint/hadolint/releases
# A workflow YAML and a Dockerfile were the only files here with no linter, which is how CI came to
# pack a bundle it had never built.
ENV PATH="/go/bin:/usr/local/go/bin:${PATH}" \
GOLANGCI_LINT_VERSION=v2.12.2 \
TASK_VERSION=v3.51.1 \
YQ_VERSION=v4.48.1 \
KUBECTL_VERSION=v1.36.2 \
K3D_VERSION=v5.9.0 \
NODE_MAJOR=22
NODE_MAJOR=22 \
ACTIONLINT_VERSION=1.7.7 \
HADOLINT_VERSION=2.14.0

RUN test "$(dpkg --print-architecture)" = "amd64" \
|| (echo "This devcontainer currently supports amd64 only." && exit 1)
Expand All @@ -65,6 +74,19 @@ RUN curl -fsSL -o /usr/local/bin/kubectl "https://dl.k8s.io/release/${KUBECTL_VE
RUN curl -fsSL -o /usr/local/bin/k3d "https://github.com/k3d-io/k3d/releases/download/${K3D_VERSION}/k3d-linux-amd64" \
&& chmod +x /usr/local/bin/k3d

# actionlint: typechecks `${{ }}` expressions, catches invalid `runs-on`/`needs`, and shellchecks
# every `run:` block.
RUN curl -fsSL "https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" \
| tar -xzO actionlint > /usr/local/bin/actionlint \
&& chmod +x /usr/local/bin/actionlint \
&& actionlint --version

# hadolint: the same for this file. Config in .hadolint.yaml. The `# hadolint ignore=` comments below
# have, until now, been talking to a linter that was not installed.
RUN curl -fsSL -o /usr/local/bin/hadolint "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64" \
&& chmod +x /usr/local/bin/hadolint \
&& hadolint --version

# Node.js — the client's test runner AND its build (tsc). Nothing else needs it.
RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key \
| gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
Expand Down
65 changes: 48 additions & 17 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,38 @@ concurrency:
cancel-in-progress: true

jobs:
# A workflow YAML and a Dockerfile are half this repo's supply-chain surface, and were the only
# files here with no linter. The bug that packed a bundle it had never built lived in a workflow,
# invisible to every Go and TypeScript check in this pipeline.
hygiene:
name: workflows and Dockerfile
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

# Versions come from the devcontainer's ENV block, so CI and a developer's shell cannot lint
# with different rules and disagree about whether the tree is clean.
- name: install actionlint and hadolint (versions from .devcontainer/Dockerfile)
run: |
set -euo pipefail
actionlint_version="$(grep -oP 'ACTIONLINT_VERSION=\K[0-9.]+' .devcontainer/Dockerfile)"
hadolint_version="$(grep -oP 'HADOLINT_VERSION=\K[0-9.]+' .devcontainer/Dockerfile)"
echo "actionlint ${actionlint_version}, hadolint ${hadolint_version}"
curl -fsSL "https://github.com/rhysd/actionlint/releases/download/v${actionlint_version}/actionlint_${actionlint_version}_linux_amd64.tar.gz" \
| tar -xzO actionlint > /tmp/actionlint
sudo install -m 0755 /tmp/actionlint /usr/local/bin/actionlint
sudo curl -fsSL -o /usr/local/bin/hadolint \
"https://github.com/hadolint/hadolint/releases/download/v${hadolint_version}/hadolint-Linux-x86_64"
sudo chmod +x /usr/local/bin/hadolint

# No path argument: actionlint discovers every workflow itself, so a new one is checked the day
# it lands rather than the day someone remembers to list it.
- run: actionlint

- run: hadolint .devcontainer/Dockerfile

# The point of the monorepo: one contract, and a change to it must be seen by BOTH
# implementations in the same commit. Both jobs read conformance/gen/, which is built
# from conformance/{bodies,fixtures}/*.yaml — so a protocol change that breaks either
Expand Down Expand Up @@ -146,6 +178,7 @@ jobs:
- name: no module may carry a `replace`
run: |
set -euo pipefail
# shellcheck disable=SC2016 # the backticks below are literal prose, not a substitution.
if grep -nE '^[[:space:]]*replace([[:space:]]|\()' gateway/go.mod gateway/kube/go.mod; then
echo '::error::a `replace` directive applies ONLY to the main module. It works in this checkout and is IGNORED by everyone who runs `go get`, so it makes the module unconsumable while every test here still passes. go.work is how this repo points the modules at each other; see the comment at the top of it.'
exit 1
Expand Down Expand Up @@ -217,6 +250,12 @@ jobs:
node-version: "22"
cache: npm
cache-dependency-path: packages/krm-stream/package-lock.json
# task, so the build+pack step below is the SAME task a developer runs. The two used to be
# hand-copied shell lines in two files, and they drifted the moment the build grew a step.
- uses: arduino/setup-task@c0bc642852239c2689f73f4ea6459c29405f3c52 # v3.0.0
with:
version: 3.x
repo-token: ${{ secrets.GITHUB_TOKEN }}
# npm ci, not npm install: the lockfile is the input, and a build that is allowed to quietly
# resolve a different tree than the one committed is not the build we tested.
- run: npm ci --no-audit --no-fund
Expand All @@ -230,24 +269,16 @@ jobs:
working-directory: packages/krm-stream
- run: npx biome ci
working-directory: packages/krm-stream
# The build, last: it is what ships, and it must stay plain ESM a browser can import with no
# bundler and no runtime dependency.
- run: npx tsc
working-directory: packages/krm-stream

# Pack the official client HERE, in the job that just tested it, and hand it to the release
# workflow as an artifact. release.yml publishes these exact bytes rather than rebuilding it:
# the things on npm are then the things this pipeline proved, and not a second build that
# merely ought to match.
# The same task a developer runs. It builds (tsc AND the esbuild flatten), packs, and asserts
# every entry point in the exports map is really in the tarball. Nothing else can prove that: a
# missing ./bundle builds, tests, packs and publishes green, then fails for the first consumer
# who imports it. `npx tsc` alone used to produce exactly that tarball.
#
# The versions inside them are already correct on a release commit — release-please bumps
# package.json in the release PR, so by the time this runs on main the tree IS the release.
- name: pack the tarballs that will be published
# mkdir first: --pack-destination does not create the directory, it fails with ENOENT.
run: |
set -euo pipefail
mkdir -p /tmp/npm
npm pack --pack-destination /tmp/npm ./packages/krm-stream
# release.yml publishes these bytes rather than rebuilding, so npm gets what this pipeline
# proved. Versions are already correct here: release-please bumps package.json in the release PR.
- name: build, pack, and prove the tarball is complete
run: task pack-client PACK_DIR=/tmp/npm

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: npm-packages
Expand Down
64 changes: 64 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# CodeQL: static analysis for the two languages this repo actually ships.
#
# The reason to have it is not the Scorecard point. The gateway parses untrusted input from a browser
# (a scope query, a merge patch) and hands back objects it has projected; the client merges a stream
# into a document a user is editing. That is where taint tracking earns its runtime.
name: codeql

on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
# The schedule matters: it re-runs the current queries against unchanged code, which is how a
# newly-published query finds an old bug that no push would have re-examined.
- cron: "40 4 * * 2"

permissions: read-all

jobs:
analyze:
name: analyze (${{ matrix.language }})
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
security-events: write # the whole point: write the findings to code scanning
packages: read
contents: read

strategy:
fail-fast: false
matrix:
include:
- language: go
build-mode: autobuild
- language: javascript-typescript
build-mode: none # interpreted; CodeQL reads the source directly

steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

# go.work ties gateway/ and gateway/kube/ together and autobuild honours it, so both modules are
# analysed. kube being a separate module must not make it the one nobody scans.
- uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
if: matrix.language == 'go'
with:
go-version-file: gateway/go.mod
cache: false

- name: initialize codeql
uses: github/codeql-action/init@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# security-extended, not the default suite: here a false positive costs a few minutes and a
# missed leak costs a Secret.
queries: security-extended

- name: analyze
uses: github/codeql-action/analyze@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with:
category: /language:${{ matrix.language }}
6 changes: 6 additions & 0 deletions .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@
# This is not decoration. A library whose whole pitch is "import this into your browser and trust what
# it tells you about your cluster" is asking for a lot of trust, and it should be able to show its
# working.
#
# There is deliberately NO repo_token here. A PAT is needed only to read CLASSIC branch protection;
# `main` uses a repository RULESET instead, which the default GITHUB_TOKEN can read, so
# Branch-Protection scores without one. That is the better posture, not a shortcut: a PAT with repo
# scope is a long-lived write credential stored as a secret, in a workflow whose whole job is to
# attest that the supply chain is sound.
name: scorecard

on:
Expand Down
12 changes: 12 additions & 0 deletions .hadolint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# hadolint configuration: https://github.com/hadolint/hadolint#configure
#
# Lint locally with `task lint-dockerfiles`; it is part of `task lint`.
ignored:
# DL3008, "pin versions in apt-get install". Deliberately not done. .devcontainer/Dockerfile is a
# tooling image: not shipped, not published, and in the supply chain of nothing this repo publishes.
# Pinning distro packages buys a rebuild that breaks the first time a security update lands. The
# versions that matter are pinned in the ENV block, and the base image by digest.
- DL3008

# Warnings and errors fail; info and style suggestions surface without blocking.
failure-threshold: warning
116 changes: 85 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,55 @@
[![CI](https://github.com/ConfigButler/krm-stream/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/ConfigButler/krm-stream/actions/workflows/ci.yml)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/ConfigButler/krm-stream/badge)](https://scorecard.dev/viewer/?uri=github.com/ConfigButler/krm-stream)
[![CodeQL](https://github.com/ConfigButler/krm-stream/actions/workflows/codeql.yml/badge.svg?branch=main)](https://github.com/ConfigButler/krm-stream/actions/workflows/codeql.yml)
[![npm](https://img.shields.io/npm/v/%40configbutler%2Fkrm-stream?logo=npm&color=cb3837)](https://www.npmjs.com/package/@configbutler/krm-stream)
[![Runtime dependencies](https://img.shields.io/badge/runtime%20deps-0-2ea44f)](packages/krm-stream/package.json)
[![Go](https://img.shields.io/badge/go-1.26-blue?logo=go)](gateway/go.mod)
[![TypeScript](https://img.shields.io/badge/typescript-ESM-3178c6?logo=typescript&logoColor=white)](packages/krm-stream)
[![License](https://img.shields.io/github/license/ConfigButler/krm-stream)](https://www.apache.org/licenses/LICENSE-2.0)
[![Open Issues](https://img.shields.io/github/issues/ConfigButler/krm-stream)](https://github.com/ConfigButler/krm-stream/issues)

# krm-stream

Live Kubernetes resource updates for browser apps, with three-way merges for form edits.
Live Kubernetes resource updates for browser apps.

`krm-stream` turns a Kubernetes watch into a small, browser-safe stream. It ships a Go gateway, a
headless TypeScript store, and shared conformance fixtures so your product can show live state while
people are editing it.
headless TypeScript store with zero runtime dependencies, and shared conformance fixtures, so your
product can show live cluster state while people are editing it.

## Is this for you?

**Yes, if:**

- You want to consume a Kubernetes **watch from a browser**, without handing the browser a cluster
credential or turning on CORS across your API server.
- You want to **live-edit Kubernetes resources**, where a concurrent server change is merged into
what the user is typing rather than clobbering it, and a genuine conflict is surfaced instead of
silently resolved. That is the [three-way merge](docs/glossary.md).
- You want to **bound the number of real watches** on your API server. Ten tabs on one namespace
should be one upstream watch, not ten.
- You are building a **product**, not a Kubernetes dashboard. The store is headless and picks no UI
framework.

**Probably not, if:**

- You just want a **generic three-way merge library**. This one knows what a `resourceVersion` is,
that `spec.containers` is keyed by `name` and not by index, and that a redacted field must never be
written back. That knowledge is the whole point; if you do not want it, it is weight.
- You want a **ready-made Kubernetes dashboard**. Use [Headlamp](https://headlamp.dev/). See
[alternatives](docs/alternatives.md).
- You want to **write to the cluster from the browser**. krm-stream is the read-and-edit half: it
hands your application a validated merge patch, and your application performs the write. Though if
you are doing that, you probably want this library anyway, because it is the thing that tells you
the patch is safe to apply. See [saving edits safely](docs/saving.md).

## KRM, briefly
## What is KRM?

KRM means **Kubernetes Resource Model**: the API objects that describe a system, such as a
`Deployment`, `Service`, `ConfigMap`, or a custom resource. Each object has identity and desired
state, and Kubernetes continuously reports observed state. Kubernetes calls these objects records
of intent; [its object model is a good starting point](https://kubernetes.io/docs/concepts/overview/working-with-objects/).
**KRM** is the Kubernetes Resource Model: the shape every Kubernetes object has (`apiVersion`,
`kind`, `metadata`, a desired `spec`, an observed `status`). Custom resources use the same shape,
which is why this works for your product's own objects, a `Database`, a `FeatureFlag`, a `Tenant`,
and not only for cluster infrastructure.

That model is useful far beyond infrastructure. A platform can model an application, an environment,
a database request, a feature rollout, access policy, or a business workflow as KRM resources. The
same live, conflict-aware editing experience should work wherever a resource expresses intent and a
controller reports what became true.
Never touched a cluster? The [glossary for frontend developers](docs/glossary.md).

## Why a gateway

Expand Down Expand Up @@ -65,6 +98,41 @@ credential or a raw API-server URL.

## Start here

There are two halves, and they are usually two different people.

### The browser half

No bundler, no framework, no Kubernetes client. `EventSource` is native, and the store is plain ESM:

```ts
import { LiveResourceStore, connectWithEventSource, resourceStreamURL } from "@configbutler/krm-stream";

const store = new LiveResourceStore();

connectWithEventSource(
resourceStreamURL("/resource-stream/v1", {
target: "production",
version: "v1",
resource: "configmaps",
namespace: "app",
}),
store,
{
onChange: (change) => render(change.uid), // what moved, and which resource it moved on
},
);

// The user edits. The server keeps changing underneath them. Neither wins by accident.
store.setValue(uid, ["spec", "replicas"], 3);
store.conflicts(uid); // paths where the server disagreed with an edit the user actually made
store.patch(uid); // an RFC 7386 merge patch of just their changes, or null
```

If you have no bundler at all and vendor the library by copying it, import
[`@configbutler/krm-stream/bundle`](packages/krm-stream/README.md): the same API in one file.

### The server half

Mount a scoped stream endpoint in an existing Go application:

```go
Expand All @@ -84,26 +152,12 @@ mux.Handle("/resource-stream/v1", gateway.Handler(gateway.Options{
}))
```

Consume the stream without choosing a UI framework:

```ts
import { LiveResourceStore, connectWithEventSource, resourceStreamURL } from "@configbutler/krm-stream";

const store = new LiveResourceStore();
connectWithEventSource(
resourceStreamURL("/resource-stream/v1", {
target: "production",
version: "v1",
resource: "configmaps",
namespace: "app",
}),
store,
);
```
The Go side owns identity, authorization, the Kubernetes credential, and the scope a caller is
allowed to ask for. It never lets the browser choose which cluster to talk to.

`LiveResourceStore` keeps server truth and a local draft separate, reconciles live updates with a
three-way merge, records conflicts, and builds RFC 7386 merge patches. The host applies any patch
through its own save endpoint.
`LiveResourceStore` keeps server truth and the local draft separate, reconciles live updates with a
three-way merge, records conflicts, and builds RFC 7386 merge patches. Your application applies any
patch through its own save endpoint, which is the one place a write can happen.

## Packages

Expand Down
Loading