feat(devcontainer): one-click osprey demo in github codespaces

.devcontainer/README.md

@@ -0,0 +1,64 @@
+# Osprey demo in this codespace
+
+This codespace runs the whole Osprey stack, with synthetic demo data flowing through
+it. You don't need to type any commands.
+
+## What happens
+
+1. **Preparing** (first launch only, a few minutes, skipped when a prebuild exists):
+   the codespace downloads and builds Osprey's services.
+2. **Starting** (every launch, 1–3 minutes): a terminal named *postStartCommand*
+   shows each service coming up.
+3. **Ready**: the terminal prints **Osprey is ready** with the UI's address, and the
+   Osprey UI opens in a new browser tab. If your browser blocked the tab, open the
+   **Ports** tab at the bottom of this window and click the globe icon next to
+   **Osprey UI (5002)**.
+
+## What to try
+
+The Query page opens empty: click **Submit Query** to load the last day of events.
+
+Synthetic posts arrive about once a second. The demo rule, `ContainsHello`, bans and
+labels any user whose post contains "hello".
+
+- Watch posts arrive in the **event stream** on the right of the Query page.
+- Query `RegexMatch(target=PostText, pattern="hello")` to see only the posts the rule matched.
+- Open the **Rules Visualizer**, search for `meow`, and turn on **Show Upstream Nodes**.
+
+## Stopping and coming back
+
+- A codespace stops by itself after 30 idle minutes (your account may set a different
+  timeout). You can also stop it from <https://github.com/codespaces> → **⋯** → **Stop codespace**.
+- Reopen it from the same page. The demo restarts on its own, keeping the data it had.
+- When you're finished for good, **Delete** it there, so it stops using your storage allowance.
+
+Full guide, costs and troubleshooting: [docs/development/codespaces.md](../docs/development/codespaces.md).
+
+## For engineers
+
+How [devcontainer.json](devcontainer.json) is put together:
+
+- **Image**: a plain Ubuntu devcontainer with the Docker-in-Docker feature. Osprey runs
+  as Docker Compose services inside it, exactly as on a laptop.
+- **`hostRequirements`**: 4 cores / 16 GB. The running stack uses about 6 GiB, plus
+  image builds; the 2-core / 8 GB machine runs out of memory.
+- **`containerEnv`**: points `docker compose` at the demo stack from any directory.
+- **Lifecycle**: `updateContentCommand` ([prepare.sh](scripts/prepare.sh)) pulls and
+  builds every image. It's the slow part, and a prebuild runs it ahead of time.
+  `postStartCommand` ([start.sh](scripts/start.sh)) runs on every start or resume:
+  `docker compose up -d`, then waits for the API, the UI and the first ingested event.
+  `postAttachCommand` ([status.sh](scripts/status.sh)) tells each new connection
+  whether the demo is ready and where.
+- **Ports**: only the UI (5002) is forwarded, and it opens in the browser. Everything
+  else is silent; the browser never needs another port.
+
+- `bash .devcontainer/scripts/start.sh` brings the stack up again and waits for it;
+  `bash .devcontainer/scripts/status.sh` prints whether it's ready.
+- `docker compose` in the terminal already targets the demo stack (`COMPOSE_FILE` and
+  `COMPOSE_PROFILES` are set), e.g. `docker compose ps`, `docker compose logs -f osprey-worker`.
+- [docker-compose.codespaces.yaml](docker-compose.codespaces.yaml) layers the demo's changes
+  over [docker-compose.yaml](../docker-compose.yaml): the UI calls the API through its own
+  dev server at `/api`, and the Druid web console is off (`docker compose --profile
+  druid-console up -d druid-router` turns it on, at port 8888).
+- The container doesn't include Python, uv or Node; it's built for running the demo.
+  See [Local Development](../docs/development/local.md) for a full development setup.

.devcontainer/devcontainer.json

@@ -0,0 +1,41 @@
+{
+  "name": "Osprey demo",
+  "image": "mcr.microsoft.com/devcontainers/base:ubuntu-24.04",
+  "features": {
+    "ghcr.io/devcontainers/features/docker-in-docker:2": {}
+  },
+  "hostRequirements": {
+    "cpus": 4,
+    "memory": "16gb",
+    "storage": "32gb"
+  },
+  "containerEnv": {
+    "COMPOSE_PROJECT_NAME": "osprey",
+    "COMPOSE_FILE": "${containerWorkspaceFolder}/docker-compose.yaml:${containerWorkspaceFolder}/.devcontainer/docker-compose.codespaces.yaml",
+    "COMPOSE_PROFILES": "test_data"
+  },
+  "updateContentCommand": "bash .devcontainer/scripts/prepare.sh",
+  "postStartCommand": "bash .devcontainer/scripts/start.sh",
+  "postAttachCommand": "bash .devcontainer/scripts/status.sh",
+  "forwardPorts": [5002],
+  "portsAttributes": {
+    "5002": {
+      "label": "Osprey UI",
+      "onAutoForward": "openBrowserOnce"
+    },
+    "5004": { "label": "Osprey UI API", "onAutoForward": "silent" },
+    "5001": { "label": "Osprey worker", "onAutoForward": "silent" },
+    "8081": { "label": "Druid coordinator", "onAutoForward": "silent" }
+  },
+  "otherPortsAttributes": { "onAutoForward": "silent" },
+  "customizations": {
+    "codespaces": {
+      "openFiles": [".devcontainer/README.md"]
+    },
+    "vscode": {
+      "settings": {
+        "workbench.startupEditor": "none"
+      }
+    }
+  }
+}

.devcontainer/docker-compose.codespaces.yaml

@@ -0,0 +1,35 @@
+# Overrides docker-compose.yaml for the Codespaces / devcontainer demo.
+# devcontainer.json sets COMPOSE_FILE so plain `docker compose` picks this up.
+
+services:
+  osprey-ui-api:
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "python3",
+          "-c",
+          "import urllib.request; urllib.request.urlopen('http://localhost:5004/_health')",
+        ]
+      interval: 5s
+      timeout: 5s
+      retries: 60
+      start_period: 30s
+
+  osprey-ui:
+    # Call the API on the UI's own origin: the dev server proxies /api to the API
+    # container. In a codespace, port 5004 is a separate, login-protected origin
+    # that the browser can't call from the UI's page.
+    environment:
+      - REACT_APP_API_BASE_URL=/api
+      - OSPREY_UI_API_PROXY_TARGET=http://osprey-ui-api:5004
+    # Don't publish the UI port (which opens the browser tab) until the API answers.
+    depends_on:
+      osprey-ui-api:
+        condition: service_healthy
+
+  # The Druid web console isn't part of the demo; skip it to save memory.
+  # `docker compose --profile druid-console up -d druid-router` starts it.
+  druid-router:
+    profiles:
+      - druid-console

.devcontainer/scripts/lib.sh

@@ -0,0 +1,102 @@
+#!/bin/bash
+# Shared helpers for the devcontainer lifecycle scripts. Sourced, not run.
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT" || exit 1
+
+# devcontainer.json sets these; default them so the scripts also work when run by hand.
+export COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME:-osprey}"
+export COMPOSE_FILE="${COMPOSE_FILE:-$REPO_ROOT/docker-compose.yaml:$REPO_ROOT/.devcontainer/docker-compose.codespaces.yaml}"
+export COMPOSE_PROFILES="${COMPOSE_PROFILES:-test_data}"
+export DOCKER_CLI_HINTS=false
+
+BOLD='\033[1m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+RED='\033[0;31m'
+BLUE='\033[0;34m'
+NC='\033[0m'
+
+START_SECONDS=$SECONDS
+
+step() { echo -e "${YELLOW}▶ $*${NC}"; }
+ok() { echo -e "${GREEN}✓ $*${NC} ${BLUE}($((SECONDS - START_SECONDS))s)${NC}"; }
+fail() { echo -e "${RED}✗ $*${NC}"; }
+
+# The address people open in their browser. In a codespace that's the forwarded
+# *.app.github.dev URL; anywhere else it's localhost.
+ui_url() {
+    if [ -n "${CODESPACE_NAME:-}" ]; then
+        echo "https://${CODESPACE_NAME}-5002.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev}/"
+    else
+        echo "http://localhost:5002/"
+    fi
+}
+
+wait_for_docker() {
+    local attempt
+    for attempt in $(seq 1 60); do
+        docker info >/dev/null 2>&1 && return 0
+        sleep 2
+    done
+    fail "Docker didn't start inside the codespace."
+    return 1
+}
+
+# wait_for <description> <timeout seconds> <command...>
+wait_for() {
+    local what=$1 timeout=$2
+    shift 2
+    local deadline=$((SECONDS + timeout))
+    step "Waiting for $what..."
+    until "$@" >/dev/null 2>&1; do
+        if [ $SECONDS -ge $deadline ]; then
+            fail "$what isn't ready after ${timeout}s."
+            return 1
+        fi
+        sleep 3
+    done
+    ok "Ready: $what"
+}
+
+api_ready() { curl -fsS http://localhost:5004/_health; }
+ui_ready() { curl -fsS http://localhost:5002/ && curl -fsS http://localhost:5002/api/config; }
+
+# True once Druid has ingested an event from the last few minutes, i.e. data is flowing
+# now (after a resume, older events are already there).
+events_ready() {
+    local count
+    count=$(curl -fsS -X POST -H 'Content-Type: application/json' \
+        -d '{"query":"SELECT COUNT(*) AS n FROM \"osprey.execution_results\" WHERE __time > CURRENT_TIMESTAMP - INTERVAL '"'"'5'"'"' MINUTE","resultFormat":"array"}' \
+        http://localhost:8082/druid/v2/sql 2>/dev/null | tr -dc '0-9')
+    [ -n "$count" ] && [ "$count" -gt 0 ]
+}
+
+# The UI with the last day of events already queried, grouped by user.
+demo_url() {
+    local start end
+    start=$(date -u -d yesterday +%Y-%m-%dT00:00:00.000Z)
+    end=$(date -u -d tomorrow +%Y-%m-%dT00:00:00.000Z)
+    echo "$(ui_url)?start=${start}&end=${end}&interval=day&queryFilter=&topn=UserId"
+}
+
+print_ready_banner() {
+    echo
+    echo -e "${GREEN}${BOLD}==============================================${NC}"
+    echo -e "${GREEN}${BOLD}  Osprey is ready${NC}"
+    echo -e "${GREEN}${BOLD}==============================================${NC}"
+    echo
+    echo -e "  Open the UI:  ${BOLD}$(ui_url)${NC}"
+    echo -e "  (or the ${BOLD}Ports${NC} tab → ${BOLD}Osprey UI${NC} → globe icon)"
+    echo -e "  Then click ${BOLD}Submit Query${NC} to load the last day of events."
+    echo
+    echo "  Or open it with that query already run:"
+    echo "  $(demo_url)"
+    echo
+    echo "  Synthetic posts arrive about once a second. Posts containing"
+    echo "  \"hello\" fire the ContainsHello rule. To see only those, query:"
+    echo -e "  ${BOLD}RegexMatch(target=PostText, pattern=\"hello\")${NC}"
+    echo
+    echo "  Guide: docs/development/codespaces.md"
+    echo
+}

.devcontainer/scripts/prepare.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+# updateContentCommand: download and build every image the demo needs, so that
+# starting the stack later is quick. Runs once when the codespace (or its prebuild)
+# is created, and again when a prebuild picks up new commits.
+set -euo pipefail
+source "$(dirname "$0")/lib.sh"
+
+echo -e "${BOLD}Preparing the Osprey demo. The first time, this takes several minutes.${NC}"
+wait_for_docker
+
+# Most images come from Docker Hub, which rate-limits anonymous pulls per IP address.
+# Signing in (from Codespaces secrets, if you've set them) avoids the limit.
+if [ -n "${DOCKERHUB_USERNAME:-}" ] && [ -n "${DOCKERHUB_TOKEN:-}" ]; then
+    echo "$DOCKERHUB_TOKEN" | docker login --username "$DOCKERHUB_USERNAME" --password-stdin >/dev/null \
+        && ok "Signed in to Docker Hub as $DOCKERHUB_USERNAME"
+fi
+
+pull_failed() {
+    fail "Couldn't download or build the demo's images."
+    echo "  If the error above mentions \"rate limit\" or \"toomanyrequests\", Docker Hub is"
+    echo "  limiting anonymous downloads. Wait an hour and rebuild the codespace, or add"
+    echo "  DOCKERHUB_USERNAME and DOCKERHUB_TOKEN as Codespaces secrets."
+    echo "  More help: docs/development/codespaces.md#troubleshooting"
+    exit 1
+}
+
+step "Downloading service images (Kafka, Druid, Postgres, ...)"
+docker compose pull --ignore-buildable --quiet || pull_failed
+ok "Images downloaded"
+
+step "Building the Osprey worker, API and UI images"
+docker compose build --quiet || pull_failed
+ok "Images built"
+
+echo -e "${GREEN}Preparation finished. The demo starts next.${NC}"

.devcontainer/scripts/start.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# postStartCommand: start the demo stack and wait until the UI has data to show.
+# Runs on every start, including when a stopped codespace resumes.
+set -uo pipefail
+source "$(dirname "$0")/lib.sh"
+
+on_failure() {
+    echo
+    fail "The Osprey demo didn't start cleanly."
+    echo "  Try: Ctrl+Shift+P (Cmd+Shift+P on a Mac) → \"Codespaces: Rebuild Container\"."
+    echo "  To see which service is failing, run:  docker compose ps -a"
+    echo "  More help: docs/development/codespaces.md#troubleshooting"
+    exit 1
+}
+
+echo -e "${BOLD}Starting the Osprey demo. This takes 1-3 minutes.${NC}"
+echo "The Osprey UI opens in a new browser tab once it's ready; this terminal"
+echo "keeps showing progress."
+echo
+
+wait_for_docker || on_failure
+
+step "Starting services"
+docker compose up -d --quiet-pull || on_failure
+ok "Services started"
+
+wait_for "the Osprey API" 300 api_ready || on_failure
+wait_for "the Osprey UI" 300 ui_ready || on_failure
+# Druid needs a minute or two to start ingesting; the UI works meanwhile, its
+# charts are just empty. A slow first event isn't a failure.
+if ! wait_for "the first demo events in the charts" 300 events_ready; then
+    echo "  The UI works, but its charts stay empty until events arrive."
+    echo "  If they haven't after a few minutes, see the troubleshooting guide."
+fi
+
+print_ready_banner

.devcontainer/scripts/status.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+# postAttachCommand: tell whoever just connected what state the demo is in.
+source "$(dirname "$0")/lib.sh"
+
+if ! docker info >/dev/null 2>&1 || ! api_ready >/dev/null 2>&1 || ! ui_ready >/dev/null 2>&1; then
+    echo -e "${YELLOW}${BOLD}The Osprey demo is still starting.${NC}"
+    echo "Progress is shown in the \"postStartCommand\" terminal (or View Creation Log)."
+    echo -e "When it's ready, the UI opens at ${BOLD}$(ui_url)${NC}"
+    exit 0
+fi
+print_ready_banner

.github/dependabot.yml

@@ -25,3 +25,13 @@ updates:
       github-actions:
         patterns:
           - "*"
+
+  - package-ecosystem: "devcontainers"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    cooldown:
+      default-days: 7
+    labels:
+      - "dependencies"
+      - "dependabot"

AGENTS.md

@@ -42,6 +42,10 @@ docker compose up -d
 # with coordinator:
 ./start.sh --with-coordinator
 
+# Same stack inside the devcontainer GitHub Codespaces uses (.devcontainer/; Docker-in-Docker,
+# adds .devcontainer/docker-compose.codespaces.yaml and the test-data producer)
+npx @devcontainers/cli up --workspace-folder .
+
 # UI dev server (Corepack auto-resolves pnpm from osprey_ui/package.json's packageManager field)
 cd osprey_ui && corepack enable && pnpm install --frozen-lockfile && pnpm start
 
@@ -51,6 +55,8 @@ cd osprey_ui && corepack enable && pnpm install --frozen-lockfile && pnpm start
 
 UI: http://localhost:5002 · UI API: http://localhost:5004 · Worker (port 5001)
 
+The UI dev server proxies `/api` to the UI API (`OSPREY_UI_API_PROXY_TARGET`, default `http://localhost:5004`); the devcontainer sets `REACT_APP_API_BASE_URL=/api` so the browser only talks to port 5002. If you change how the UI reaches the API, keep the Codespaces path in `.devcontainer/` and `docs/development/codespaces.md` working.
+
 ## Testing
 
 Run the full integration suite (spins up all services via docker compose; ~8 GB RAM):

CHANGELOG.md

@@ -8,6 +8,10 @@ For more information about each release including git tags and artifacts, see [R
 
 ## [Unreleased]
 
+### Added
+
+- One-click demo in GitHub Codespaces, with sample data flowing ([#593](https://github.com/roostorg/osprey/pull/593) by [@reitblatt](https://github.com/reitblatt))
+
 ### Fixed
 
 - `osprey-cli` no longer crashes at import with gevent's "cannot release un-acquired lock" ([#574](https://github.com/roostorg/osprey/pull/574) by [@reitblatt](https://github.com/reitblatt))

README.md

@@ -22,6 +22,10 @@ If you have [Docker Compose](https://docs.docker.com/compose/install), get a dem
 curl -sSL https://raw.githubusercontent.com/roostorg/osprey/main/demo.sh | bash
 ```
 
+Or try it in your browser with nothing to install: open the demo in GitHub Codespaces, wait a few minutes, and the Osprey UI opens with sample data flowing. [Step-by-step guide](https://roostorg.github.io/osprey/latest/development/codespaces.html).
+
+[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/roostorg/osprey?quickstart=1)
+
 See the [Getting Started guide](https://roostorg.github.io/osprey/latest/development/) for more, including what to try in the UI. See the [full documentation](https://roostorg.github.io/osprey/latest/) for a user guide, development setup, basic concepts, integration information, and more.
 
 ## Adopters

docs/README.md

@@ -16,7 +16,7 @@ The docs are versioned; other versions are available at the [documentation site
 
 [![The Query page with an SML filter for post-creation events and its history of past queries, beside a timeseries chart showing matching event volume in fifteen-minute buckets](images/query-and-charts.png)](images/query-and-charts.png)
 
-The fastest way to get a feel for Osprey is the one-command demo in [Getting Started](development/), which brings up the full stack on live sample data in a few minutes.
+The fastest way to get a feel for Osprey is the one-command demo in [Getting Started](development/), which brings up the full stack on live sample data in a few minutes. To skip installing anything, [open it in GitHub Codespaces](development/codespaces.md) instead.
 
 ## Contributing