diff --git a/SUMMARY.md b/SUMMARY.md index 895cc21..1985087 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -80,6 +80,7 @@ * [Creating a preview](verify/how-to-guides/creating-a-preview.md) * [Managing previews](verify/how-to-guides/managing-previews.md) * [Seed data for previews](verify/how-to-guides/seed-data-for-previews.md) + * [Registering an external preview](verify/how-to-guides/registering-an-external-preview.md) * [Running with remote agents](verify/how-to-guides/running-with-remote-agents.md) * [Configuring branch protection](verify/how-to-guides/configuring-branch-protection.md) * [Fixing verification failures](verify/how-to-guides/fixing-verification-failures.md) diff --git a/verify/concepts/previews.md b/verify/concepts/previews.md index e8d2757..bf499cf 100644 --- a/verify/concepts/previews.md +++ b/verify/concepts/previews.md @@ -4,6 +4,8 @@ A **preview** is an ephemeral environment Verify builds on demand and runs scena A preview is short-lived: built for the branch's current commit, used by the scenario runner (and optionally a human reviewer), and torn down when the branch moves to a new commit. A re-run on the same commit reuses the running preview — including its state. See [Lifecycle](#lifecycle) for the exact contract. +Aviator can build the environment itself — the **sandbox** method, described through most of this page — or use one your own CI deploys, the **endpoint** method. See [Externally hosted previews](#externally-hosted-previews). + ### Optional, not required Previews are optional. Verify works on day one with code-scan alone — without a preview, every criterion is routed to code-scan (static analysis of the diff) and you get verdicts on structural criteria from the first PR. @@ -43,6 +45,33 @@ A preview is composed of inputs from three places: a preview image, your secret Aviator stitches these together into a single ephemeral container. Your Verify settings are the contract — see [Preview YAML reference](../reference/preview-yaml.md) for every field. +### Externally hosted previews + +Many teams already deploy an ephemeral environment for every pull request from their own CI. Rather than have Aviator build a second one, a preview can be declared as **externally hosted** — `method: endpoint` in the config. The environment stays yours; Aviator only borrows it. + +The split of responsibility: + +| Your CI does | Aviator does | +| ---------------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| Builds and deploys the environment for the pull request | Correlates the registration to the verification session for that PR or branch | +| Registers its URL (and the commit it's running) with Aviator | Waits for a registration when a run needs one | +| Destroys the environment on its own schedule | Drives the app at the registered URL and records the evidence | +| Optionally deregisters early when it tears the environment down | Stops using the registration at its expiry | + +Aviator never builds or tears down an externally hosted preview, and never holds infrastructure credentials for it — the environment is already running by the time Aviator sees it. + +**Waiting, then degrading.** Verification and the deploy pipeline run on their own clocks, so a run that needs the preview waits for one to be registered, up to a configured timeout. If the registration arrives in time, scenarios run against it. If it doesn't, verification still completes on code analysis alone — a degraded result rather than a failure — and a registration that lands afterwards starts a follow-up verification automatically. A registration also has to match the commit being verified: one built from a stale commit never drives the run. + +**Choosing between the two methods:** + +| Choose sandbox when… | Choose endpoint when… | +| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ | +| You don't already build per-PR environments and don't want to | You already run per-PR ephemeral environments in CI | +| The service boots from an image plus a setup script | The environment needs infrastructure Aviator can't host — managed data stores, private networks, a multi-service topology | +| You want previews with no CI work at all | You want verification to run against the same environment your team already reviews | + +See [Registering an external preview](../how-to-guides/registering-an-external-preview.md) for the registration call and its semantics. + ### Multiple previews per repo A repo can have more than one preview configured. Common patterns: @@ -86,3 +115,4 @@ Previews look like CI environments but they're not the same thing: * [Creating a preview](../how-to-guides/creating-a-preview.md) — walkthrough * [Managing previews](../how-to-guides/managing-previews.md) — bake vs. setup, refresh, cleanup * [Seed data for previews](../how-to-guides/seed-data-for-previews.md) — fixtures and deterministic state +* [Registering an external preview](../how-to-guides/registering-an-external-preview.md) — using an environment your CI deploys diff --git a/verify/how-to-guides/creating-a-preview.md b/verify/how-to-guides/creating-a-preview.md index 605d5ed..f40b7b2 100644 --- a/verify/how-to-guides/creating-a-preview.md +++ b/verify/how-to-guides/creating-a-preview.md @@ -4,6 +4,8 @@ This guide walks through adding a preview to a repo end-to-end: registering an i By the end, you'll have a working `default` preview that scenarios can run against. +> This guide covers previews Aviator builds and runs. If your CI already deploys an ephemeral environment for every pull request, you can point Verify at that instead — see [Registering an external preview](registering-an-external-preview.md). + > Previews are optional. Verify works without one — every criterion routes to code-scan in that case. Add a preview when behavioral verdicts (endpoint contracts, UI flows, error shapes) start mattering for your team. See [Concepts: Previews](../concepts/previews.md). **Time:** ~15 minutes diff --git a/verify/how-to-guides/registering-an-external-preview.md b/verify/how-to-guides/registering-an-external-preview.md new file mode 100644 index 0000000..56b1879 --- /dev/null +++ b/verify/how-to-guides/registering-an-external-preview.md @@ -0,0 +1,160 @@ +# Registering an external preview + +Some teams already deploy an ephemeral environment for every pull request from their own CI. An **endpoint preview** lets Verify use that environment instead of building one: your pipeline deploys the app as it always has, then registers the resulting URL with Aviator, and verification runs against it. + +Aviator never builds or tears down an endpoint preview. It correlates the registration to the verification session, waits for it when a run needs it, drives the app at that URL, and stops using it when the registration expires. + +For the alternative — Aviator booting a container from a preview image — see [Creating a preview](creating-a-preview.md). For the concept, see [Concepts: Previews](../concepts/previews.md). + +**Time:** ~20 minutes + +**Prerequisites:** + +* The repo connected to Aviator — see [Connect a repository](connect-a-repository.md) +* A CI pipeline that already deploys a reachable per-PR environment +* An Aviator API token — see [API authentication](../../api/reference/authentication.md) + +### Step 1: Declare an endpoint preview + +Endpoint previews are configured in your **Aviator Verify settings** (**Verify → Settings → Verify**, with the repo selected). Add a `preview` entry with `method: endpoint`: + +```yaml +verify: + preview: + - name: ci + method: endpoint + wait_timeout_sec: 1800 # how long a run waits for a registration + expires_default_sec: 7200 # registration lifetime when a registration omits expires_at +``` + +Endpoint entries take none of the sandbox fields (`image`, `port`, `setup`, `teardown`, `secrets`) — the environment isn't Aviator's to build. See [Preview YAML reference](../reference/preview-yaml.md) for the full schema. + +Registrations are rejected until this entry exists: a repo with no `method: endpoint` preview configured returns `400`. + +### Step 2: Register the preview from CI + +Make the registration the **last step of the deploy job**, after the environment is deployed and reachable. Registering earlier means Verify may start driving a URL that isn't serving yet. + +`POST https://api.aviator.co/api/v1/verify/preview-environment` + +Authenticate with `Authorization: Bearer `, the same as the rest of the [REST API](../../api/reference/json-api.md). + +#### Request body + +| Field | Type | Required | Description | +| --------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------ | +| `repository` | object | yes | `{"org": "...", "name": "..."}` — the repository the preview was built from. | +| `branch` | string | yes | The branch the preview was built from. Used to correlate the registration when no PR number is supplied. | +| `pr_number` | integer | no | The pull request number. Preferred: when present it takes precedence over `branch` for correlation. | +| `commit_sha` | string | yes | The commit the environment is running. A registration only drives verification while this matches the head of the branch being verified. | +| `preview_url` | string | yes | Absolute `http(s)` base URL of the live environment. Must be reachable from Aviator. | +| `secrets` | object | no | Credentials for driving the app as a signed-in test user. See [Credentials](#credentials-for-signing-in). | +| `expires_at` | string | no | ISO 8601 timestamp. When omitted, the config's `expires_default_sec` applies. | + +`branch` is required even when you send `pr_number`, because every CI job knows its branch — including jobs that run before a pull request exists. + +#### Responses + +| Status | Meaning | +| ------ | -------------------------------------------------------------------------------------------------------------- | +| `202` | Registration accepted. The body carries `registration_id`, `status`, `url`, `message`, and `expires_at`. | +| `400` | Invalid payload — including a repo whose Verify config declares no `method: endpoint` preview. | +| `401` | Missing or invalid API token. | +| `404` | Repository not found for this account. | + +A `202` body looks like: + +```json +{ + "registration_id": 412, + "status": "attached", + "url": "https://app.aviator.co/r/1042", + "message": "Preview registered and attached to this Verify session.", + "expires_at": "2026-09-05T18:00:00+00:00" +} +``` + +`status` tells you what the registration found: + +* **`attached`** — a verification session already existed for this pull request or branch, and the registration is now driving it. `url` links to that session. +* **`buffered`** — the registration arrived before the session existed. It's held and attaches automatically once the session is created, so CI outrunning session creation is expected and safe. + +Registrations are append-only: re-POSTing records a new registration and the newest one wins, so blind CI retries are safe. + +### Step 3: Add the CI step + +A GitHub Actions example — adapt the shell and variables to your provider: + +```yaml +- name: Register preview with Aviator + run: | + curl -sS -X POST https://api.aviator.co/api/v1/verify/preview-environment \ + -H "Authorization: Bearer $AVIATOR_API_TOKEN" \ + -H "Content-Type: application/json" \ + -d @- <Preview anatomy: inputs, the preview container, and what consumes it

How the YAML fields translate to a running preview

-### Examples +### Sandbox examples **Minimal:** @@ -127,8 +141,34 @@ verify: - QUEUE_URL ``` +### Endpoint previews (`method: endpoint`) + +An endpoint preview is hosted outside Aviator. Your CI pipeline builds and deploys the environment for the pull request and registers its URL with Aviator; Aviator correlates that registration to the verification session, waits for it when a run needs it, drives it, and honors its expiry. It never builds or tears the environment down. + +```yaml +verify: + preview: + - name: ci + method: endpoint + wait_timeout_sec: 1800 # how long a verification run waits for a registered preview + expires_default_sec: 7200 # registration lifetime when a registration omits expires_at +``` + +| Field | Type | Required | Description | +| --------------------- | ------ | -------- | ---------------------------------------------------------------------------------------------------------- | +| `name` | string | no | Unique name within the repo. Defaults to `default`. Scenarios target a preview by this name. | +| `method` | string | yes | Must be `endpoint`. | +| `wait_timeout_sec` | int | no | How long a verification run waits for a registered preview before falling back to code analysis. Defaults to 1800. Range 60–7200. | +| `expires_default_sec` | int | no | How long a registration stays current when it doesn't supply its own `expires_at`. Defaults to 7200. Range 300–86400. | +| `verify_skill` | string | no | Repo-relative path to this preview's [Verify skill](../how-to-guides/writing-a-skill-md.md) entry point. Overrides the default `.aviator/verify/skills/.md` lookup. | + +Endpoint entries take none of the sandbox fields — `image`, `port`, `setup`, `teardown`, and `secrets` are all rejected, because the environment isn't Aviator's to build. Credentials for driving the app come from the registration itself. + +Registering a preview is an API call from your CI pipeline. See [Registering an external preview](../how-to-guides/registering-an-external-preview.md) for the request shape, the waiting behavior, and the expiry and deregistration semantics. + ### See also * [Concepts: Previews](../concepts/previews.md) * [Creating a preview](../how-to-guides/creating-a-preview.md) * [Managing previews](../how-to-guides/managing-previews.md) +* [Registering an external preview](../how-to-guides/registering-an-external-preview.md)