feat: execution-aware standalone aggregation core #766 (partial)

CHANGELOG.md

@@ -9,12 +9,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **Standalone manifest aggregation (unreleased v0.41 source preview, #766)** -
+  Aggregate explicit public completed manifest/evidence references without
+  agents, credential helpers or private worker ownership. Aggregate schema 1.0
+  retains separate member/manifest/execution-record populations, repeated
+  executions, compatible views, original failed/cancelled outcomes and explicit
+  omissions/conflicts. Local CLI/API/Lab composition and canonical JSON exports
+  distinguish admission completeness from pass/attestation and withhold partial
+  or cached historical usage. No automatic migration invents execution records,
+  evidence or billing. See [ordering, compatibility and migration](docs/MANIFEST_AGGREGATION.md).
+
 - **Checkpoint embedding API (unreleased partial #765)** - Recover immutable
   before-unit/finalized-unit checkpoints and compatibility-checked restore atop
   the attempt core. Finalized restore reuses the original terminal; before-unit
   restore rejects a recorded dispatch even without completion, preventing
-  uncertain execution from silently rerunning. Public CLI/API/Lab and full
-  production-CLI/platform acceptance remain pending. See
+  uncertain execution from silently rerunning. Source commands `runner checkpoint
+  create` and `runner checkpoint resume` emit capture-approved results. Protected
+  HTTP operations and the source Lab preserve original outcomes, unavailable
+  progress, and exact JSON exports without raw source inspection. Linux clean-CLI
+  boundary-kill/fresh-process controls cover offline mock work; full roadmap,
+  native-platform acceptance, and independent approval remain pending. See
   [checkpoint contracts and migration limits](docs/EXECUTION_CHECKPOINTS.md).
 
 - **Logical execution-attempt embedding API (unreleased partial #764)** - Recover

README.md

@@ -30,7 +30,10 @@ adds immutable reservations and explicit verified reuse without changing
 `runner run`. Public submission commands and Lab lineage integration remain
 pending; this is not available in installed v0.40.1 artifacts.
 The [checkpoint embedding preview](docs/EXECUTION_CHECKPOINTS.md) adds compatible
-whole-unit recovery; its public CLI/API/Lab integration is also pending.
+whole-unit recovery. Its unreleased source CLI adds `runner checkpoint create`
+and `runner checkpoint resume`, with protected HTTP create/resume operations.
+The source Lab can create/resume checkpoints and export protected results.
+Ordered evaluation-level lineage and full platform acceptance remain pending.
 
 **v0.40.0 adds** bounded
 runner/text execution, selected AgentV assertion conversion, task-level MCP
@@ -291,6 +294,26 @@ canonical workflow, target-specific validation steps, and a reusable delegation
 
 ## Commands
 
+### `waza aggregate` (unreleased v0.41 source preview)
+
+Aggregate explicit completed public manifests from an existing local immutable
+store without starting agents or requiring the original worker:
+
+```bash
+waza aggregate --store .waza-artifacts "$DIGEST" -o aggregate.json
+waza aggregate --store .waza-artifacts --inputs inputs.json -o qualified.json
+```
+
+Input reference pairs, verified manifests and verified execution records are
+separate populations. Repeated executions and compatible views are retained;
+missing/conflicting inputs remain in emitted reports and exit nonzero.
+Complete admission is not a passing task or execution attestation. Historical
+usage is unavailable rather than a partial or cached subtotal. Lab's
+**Runner** view's **Aggregate completed public manifests** panel exposes composition,
+diagnostics and the exact JSON download.
+See [worked examples](site/src/content/docs/guides/manifest-aggregation.mdx) and
+[ordering, compatibility and migration](docs/MANIFEST_AGGREGATION.md).
+
 ### `waza artifact` (unreleased v0.41 source preview)
 
 Publish immutable artifact manifests without executing agents. The opt-in store
@@ -349,7 +372,7 @@ waza runner run < examples/runner/unit.json
 
 The v0.40.0 release implements `negotiation`, `exact-unit`, `inline.mock-text`,
 `terminal-result` and opt-in `inline.copilot-text`.
-Only `runner run` starts an engine; each dispatch runs its selected task/trial/
+In v0.40.0, only `runner run` starts an engine; each dispatch runs its selected task/trial/
 attempt once, without discovery, cache or retries. Admitted receipts include
 versioned [logical identities](site/src/content/docs/guides/runner-identities.mdx)
 derived from the original captured input, not proof of execution or deduplication.
@@ -392,6 +415,13 @@ The task entry must be eval-relative, and successful adaptation passes shared
 offline admission within the complete 4 MiB emitted-byte limit.
 See the [compatibility table and before/after examples](site/src/content/docs/guides/runner-compatibility.mdx).
 
+**Unreleased checkpoint source preview:** `waza runner checkpoint create --store
+.waza-artifacts < submission.json` publishes whole-unit checkpoints;
+`waza runner checkpoint resume CHECKPOINT_SHA256 --store .waza-artifacts <
+submission.json` admits compatible recovery. These commands require a clean
+VCS-stamped CLI and a captured submission wrapper, not a bare runner request.
+See the [checkpoint workflow and limits](site/src/content/docs/guides/immutable-artifacts.mdx#checkpoint-public-source-preview).
+
 The [local container reference](site/src/content/docs/guides/runner-container.mdx)
 keeps the default scratch image mock-only and provides an explicit
 `native-reference` target for the exact-unit native Copilot text profile. The

cmd/waza/cmd_aggregate.go

@@ -0,0 +1,94 @@
+package main
+
+import (
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+
+	"github.com/microsoft/waza/internal/artifactmanifest"
+	"github.com/microsoft/waza/internal/manifestaggregate"
+	"github.com/spf13/cobra"
+)
+
+func newAggregateCommand() *cobra.Command {
+	var directory, inputPath, outputPath string
+	cmd := &cobra.Command{
+		Use:          "aggregate [MANIFEST_SHA256...]",
+		Short:        "Aggregate public completed manifests without agents or worker state",
+		SilenceUsage: true,
+		Long:         "Aggregate explicit references from an existing local immutable store. Optional evidence sidecars qualify recorded categories and execution records. Admission completeness is not a pass verdict or execution attestation. Recorded historical usage is not new SDK billing.",
+		RunE: func(cmd *cobra.Command, args []string) error {
+			inputs := make([]manifestaggregate.Input, 0, len(args))
+			if inputPath != "" {
+				if len(args) != 0 {
+					return fmt.Errorf("%w: --inputs and positional digests are mutually exclusive", manifestaggregate.ErrInvalid)
+				}
+				data, err := readArtifactFile(inputPath, artifactmanifest.MaxManifestBytes)
+				if err != nil {
+					return err
+				}
+				inputs, err = manifestaggregate.ParseInputs(data)
+				if err != nil {
+					return err
+				}
+			} else {
+				for _, digest := range args {
+					inputs = append(inputs, manifestaggregate.Input{Manifest: digest})
+				}
+			}
+			inputs, err := manifestaggregate.Normalize(inputs)
+			if err != nil {
+				return err
+			}
+			sources, closeSources, err := manifestaggregate.OpenLocal(directory)
+			if err != nil {
+				return err
+			}
+			report, buildErr := manifestaggregate.Build(cmd.Context(), sources, inputs, manifestaggregate.DefaultLimits())
+			closeErr := closeSources()
+			if closeErr != nil {
+				// Do not emit a success-shaped report after a local close failure.
+				return errors.Join(buildErr, fmt.Errorf("closing aggregate sources: %w", closeErr))
+			}
+			if report == nil {
+				return buildErr
+			}
+			wire, err := manifestaggregate.Encode(*report)
+			if err != nil {
+				return errors.Join(buildErr, err)
+			}
+			if outputPath != "" {
+				if err := writeAggregateOutput(outputPath, wire); err != nil {
+					return errors.Join(buildErr, err)
+				}
+			}
+			_, emitErr := cmd.OutOrStdout().Write(append(wire, '\n'))
+			return errors.Join(buildErr, emitErr)
+		},
+	}
+	cmd.Flags().StringVar(&directory, "store", ".waza-artifacts", "Existing local immutable store (never reads ambient configuration or starts credentials)")
+	cmd.Flags().StringVar(&inputPath, "inputs", "", "JSON array of manifest_sha256 and optional evidence_sha256 reference pairs")
+	cmd.Flags().StringVarP(&outputPath, "output", "o", "", "Create a new canonical aggregate JSON file; existing files are never replaced")
+	return cmd
+}
+
+func writeAggregateOutput(path string, wire []byte) error {
+	root, err := os.OpenRoot(filepath.Dir(path))
+	if err != nil {
+		return fmt.Errorf("opening aggregate output parent: %w", err)
+	}
+	file, err := root.OpenFile(filepath.Base(path), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
+	if err != nil {
+		return errors.Join(fmt.Errorf("creating aggregate output: %w", err), root.Close())
+	}
+	written, writeErr := file.Write(wire)
+	if writeErr == nil && written != len(wire) {
+		writeErr = io.ErrShortWrite
+	}
+	if err := errors.Join(writeErr, file.Close(), root.Close()); err != nil {
+		return fmt.Errorf("aggregate output may be partial at %q: %w", path, err)
+	}
+	return nil
+}

cmd/waza/cmd_aggregate_offline_test.go

@@ -0,0 +1,110 @@
+package main
+
+import (
+	"bytes"
+	"errors"
+	"net/http"
+	"os"
+	"path/filepath"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/microsoft/waza/internal/artifactmanifest"
+	"github.com/microsoft/waza/internal/manifestaggregate"
+	"github.com/microsoft/waza/internal/testutil/manifestfixture"
+	"github.com/spf13/cobra"
+	"github.com/stretchr/testify/require"
+)
+
+type aggregateNetworkGuard struct {
+	calls     atomic.Int64
+	contacted chan struct{}
+}
+
+func (g *aggregateNetworkGuard) RoundTrip(*http.Request) (*http.Response, error) {
+	g.calls.Add(1)
+	select {
+	case g.contacted <- struct{}{}:
+	default:
+	}
+	return nil, errors.New("network forbidden in aggregate offline control")
+}
+
+func TestAggregateUpdateCheckOnlyExcludesExactCommandAndAncestors(t *testing.T) {
+	t.Setenv("WAZA_NO_UPDATE_CHECK", "")
+	root := newRootCommand()
+	for _, control := range []struct {
+		name string
+		want bool
+	}{{"aggregate", false}, {"models", true}, {"results", true}, {"artifact", true}, {"runner", false}, {"preflight", false}} {
+		command, _, err := root.Find([]string{control.name})
+		require.NoError(t, err)
+		require.Equal(t, control.name, command.Name())
+		require.Equal(t, control.want, shouldRunUpdateCheck(command, false), control.name)
+	}
+	aggregate, _, err := root.Find([]string{"aggregate"})
+	require.NoError(t, err)
+	child := &cobra.Command{Use: "child"}
+	aggregate.AddCommand(child)
+	require.False(t, shouldRunUpdateCheck(child, false))
+	require.True(t, shouldRunUpdateCheck(&cobra.Command{Use: "aggregate-other"}, false))
+	require.True(t, shouldRunUpdateCheck(root, false))
+}
+
+func TestAggregateRegisteredRootIsOfflineWithoutUpdateOptOut(t *testing.T) {
+	directory := t.TempDir()
+	store := filepath.Join(directory, "store")
+	require.NoError(t, os.Mkdir(store, 0o700))
+	backend, err := artifactmanifest.OpenLocal(store)
+	require.NoError(t, err)
+	source := manifestfixture.Publish(t, backend, "../../examples/runner/unit.json", nil, manifestfixture.RecordedNonzero(1))
+	require.NoError(t, backend.Close())
+	t.Chdir(directory)
+	require.NoError(t, os.WriteFile(".waza.yaml", []byte("invalid: [ambient configuration must not be read"), 0o600))
+	home := t.TempDir()
+	t.Setenv("HOME", home)
+	t.Setenv("USERPROFILE", home)
+	t.Setenv("WAZA_NO_UPDATE_CHECK", "")
+	oldVersion := version
+	version = "0.1.0"
+	t.Cleanup(func() { version = oldVersion })
+	guard := &aggregateNetworkGuard{contacted: make(chan struct{}, 1)}
+	transport := http.DefaultTransport
+	http.DefaultTransport = guard
+	t.Cleanup(func() { http.DefaultTransport = transport })
+	output := filepath.Join(directory, "saved.json")
+	root := newRootCommand()
+	var stdout, stderr bytes.Buffer
+	root.SetOut(&stdout)
+	root.SetErr(&stderr)
+	root.SetArgs([]string{"aggregate", "--store", store, source.Manifest, "-o", output})
+	require.NoError(t, root.Execute())
+	require.False(t, root.PersistentFlags().Lookup("no-update-check").Changed)
+	require.Equal(t, "false", root.PersistentFlags().Lookup("no-update-check").Value.String())
+	saved, err := os.ReadFile(output)
+	require.NoError(t, err)
+	require.Equal(t, append(bytes.Clone(saved), '\n'), stdout.Bytes())
+	report, err := manifestaggregate.Parse(saved)
+	require.NoError(t, err)
+	require.Equal(t, "complete", report.State)
+	require.Equal(t, 1, report.Counts.VerifiedManifests)
+	require.Equal(t, source.Manifest, report.Members[0].Manifest)
+	require.Empty(t, stderr.String())
+	require.Zero(t, guard.calls.Load(), "registered root must not start its update checker")
+	require.NoDirExists(t, filepath.Join(home, ".waza"))
+
+	// Prove the same unsuppressed root hook can reach the rejecting transport.
+	control := newRootCommand()
+	models, _, err := control.Find([]string{"models"})
+	require.NoError(t, err)
+	require.True(t, shouldRunUpdateCheck(models, false))
+	control.PersistentPreRun(models, nil)
+	select {
+	case <-guard.contacted:
+	case <-time.After(5 * time.Second):
+		t.Fatal("released-version, fresh-home update-check positive control did not reach transport")
+	}
+	control.PersistentPostRun(models, nil)
+	require.EqualValues(t, 1, guard.calls.Load())
+}