REF-002: make storage v3 the only supported layout

CHANGELOG.md

@@ -25,8 +25,43 @@ All notable changes to this project will be documented in this file.
 - DOC-003: referee one-shot `prove-academic-review.sh` (Python only; Guide/Neo4j
   optional, not required).
 
+### Fixed
+
+- BUG-001: `sdlc-engine db export --format sql` no longer raises `NameError`;
+  `db_query.py` imports `utc_now`, the uncalled `context_linked_to_section`
+  (sole user of the graph constants) is removed, and an `export_sql`
+  round-trip test covers the CLI path (#308).
+- REF-002: unit, integration, E2E, and shell CI fixtures now create and inspect
+  framework state under `<root>/sdlc-spdd/`; installer configs and runtime
+  state resolve through `Project.sdlc_dir`.
+
+### Removed
+
+- REF-002: pre-v3 consolidation, layout migration, TSV registry fallback,
+  migration-only parsers and CLI verbs, duplicate-canvas synchronization, and
+  executable root-layout fallbacks. Storage v3 is the only runtime layout.
+
 ### Changed
 
+- Milestone 3 (one flow on storage v3) opened from SPIKE-005: Python
+  `sdlc-engine` is the only engine, storage v3 the only persistence model;
+  pre-v3 compatibility and the bash workflow twin are removed, not
+  maintained (#308).
+- CHORE-004: ruff selects full pyflakes (`F,E9`) in `pyproject`, CI, and the
+  suite; `check-complexity.py` runs against the PR base on pull requests; new
+  `shellcheck` workflow (`bash -n` + `-S error`) over scripts, installed
+  runtime templates, tests, and `install.sh` (#308).
+- REF-002 T01: `sdlc-engine archive` **removes** canvas, analysis, review,
+  sync, hot session briefs, and workflow `.state`, and appends the `archived`
+  registry event. No `spdd/*/archive/` folders and no `agent-context/sessions`
+  sweep; git history is the record. Requirements and `lessons.jsonl` are
+  untouched (#308).
+- REF-002: `Project.home` and the shell path helpers resolve only
+  `SDLC_HOME` or `<root>/sdlc-spdd`; ADF, local session, issue, roadmap,
+  playground, Guide, viewer, and integration runtime paths follow that home.
+- CHORE-006: `.cursor/commands`, `.claude/commands`, and `.github/prompts`
+  regenerated from `templates/`; `tests/test-command-specs.sh` diffs every slug
+  in all three dogfood packs (#308).
 - TEST-003 T05 close-out: canvas, requirement, and review match `main`
   (`e716100` / #259). System review
   `sdlc-spdd/spdd/reviews/academic-freeze-system-review.md` lists remaining
@@ -61,7 +96,8 @@ All notable changes to this project will be documented in this file.
   importable, even if `SDLC_ENGINE=shell`. `SDLC_GATE_ENGINE=shell` is the
   labeled pre-FEAT-014 substring fallback and is **not** an evaluation
   condition (`tests/test-sdlc-workflow.sh` uses it for hermetic bash CLI).
-  Python `archive` moves contracts under `project.home` (storage v3).
+  Python `archive` moved contracts under `project.home` (superseded by
+  REF-002 T01: archive deletes).
 - Ops console `/` defaults to the Vue3 build (`console-ui/dist`); `sdlc.sh console` builds dist when needed. The Flask HTML template (`installer/pages.py`) is removed — Flask remains the `/api/*` BFF and the ADF Viewer
 - `sdlc.sh console --playground` seeds a disposable SPDD tree under `.sdlc/console-playground` so Vue tabs can be clicked without installing into an app
 - Playground Guide / Jira / GitHub are in-process fakes (`installer/playground_fakes.py`) so those tabs show live UP/sync mocks without Docker, JVM, or tracker APIs

README.md

@@ -303,9 +303,6 @@ your-app/
   adf/                       Optional checked-in ADF JSON
 ```
 
-Legacy sprawled installs (framework folders at the repo root, `agent-context/`
-trees, `work-registry.tsv`) keep working read-only and are consolidated by
-`upgrade` + `sdlc-engine storage migrate`.
 
 | Path (this repo) | Purpose |
 | ---------------- | ------- |

STARTER-SPEC.md

@@ -247,7 +247,6 @@ Create the repo with this structure:
     │   ├── install-cursor-commands.sh
     │   ├── create-feature.sh
     │   ├── validate-reasons-canvas.sh
-    │   ├── sync-agent-context.sh
     │   └── detect-stack.sh
     ├── examples/
     │   ├── spring-boot-order-api/
@@ -1419,33 +1418,6 @@ Output should be written to:
 
     spdd/memory/lessons.jsonl
 
-### scripts/sync-agent-context.sh
-
-Purpose:
-
-Synchronize canonical canvas, feature workspace, progress logs, reviews, and memory.
-
-Usage:
-
-    ./scripts/sync-agent-context.sh --work-id FEAT-001-order-status-api
-
-Behavior:
-
-- Ensure feature folder exists.
-- Ensure canonical canvas exists.
-- Compare feature canvas and canonical canvas.
-- Report drift.
-- Optionally copy canonical canvas to feature folder.
-- Optionally copy feature canvas to canonical canvas.
-- Do not overwrite without explicit option.
-
-Options:
-
-    --from-canvas
-    --from-feature
-    --dry-run
-    --force
-
 ## GitHub Project Setup
 
 Create GitHub issues for the initial repo build.

TESTING.md

@@ -128,7 +128,6 @@ In orchestrator repo:
 
 - `validate-command-adapters` (`.github/workflows/validate-command-adapters.yml`)
 - `test-adapter-install` (`.github/workflows/test-adapter-install.yml`)
-- `test-upgrade-consolidate` (`.github/workflows/test-upgrade-consolidate.yml`) — storage v3 layout upgrade
 - `test-sdlc-pointer` (`.github/workflows/test-sdlc-pointer.yml`)
 - `test-sdlc-workflow` (`.github/workflows/test-sdlc-workflow.yml`)
 - `test-archive-work` (`.github/workflows/test-archive-work.yml`)
@@ -174,8 +173,8 @@ Copilot, Claude Code) into throwaway target directories and asserts:
 
 - Single-assistant installs (`--cursor`, `--copilot`, `--claude`) produce only
   that assistant's files and no others.
-- No-flag setup/upgrade keeps the legacy Cursor + Copilot default; Claude Code
-  is installed only with `--claude` or `--all`.
+- No-flag setup/upgrade installs Cursor + Copilot by default; Claude Code is
+  installed only with `--claude` or `--all`.
 - `--all` and `upgrade --all` install all three; Cursor and Copilot files stay
   byte-identical to their templates.
 - Upgrade preserves project-owned files such as an existing root `CLAUDE.md`
@@ -199,27 +198,6 @@ Run it locally before changing any install/upgrade script or command template.
 The CI workflow also runs `bash -n` over shell scripts before executing the
 regression harness.
 
-### Storage v3 upgrade consolidation harness
-
-`./tests/test-framework-install-consolidate.sh` unit-tests consolidate/archive
-helpers in `scripts/lib/framework-install.sh` (move/merge/dest-wins, dry-run,
-archive leftovers, orchestrator-vs-target agent-context handling, harness seed).
-
-`./tests/test-upgrade-consolidate.sh` is the end-to-end layout suite:
-
-- **A.** Pure legacy sprawl (no `sdlc-spdd/` yet) → single home; root stay-set gone
-- **B.** Dual layout merge when home already exists (destination wins conflicts)
-- **C.** Idempotent second upgrade
-- **D.** `--dry-run` + `--consolidate` no-op leave the tree untouched
-- **E.** Orchestrator-shaped target archives `agent-context/`; keeps root `scripts/`
-- **F.** Fresh v3 init/setup then upgrade preserves project content
-- **G.** Nested helper unit suite
-
-Leftover `agent-context/` trees must land under
-`sdlc-spdd/.sdlc/legacy-layout-archive/` (install source is
-`templates/agent-context/`). `verify-project-install.sh` must pass after each
-real upgrade. CI: `.github/workflows/test-upgrade-consolidate.yml`.
-
 ### SDLC pointer harness
 
 `./tests/test-sdlc-pointer.sh` exercises `templates/agent-context/sdlc-pointer.sh`:
@@ -237,7 +215,7 @@ real upgrade. CI: `.github/workflows/test-upgrade-consolidate.yml`.
 - Phase/gate tracking, `next`/`advance`/`skip`/`shelf`/`resume`/`sync`
 - `sdlc.sh` wrapper delegation
 - Guarded `capture` (pointer must match)
-- Team `claim`/`release`, stale TTL, branch/PR/Jira notes in `work-registry.tsv`
+- Team `claim`/`release`, stale TTL, branch/PR/Jira notes in `spdd/memory/registry.jsonl`
 - Jira Key auto-link from `requirements/milestones/<WORK-ID>.md` on claim
 
 ### Archive completed/cancelled work harness
@@ -287,7 +265,6 @@ throwaway targets and asserts:
 - `--work-id` loads canvas, analysis, tasks, and ledger progress excerpts (storage v3)
 - `--format json` returns paths
 - `start-agent-session.sh` embeds Resolved Context and avoids redundant resume prompts
-- legacy `playbooks/` + `extensions/` trees migrate idempotently into `harness/skills/`
 
 Run locally after changing `resolve-agent-context.sh`, `scripts/lib/skills.sh`,
 harness templates, or `start-agent-session.sh`.

docs/agent-session-scripts.md

@@ -18,7 +18,7 @@ They solve four operational needs:
 | `scripts/upgrade-project.sh` | Framework-only upgrade for older initialized projects without overwriting implementation files or existing memory |
 | `sdlc-spdd/scripts/sdlc.sh` | Workflow CLI: `next`, `claim`, `resume`, `advance`, `skip`, `shelf`, `sync`, `capture`, `accept`, `team`, `list-work`, `archive`, `gate`, `db`, `issues`, `local`, `quick` |
 | `sdlc-spdd/scripts/start-agent-session.sh` | Target-local script that creates a session brief for a new agent |
-| `sdlc-spdd/scripts/resync-agent-session.sh` | Target-local script that checks or reconciles the canonical canvas, validates it, and creates a session brief |
+| `sdlc-spdd/scripts/resync-agent-session.sh` | Target-local script that validates the canvas and creates a session brief |
 | `sdlc-spdd/scripts/capture-session-memory.sh` | Stage session summary, decisions/pitfalls/patterns, and optional metrics as lesson records in `.sdlc/staged/lessons.jsonl` |
 | `sdlc-spdd/scripts/accept-lessons.sh` | Promote staged lesson records into the committed `spdd/memory/lessons.jsonl` (also `sdlc.sh accept`) |
 | `sdlc-spdd/scripts/index-spdd-analysis.sh` | Stage an `analysis` lesson record from a Fowler analysis artifact |
@@ -27,7 +27,6 @@ They solve four operational needs:
 | `sdlc-spdd/scripts/create-work-from-milestone.sh` | Target-local script that maps milestone checklist items into SDLC-SPDD work artifacts |
 | `sdlc-spdd/scripts/sync-roadmap-from-spdd.sh` | Target-local script that refreshes a managed roadmap summary from canvas metadata |
 | `sdlc-spdd/scripts/summarize-session-notes.sh` | Target-local script that imports existing session notes as staged lesson records |
-| `sdlc-spdd/scripts/sync-agent-context.sh` | Target-local low-level canvas copy synchronization |
 | `sdlc-spdd/scripts/validate-command-adapters.sh` | Target-local checker that validates Cursor/Copilot/Claude Code command-pack parity in the installed project |
 | `sdlc-spdd/scripts/verify-agent-command-effects.sh` | Target-local verifier for deterministic artifact side-effects after `/sdlc-spdd-*` command invocations and post-capture planning sync |
 | `sdlc-spdd/scripts/validate-reasons-canvas.sh` | REASONS Canvas structure validation (+ optional readiness vocabulary) |
@@ -75,7 +74,7 @@ Preview first:
 
     ./scripts/upgrade-project.sh --target /path/to/app --all --dry-run
 
-The upgrade updates framework-owned prompts, harness/skills files, target-local docs under `docs/sdlc-spdd/`, and target-local runtime scripts. It preserves application source, application docs outside `docs/sdlc-spdd/`, requirements, canvases, reviews, sync logs, the lessons ledger, existing root `CLAUDE.md`, and target workflow customizations. Legacy memory layouts are converted by `sdlc-engine storage migrate` — see [Framework upgrade](framework-upgrade.md).
+The upgrade updates framework-owned prompts, harness/skills files, target-local docs under `docs/sdlc-spdd/`, and target-local runtime scripts. It preserves application source, application docs outside `docs/sdlc-spdd/`, requirements, canvases, reviews, sync logs, the lessons ledger, existing root `CLAUDE.md`, and target workflow customizations. See [Framework upgrade](framework-upgrade.md).
 
 ## 2. Start a New Agent Session
 
@@ -138,21 +137,14 @@ Check the canonical canvas and validate it:
 
     ./sdlc-spdd/scripts/resync-agent-session.sh --target . --work-id FEAT-001-order-status-api --check-only
 
-`--check-only` runs sync check and canvas validation, then stops. It does **not** create a session brief. Run `start-agent-session.sh` next.
+`--check-only` validates the canvas, then stops. It does **not** create a session brief. Run `start-agent-session.sh` next.
 
-### Reconcile drift (creates session brief)
+### Validate and create a session brief
 
-If drift exists, reconcile and create a session brief in one step. **Default:** canonical `spdd/canvas/<WORK-ID>.md` is authoritative:
+Without `--check-only`, resync validates the single canvas and creates a fresh
+session brief for the requested phase:
 
-    ./sdlc-spdd/scripts/resync-agent-session.sh --target . --work-id FEAT-001-order-status-api --from-canvas --force --phase code
-
-When reconciling, you do **not** need a separate `start-agent-session.sh` call — resync creates the brief.
-
-The reconcile path:
-
-1. Runs `sync-agent-context.sh`.
-2. Validates the canonical canvas.
-3. Creates a fresh session brief for the requested `--phase`.
+    ./sdlc-spdd/scripts/resync-agent-session.sh --target . --work-id FEAT-001-order-status-api --phase code
 
 ## 4. Capture Current Session Memory
 

docs/diagrams/02-container.puml

@@ -12,7 +12,7 @@ System_Boundary(repo, "Project repository") {
         Container(contracts, "Contracts (committed)", "Markdown", "requirements/ + ROADMAP.md, spdd/canvas|analysis|reviews|sync, harness/ playbooks/ extensions/")
         ContainerDb(ledger, "Lessons ledger + registry (committed)", "JSONL", "spdd/memory/lessons.jsonl - accepted records (decision, pitfall, pattern, session, analysis). spdd/memory/registry.jsonl - claim/release events")
         Container(scripts, "Workflow CLI", "Bash", "scripts/sdlc.sh: capture, accept, claim, release, db; start-agent-session digest; paths.sh home resolution")
-        Container(engine, "sdlc-engine", "Python", "context retrieve/show/digest/accept/parity, storage migrate, sqlite rebuild, guide projection client")
+        Container(engine, "sdlc-engine", "Python", "context retrieve/show/digest/accept/parity, sqlite rebuild, guide projection client")
         ContainerDb(runtime, ".sdlc/ runtime (gitignored)", "Files", "sessions/current-session.md hot brief, staged/lessons.jsonl captures, index.sqlite opt-in cache (schema v5)")
     }
 }

docs/diagrams/03-component-engine.puml

@@ -4,13 +4,12 @@
 title Components - sdlc-engine (Python, storage v3)
 
 Container_Boundary(engine, "sdlc-engine") {
-    Component(cli, "cli.py", "argparse CLI", "context retrieve|show|digest|accept|parity|persist-lesson, storage migrate|status, db, registry, workflow")
+    Component(cli, "cli.py", "argparse CLI", "context retrieve|show|digest|accept|parity|persist-lesson, db, registry, workflow")
     Component(project, "project.py", "Path resolution", "Project.home: SDLC_HOME > <root>/sdlc-spdd/ > root (legacy). All artifact paths derive from home")
     Component(ledgermod, "lessons_ledger.py", "System of record", "LessonRecord (5 kinds) + LessonsLedger: stage, accept, records, digest. Validates schema, dedupes ids")
     Component(store, "context_store.py", "Orchestration", "persist_lesson stages by default; accept promotes; fan-out to sqlite + guide; parity check/repair")
-    Component(registry, "registry.py", "Work registry", "Append-only claim/release events in registry.jsonl; legacy TSV read-only fallback")
+    Component(registry, "registry.py", "Work registry", "Append-only claim/release events in registry.jsonl (only store)")
     Component(db, "db.py", "Opt-in cache", "sqlite schema v5 at .sdlc/index.sqlite; rebuilt from ledger + registry + canvases + governance docs")
-    Component(migrate, "storage_migrate.py", "One-time migration", "Legacy agent-context trees -> ledger records + .sdlc/legacy-export")
     Component(workflow, "workflow.py", "Phase gates", "Progress gate: any ledger record for work id. Retro gate: decision/pitfall/pattern present")
     Component(guideops, "installer/guide_ops.py", "Guide integration", "Projection trigger, RAG ops, ops console actions")
 }
@@ -21,7 +20,6 @@ ContainerDb_Ext(sqlite, ".sdlc/index.sqlite", "Opt-in cache")
 System_Ext(guide, "Guide DICE", "Neo4j projection")
 
 Rel(cli, store, "commands")
-Rel(cli, migrate, "storage migrate")
 Rel(store, ledgermod, "stage / accept / read")
 Rel(store, db, "upsert lesson records (when sqlite backend on)")
 Rel(store, guideops, "re-project after accept")

docs/engine-v2.md

@@ -36,7 +36,6 @@ engine/
     lessons_ledger.py   # lessons.jsonl + gitignored stage
     context_store.py    # persist/retrieve/accept/parity
     persistence.py      # CONTEXT_BACKENDS config
-    storage_migrate.py  # legacy → ledger migration
     archive.py
     canvas.py
     links.py            # milestone/canvas/registry link parsing
@@ -170,9 +169,8 @@ sdlc-engine context parity --repair
 sdlc-engine context backends
 sdlc-engine context backends --set git-pointers,guide-dice,sqlite
 
-# Legacy install migration
-sdlc-engine storage status
-sdlc-engine storage migrate [--dry-run]
+# Quiet / product-test mode
+sdlc-engine quiet-status [--quiet]
 ```
 
 ## Local SQLite cache (opt-in)

docs/framework-upgrade.md

@@ -53,27 +53,13 @@ managed grounding block inside `CLAUDE.md`:
     ...
     <!-- END SDLC-SPDD MANAGED CLAUDE GROUNDING -->
 
-## Storage migration and consolidation
+## Storage v3 layout
 
-After upgrading framework files, run the engine storage commands from the target
-project root:
+Upgrade operates on one `sdlc-spdd/` home. The lessons and registry JSONL files
+are committed under `spdd/memory/`; runtime stays gitignored under `.sdlc/`.
+Full model: [Storage v3](storage-v3.md).
 
-```bash
-# Detect legacy layouts and show what would change
-sdlc-engine storage status
-
-# One-shot: convert legacy memory trees → ledger + registry
-sdlc-engine storage migrate [--dry-run]
-
-# Move scattered framework paths into the single sdlc-spdd/ home
-./scripts/upgrade-project.sh --target . --all
-```
-
-Migration exports converted originals aside; the committed system of record becomes
-`spdd/memory/lessons.jsonl` + `spdd/memory/registry.jsonl`. Hot runtime stays
-gitignored under `.sdlc/`. Full model: [Storage v3](storage-v3.md).
-
-Verify parity after migration:
+Verify parity after upgrading:
 
 ```bash
 sdlc-engine context parity
@@ -86,8 +72,7 @@ Run from the SDLC-SPDD orchestrator repository:
 
     ./scripts/upgrade-project.sh --target /path/to/app --all
 
-For backward compatibility, omitting assistant flags upgrades Cursor and GitHub
-Copilot only. Use `--all` or `--claude` when you want Claude Code files.
+By default, omitting assistant flags upgrades Cursor and GitHub Copilot only. Use `--all` or `--claude` when you want Claude Code files.
 
 Upgrade only Cursor prompts: