Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/runbooks/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- generated-by: co-agent · source: CLAUDE.md · claude-md-sha: 82ed41aabe32 · generated-at: 2026-11-06 · DO NOT EDIT — edit CLAUDE.md then run /co-agent sync-context -->
<!-- generated-by: co-agent · source: CLAUDE.md · claude-md-sha: 204c9c831227 · generated-at: 2026-11-11 · DO NOT EDIT — edit CLAUDE.md then run /co-agent sync-context -->

> You are an external reviewer for this repo — project context below, distilled from CLAUDE.md. This file is shared verbatim by the external review panel (not a per-AI copy).

Expand Down Expand Up @@ -226,13 +226,13 @@ distinction between operational degraded data and ineligible release evidence.
Four lanes share Lambda/Steampipe limits; contention can fail the gate. Budgets are
admission bounds, not guaranteed completion. Runtime foundation records a 43-type,
57.461-second collection sample and its limits; it is not full live readiness.
Require every pinned baseline name (currently 46, source-AST checked); valid growth
Require every pinned baseline name (currently 49, source-AST checked); valid growth
is allowed up to 128 and every returned type needs strict proof. Both modes default
to the enabled host only; explicit applied targets require exact enabled registration.
Collect additionally requires measured SQL reachability with zero unreachable accounts
and fresh account-bound EC2/CloudFront known-member proof. Explicit scope preserves null for host-only/unmeasured
counts; CI permits host-only/null only for five source-AST-pinned SDK types, not
46-type coverage for every member. Only Terraform onboarding
49-type coverage for every member. Only Terraform onboarding
preflight permits approved subsets; apply reads the restored plan, not a newer secret.
Runtime release stays exact. Preserve isolated
AWS CLI environments, first chronological terminal failure and stopped admission;
Expand Down
4 changes: 2 additions & 2 deletions docs/runbooks/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,13 +280,13 @@ distinction between operational degraded data and ineligible release evidence.
Four lanes share Lambda/Steampipe limits; contention can fail the gate. Budgets are
admission bounds, not guaranteed completion. Runtime foundation records a 43-type,
57.461-second collection sample and its limits; it is not full live readiness.
The catalog must include the pinned baseline (currently 46 names, source-AST checked);
The catalog must include the pinned baseline (currently 49 names, source-AST checked);
valid growth is allowed up to 128 types and every returned type needs strict proof.
Both modes default to the enabled host only. Explicit applied targets require exact
enabled host/member registration. Collect additionally requires measured SQL reachability
with zero unreachable accounts and fresh account-bound EC2/CloudFront known-member evidence.
Host-only/unmeasured counts remain null under explicit `account_reachability_scope`;
CI permits host-only/null only for five source-AST-pinned SDK types, not 46-type
CI permits host-only/null only for five source-AST-pinned SDK types, not 49-type
coverage for every member. Terraform plan/apply onboarding
alone allows approved subsets; apply derives scope from the restored saved plan.
Runtime release never requests subset leniency. See `runtime-foundation.md#explicit-runtime-targets`.
Expand Down
2 changes: 1 addition & 1 deletion docs/runbooks/graph-read-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ healthy evidence. Qualified empty/unproven telemetry retains the previous graph.
Failed, retained, skipped or unattempted host infra still withholds trace collection.
A clean confirmed-empty host publication is valid; zero nodes alone are not proof.
Member-only gaps remain in fleet-wide counts/reasons and CLI exit 1/2. None of these
presentation rules relaxes the strict 46-type, zero-unknown runtime release gate.
presentation rules relaxes the strict 49-type, zero-unknown runtime release gate.

| Exact diagnostic | Meaning and next check |
|---|---|
Expand Down
10 changes: 5 additions & 5 deletions docs/runbooks/runtime-foundation.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ After registration, release prepare and collect require **exactly** the enabled
and every configured target, each unique. The controller never requests onboarding
leniency. Missing, disabled or unexpected enabled members stop before type collection.

Collect retains every aggregate catalog proof (currently 46), known row/unknown-attribute
Collect retains every aggregate catalog proof (currently 49), known row/unknown-attribute
counts and zero unknown attributes. In target mode, SQL types must report
`account_reachability_scope="enabled_scan_accounts"` and numeric
`unreachable_account_count=0`. This measures the enabled, renderable DB scan scope
Expand Down Expand Up @@ -295,7 +295,7 @@ Group import uses a slash; membership uses comma-separated pool/group/username.

Before invocation, the controller compares live `CodeSha256` with `runtime_deployment.inventory.sync_code_sha256`, derived from configured `source_code_hash`. Apply reviewed configuration to persist this expected fingerprint; provider observations cannot authorize unreviewed code. The controller also captures the Lambda revision and rechecks the hash and revision after all owned collection calls settle, before authenticated acceptance. A concurrent code/configuration change rejects that evidence, including a change restored to the same hash with a different revision.

Every current catalog type (46 in this version) must have succeeded after the release marker, with known counts and zero unknown attributes. Partial, failed, stale, missing or unknown evidence blocks the release. A recent success from before the marker cannot substitute. The controller obtains the complete catalog from the verified Lambda and synchronously invokes each type through at most four concurrent collectors. It never submits `type=all` or asynchronous Event batches; existing scheduled work can still contend with its calls.
Every current catalog type (49 in this version) must have succeeded after the release marker, with known counts and zero unknown attributes. Partial, failed, stale, missing or unknown evidence blocks the release. A recent success from before the marker cannot substitute. The controller obtains the complete catalog from the verified Lambda and synchronously invokes each type through at most four concurrent collectors. It never submits `type=all` or asynchronous Event batches; existing scheduled work can still contend with its calls.

After catalog discovery, authenticated preparation verifies login, DB and host registration and samples Aurora's UTC clock. That `server_time` becomes the marker before all collection calls and remains fixed across retries. The controller calibrates later clock reads from the DB sample and local request-start timestamp, conservatively, and shifts the existing overall deadline by the same offset. It never anchors at response end or allows pre-marker data. DB-request and host-check elapsed time consume the marker window; malformed or missing clock evidence stops collection. Catalog admission allows up to 450 seconds. All type attempts and their retries share the remaining collection admission window; there is no separate fifteen-minute allowance per type. An invocation needs 450 seconds remaining for the verified function timeout of at most 420 seconds plus transport. Only confirmed throttling, busy and exact superseded outcomes retry, after ten seconds. Denied, uncertain-delivery, partial, failed, unknown and malformed outcomes cannot prove collection. All admitted workers settle before private files are cleaned.

Expand Down Expand Up @@ -453,9 +453,9 @@ milliseconds and request/response times inside the observed prepare interval, wi
DB request elapsed time from zero through 35 seconds. The DB timestamp becomes
`collectionStartedAt`; subsequent time is `rawNow + (DB time - request start)`.
The existing controller deadline shifts by that same offset, preserving time remaining.
Every current catalog type (46 at this revision) must succeed after that marker with
Every current catalog type (49 at this revision) must succeed after that marker with
known counts and zero unknown attributes. The returned catalog must contain every
member of the controller's pinned baseline of 46 type names, not merely 46 arbitrary
member of the controller's pinned baseline of 49 type names, not merely 49 arbitrary
names. A source-AST regression ties that baseline to the checked-in `QUERIES` and
`SDK_SYNCS` catalogs. Valid additional returned types are allowed up to 128 total;
every returned type remains required, permitting growth without replacing a baseline member.
Expand Down Expand Up @@ -517,7 +517,7 @@ are added. Fixed diagnostics and private cleanup remain required on success and
The [owner's 2026-09-14 acceptance condition](https://github.com/aws-samples/sample-awsops/pull/67#issuecomment-5663692939)
requires all current catalog types, superseding the earlier CloudFront-only verifier
proposal. At this revision that means at least one catalog request plus at least one request
for each of the 46 types; catalog and per-type retries add calls. Four is the concurrency ceiling, not the total call count or
for each of the 49 types; catalog and per-type retries add calls. Four is the concurrency ceiling, not the total call count or
a claim of fourfold throughput. The [session contract](runtime-verifier-sessions.md#collection-effects-and-proof)
authorizes exactly catalog or a verified catalog member, never empty/all/unregistered
payloads or asynchronous `Event` invocation. No IAM scope is widened.
Expand Down
6 changes: 3 additions & 3 deletions docs/runbooks/runtime-verifier-sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ symlink/public/non-regular input files, or an existing output policy file.
Collect consumers must use `RequestResponse` on the pinned function's
unqualified ARN, without a version or alias qualifier.
The [owner acceptance condition dated 2026-09-14](https://github.com/aws-samples/sample-awsops/pull/67#issuecomment-5663692939)
requires complete post-marker collection of all 46 current catalog types. The
requires complete post-marker collection of all 49 current catalog types. The
strict controller therefore sends exactly `{"type":"catalog"}`, followed by
`{"type":"<catalog member>"}` for every validated returned type. This supersedes
the earlier catalog/CloudFront-only consumer proposal; IAM scope is unchanged.
Expand Down Expand Up @@ -212,7 +212,7 @@ raw AWS errors. Each synchronous response must have `StatusCode=200`, no

| Payload | Required result |
| --- | --- |
| `catalog` | Exactly `status: "catalog"` and a unique catalog containing every pinned baseline member (currently 46), with valid growth allowed up to 128 total. The source-AST test binds baseline membership to the checked-in collector. No result `type` or counts are expected. |
| `catalog` | Exactly `status: "catalog"` and a unique catalog containing every pinned baseline member (currently 49), with valid growth allowed up to 128 total. The source-AST test binds baseline membership to the checked-in collector. No result `type` or counts are expected. |
| Each catalog member | `status: "succeeded"`, exact requested `type`, nonnegative safe-integer `row_count`, and `unknown_attribute_count: 0` |

`busy`, `failed` (including superseded), `partial`, unknown-type errors and
Expand All @@ -236,7 +236,7 @@ unreachable accounts and fresh account-bound known-member EC2/CloudFront evidenc
`account_reachability_scope` distinguishes
`enabled_scan_accounts` measurement from host-only and unmeasured results, whose counts stay
null. CI permits host-only/null only for the five source-AST-pinned SDK types; aggregate
catalog proof does not establish 46-type coverage for every member. See the [explicit target contract](runtime-foundation.md#explicit-runtime-targets).
catalog proof does not establish 49-type coverage for every member. See the [explicit target contract](runtime-foundation.md#explicit-runtime-targets).
Each member uses one exact `/api/deployment/member-inventory` lookup, whose bounded
identity projection also rejects disabled, absent or ambiguous scan scope/evidence.
Only Terraform onboarding preflight permits approved subsets, deriving apply scope
Expand Down
6 changes: 4 additions & 2 deletions docs/runbooks/steampipe-quota-and-staleness.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ observe a complete cycle under [safe tuning](#6-안전한-튜닝--safe-tuning).
Apply only the exact reviewed plan outside active collector/runtime proof, then
confirm service stability and the effective `steampipe_limiter_config` event.
Preserve bucket 4, plugin/Lambda/collector concurrency 4, schedule, timeouts and IAM.
Require all 46 baseline types and every current catalog type to complete with zero
Require all 49 baseline types and every current catalog type to complete with zero
unknown attributes, plus authentication/DB, model and both owned worker proofs.
Do not remove fields, accept fallback data as complete, or disable gates to pass.

Expand Down Expand Up @@ -404,7 +404,7 @@ and unknown attributes still stop release, and every configured member still nee
fresh known-resource evidence alongside the aggregate catalog proof.
`collection_attempts` retains this bounded scope; null scope means no valid scope
was reported yet. These RPC/log fields do not turn the singleton ledger into a
per-account coverage report or assert that all 46 types cover each member.
per-account coverage report or assert that all 49 types cover each member.
Enabled accounts with no enabled region and `all_regions=false` are excluded by
`_enabled_target_accounts`; the metric must not imply they were measured. In explicit
target mode the renderer rejects such registered members, and the exact member-proof
Expand Down Expand Up @@ -504,6 +504,8 @@ concurrency change needs a whole-plan check showing no DNS changes.

The values are safeguards, not assertions of universal AWS quotas; service, operation, account, and Region quotas differ.

**ENI inventory sizing.** The `eni` type has no pushdown predicate. Every ENI is written in one Aurora round trip, and stale ENIs are deleted one per row, all inside the fixed `AURORA_RESERVE_S` (120 s). Fargate task churn enlarges the stale set on every run. At the 2026-11-11 measurement (about 470–520 ENIs per account in ap-northeast-2, 41 in us-east-1) this is far below the reserve. Watch the `eni` sync duration in the sync Lambda logs as accounts or regions grow. If one account approaches several thousand ENIs per region, batch the upserts and deletes before adding more targets. An `eni` overrun fails that type, and the runtime release gate requires every catalog type, so it blocks web releases until the type succeeds.

## 7. 롤백 / Rollback

롤백은 파괴적 데이터베이스 변경 없이 이전 limiter defaults 또는 AgentCore catalog를 복원하는 방식이다.
Expand Down
6 changes: 3 additions & 3 deletions scripts/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- generated-by: co-agent · source: CLAUDE.md · claude-md-sha: c60071532258 · generated-at: 2026-11-06 · DO NOT EDIT — edit CLAUDE.md then run /co-agent sync-context -->
<!-- generated-by: co-agent · source: CLAUDE.md · claude-md-sha: 73b0f08cc5cb · generated-at: 2026-11-11 · DO NOT EDIT — edit CLAUDE.md then run /co-agent sync-context -->

> You are an external reviewer for this repo — project context below, distilled from CLAUDE.md. This file is shared verbatim by Kiro, Codex, and Agy (not a per-AI copy).

Expand Down Expand Up @@ -262,7 +262,7 @@ verification. Dev releases verify exact ECS/image proof first and pass `EXPECTED
Full releases require collect mode (including login/DB); prepare never establishes full readiness.
Explicit activation remains separate and inactive prerequisites cannot be skipped.
It binds dev source/account/actual role, applied runtime identity and ARM64 web digest.
Require the pinned 46-name baseline, source-AST checked; valid growth is allowed up
Require the pinned 49-name baseline, source-AST checked; valid growth is allowed up
to 128 types. Every returned type
needs post-marker succeeded evidence, known counts and
zero unknowns, with at most four concurrent in-flight synchronous owned calls. Prepare obtains authenticated
Expand All @@ -274,7 +274,7 @@ host/member registration, measured SQL reachability over enabled scan accounts w
unreachable accounts and fresh account-bound EC2/CloudFront member evidence from one
exact minimal `/api/deployment/member-inventory` lookup. Explicit scope preserves null for
host-only/unmeasured counts; CI allows host-only/null only for five source-AST-pinned
SDK types, not 46-type coverage for every member. Only Terraform onboarding preflight permits approved subsets;
SDK types, not 49-type coverage for every member. Only Terraform onboarding preflight permits approved subsets;
apply reads the restored saved plan, never a newer secret. Runtime release stays exact.
AWS CLI children use an explicit
credential/settings allowlist, pinned path, disabled config/credential files/metadata
Expand Down
6 changes: 3 additions & 3 deletions scripts/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,9 +405,9 @@ full readiness (including login/DB); prepare only verifies
authenticated login/DB/host registration. Explicit runtime/readiness activation remains
separate, and inactive prerequisites cannot be skipped.
The controller verifies dev source/account/role, applied runtime metadata and ARM64
web digest, requires every pinned baseline member (currently 46, source-AST checked)
web digest, requires every pinned baseline member (currently 49, source-AST checked)
and permits valid growth up to 128 types, then drives every returned
type (currently 46) with at most four
type (currently 49) with at most four
concurrent in-flight synchronous invocations. It requires succeeded results, known counts and zero unknowns.
It samples the authenticated DB clock before collecting, anchors calibration at request
start, shifts the existing deadline by the same offset, and retains strict post-marker
Expand All @@ -418,7 +418,7 @@ require the exact enabled host/member registry before type calls, measured SQL r
with zero unreachable accounts, and fresh account-bound known EC2/CloudFront member rows.
`account_reachability_scope` distinguishes `enabled_scan_accounts` measurement from host-only
and unmeasured results; the latter counts stay null. CI permits host-only/null only for
the five source-AST-pinned SDK types. The aggregate 46-type proof does not establish 46-type coverage for every member.
the five source-AST-pinned SDK types. The aggregate 49-type proof does not establish 49-type coverage for every member.
Member evidence uses one exact, bounded `/api/deployment/member-inventory` response,
never a scan of full inventory pages.
Only Terraform plan/apply onboarding preflight permits approved subsets; apply reads
Expand Down
3 changes: 2 additions & 1 deletion scripts/v2/ci/runtime-release.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ export const REQUIRED_CATALOG_TYPES = Object.freeze([
'transit_gateway_attachment', 'vpc_peering_connection', 'vpc_endpoint',
'elasticache_replication_group', 'iam_policy', 'neptune_cluster', 'msk', 'waf',
'waf_rule_group', 'waf_ip_set', 'cloudwatch_alarm', 'cloudtrail', 'apigatewayv2_api',
'apigatewayv2_integration', 'apigatewayv2_route', 'ebs_snapshot', 's3',
'apigatewayv2_integration', 'apigatewayv2_route', 'ebs_snapshot', 'eni', 'rds_cluster',
'rds_proxy', 's3',
'opensearch_serverless', 'cloudfront_vpc_origin', 'alb_listener_rule', 's3_public_access',
]);
export const MIN_CATALOG_TYPES = REQUIRED_CATALOG_TYPES.length;
Expand Down
Loading
Loading