Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs-site/docs/monitoring/inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ AWS 리소스의 수량 변화를 일별로 추적하고 비용 영향을 추정
차트 시리즈는 고정 목록이 아니라 최신 스냅샷 수량 기준으로 동적으로 순위가 매겨집니다:
- **Core Resources**: 수량 상위 5개 실제 리소스 타입 — 기본 표시
- **Other Resources**: 다음 순위 최대 3개 타입 — 기본 숨김(칩 클릭으로 표시)
- 네트워크 인터페이스(ENI)는 전체 합계에 포함되지 않으므로 Core에 들지 않고, 순위와 관계없이 Other에 추가됩니다
- 나머지 타입은 차트에는 표시되지 않지만 아래 수량 변화 테이블에는 전부 나열됩니다
### 보안 시리즈 (기본 숨김, 별도 토글 그룹)
- Public S3 Buckets, Open Security Groups, Unencrypted EBS — 보안 페이지와 동일 판정 기준의 파생 카운트, 전체 합계(total) 미포함
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ A page for tracking daily changes in AWS resource counts and estimating cost imp
Chart series are ranked dynamically by the latest snapshot counts, not a fixed list:
- **Core Resources**: the top 5 real resource types by count — shown by default
- **Other Resources**: up to the next 3 types — hidden by default (click a chip to show)
- Network interfaces (ENIs) are left out of the fleet total, so they never join Core and are always added to Other
- Remaining types don't chart, but all of them appear in the delta table below
### Security Series (hidden by default, own toggle group)
- Public S3 Buckets, Open Security Groups, Unencrypted EBS — derived counts using the Security page's criteria, excluded from the overall total
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ AWS リソースの数量変化を日次で追跡し、コスト影響を推定
チャート系列は固定リストではなく、最新スナップショット数量で動的にランク付けされます:
- **Core Resources**: 数量上位 5 つの実リソースタイプ — デフォルト表示
- **Other Resources**: 続く最大 3 タイプ — デフォルト非表示(チップをクリックで表示)
- ネットワークインターフェイス(ENI)は合計に含まれないため Core には入らず、順位に関係なく Other に追加されます
- 残りのタイプはチャートには表示されませんが、下の数量変化テーブルにはすべて表示されます
### セキュリティ系列(デフォルト非表示、独立トグルグループ)
- Public S3 Buckets, Open Security Groups, Unencrypted EBS — セキュリティページと同じ判定基準の派生カウント、合計(total)には含まれません
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import Screenshot from '@site/src/components/Screenshot';
图表序列按最新快照数量动态排序,而不是固定列表:
- **Core Resources**: 数量前 5 的实际资源类型 — 默认显示
- **Other Resources**: 其后的最多 3 个类型 — 默认隐藏(点击标签显示)
- 网络接口(ENI)不计入总数,因此不会进入 Core,并且无论排名如何都会加入 Other
- 其余类型不出现在图表中,但全部列在下方的数量变化表中
### 安全序列(默认隐藏,独立开关组)
- Public S3 Buckets、Open Security Groups、Unencrypted EBS — 采用安全页面判定标准的派生计数,不计入总数
Expand Down
2 changes: 1 addition & 1 deletion docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ without a manually held transaction. PostgreSQL supplies one MVCC snapshot for t
statement; `statement-snapshot` describes this guarantee, not a requested transaction
isolation level. Parameters and scope filters are unchanged. `view=agg` remains separate.

Topology reads target groups/ECS tasks/subnets in at most 20 pages of 500 under one
Topology reads target groups/ECS tasks/subnets/network interfaces in at most 20 pages of 500 under one
30-second browser load budget shared with EKS. All inventory and VPC/security-group
enrichment requests share two lanes per load; critical pages run sequentially within a
lane. This bounds this loader’s fan-out against the shared pool. Each critical page must carry the marker;
Expand Down
2 changes: 1 addition & 1 deletion docs/runbooks/graph-read-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ Flow and infra input allow 8192 rows within the same bounded read envelope. The
All listed sources feed the builders: a failed/missing source cannot be dropped to authorize replacement,
and elapsed retentions never authorize an unproven sweep. Larger generations need a separately reviewed capacity path.

**Enrichment sources.** The one exception is an explicit enrichment tier that only adds detail to nodes built from core sources. Flow enrichment is `eni`, `rds`, `rds_cluster`, `rds_proxy`, `vpc_endpoint` and `nat_gateway`; it resolves load-balancer IP-target ownership. Infra enrichment is `rds_cluster` and `rds_proxy`; it adds DB endpoint nodes for trace binding. These types are evaluated outside `details.sources` and kept in the unprojected `details.enrichmentSources`. A missing, failed, running, unconfirmed, stale or capped enrichment source is dropped whole. The resolver receives it as incomplete, so lookups report `owner_inventory_incomplete` and RDS placement inference (which needs `rds` and `rds_proxy`) is not made. Dropping enrichment never changes the publication status, freshness or reason vocabulary. A core source keeps the rule above.
**Enrichment sources.** The one exception is an explicit enrichment tier that only adds detail to nodes built from core sources. Flow enrichment is `eni`, `rds`, `rds_cluster`, `rds_proxy`, `vpc_endpoint` and `nat_gateway`; it resolves load-balancer IP-target ownership. Infra enrichment is `rds_cluster` and `rds_proxy`; it adds DB endpoint nodes for trace binding. These types are evaluated outside `details.sources` and kept in the unprojected `details.enrichmentSources`. A missing, failed, running, unconfirmed, stale or capped enrichment source is dropped whole. The resolver receives it as incomplete, so lookups report `owner_inventory_incomplete` and RDS placement inference (which needs `rds` and `rds_proxy`) is not made. Dropping enrichment never changes the publication status, freshness or reason vocabulary. A core source keeps the rule above. These rules describe the materialized graph only. The live `/topology` page reads `eni`, `vpc_endpoint` and `nat_gateway` itself, accepts them only from a succeeded sync, and never reads the RDS sources: it passes them to the resolver as incomplete, so RDS placement inference happens only in the saved graph.

## Browser recovery and source evidence

Expand Down
2 changes: 1 addition & 1 deletion docs/runbooks/source-sync-observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ Refresh retains the previous graph, or source details do not explain partial cov
**Interpretation:** inventory rows and the global per-type sweep ledger are read by one
SQL statement through `pool.query`, so each page uses one PostgreSQL statement snapshot
without holding a connection across application-managed transaction commands. Critical
target-group/ECS-task/subnet pages require `consistency: "statement-snapshot"` and a stable
target-group/ECS-task/subnet/network-interface pages require `consistency: "statement-snapshot"` and a stable
succeeded ledger version across pages. The ledger is keyed under `self` for the whole
account sweep; its count is neither the selected account's count nor this page's count.
The collector marks that ledger running before mutating rows. This supports cross-page
Expand Down
14 changes: 14 additions & 0 deletions web/app/api/inventory/trend/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,20 @@ describe('GET /api/inventory/trend', () => {
expect(body.types).toEqual(['lambda', 'ec2']);
});

it('ranks ENIs (left out of the fleet total) below the resources they outnumber', async () => {
verifyUser.mockResolvedValue({ sub: 'u' });
query.mockResolvedValueOnce({ rows: [
{ d: '2026-07-01', resource_type: 'eni', n: 970 },
{ d: '2026-07-01', resource_type: 'ec2', n: 40 },
{ d: '2026-07-01', resource_type: 'lambda', n: 12 },
] });
query.mockResolvedValue({ rows: [] }); // coverage query
const { GET } = await import('./route');
const body = await (await GET(req())).json();
expect(body.trend[0].total).toBe(52);
expect(body.types).toEqual(['ec2', 'lambda', 'eni']);
});

// every request issues 2 data queries (trend GROUP BY + per-day account coverage);
// '__all__' prepends the accounts-table resolution query
it('clamps days into [1, 90] and defaults to 14 (accounts default: self)', async () => {
Expand Down
12 changes: 7 additions & 5 deletions web/app/api/inventory/trend/route.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { verifyUser } from '@/lib/auth';
import { getPool } from '@/lib/db';
import { countsTowardFleetTotal, isDerivedTrendType } from '@/lib/trend-utils';
import { countsTowardFleetTotal } from '@/lib/trend-utils';

export const dynamic = 'force-dynamic';

Expand Down Expand Up @@ -135,10 +135,12 @@ export async function GET(request: Request) {
const recentPts = trend.filter((pt) => lastMs - new Date(pt.date).getTime() < 2 * 86_400_000);
const recent = (t: string) => recentPts.some((pt) => typeof (pt as Record<string, unknown>)[t] === 'number');
const types = [...latestByType.keys()].sort((a, b) => {
// Derived security series rank BELOW every real resource type: they must never claim a
// Core top-5 chip slot from an actual resource (their counts overlap the base series).
const da = isDerivedTrendType(a) ? 1 : 0;
const db = isDerivedTrendType(b) ? 1 : 0;
// Types outside the fleet total rank BELOW every fleet-counted type: derived security
// series (their counts overlap the base series) and attachment-only ENIs must never
// outrank the resources whose total they are left out of. Within that lower group they
// order by the same recency/count rules.
const da = countsTowardFleetTotal(a) ? 0 : 1;
const db = countsTowardFleetTotal(b) ? 0 : 1;
if (da !== db) return da - db;
const ra = recent(a) ? 1 : 0;
const rb = recent(b) ? 1 : 0;
Expand Down
9 changes: 3 additions & 6 deletions web/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import MultiLineTrend from '@/components/charts/MultiLineTrend';
import SegmentedControl from '@/components/ui/SegmentedControl';
import AiOps from '@/components/overview/AiOps';
import { useActiveScope, scopeParams, type ScopeSelection } from '@/lib/account-context';
import { nearestSnapshot, netChange, covCompleteForScope, countsTowardFleetTotal, isDerivedTrendType, sameAccountSet, DERIVED_TREND_TYPES, type TrendCoverage } from '@/lib/trend-utils';
import { nearestSnapshot, netChange, covCompleteForScope, countsTowardFleetTotal, sameAccountSet, trendLegendGroups, DERIVED_TREND_TYPES, type TrendCoverage } from '@/lib/trend-utils';
import { estimateCostImpact, COST_IMPACT_WEIGHTS } from '@/lib/cost-impact';
import { useI18n } from '@/components/shell/LanguageProvider';
import { localeOf } from '@/lib/i18n';
Expand Down Expand Up @@ -350,11 +350,9 @@ function ScopedHome({ scope }: { scope: ScopeSelection }) {
// the chart entirely. The palette has 8 hues; indices past 8 wrap (colorFor is modulo), a
// conscious trade — derived series are default-hidden, so duplicated hues only co-occur
// when a user opts a security series in. Colors stay pinned to each series' index.
const realTrendTypes = (resTrend?.types ?? []).filter((t) => !isDerivedTrendType(t));
const derivedTrendTypes = (resTrend?.types ?? []).filter((t) => isDerivedTrendType(t));
const trendTypes = [...realTrendTypes.slice(0, 8), ...derivedTrendTypes];
// Attachment-only types (ENIs) are appended like the derived series, into Other.
const { trendTypes, coreTypes, otherTypes, derivedTrendTypes } = trendLegendGroups(resTrend?.types ?? []);
const trendSeries = trendTypes.map((t) => ({ key: t, label: INV_LABEL(t) }));
const coreTypes = realTrendTypes.slice(0, 5); // never a derived series, even on a tiny fleet
// Chart honesty (gap L124): a (day, type) whose account coverage is less than the RESOLVED
// scope is DROPPED from that day's point — the summed line would otherwise dip on an
// account's silent day, presenting a sync artifact as a fleet change (the same rule the
Expand Down Expand Up @@ -391,7 +389,6 @@ function ScopedHome({ scope }: { scope: ScopeSelection }) {
const trendScopeNarrowed = Boolean(resTrend?.degraded) || Boolean(
resTrend?.accounts && selectedValidAccts && resTrend.accounts.length < selectedValidAccts.length,
);
const otherTypes = trendTypes.slice(5).filter((t) => !isDerivedTrendType(t));
// L127: 7d net change (lib/trend-utils netChange — coverage-parity diff with honest-degrade
// branches; see its doc). The trend data IS account-scoped now (gap L124 — the fetch above
// passes the accounts scope), but snapshots still have no REGION dimension — so a narrowed
Expand Down
12 changes: 7 additions & 5 deletions web/app/topology/page-ownership.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ describe('bounded ownership inventory paging', () => {
const pending: (() => void)[] = [];
let active = 0, peak = 0, completed = 0;
const requests = serve({
inventory: Object.fromEntries(['target_group', 'ecs_task', 'subnet'].map(type => [type, largeInventory(type)])),
inventory: Object.fromEntries(['target_group', 'ecs_task', 'subnet', 'eni'].map(type => [type, largeInventory(type)])),
inventoryReply: async (_url, body, signal) => {
active++; peak = Math.max(peak, active);
await new Promise<void>(resolve => {
Expand All @@ -229,17 +229,19 @@ describe('bounded ownership inventory paging', () => {
try {
await waitFor(() => expect(pending.length).toBeGreaterThan(0));
expect(peak).toBeLessThanOrEqual(2);
while (completed < 27) {
while (completed < 25) {
await waitFor(() => expect(pending.length).toBeGreaterThan(0));
await act(async () => { pending.splice(0).forEach(resolve => resolve()); });
expect(peak).toBeLessThanOrEqual(2);
}
await screen.findByRole('option', { name: 'ECS · ecs-app' });
const inventoryReads = requests.filter(url => url.pathname.startsWith('/api/inventory/'));
expect(new Set(inventoryReads.map(url => url.pathname)).size).toBe(24);
expect(inventoryReads).toHaveLength(27);
expect(new Set(inventoryReads.map(url => url.pathname)).size).toBe(21);
expect(inventoryReads).toHaveLength(25);
// RDS placement inference is saved-graph-only: the live page never reads RDS sources.
expect(inventoryReads.some(url => /\/rds/.test(url.pathname))).toBe(false);
expect(inventoryReads.every(url => url.searchParams.get('accounts') === 'self')).toBe(true);
for (const type of ['target_group', 'ecs_task', 'subnet']) {
for (const type of ['target_group', 'ecs_task', 'subnet', 'eni']) {
expect(inventoryReads.filter(url => url.pathname.endsWith(`/${type}`))
.map(url => url.searchParams.get('offset'))).toEqual(['0', '500']);
}
Expand Down
12 changes: 6 additions & 6 deletions web/app/topology/page.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,9 @@ describe('sample topology evidence', () => {
render(<TopologyPage />);
await screen.findByLabelText('Inventory collection evidence');
const text = screen.getByLabelText('Inventory collection evidence').textContent;
expect(text).toContain('Aggregate sync runs: unknown (21)');
expect(text).toContain('Aggregate sync runs: unknown (17)');
expect(text).toContain('Run health unknown');
expect(text).toContain('Inventory read failures: target_group: failed, ecs_task: failed, subnet: failed');
expect(text).toContain('Inventory read failures: target_group: failed, ecs_task: failed, subnet: failed, eni: failed');
expect(screen.queryByTestId('target')).toBeNull();
});
it.each(['entry-only', 'no-entry'])('separates %s onboarding coverage from EKS read failure', async clusterAccess => {
Expand Down Expand Up @@ -211,7 +211,7 @@ describe('sample topology evidence', () => {
serve({ runStatus: 'running' });
await ready();
const text = screen.getByLabelText('Inventory collection evidence').textContent;
expect(text).toContain('Aggregate sync runs: running (24)');
expect(text).toContain('Aggregate sync runs: running (21)');
expect(text).not.toContain('Aggregate sync issues:');
expect(text).not.toContain('Run health unknown');
});
Expand All @@ -228,7 +228,7 @@ describe('sample topology evidence', () => {
serve({ runStatus, rowCapture: null });
await ready();
const evidence = screen.getByLabelText('Inventory collection evidence');
expect(evidence.textContent).toContain(`Aggregate sync runs: ${runStatus} (24)`);
expect(evidence.textContent).toContain(`Aggregate sync runs: ${runStatus} (21)`);
expect(evidence.textContent).toContain(`Aggregate sync issues: route53: ${runStatus}`);
expect(evidence.textContent).toContain('Run health unknown for this account scope');
expect(evidence.textContent).not.toContain(new Date(captured).toLocaleString());
Expand All @@ -238,7 +238,7 @@ describe('sample topology evidence', () => {
serve({ runStatus: 'succeeded' });
await ready();
const text = screen.getByLabelText('Inventory collection evidence').textContent;
expect(text).toContain('Aggregate sync runs: succeeded (24)');
expect(text).toContain('Aggregate sync runs: succeeded (21)');
expect(text).toContain('Run health unknown for this account scope');
expect(text).not.toContain('Aggregate sync issues:');
});
Expand All @@ -249,7 +249,7 @@ describe('sample topology evidence', () => {
await act(async () => setActiveScope({ ...DEFAULT_SCOPE, regions: ['ap-northeast-2'] }));
const requests = vi.mocked(fetch).mock.calls.slice(count).map(([url]) => new URL(String(url), 'http://localhost'))
.filter(url => url.pathname.startsWith('/api/inventory/'));
expect(requests).toHaveLength(24);
expect(requests).toHaveLength(21);
expect(requests.every(url => url.searchParams.get('regions') === 'ap-northeast-2'
&& url.searchParams.get('accounts') === 'self' && url.searchParams.get('includeGlobal') === '1')).toBe(true);
});
Expand Down
Loading
Loading