Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md

Large diffs are not rendered by default.

24 changes: 24 additions & 0 deletions docs/reference/05-agentcore.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,30 @@ network object and a policy word directly after it (only whitespace or 의/에
principal is named; any other security subject ("보안 그룹과 퍼블릭 노출된 S3", "보안 그룹과 버킷 정책",
"… and the account password policy") keeps both candidates for the classifier.

**Resource-first routing.** A prompt naming an AWS resource ID or ARN (ENI/SG/subnet/VPC-family IDs
and EC2 instance IDs, plus IAM/EKS/ECS/RDS/ELB/CloudFront/CloudFormation ARNs) routes to the owning
section after a pin (`web/lib/route-resources.ts`, `web/lib/route.ts` `resourceRoute`, method
`resource`). Before keyword matching (`stripResources`) every network ID is replaced by a
network-object word (sg, eni, nacl, route table, or `network-resource` for
vpc/subnet/vpce/tgw/nat/igw/pcx/vpn/vgw/cgw), an instance ID by a neutral token and an ARN by its
service name; a role/user ARN (IAM, or an STS assumed-role) also keeps a `role` word, so it counts
as a named IAM principal ("Can `arn:aws:sts::…:assumed-role/…` modify sg-…?" goes to security).
ARN text stops at whitespace, a comma or CJK text, so comma-joined words or ARNs stay separate.
Resource-name text is never a keyword, an ARN of a service without its own resource rule (for
example DynamoDB) still routes by its service name, and every network ID follows the network-object
rule exactly like the object's name: "`sg-…` 정책" and "`subnet-…` 권한" go to network, while
"`sg-…` 과 버킷 정책" keeps both network and security as classifier candidates. The resource decides
alone only when no IAM principal is named (unless it is an IAM ARN), at most one area's keywords
match (a list/count phrasing such as "list all … sg-…" does not count), and that area is absent,
the resource's own, or `security` through `보안` directly after an instance ID ("`i-…` 보안 점검").
Otherwise the keyword rules, the hybrid classifier and ADR-003 fan-out decide: "which IAM role can
modify sg-…", "why is `i-…` access denied?" and "`i-…` 퍼블릭 노출" go to security, "`i-…` CPU 지표"
to monitoring, "`sg-…` 비용" to cost, and "eni-… 와 RDS 연결" to the classifier. An ops-owned
resource (EC2 instance, load balancer, CloudFront ARN) is weak evidence like a lone ops keyword: on
the hybrid path the classifier arbitrates and the resource's section is the fallback. When several
resources are named, the first-named one decides. An enabled custom agent whose keywords match still
outranks resource routing (`web/app/api/chat/route.ts` precedence).

### Tool result completeness contract

Capped list results from the domain MCP Lambdas carry completeness metadata so the chat model never
Expand Down
53 changes: 53 additions & 0 deletions web/lib/route-resources.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import { describe, it, expect } from 'vitest';
import { resourceSection, stripResources } from './route-resources';
import { sectionByKey } from './sections';

describe('resourceSection', () => {
it('routes a named ENI to network', () => {
expect(resourceSection('ENI <eni-058d44a138cb9f099>의 보안 그룹 …')).toBe('network');
});
it.each([
'sg-0123456789abcdef0', 'acl-0a1b2c3d', 'rtb-0a1b2c3d4e5f60718', 'subnet-0a1b2c3d',
'vpc-0a1b2c3d', 'vpce-0a1b2c3d', 'tgw-attach-0a1b2c3d4e5f60718', 'nat-0a1b2c3d',
'igw-0a1b2c3d', 'pcx-0a1b2c3d',
])('routes %s to network', (id) => {
expect(resourceSection(`${id} 확인`)).toBe('network');
});
it('routes an instance ID to ops', () => {
expect(resourceSection('i-0a1b2c3d4e5f60718 점검')).toBe('ops');
});
it('routes ARNs by service', () => {
expect(resourceSection('arn:aws:iam::123456789012:role/x')).toBe('security');
expect(resourceSection('arn:aws:eks:ap-northeast-2:123456789012:cluster/c')).toBe('container');
expect(resourceSection('arn:aws:ecs:ap-northeast-2:123456789012:service/c/s')).toBe('container');
expect(resourceSection('arn:aws:rds:ap-northeast-2:123456789012:db:d')).toBe('data');
expect(resourceSection('arn:aws:elasticloadbalancing:ap-northeast-2:123456789012:loadbalancer/app/a/1')).toBe('ops');
expect(resourceSection('arn:aws:cloudformation:ap-northeast-2:123456789012:stack/s/1')).toBe('iac');
});
it('first named resource wins', () => {
expect(resourceSection('i-0a1b2c3d4e5f60718 의 eni-0a1b2c3d')).toBe('ops');
});
it('returns undefined when no resource is named', () => {
expect(resourceSection('sg-name-prod 규칙')).toBeUndefined();
expect(resourceSection('vpc-endpoint 설명')).toBeUndefined();
expect(resourceSection('안녕하세요')).toBeUndefined();
});
it('every resource rule targets a known section', () => {
for (const x of ['eni-0a1b2c3d', 'i-0123456789abcdef0', 'arn:aws:iam::1:role/x', 'arn:aws:eks:r:1:cluster/c',
'arn:aws:rds:r:1:db:d', 'arn:aws:cloudfront::1:distribution/E1', 'arn:aws:cloudformation:r:1:stack/s/1']) {
const key = resourceSection(x);
expect(key).toBeDefined();
expect(sectionByKey(key as string)).toBeTruthy();
}
});
it('comma-joined ARNs are reduced to their own service names', () => {
const stripped = stripResources('arn:aws:rds:ap-northeast-2:123456789012:db:mydb,arn:aws:iam::123456789012:role/x');
expect(stripped).toContain(' rds ');
expect(stripped).toContain(' iam ');
});
it('a role/user ARN keeps a principal word', () => {
expect(stripResources('arn:aws:sts::123456789012:assumed-role/Admin/session')).toContain(' sts role ');
expect(stripResources('arn:aws:iam::123456789012:user/alice')).toContain(' iam role ');
expect(stripResources('arn:aws:rds:ap-northeast-2:123456789012:db:mydb')).not.toContain('role');
});
});
56 changes: 56 additions & 0 deletions web/lib/route-resources.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Resource-first routing: a prompt that names a concrete AWS resource (ID or ARN) belongs to the
// section that owns that resource's tools, regardless of incidental keywords ("i-0abc 보안 점검"
// is an instance question, not an IAM one).

const ID_TAIL = '-(?:[0-9a-f]{17}|[0-9a-f]{8})\\b';
const NETWORK_PREFIXES = 'eni|sg|acl|rtb|subnet|vpc|vpce|tgw-attach|tgw-rtb|tgw|nat|igw|pcx|vpn|vgw|cgw';
const NETWORK_ID = new RegExp(`\\b(?:${NETWORK_PREFIXES})${ID_TAIL}`, 'i');
const INSTANCE_ID = new RegExp(`\\bi${ID_TAIL}`, 'i');
// An ARN's resource part uses ASCII ARN characters only; the match stops at whitespace, a comma or
// CJK text, so comma-joined words or ARNs stay separate (an IAM name that itself contains a comma
// is cut at the comma).
const ARN = /arn:aws[a-z-]*:([a-z0-9-]+):[\w:/.+=@*-]*/gi;
// Object word that replaces a network ID before keyword matching, so "sg-… 정책" reads like "sg 정책".
const NETWORK_ID_WORD: Record<string, string> = { eni: 'eni', sg: 'sg', acl: 'nacl', rtb: 'route table' };
const NETWORK_ID_G = new RegExp(`\\b(${NETWORK_PREFIXES})${ID_TAIL}`, 'gi');
const INSTANCE_ID_G = new RegExp(INSTANCE_ID.source, 'gi');
/** Token that replaces an EC2 instance ID; no keyword rule matches it. */
export const INSTANCE_TOKEN = 'ec2-instance';
/** Object word for network IDs without a dedicated word (vpc, subnet, vpce, tgw, nat, igw, pcx, vpn, vgw, cgw). */
export const NETWORK_RESOURCE_TOKEN = 'network-resource';
// A role/user ARN is itself an IAM principal: keep that word so a question about what it may do stays
// an IAM question ("Can arn:aws:sts::…:assumed-role/Admin/x modify sg-…?").
const ARN_PRINCIPAL = /:(?:assumed-role|role|user)\//i;

// Ordered list; the match with the lowest index in the prompt wins (first-named resource).
const RESOURCE_RULES: { re: RegExp; key: string }[] = [
{ re: NETWORK_ID, key: 'network' },
{ re: INSTANCE_ID, key: 'ops' },
{ re: /arn:aws[a-z-]*:iam:/i, key: 'security' },
{ re: /arn:aws[a-z-]*:(?:eks|ecs):/i, key: 'container' },
{ re: /arn:aws[a-z-]*:rds:/i, key: 'data' },
{ re: /arn:aws[a-z-]*:(?:elasticloadbalancing|cloudfront):/i, key: 'ops' },
{ re: /arn:aws[a-z-]*:cloudformation:/i, key: 'iac' },
];

/** Section key of the first-named recognised AWS resource in the prompt, else undefined. */
export function resourceSection(prompt: string): string | undefined {
let best: { index: number; key: string } | undefined;
for (const { re, key } of RESOURCE_RULES) {
const m = re.exec(prompt);
if (m && (best === undefined || m.index < best.index)) best = { index: m.index, key };
}
return best?.key;
}

/** The prompt prepared for keyword matching: a network ID becomes its object word (sg, eni, nacl,
* route table, or `network-resource` for the other prefixes), an instance ID a neutral token and an
* ARN its service name (plus `role` for a role/user ARN, an IAM principal), so the network-object
* rule applies to every network ID exactly as to "sg" and `arn:aws:dynamodb:…` still matches the
* data rule, while resource-name text is never a keyword. */
export function stripResources(prompt: string): string {
return prompt
.replace(ARN, (m, service: string) => ` ${service}${ARN_PRINCIPAL.test(m) ? ' role' : ''} `)
.replace(NETWORK_ID_G, (_m, prefix: string) => ` ${NETWORK_ID_WORD[prefix.toLowerCase()] ?? NETWORK_RESOURCE_TOKEN} `)
.replace(INSTANCE_ID_G, ` ${INSTANCE_TOKEN} `);
}
123 changes: 123 additions & 0 deletions web/lib/route.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { describe, it, expect, vi } from 'vitest';
import { pickGateway, classifyRoute, matchedSections, ACTIVE_FALLBACK } from './route';
import { stripResources } from './route-resources';

describe('pickGateway', () => {
it('honors an explicit pin over keywords', () => {
Expand Down Expand Up @@ -457,3 +458,125 @@ describe('classifyRoute follow-up continuity (hybrid)', () => {
expect(r.primary).toBe('security');
});
});

describe('resource-first routing', () => {
it('a named resource beats keywords; pin still wins', () => {
expect(pickGateway('i-0a1b2c3d4e5f60718 보안 점검')).toBe('ops');
expect(pickGateway('eni-0a1b2c3d', 'cost')).toBe('cost');
});
it('classifyRoute short-circuits on a named resource without calling the classifier', async () => {
const classify = vi.fn();
const r = await classifyRoute('eni-0a1b2c3d 상태', undefined, { llmEnabled: true, classify });
expect(r).toMatchObject({ primary: 'network', method: 'resource', multiDomain: false });
expect(classify).not.toHaveBeenCalled();
});
it('an ops-owned resource lets the hybrid classifier arbitrate', async () => {
const prompt = 'i-0123456789abcdef0 가 자꾸 죽어';
const classify = vi.fn(async () => [{ key: 'monitoring', score: 0.8 }]);
const r = await classifyRoute(prompt, undefined, { llmEnabled: true, classify });
expect(r).toMatchObject({ primary: 'monitoring', method: 'llm' });
expect(classify).toHaveBeenCalledTimes(1);
const failing = vi.fn(async () => { throw new Error('x'); });
expect(await classifyRoute(prompt, undefined, { llmEnabled: true, classify: failing }))
.toMatchObject({ primary: 'ops', method: 'resource' });
expect(await classifyRoute(prompt)).toMatchObject({ primary: 'ops', method: 'resource' });
});

// A named resource must not override an explicit subject from another area: IDs become object words
// and ARNs their service name, and the resource's section wins alone only when no IAM principal is
// named (unless it is an IAM ARN), at most one area's keywords match (a list/count phrasing is
// ignored), and that area is absent, the resource's own, or `보안` directly after an instance ID.
const otherAreaIntent: [string, string][] = [
['i-0123456789abcdef0 의 CPU 사용률 지표 보여줘', 'monitoring'],
['i-0123456789abcdef0 CloudTrail 누가 변경했는지', 'monitoring'],
['sg-0123456789abcdef0 관련 비용', 'cost'],
['arn:aws:eks:ap-northeast-2:123456789012:cluster/c 비용 분석', 'cost'],
['i-0123456789abcdef0 의 보안 그룹 규칙 확인', 'network'],
['어떤 IAM role이 arn:aws:rds:ap-northeast-2:123456789012:db:mydb에 접근 가능한가요?', 'security'],
['which role can access arn:aws:rds:ap-northeast-2:123456789012:db:mydb', 'security'],
['i-0123456789abcdef0 에 붙은 IAM 권한 확인', 'security'],
['Which IAM role may modify sg-0123456789abcdef0?', 'security'],
['why is i-0123456789abcdef0 access denied?', 'security'],
// like "보안 그룹 접근이 denied" (network first; the hybrid classifier still sees security)
['sg-0123456789abcdef0 접근이 denied 돼요', 'network'],
['which principal can modify sg-0123456789abcdef0?', 'security'],
['Can arn:aws:sts::123456789012:assumed-role/Admin/session modify sg-0123456789abcdef0?', 'security'],
];
it.each(otherAreaIntent)('pickGateway keeps another area’s keyword: %s → %s', (prompt, want) => {
expect(pickGateway(prompt)).toBe(want);
});
it.each(otherAreaIntent)('classifyRoute keeps another area’s keyword: %s → %s', async (prompt, want) => {
const r = await classifyRoute(prompt);
expect(r.primary).toBe(want);
expect(r.method).not.toBe('resource');
});
it('a bare policy word next to a network ID keeps the network route', async () => {
for (const p of ['eni-0a1b2c3d 에 대한 권한 확인', 'sg-0123456789abcdef0 의 정책 확인']) {
expect(pickGateway(p)).toBe('network');
expect(await classifyRoute(p)).toMatchObject({ primary: 'network' });
}
});
it('every network ID prefix follows the network-object rule', async () => {
for (const p of ['vpc-0a1b2c3d 보안 점검', 'subnet-0a1b2c3d 권한 확인', 'vpce-0a1b2c3d4e5f60718 정책 확인']) {
expect(pickGateway(p)).toBe('network');
}
const classify = vi.fn(async () => [{ key: 'security', score: 0.9 }]);
const r = await classifyRoute('subnet-0a1b2c3d 권한 확인', undefined, { llmEnabled: true, classify });
expect(r.primary).toBe('network');
expect(classify).not.toHaveBeenCalled();
});
it('a resource ID follows the network-object rule for policy words', async () => {
const classify = vi.fn(async () => [{ key: 'network', score: 0.6 }, { key: 'security', score: 0.5 }]);
const r = await classifyRoute('sg-0123456789abcdef0 과 버킷 정책 검토해줘', undefined, { llmEnabled: true, classify });
expect(classify).toHaveBeenCalledTimes(1);
expect(r.method).toBe('llm');
expect(pickGateway('i-0123456789abcdef0 이 퍼블릭에 노출됐나?')).toBe('security');
expect(pickGateway('sg-0123456789abcdef0 의 정책 확인')).toBe('network');
expect(await classifyRoute('sg-0123456789abcdef0 의 정책 확인')).toMatchObject({ primary: 'network' });
});
it('a list/count phrasing about a named resource stays with the resource', async () => {
expect(pickGateway('list all rules for sg-0123456789abcdef0')).toBe('network');
expect(await classifyRoute('list all rules for sg-0123456789abcdef0'))
.toMatchObject({ primary: 'network', method: 'resource' });
expect(pickGateway('eni-0a1b2c3d 몇 개 붙어 있어?')).toBe('network');
});
it('an unrecognised ARN keeps its service keyword', () => {
expect(pickGateway('arn:aws:dynamodb:ap-northeast-2:123456789012:table/x 점검')).toBe('data');
expect(pickGateway('arn:aws:eks:ap-northeast-2:123456789012:cluster/c,비용분석해줘')).toBe('cost');
expect(pickGateway('arn:aws:eks:ap-northeast-2:123456789012:cluster/c,cost analysis')).toBe('cost');
const stripped = stripResources('arn:aws:dynamodb:ap-northeast-2:123456789012:table/x 점검');
expect(stripped).toContain('dynamodb');
expect(stripped).not.toContain('123456789012');
});
it('a resource question naming several areas runs the classifier path unchanged', async () => {
const classify = vi.fn(async () => [{ key: 'cost', score: 0.9 }]);
const r = await classifyRoute('arn:aws:eks:ap-northeast-2:123456789012:cluster/c 비용 분석과 파드 상태', undefined,
{ llmEnabled: true, classify });
expect(classify).toHaveBeenCalledTimes(1);
expect(r).toMatchObject({ primary: 'cost', method: 'llm' });
});
it('the security collision and own-section keywords still route by resource', async () => {
const sec = await classifyRoute('i-0a1b2c3d4e5f60718 보안 점검');
expect(sec).toMatchObject({ primary: 'ops', method: 'resource' });
const own = await classifyRoute('ENI <eni-058d44a138cb9f099>의 보안 그룹, 네트워크 ACL, 라우팅 설정을 조회해 주세요.');
expect(own).toMatchObject({ primary: 'network', method: 'resource' });
});
it('a resource question spanning several areas keeps the classifier and multi-domain fan-out', async () => {
const classify = vi.fn(async () => [{ key: 'network', score: 0.6 }, { key: 'data', score: 0.5 }]);
const r = await classifyRoute('eni-0a1b2c3d 와 RDS 연결 안 돼', undefined, { llmEnabled: true, classify });
expect(classify).toHaveBeenCalledTimes(1);
expect(r.method).toBe('llm');
expect(r.multiDomain).toBe(true);
expect(r.selected.map((s) => s.key)).toEqual(['network', 'data']);
});
it('resource IDs become object words and ARNs their service name', () => {
const stripped = stripResources('sg-0123456789abcdef0 와 arn:aws:rds:ap-northeast-2:123456789012:db:mydb 확인');
expect(stripped).toContain(' sg ');
expect(stripped).toContain(' rds ');
expect(stripped).not.toContain('sg-0123');
expect(stripped).not.toContain('arn:');
expect(stripped).not.toContain('mydb');
expect(stripped).not.toContain('123456789012');
expect(matchedSections('sg-0123456789abcdef0 비용')).toEqual(['cost', 'network']);
});
});
Loading
Loading