Skip to content

aws-transform-agent-toolkit: add connector steering for S3 access - #217

Open
lzongren wants to merge 5 commits into
kirodotdev:mainfrom
lzongren:aws-transform-connector-steering
Open

lzongren wants to merge 5 commits into
kirodotdev:mainfrom
lzongren:aws-transform-connector-steering

Conversation

@lzongren

@lzongren lzongren commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Steering for the platform S3 connector in the AWS Transform Agent Toolkit power. It documents the toolkit change awslabs/agent-builder-toolkit-aws-transform#103, released as agent-builder-agentic-mcp-aws-transform 1.1.0 and agent-builder-sdk-aws-transform 1.2.0.

File Change
steering/connectors.md (new) S3 only: the flow, the MCP tool create_platform_connector_hitl_task, a code sample with the SDK class ConnectorClient, the rules (approval, workspace scope, 1-hour expiry, write prefix, secrets, connector details), and the "not authorized to access this type of connector" error
POWER.md One line in "When to Load Steering Files"
steering/subagent-patterns.md Removes four helpers that exist nowhere (connector_s3_tools, list_s3_connectors, download_from_connector, upload_to_connector) and the "fall back to direct S3" rule; points to connectors.md
steering/troubleshooting.md Entry 29: the cause is a workload-owned connector type; the fix is the platform connector
steering/agent-registration.md Agent card example: my_service|s3|1 → platform|s3|1
 model ─ create_platform_connector_hitl_task(step_id, connector="s3") ─► HITL task
 user creates or selects the connector ─► account administrator approves (PENDING → ACTIVE) ─► user submits the task
 model ─ retrieve_hitl_output_as_string(hitl_task_id) ─► {"connectorId"}
 agent code ─ ConnectorClient.get_connector ─► ConnectorClient.get_temporary_credentials_for_connector ─► boto3 S3 client

Validation

Check Result
Every tool, class, method, field and type id in the new text Found in the toolkit source at the head of the toolkit pull request
Code sample In a new virtual environment with only the two released packages from PyPI: every import resolves; run against the real ConnectorClient class with a fake API client, ACTIVE returns the S3 client and bucket, PENDING and REJECTED raise before the credential call. With the root logger at DEBUG, botocore.parsers is at INFO during get_connector and during the credential call.
Facts (works with no connector declared in the agent card, 1-hour credentials, write only under transform-output/<jobId>/, the error text and where it occurs, no credentials before ACTIVE, Submit enabled only after the approval, workspace scope) Measured in a real AWS Transform job and in the web application; see the toolkit pull request

Minimum versions

Package Version Holds
agent-builder-agentic-mcp-aws-transform 1.1.0 The tool create_platform_connector_hitl_task
agent-builder-sdk-aws-transform 1.2.0 The class ConnectorClient

Add steering/connectors.md: the platform S3 connector flow (the MCP tool
create_platform_connector_hitl_task, then the SDK class Connector for
temporary credentials), its limits, and the "not authorized to access this
type of connector" error.

The old guidance named four helpers that do not exist and told agents to
fall back to direct S3. Remove both from subagent-patterns.md and
troubleshooting.md and point to the new file. Use the platform|s3|1 type
in the agent card example.
@lzongren

Copy link
Copy Markdown
Contributor Author

Result of the end-to-end run for the toolkit change that this steering documents: awslabs/agent-builder-toolkit-aws-transform#103 at commit 14a5289 (original comment). "This commit" and "the PR commit" below mean that toolkit commit. The run did not use the steering text of this pull request.


Simulated builder: ✅ PASS at 14a5289

In the scenario e2e/scenarios/platform-connector.md, a simulated partner developer uses Kiro web and a test AWS account to install the toolkit from this commit, build, deploy and register an agent, use it in an AWS Transform job, and clean up. The verdict comes from the evidence below, not from the agent's own report.

Check Result
toolkit installed from the PR commit ✅
account: image build SUCCEEDED ✅
account: runtime READY ✅
account: registered 1.0.0 ACTIVE as job orchestrator ✅
feature: reply to ask-access holds the expected text ✅
agent log: holds every expected line and no excluded line ✅
cleanup: agent deregistered ✅
no deviations from the scenario (0) ✅

Screenshot: Job chat: what the agent's reply rendered

Job chat: what the agent's reply rendered

Time: 24 min (install 5m · discover 1m · build 4m · deploy 5m · register 2m · use 5m · cleanup 2m).

Simulated builder · Kiro web · evidence kept in the test account

@youtuyy youtuyy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (before ready): Please set the minimum versions in connectors.md to agent-builder-agentic-mcp-aws-transform 1.1.0 and agent-builder-sdk-aws-transform 1.2.0, matching the bumps in awslabs/agent-builder-toolkit-aws-transform#103.

suggestion: In subagent-patterns.md, the layout still lists s3_tools.py # Direct S3 upload/download, but the section that explained it is gone. Could you add "agent's own buckets only, never the customer's", or remove the line?

nit: In "The tool", "has it with no code change" should be "has it after upgrading to the minimum version below". The agent image has to ship the new MCP server.

question: Does declaring platform|s3|1 in the agent card's connectors do anything? The toolkit's real job ran with connectors: []. A word on whether the declaration is optional would help builders.

praise: These all match the code at the head of the toolkit pull request:

  • the tool name and its arguments, and the platform|s3|1 id;
  • the result keys (hitl_task_id, payload) and the credential key names;
  • the subagent note (put_job_plan is in the default subagent exclusions).

…eview

The SDK class is now ConnectorClient and the artifact helper is gone (review of awslabs/agent-builder-toolkit-aws-transform#103). Set the minimum versions (agentic MCP server 1.1.0, SDK 1.2.0). The sample keeps botocore at INFO, accepts only an S3 bucket ARN and reads the account connection safely. Say that the card declaration is optional, check the status after submit, and keep connector details and the own-bucket S3 tools apart from the customer's bucket.
…the re-test

The toolkit re-test showed that a connector belongs to its workspace, that the
credentials call refuses a connector that is not ACTIVE, and that the web application
enables Submit only after the approval.
@lzongren

Copy link
Copy Markdown
Contributor Author

Thanks. The changes are in 799f2a5 and 8c5b111:

Point Change
Minimum versions agent-builder-agentic-mcp-aws-transform 1.1.0 and agent-builder-sdk-aws-transform 1.2.0 (class ConnectorClient)
s3_tools.py line # Direct S3 upload/download: the agent's own buckets, never the customer's
"with no code change" "has it after upgrading to the minimum versions below"
Card declaration Not needed. The real job and both web-application runs used "connectors": [] and got credentials. I did not test a card that declares it. The steering now says "You do not need to declare it in the agent card".
From the toolkit review ConnectorClient name. The sample matches the toolkit FAQ. Rules for connectionToken, the approval order and the workspace scope.

I will mark this ready when the toolkit release is on PyPI.

@youtuyy youtuyy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. The four points from the last round are addressed: the versions are filled in, the s3_tools.py line says whose buckets it is for, "no code change" became "after upgrading", and the agent card question is answered in connectors.md. I checked the text against the toolkit's main branch after its connector change merged and ran the sample and the tool with stub clients: names, arguments, result keys, status values and error paths all match.

Two things before this leaves draft: the minimum versions are not published yet, and the sample should raise the botocore log level before its first call. One nit on spelling out the upgrade.

| Package | Provides | Minimum version |
|---------|----------|-----------------|
| `agent-builder-agentic-mcp-aws-transform` | The tool `create_platform_connector_hitl_task` | 1.1.0 |
| `agent-builder-sdk-aws-transform` | The class `ConnectorClient` | 1.2.0 |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking for merge, not for the text: the two minimum versions are not on PyPI yet. Today the latest are agent-builder-agentic-mcp-aws-transform 1.0.2 and agent-builder-sdk-aws-transform 1.1.0; 1.0.2 has no create_platform_connector_hitl_task and 1.1.0 has no agent_builder_sdk.agentic_framework.connector. Until 1.1.0 and 1.2.0 are published, a reader cannot do the upgrade this file asks for and the sample fails on its import. Please keep this a draft until both are on the index.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both are on PyPI now: agent-builder-agentic-mcp-aws-transform 1.1.0 and agent-builder-sdk-aws-transform 1.2.0. In a new virtual environment with only these two from PyPI, the sample's imports resolve, and with stub clients the sample returns the S3 client for ACTIVE and stops before the credential call for PENDING and REJECTED. I marked the PR ready for review.

creds = connector_client.get_temporary_credentials_for_connector(connector_id)
# At DEBUG, botocore logs the access key id and the session token of each request, and
# the SDK's configure_logging() sets DEBUG when STAGE is not prod.
logging.getLogger("botocore").setLevel(logging.INFO)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this line to the top of the function, before get_connector. The SDK lowers the botocore.parsers logger only around the credentials call. With configure_logging() and a non-prod STAGE, that logger is still at DEBUG during the first get_connector call, and botocore logs the response body there, which holds connectionToken (the field the "Connector details" rule keeps away from the model). I ran the sample with stub clients to confirm: DEBUG is enabled during get_connector, off during the credentials call, off after the sample. The same order is in the SDK docstring and the toolkit guide, so it may be worth the same change there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the top of the function in 7b9d2e7. Same run as above, with the root logger at DEBUG: botocore.parsers is at INFO during get_connector and during the credential call. The toolkit FAQ and the ConnectorClient docstring get the same order in awslabs/agent-builder-toolkit-aws-transform#121, which also puts get_connector under the SDK's log guard.


## Minimum versions

| Package | Provides | Minimum version |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could this say how to upgrade? The Dockerfile steering in this power installs the packages unpinned in a cached layer, and the install commands say >=1.0.0, so a rebuild of an existing image can keep the old versions and the tool is missing with no error. Suggestion: "Pin agent-builder-agentic-mcp-aws-transform>=1.1.0 and agent-builder-sdk-aws-transform>=1.2.0, and rebuild the image without the cache."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added after the versions table in 7b9d2e7: pin both minimum versions in the pip install line of the Dockerfile, and rebuild the image without the cache, because a cached install layer keeps the old versions with no error.

…irst

- The sample sets botocore to INFO before get_connector, whose response
  holds the connection token.
- Says how to upgrade: pin the minimum versions and rebuild the image
  without the cache.
- The KMS key setting takes a key ARN, not an alias ARN.
- An agent allowlisted for a workload-owned S3 type keeps it by naming
  that connector in its system prompt, or by hiding the new tool.

@youtuyy youtuyy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the three points are fixed. I checked each one: 1.1.0 and 1.2.0 are on PyPI and their wheels match the toolkit's main; I ran the sample against those packages with stub clients and with botocore's real parser at DEBUG on a non-prod stage, and no connection token, key or session token reaches the logs (the previous order did log the token); the upgrade steps read well. One sentence to correct on where excluded_mcp_tool_names is accepted, one question on the version behind the "by name" sentence, one nit. Approving.

(AI-assisted review)

connector type (see the error below). An agent that is allowlisted for such a type keeps it when
its system prompt asks for that connector by name (for example "the mainframe modernization S3
connector"), or when it hides the new tool with
`excluded_mcp_tool_names={"create_platform_connector_hitl_task"}`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this say where the argument goes? On SDK 1.2.0 only the BaseOrchestrator and BaseSubagent constructors (and create_default_async_orchestrator_with_subagent) take excluded_mcp_tool_names. create_default_orchestrator and create_default_async_orchestrator, which orchestrator-patterns.md shows, raise TypeError, and create_default_subagent / create_default_async_subagent accept it through **kwargs and ignore it, so the tool stays visible with no error. Suggestion: "or when it hides the new tool by passing excluded_mcp_tool_names={"create_platform_connector_hitl_task"} to the BaseOrchestrator or BaseSubagent constructor (the create_default_* helpers do not forward it)".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 194468b with your list. The sentence names the BaseOrchestrator, AsyncBaseOrchestrator, BaseSubagent and AsyncBaseSubagent constructors and create_default_async_orchestrator_with_subagent, and says that the orchestrator helpers raise TypeError and the subagent helpers ignore it. I checked each signature on SDK 1.2.0. The toolkit FAQ gets the same sentence in awslabs/agent-builder-toolkit-aws-transform#121, and the release notes of agentic-mcp-server 1.1.0 and sdk 1.2.0 are corrected.

submitted, then read `connectorId` and pass it to your custom tool.
- **Other tool:** do not use `create_s3_connector_hitl_task` here. It asks for a workload-owned
connector type (see the error below). An agent that is allowlisted for such a type keeps it when
its system prompt asks for that connector by name (for example "the mainframe modernization S3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: which version of the agentic MCP server was this tested with? The tool descriptions that tell the model to keep a mainframe or .NET connector when the instructions ask for it are in the toolkit's follow-up (1.1.1). In 1.1.0, the minimum named below, the description of create_s3_connector_hitl_task says every other agent must use the platform tool. If the behaviour needs 1.1.1, the sentence could say so.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with 1.1.0 from PyPI, in a new virtual environment, with the offline tool-choice sample (10 runs per prompt, scripted tool results). A mainframe agent whose system prompt asks for "the mainframe modernization S3 connector" got create_s3_connector_hitl_task in 10 of 10 runs, the same as with the follow-up's docstrings. So the sentence holds at the minimum version, and I left it as it is. A mainframe prompt that names no connector split 5 and 5 on 1.1.0 (6 and 4 with the follow-up), which is why the sentence asks for the name.


To upgrade an agent, pin `agent-builder-agentic-mcp-aws-transform>=1.1.0` and
`agent-builder-sdk-aws-transform>=1.2.0` in the `pip install` line of its Dockerfile, and rebuild the
image without the cache (for example `docker build --no-cache`). The install layer is cached, so a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: changing the pip install line already invalidates the cached layer, and the power's build_agent_image tool has no cache switch, so the pin is the step that matters. --no-cache could be marked as needed only for a hand-run docker build.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 194468b: the pin is the step that matters, and --no-cache is only for a hand-run docker build of an unchanged Dockerfile.

- Name the constructors that take excluded_mcp_tool_names. The other
  create_default_* helpers raise TypeError or ignore it, so the tool
  stayed visible for agents built with them.
- Upgrade: the pin is the step that matters; --no-cache only for a
  hand-run docker build of an unchanged Dockerfile.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants