Conversation
tool_from_function derived a tool's description from the first line of the docstring only, so everything after the summary — the prose that actually says what a tool covers, when to use it, and what it will not do — never reached the model. The description is now built from the docstring's summary plus its extended description, with Returns:/Yields: appended as a single line. Args: stays out since it is already encoded in the parameter schema, and Raises:/Examples:/ Notes: are dropped to keep descriptions dense. Descriptions are capped at MAX_TOOL_DESCRIPTION_CHARS (1024, OpenAI's limit) and truncated on a paragraph, sentence, or word boundary; pass max_description_chars=None to opt out. ToolRegistry.register() threads the cap through. _parse_docstring_params is rewritten on the new section splitter, which also fixes Args: not terminating at multi-word headers such as See Also. Two renderers assumed single-line descriptions and now indent continuation lines: ToolDefinition.to_prompt_format() and the XML tool-grammar block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tukuy derives a skill's JSON Schema from the raw `__annotations__`, so a
skill defined in a module using `from __future__ import annotations` hands
it the *string* `"int"` rather than `int` — and every such parameter
silently degraded to `{"type": "string"}`. That style is used throughout
this codebase, so it was the common case rather than an edge case.
Previously this only misinformed the model. Now that tool arguments are
validated against the advertised schema it rejects correct calls outright
(`double(x=7)` -> "7 is not of type 'string'").
Re-derive the parameter fragments via `typing.get_type_hints`, which
resolves the strings, and merge them over tukuy's output. Only properties
tukuy already emitted are touched, so parameters it deliberately hides
(`context`, config params) stay hidden, and non-shape keys it attached
(description, default) are preserved.
Also reduce `integrations/tukuy_bridge.py` to a re-export shim. It was a
byte-identical copy of the canonical module, which meant two *distinct*
`current_tool_call_id` ContextVars — a tool executed via one module's var
was invisible to code reading the other's, and this fix would have applied
to only one copy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…args
Three gaps in the driver layer surfaced by a tool-calling audit:
`tool_choice` was silently dropped by the OpenAI, Claude and Google
drivers (only minor raw-HTTP drivers forwarded it), and
`parallel_tool_calls` was exposed nowhere. Both now travel through a
shared `_translate_tool_choice` that maps the normalized form onto each
provider's wire format, warning and ignoring rather than sending
something the API will reject. `tool_choice="required"` on a final round
is the standard cure for an agent that never stops, so it needs to
actually reach the API.
`stop_reason` was passed through raw, so callers had to know each
provider's vocabulary (`tool_calls` vs `tool_use` vs numeric Gemini enums)
and truncation was invisible to the agent loop. Responses now carry a
normalized reason (`end_turn`, `tool_use`, `max_tokens`, `content_filter`,
`error`) with the provider's original value preserved in
`meta["raw_stop_reason"]` / `usage["raw_stop_reason"]`. Unknown strings
pass through unchanged so no information is lost.
Malformed or truncated tool arguments were logged and then executed as
`{}` — a `write_file` call truncated at `max_tokens` ran with empty
arguments instead of the model being told to retry. Both the buffered and
streaming paths now attach `arguments_error` / `truncated` to the tool
call so the conversation layer can feed the problem back to the model.
Also accept single-quoted attributes in the prompted-tool XML grammar and
give `ToolGrammar` an `open_prefix` so the streaming parser can hold back
a partial delimiter split across chunks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`_python_type_to_json_schema` degraded most non-trivial annotations to
`{"type": "string"}`: `int | None`, `Literal`, `Enum`, nested pydantic
models, dataclasses, `TypedDict`, `dict[str, X]` and container element
types. The schema handed to the model therefore disagreed with what the
tool actually accepted, and coercion existed but was wired only to
extraction, never to tool arguments.
Now emitted faithfully: unions as `anyOf` (with `{"type": "null"}` for
NoneType), `Literal`/`Enum` as `enum` (with a shared `type` when all
values agree), containers with `items`/`prefixItems`/`additionalProperties`,
date-ish scalars with a `format`, and nested structured types expanded via
`pydantic.TypeAdapter`. Arguments are coerced towards the annotation and
validated against the schema, returning an LLM-friendly message naming the
offending argument so the model can self-correct instead of a bare
TypeError escaping the tool.
A *missing* annotation now yields the empty (unconstrained) schema rather
than a string. An unannotated parameter means "type unknown", and since
arguments are now validated, claiming `string` rejected every correct
non-string call — `lambda x: x + 1` could no longer be called with an int.
Adds direct coverage for the type mapping, which previously had none: 38
cases across unions, enumerations, containers, formats and nested models,
plus end-to-end checks that a tool's advertised schema matches what it
accepts and survives conversion to OpenAI wire format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An async `on_approval_needed` handler auto-approved everything. The
callback was typed as possibly-awaitable but invoked bare from the sync
agent, so an `async def` handler returned a coroutine — always truthy —
and every dangerous tool was silently approved, defeating the only
human-in-the-loop mechanism. Callbacks now run through a bridge that
drives awaitables to completion, carrying contextvars across the thread
hop when a loop is already running.
`Agent.stop()` was dead code: `_stop_requested` was written in six places
and read nowhere, and the one test asserted only that the flag was set, so
the suite certified a no-op. `stop()` now forwards to the conversation
driving the run. It is tracked in `_active_conversation` rather than
`_conversation`, because the latter is only populated when
`persistent_conversation=True` — forwarding through it alone left `stop()`
a silent no-op for the default agent. A `stop()` arriving before the
conversation exists is replayed onto it rather than lost to the race.
Exhausting `max_iterations` raised RuntimeError and discarded the whole
run. The loop now takes one final turn with tools removed, so the model
answers with what it has and `max_rounds_reached` records why — the worst
case was exactly the weak local models that prompted tools target.
Async tools ran coroutines via `asyncio.run` on a throwaway thread, which
blocked the loop, dropped contextvars (tukuy `SecurityContext`,
`current_tool_call_id`) and could deadlock; they now await natively.
Adds a per-tool timeout, so a hung tool no longer wedges a run forever,
exposed as `Agent(tool_timeout=...)` and overridable per call via
`options={"tool_timeout": ...}`. Independent async calls execute
concurrently via `asyncio.gather`; `zip(tool_calls, results)` pairs
results with `strict=True` so a mis-pairing fails loudly instead of
attaching a result to the wrong `tool_call_id`.
Remaining sync/async drift and loop hardening: history trimming no longer
orphans a `tool` message from its assistant `tool_calls` (which providers
reject with a 400), malformed and truncated arguments are reported back to
the model instead of executed as `{}`, tool results have a default size
bound, `current_tool_call_id` is set on the sync path too, depth guards
apply to every run path, `RunContext` is rebuilt per tool invocation so
tools see live iteration and history, and `AgentStep` records the real
tool name and a populated `duration_ms` instead of the call id and a dead
field.
Agent-level pre-call budget enforcement is removed: it read a freshly
created, always-empty UsageSession and so never triggered. Enforcement
lives in the Conversation, which sees real usage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Prompture had no MCP support, while every peer framework (OpenAI Agents SDK, Pydantic AI, LangChain) now consumes MCP servers. It maps cleanly onto the existing ToolRegistry interface — `list_tools()` becomes ToolDefinitions and `call_tool()` becomes the tool function — so MCP tools are called through the normal tool path with no special casing in the agent loop. `register_mcp_tools(registry, session)` registers every tool an initialized `mcp.ClientSession` advertises, with an optional `prefix` to avoid collisions across servers. `mcp_session_from_stdio()` is a convenience context manager for stdio servers, and `register_mcp_tools_sync()` covers sync `Agent` code outside an event loop. Results are flattened to strings: text blocks joined, non-text blocks (images, resources) reduced to a short placeholder, and MCP call errors returned as `Error: ...` so the model can react to them. Behind the optional `prompture[mcp]` extra, exported lazily so importing `prompture.integrations` never requires the dependency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both streaming tests asserted a 200 but got 501, which is the endpoint
correctly reporting that `sse-starlette` is unavailable rather than a
defect. `sse-starlette` ships in the optional `serve` extra, so a valid
install without it left the suite red.
Skip via `importorskip`, matching the `importorskip("fastapi")` already at
the top of both modules. The non-streaming tests need no extra and keep
running.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI caught `test_typed_dict_expands` failing on Python 3.11: the schema had
no `properties` at all. pydantic refuses `typing.TypedDict` below 3.12 (it
requires the `typing_extensions` variant), so `TypeAdapter` raised, the
except branch logged at debug level and fell through to the generic
`{"type": "string"}` return.
So on two of the four supported Python versions a TypedDict tool parameter
was advertised to the model as a bare string — the exact silent degradation
the richer type mapping was written to eliminate. It only looked fixed
because local development runs 3.14, where pydantic accepts it.
Rebuild the object schema from resolved type hints when TypeAdapter cannot,
so field names and types survive on every supported version. Optionality is
read per kind — `__required_keys__` for TypedDict, absent default/
default_factory for dataclasses, `is_required()` for models — and `required`
is omitted entirely when it cannot be determined, so nothing is
over-constrained.
The fallback is exercised on all versions by forcing TypeAdapter to raise,
rather than being covered only where pydantic happens to refuse; otherwise
the path would stay untested on 3.12+ and regress unnoticed again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR hardens Prompture’s agent tool-calling pipeline end-to-end: it improves tool JSON Schema fidelity and argument validation, makes stop/approval/timeout behavior actually take effect across sync + async runs, normalizes provider stop reasons and tool-choice wiring, and adds optional MCP server tool integration via the normal ToolRegistry path.
Changes:
- Expand tool schema generation + add argument coercion/validation to prevent malformed calls from executing and to give correctable feedback.
- Rework conversation/agent tool loops for cooperative stop, graceful max-rounds completion, per-tool timeouts, concurrency correctness, and richer step metadata (tool name + duration).
- Add MCP bridge (optional extra) and plumb tool-choice + stop-reason normalization through multiple drivers (OpenAI-compatible, Anthropic, Google, etc.).
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_tukuy_bridge.py | Adds regression coverage for PEP-563 annotation resolution in tukuy bridge tool schemas. |
| tests/test_tools_schema.py | Adds extensive schema-generation and validation/coercion test coverage. |
| tests/test_tool_use.py | Tests richer docstring descriptions + graceful max-rounds behavior. |
| tests/test_simulated_tools.py | Updates simulated-tools loop tests to expect graceful completion. |
| tests/test_server.py | Skips SSE tests when the serve extra isn’t installed. |
| tests/test_openai_server.py | Skips SSE tests when the serve extra isn’t installed. |
| tests/test_mcp_bridge.py | Adds mocked unit tests for MCP tool discovery/registration/execution. |
| tests/test_live_events.py | Updates expectations for normalized stop reasons + raw preservation. |
| tests/test_conversation_robustness.py | Adds robustness contracts for malformed args, stop, timeouts, trimming, ordering, budgets. |
| tests/test_agent.py | Adds broad agent-level regression coverage for approval/stop/timeout/context/steps. |
| pyproject.toml | Adds mcp optional extra and includes it in all. |
| prompture/integrations/tukuy_bridge.py | Replaces duplicated code with a deprecated re-export alias. |
| prompture/integrations/mcp_bridge.py | Implements MCP tool bridge (registration + stdio session helper). |
| prompture/integrations/init.py | Lazily exports MCP bridge functions to avoid hard dependency. |
| prompture/extraction/tukuy_bridge.py | Repairs tukuy-emitted parameter schemas using resolved type hints. |
| prompture/drivers/openai_driver.py | Normalizes stop reasons, preserves raw stop reason, applies tool-choice options. |
| prompture/drivers/groq_driver.py | Normalizes stop reasons, preserves raw stop reason, applies tool-choice options. |
| prompture/drivers/google_driver.py | Adds tool-choice wiring, stop-reason normalization, and richer HTTP error type. |
| prompture/drivers/claude_driver.py | Adds tool-choice wiring, stop-reason normalization, and truncation-aware streamed tool parsing. |
| prompture/drivers/base.py | Adds shared stop-reason normalization, tool-choice translation, tool-call parsing with errors, HTTP error type. |
| prompture/drivers/async_openai_driver.py | Mirrors OpenAI buffered tool-call changes for async driver. |
| prompture/drivers/async_google_driver.py | Mirrors Google driver changes for async driver. |
| prompture/drivers/async_claude_driver.py | Mirrors Claude driver changes for async driver. |
| prompture/drivers/async_base.py | Improves streamed event generation for tool calls (ids + truncation flags). |
| prompture/drivers/_prompted_tool_stream.py | Makes prompted-tool streaming parser prefix-aware for chunk-split delimiters. |
| prompture/drivers/_openai_compat_stream.py | Adds thinking deltas, robust tool-stop finalization, and normalized MessageStop. |
| prompture/agents/types.py | Updates RunContext/AgentCallbacks docs to match new execution semantics. |
| prompture/agents/tool_grammars.py | Improves XML tool grammar parsing/rendering (quotes, multiline descriptions, prefix). |
| prompture/agents/live_events.py | Extends ToolUseStop event with truncation + raw stop reason metadata. |
| prompture/agents/conversation.py | Adds cooperative stop, graceful max-rounds, timeouts, history trimming, malformed-arg handling. |
| prompture/agents/async_conversation.py | Async parity for stop/max-rounds/timeouts/history trimming + parallel tool execution. |
| prompture/agents/async_agent.py | Fixes async tool/callback execution semantics; adds live contexts + step timing/name improvements. |
| prompture/agents/agent.py | Sync parity for stop/approval semantics; adds live contexts + step timing/name improvements. |
| docs/INTEGRATIONS.md | Documents MCP tool integration and installation flow. |
Comment on lines
+499
to
+502
| pool = ThreadPoolExecutor(max_workers=1) | ||
| try: | ||
| future = pool.submit(self._tools.execute, name, arguments) | ||
| return future.result(timeout=timeout) |
Comment on lines
+109
to
+112
| """End of one assistant turn. ``stop_reason`` uses the shared vocabulary | ||
| (``end_turn``, ``tool_use``, ``max_tokens``, ``content_filter``, | ||
| ``error``); drivers preserve the provider's raw value in | ||
| ``usage["raw_stop_reason"]``.""" |
Comment on lines
+99
to
+102
| text = _serialize_call_result(result) | ||
| if getattr(result, "isError", False): | ||
| return f"Error from MCP tool '{mcp_name}': {text}" | ||
| return text |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tool loop had a human-in-the-loop approval gate that approved everything, a
stop()that stopped nothing, and schemas that described half your parameters as strings regardless of what they were — three ways of being confidently wrong. This PR makes the loop tell the truth: honest parameter schemas with argument validation, approval and cancellation that actually take effect, per-tool timeouts, graceful exits instead of exceptions, andtool_choiceplumbed through to the providers that support it. It also adds an MCP client, so agents can consume tools from any MCP server through the ordinary tool path.Highlights
async defhandler returned a coroutine that read as "approved", so every tool guarded by human review ran unattendedmcpextraTechnical changes
Tool schema generation
_python_type_to_json_schemato emit unions (bothtyping.Unionand PEP-604) asanyOfwith{"type": "null"}forNoneType,LiteralandEnumasenumvia_enum_schema(adding a sharedtypeonly when every value agrees),list/tuple/dictwithitems/prefixItems/additionalProperties,datetime/date/time/UUIDasstringplus aformat, and nested pydantic models, dataclasses andTypedDictexpanded throughpydantic.TypeAdapter. All of these previously collapsed to{"type": "string"}_structured_fallback_schemaand_structured_required_fields, which rebuild an object schema fromget_type_hintswhenTypeAdapterrefuses a structured type — pydantic rejectstyping.TypedDictbelow Python 3.12, so on two of the four supported versions such a parameter was advertised as a bare string. Optionality is read per kind (__required_keys__, absent dataclassdefault/default_factory,FieldInfo.is_required()) andrequiredis omitted entirely when undeterminable{"type": "string"}; an unannotated parameter means "type unknown", and since arguments are now validated, claimingstringrejected every correct non-string call_coerce_and_validate_arguments, which coerces each argument toward its annotation, rejects unknown keys, and validates against the tool's schema withjsonschema.Draft7Validator, returning a message naming the offending argument. Wired into bothToolRegistry.executeandaexecute_strict_parametersand_make_nullablefor OpenAI strict tool use, forcing originally-optional parameters nullable since strict mode moves every property intorequired_split_docstring,_section_key,_docstring_descriptionand_truncate_description, withReturns:/Yields:appended,Args:omitted (already in the parameter schema), and aMAX_TOOL_DESCRIPTION_CHARScap of 1024 truncated on a paragraph, sentence or word boundary. Rewrite_parse_docstring_paramson the new section splitter, which also fixesArgs:failing to terminate at multi-word headers such asSee Also:ToolDefinition.to_prompt_format()and the XML grammar renderer, both of which assumed a single-line descriptionAgent loop
_invoke_cb_sync, which detects an awaitable callback result and drives it to completion — directly viaasyncio.run, or on a worker thread carrying acontextvars.copy_context()snapshot when a loop is already running. Applied toon_tool_start,on_tool_endandon_approval_needed; the last of these is why anasync defapproval handler auto-approved_active_conversationvia_track_active, sostop()reaches non-persistent runs;_conversationis only populated underpersistent_conversation=True. Astop()arriving before the conversation is built is replayed onto it, and the field is reset at every run entry (_execute,_execute_iter,_execute_stream,_execute_live)tool_timeoutconstructor parameter onAgent/AsyncAgent, forwarded into the conversation and overridable per call throughoptions={"tool_timeout": ...}_make_live_ctx_fnsoRunContextis rebuilt per tool invocation, giving tools the current iteration, message history and usage rather than a snapshot from the start of the runtool_timingslist through the tool wrappers to populateAgentStep.duration_ms(previously a dead field), and build anid_to_namemap from assistanttool_callsmessages soAgentStep.tool_namerecords the tool name instead of the call id_agent_depthguard and reset to every run path, not justrun_live_async_fnhook on the current event loop, replacingasyncio.runon a throwaway thread — which blocked the loop, dropped contextvars (tukuySecurityContext,current_tool_call_id) and could deadlock_build_conversationwhen the agent's driver has since been swapped, so a budget fallback is not silently ignoredUsageSession; enforcement lives inConversation._check_budget, which sees real usageConversation
request_stop()and amax_rounds_reachedproperty toConversation/AsyncConversationRuntimeErroron exhausted tool rounds with_final_answer_without_tools/_final_answer_simulated, which take one more driver call with the tools removed and instruct the model to answer from the results already gathered. Shared by max-rounds exhaustion and cooperative stop_call_tool, enforcing the timeout with a single-workerThreadPoolExecutorandshutdown(wait=False)to detach a thread Python cannot kill; the async path usesasyncio.wait_for_execute_tool_call, which setscurrent_tool_call_idon the sync path as well (previously async-only) and convertsTimeoutErrorinto a tool result the model can react to_malformed_arguments_message, which returns a retry instruction instead of executing a call whose driver flaggedarguments_errorortruncated_trim_history, dropping leading orphanedtoolmessages after the sliding-window slice; a naive tail-slice could cut between an assistanttool_callsmessage and its results, which providers reject with a 400max_tool_result_lengthdefault fromNoneto16000, bounding tool results out of the boxAsyncConversation._run_tool_callsviaasyncio.gather, with asequential_toolsopt-out, and pair results usingzip(..., strict=True)so a mis-pairing fails loudly rather than attaching a result to the wrongtool_call_id_full_tool_resultsinclear()so a reset conversation does not retain prior resultsDriver layer
_normalize_stop_reasonwith_STOP_REASON_MAPand_GOOGLE_STOP_REASON_MAP(including Gemini's numericFinishReasonvalues), mapping providers onto a shared vocabulary ofend_turn,tool_use,max_tokens,content_filteranderror, upgrading totool_usewhen a provider reports end-of-turn alongside tool calls, and passing unknown strings through unchanged. The provider's raw value is preserved inmeta["raw_stop_reason"]andusage["raw_stop_reason"]_translate_tool_choiceand_apply_openai_tool_optionsto forwardtool_choiceandparallel_tool_calls, translating the normalized form into each provider's wire shape and warning rather than sending a payload the API will reject. Applied across the OpenAI, Azure, Claude, Google, Groq, Grok, Moonshot, ModelScope, MiniMax and Zai drivers_parse_tool_arguments_with_errorand_tool_call_dict, attachingarguments_errorandtruncatedto a tool call so malformed ormax_tokens-truncated arguments are reported instead of executed as{}. Applied to the buffered path and both streaming finalizersDriverHTTPErrorcarrying status and response body for raw-HTTP driver failurestruncatedandraw_stop_reasonfields to theToolUseStoplive eventToolGrammar.open_prefixso the streaming parser holds back a trailing run that could still grow into an opening delimiter split across chunksMCP
prompture/integrations/mcp_bridge.pywithregister_mcp_tools,register_mcp_tools_syncand themcp_session_from_stdioasync context manager. MCPinputSchemais already JSON Schema and passes through unchanged; names are sanitized to^[a-zA-Z0-9_-]{1,64}$with optional prefixing to namespace multiple servers_async_fnhook soAsyncAgent/AsyncConversationawait it natively, with the sync path raising a clear error when called from inside a running loop_serialize_content_block/_serialize_call_result: text blocks joined, non-text blocks reduced to a short placeholder, and MCP call errors returned asError: ...strings the model can react to__getattr__inprompture/integrations/__init__.py, so importing the package never requires the optional dependencymcp = ["mcp>=1.0"]extra and include it inall; document the flow indocs/INTEGRATIONS.mdtukuy bridge
_resolved_param_schemasand_repair_parameter_types, re-deriving skill parameter types throughget_type_hints. tukuy builds its schema from raw__annotations__, so a skill in a module usingfrom __future__ import annotationshanded it the string"int"and every parameter degraded to{"type": "string"}— harmless when it only misinformed the model, but a hard failure once arguments are validated. Only properties tukuy already emitted are touched, so parameters it deliberately hides stay hidden, and non-shape keys it attached are preservedprompture/integrations/tukuy_bridge.pyto a re-export shim. It was a byte-identical copy ofprompture/extraction/tukuy_bridge.py, which meant two distinctcurrent_tool_call_idContextVars — a tool executed through one module's var was invisible to code reading the other's, and any fix would have applied to only one copy