Skip to content

Add function call resolution loop and withMessages() to PromptBuilder - #273

Open
gziolo wants to merge 13 commits into
trunkfrom
add/function-call-resolution-loop
Open

gziolo wants to merge 13 commits into
trunkfrom
add/function-call-resolution-loop

Conversation

@gziolo

@gziolo gziolo commented Aug 5, 2026 •

Copy link
Copy Markdown
Member

Fixes #272.

Adds a bounded function call loop to PromptBuilder and withMessages() for callers that want to manage the conversation themselves.

$result = AiClient::prompt('What is this site about?')
    ->usingFunctionDeclarations(...$declarations)
    ->usingFunctionCallResolver($resolver)
    ->usingMaxFunctionCallIterations(3) // Default: 5 tool rounds.
    ->generateTextResult();

Use generateTextResult() to inspect calls when the loop stops early. generateText() throws if the final response has no text.

Design

PromptBuilder handles configuration, model selection, and lifecycle events. An internal FunctionCallRunner handles the loop. The public extension point is:

interface FunctionCallResolverInterface
{
    public function canResolve(FunctionCall $functionCall): bool;
    public function resolve(FunctionCall $functionCall): FunctionResponse;
}

canResolve() must have no side effects. The runner checks all calls before executing any. If one is unsupported, none in that round execute. Supported calls run sequentially. Each response must match its call by ID, or by name when no ID is present. Any response name must also match.

withMessages() appends complete messages. withHistory() continues to prepend them.

Behavior

  • Automatic resolution applies only to text generation and requires one candidate. Support checks return false for an invalid candidate count, and generation rejects it before model discovery. Unexpected multiple response candidates are also rejected before tool execution.
  • Each tool round executes the calls, appends their responses, and sends a follow-up request using the same model. The builder's original conversation stays unchanged.
  • The loop stops when there are no calls, a call is unsupported or incomplete, or the round limit is reached. Calls require a toolCalls finish reason and a non-empty name. The default limit allows five tool rounds, plus the initial model request.
  • Model discovery requires chat history support. Explicitly selected models follow the existing selection rules.
  • Each request dispatches BeforeGenerateResultEvent and AfterGenerateResultEvent. There is no extra event for the aggregate result.
  • The final result includes total token usage. Its functionCallResolution data contains rounds, stopReason, resolvedCalls, and the conversation in messages.

Errors and retries

Checking support does not make a batch atomic. A later tool or model request can fail after earlier tools have run.

Exceptions stop the loop and pass to the caller. Completed tools are not rolled back. The SDK does not retry tools or return partial progress after an exception. Retrying the original prompt can repeat actions, so applications that need recovery must record tool outcomes and manage retries. Resolvers can return expected execution errors in a FunctionResponse so the model can handle them.

Provider compatibility

The shared OpenAI-compatible serializer now expands multiple function responses into separate tool messages. It rejects malformed JSON arguments instead of turning them into null and allowing execution.

The official OpenAI provider uses the Responses API and needs a separate fix. It must combine parallel output items into one candidate, preserve incomplete response status, emit each call and response as a top-level input item, and reject malformed JSON arguments. That provider change is not included here.

Scope

This supports the WordPress ability resolution work in Trac #64865 and wordpress-develop #12658. WordPress can supply a resolver for registered abilities instead of managing follow-up requests through lifecycle events. Tool execution stays under the caller's control. Persistent execution state and automatic recovery are outside this PR.

Validation

  • composer test:unit: 1,232 tests and 4,507 assertions passed after rebasing onto trunk.
  • composer phpcs: passed.
  • composer phpstan -- --debug: passed. Debug mode avoids the local sandbox's socket restriction.
  • git diff --check origin/trunk...HEAD: passed.

Tests cover multiple calls and rounds, stop conditions, malformed arguments, response matching, exceptions, token usage, event order, builder reuse, model discovery, and message ordering. Scripted test models reject unexpected requests unless repeating is explicitly enabled. No live provider API calls were made.

AI assistance

Claude Code and Codex assisted with implementation, review, tests, documentation, and this description.

@codecov

codecov Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.77419% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.86%. Comparing base (77a5995) to head (63988de).

Files with missing lines Patch % Lines
src/Tools/Execution/FunctionCallRunner.php 94.59% 4 Missing ⚠️
src/Builders/PromptBuilder.php 97.67% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##              trunk     #273      +/-   ##
============================================
+ Coverage     86.58%   86.86%   +0.27%     
- Complexity     1383     1429      +46     
============================================
  Files            69       70       +1     
  Lines          4449     4574     +125     
============================================
+ Hits           3852     3973     +121     
- Misses          597      601       +4     
Flag Coverage Δ
unit 86.86% <96.77%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support in PromptBuilder for (1) appending messages to an existing conversation and (2) automatically resolving model-requested function calls via a bounded multi-round loop, returning loop metadata in the final result’s additional data.

Changes:

  • Add PromptBuilder::withMessages() to append full Message instances (counterpart to withHistory() which prepends).
  • Introduce FunctionCallResolverInterface and integrate an optional function call resolution loop into text generation, including token-usage aggregation and transcript exposure.
  • Add comprehensive unit tests plus supporting mocks/helpers for scripted multi-round text generation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Builders/PromptBuilder.php Adds withMessages(), resolver configuration, and the multi-round function call resolution loop with transcript + token aggregation.
src/Tools/Contracts/FunctionCallResolverInterface.php Defines the resolver extension point (canResolve() + resolve()) used by the loop.
tests/unit/Builders/PromptBuilderFunctionCallResolutionTest.php Covers loop behavior (happy path, transcript ordering, unresolved calls, max iterations, token aggregation, non-text capabilities) and withMessages() ordering.
tests/traits/MockModelCreationTrait.php Adds a scripted text-generation mock model helper to simulate multi-round responses deterministically.
tests/mocks/MockFunctionCallResolver.php Adds a mock resolver that records checked/resolved calls and can be customized via callbacks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/Builders/PromptBuilder.php Outdated
@jeffpaul jeffpaul added this to the 1.5.0 milestone Aug 26, 2026
@gziolo
gziolo force-pushed the add/function-call-resolution-loop branch 2 times, most recently from 6fcc530 to 1d0f4df Compare September 3, 2026 10:03
@gziolo
gziolo marked this pull request as ready for review September 3, 2026 11:52
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: gziolo <gziolo@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Thought-token aggregation must preserve unknown usage instead of reporting a misleading partial total.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/Builders/PromptBuilder.php Outdated
gziolo and others added 11 commits October 20, 2026 13:54
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gziolo
gziolo force-pushed the add/function-call-resolution-loop branch from 02b40a5 to 9663ef4 Compare October 20, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Enhancement A suggestion for improvement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a function call resolution loop and a way to append messages to PromptBuilder

3 participants