feat(uipath-maestro-flow): one canonical headless preamble on every zero-shot flow task by rockymadden · Pull Request #3088 · UiPath/skills

rockymadden · 2026-09-04T16:39:11Z

Replaces #3086 and #3079. Same problem, an approach that works under every runner without a second config or a cross-repo change.

Problem

In the 2026-09-04 nightly, 5 of 8 skill-flow-* tasks built and validated a flow, reported success, and never executed it. The checker then ran flow debug and found a null End-node output mapping, a faulted script, and an empty result. flow validate had passed on all of them.

Every one of those prompts contained "Do NOT ask for approval, confirmation, or feedback." That phrasing forbids asking. It does not say nobody is there to ask, so an agent can honor it and still stop at a consent gate waiting for a reply that never arrives.

Measured across the 128-task suite:

	count
Tasks stating the run is headless	0
Non-simulated tasks	119
...saying nothing about autonomy	51
...saying something, across 9 wording variants	68

The ninth variant was found by the guard in this PR, not by the sweep that preceded it.

Why the task prompt and not an experiment config

Two earlier attempts put this in a config. Both failed on the same thing: flow tasks run under nightly.yaml via daily.sh in coder_eval_uipath, smoke.yaml on every PR, default.yaml locally, and whatever a dispatch selects.

coder_eval has no pattern-scoped defaults — defaults and variants both apply run-wide, and task-level is the only per-task granularity. So a config carrying this either misses those runners or reaches the 319 simulated tasks of every other skill, which is what the Slack thread vetoed.

Keeping it in the prompt makes it travel with the task. It works everywhere today, with no new config, no workflow input, no dispatch guard, and no change to coder_eval_uipath.

The original objection to n copies was to hand-maintaining them. A test removes that.

Changes

One canonical paragraph on all 119 zero-shot tasks. It states that nobody is present, that the task's implied actions are authorized including tenant writes and real messages, and the two things to hold back on: do not delete or overwrite what this run did not create, and exhaust the documented resolution path before giving up on a lookup.

_shared/test_headless_preamble.py enforces four things, each verified by breaking it:

Guard	Verified by
Every zero-shot task has it	removing it from one task → fails
Wording is identical everywhere	rewording one task → fails
Simulated tasks do not have it	adding it to one → fails
No superseded variant returns	caught a real 9th variant during development

Skill side

Both independent of headlessness and true with a user watching:

flow debug consent comes from the mandate. A request to build something that does X is a request for it to work; building plus validating does not discharge it. Debug also overwrites the Studio Web solution behind the local .uipx SolutionId — confirmed in 3 of 8 runs and by the eval suite's own cleanup_solutions.py — so never debug a solution this run did not scaffold.
"Publish to Studio Web" is no longer marked (default) in either What's next dropdown. Rule feat(CodedAgents): group skill by functionality #5's non-interactive fallback takes the marked option, which would have auto-published to a tenant. Both menus also stopped gating Debug on consent, which contradicted rule feat: new coded agent skills #2.

Modifies Critical Rules 2 and 5, called out per CONTRIBUTING.

Verification

775 pytest pass across tests/tasks/uipath-maestro-flow/.
119 zero-shot tasks carry the preamble; 9 simulated tasks correctly do not.
All maintenance checkers clean apart from one anchor that pre-exists on main; both flavors compose.
Swept for orphaned list markers left by the strip; four found and removed.
No eval run. This changes agent behavior on the same task set, so a skill-flow-* nightly is the real check.

What this costs

~650 characters of repeated prompt text across 119 files. That is the trade for an instruction that reaches every runner without asking another repo to change first.

Follow-ups (separate)

greenfield.md Step 2: assert one .flow + one project.uiproj after scaffold. bindings-no-duplicates passed its actual regression and failed only on a stray duplicate project.
script/impl.md: extend the node -e "new Function(...)" check from syntax to a value test. Would have caught the off-by-one that shipped "Suite 2000" as a city name.
variables-and-expressions.md: document the =js: boundary — an End-node object spread returned null with a clean validate. Needs a repro.
HITL node-selection: a quick-form is not a way to "display" a value; dice_roller never terminates.
Eval tenant credential preflight. A revoked Outlook grant and a hibernating ServiceNow instance currently look identical to agent error.
test-task-template.yaml still tells authors to hand-write autonomy language, which the canonical preamble supersedes. Left alone here as a separate call.

🤖 Generated with Claude Code

…ero-shot flow task In the 2026-09-04 nightly, 5 of 8 `skill-flow-*` tasks built and validated a flow, reported success, and never executed it. The checker then ran `flow debug` and found a null End-node output mapping, a faulted script, and an empty result. `flow validate` had passed on all of them. Every one of those prompts contained "Do NOT ask for approval, confirmation, or feedback". That phrasing forbids asking. It does not say nobody is there to ask, so an agent can honor it and still stop at a consent gate waiting for a reply that never arrives. Measured across the 128-task suite: 0 tasks said the run was headless, 51 of the 119 non-simulated tasks said nothing about autonomy at all, and the 68 that did were spread across 9 wording variants — the ninth found by the guard added here, not by the sweep. Task prompts, not an experiment config: flow tasks run under nightly.yaml via daily.sh in coder_eval_uipath, smoke.yaml on every PR, default.yaml locally, and whatever a dispatch selects. coder_eval has no pattern-scoped defaults, so a config carrying this would either miss those runners or reach the 319 simulated tasks of every other skill. Keeping it in the prompt makes it travel with the task and work everywhere today, with no cross-repo change. - One canonical paragraph on all 119 zero-shot tasks. It states that nobody is present, that the task's implied actions are authorized including tenant writes, and the two things to hold back on: do not delete or overwrite what this run did not create, and exhaust the documented resolution path before giving up on a lookup. - _shared/test_headless_preamble.py enforces presence, identical wording, absence on the 9 simulated tasks, and that no superseded variant comes back. Each guard verified by breaking it. - test-task-template.yaml points authors at the canonical block instead of telling them to hand-write autonomy language. Skill side, both true with a user watching: - `flow debug` consent comes from the mandate. A request to build something that does X is a request for it to work. Debug also overwrites the Studio Web solution behind the local .uipx SolutionId, so never debug a solution this run did not scaffold. - "Publish to Studio Web" is no longer marked `(default)` in either What's next dropdown — rule #5's non-interactive fallback takes the marked option, which would have auto-published to a tenant. Both menus also stopped gating Debug on consent, which contradicted rule #2. Modifies Critical Rules 2 and 5, per CONTRIBUTING. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-09-04T16:39:47Z

Claude finished @rockymadden's task in 4m 44s —— View job

Summary

Adds one canonical headless-run paragraph to all 119 zero-shot flow task prompts, replacing 9 ad-hoc autonomy variants (and 51 tasks that said nothing). Backed by a four-guard pytest suite that enforces presence, exact wording, simulated-task exclusion, and no regression to superseded variants. Also updates SKILL.md Critical Rules 2 and 5 to align flow debug consent with the mandate pattern, removes the (default) marker from "Publish to Studio Web" in both dropdown tables, and updates test-task-template.yaml to point authors at the canonical block.

Change-by-Change Review

1. `tests/tasks/uipath-maestro-flow/_shared/test_headless_preamble.py` (new)

Severity: OK

Well-structured test module with a thorough docstring explaining the history, the problem, and why the preamble lives in the prompt rather than an experiment config. Four guards cover the key invariants:

test_every_zero_shot_task_states_the_run_is_headless — presence
test_the_wording_is_identical_everywhere — exact match
test_simulated_tasks_are_not_told_nobody_is_present — exclusion
test_no_task_reintroduces_a_superseded_variant — drift detection

The _SUPERSEDED regex catches both known old variants. The docstring correctly explains the regex-not-PyYAML choice (CI installs only pytest).

One minor style note (Low): The all(n in ... for n in [single_element]) pattern on line 71 wraps a single membership check in all() over a one-element list. Functionally correct but slightly obscures intent — a direct in check would read more clearly. Not worth blocking on.

2. `skills/uipath-maestro-flow/SKILL.md` — Critical Rules 2 and 5

Severity: OK

Rule #2: Changed from "Do not run flow debug without explicit user consent" to "flow debug consent comes from the mandate." The new wording correctly delineates when debug is authorized (user asked for something that works) vs. when to ask (request stops at build/validate) vs. headless behavior (report debug as the step not run if nobody to ask and mandate doesn't imply it). Also adds the Studio Web overwrite warning — important safety detail.

Rule #5: Explicitly carves flow debug out of the consent-gate list and cross-references rule #2. The interaction is well-structured: rule #2 is the specific authority for debug, rule #5's fallback handles everything else.

Removing (default) from "Publish to Studio Web" is a good safety change — prevents rule #5's non-interactive fallback from auto-publishing to a tenant.

3. `skills/uipath-maestro-flow/references/author/greenfield.md` and `brownfield.md`

Severity: OK

Both "What's next" dropdown tables updated consistently:

(default) removed from "Publish to Studio Web"
"Debug the solution" description changed from "Confirm consent first" to "Consent comes from the mandate"
Cross-reference to SKILL.md rule preserved

Both files make the same changes symmetrically. Links verified — ../../SKILL.md resolves correctly from both locations.

4. `skills/uipath-maestro-flow/references/operate/run.md`

Severity: OK

Blockquote updated from "Confirm consent first" to "Consent comes from the mandate." Adds the Studio Web overwrite warning and references rule #2. Consistent with the SKILL.md change.

5. 119 task YAML files — headless preamble addition

Severity: OK

Spot-checked across multiple categories (bindings, connector_features, datafabric_connector, e2e, edit, evaluate, hitl, ixp, multi_node, single_node, smoke, voice, context-grounding, connector_trigger). All carry the identical canonical paragraph. Superseded variants ("Do NOT ask for approval", "Do NOT pause between planning") are cleanly removed — grep confirms zero remaining occurrences in task YAMLs.

Where old prompts used inline strings (e.g., drive_to_slack.yaml), the PR reformats to block scalar style (|) for readability — good cleanup.

No orphaned list markers found (verified with grep for bare - lines).

Zero simulated/interactive tasks carry the preamble (verified: 0 matches under interactive/).

6. `tests/templates/test-task-template.yaml`

Severity: OK

Template now warns against hand-writing autonomy language and points to the canonical block in test_headless_preamble.py. Removes the old "Do NOT ask for approval" / "Do NOT pause between planning" boilerplate from the example prompt. Good — new task authors will follow the right pattern.

What's Missing

Nothing identified. The PR is well-scoped. CODEOWNERS is not relevant (no new skill path). The _shared/ directory already has __init__.py and pytest uses --import-mode=importlib (per tests/pytest.ini), so test discovery works. All relative links resolve. No secrets or personal paths.

Area Ratings

Area	Status	Notes
Frontmatter	OK	No frontmatter changes
E2E Tests	OK	119 task prompts updated; test infrastructure added
Skill Body	OK	Critical Rules 2 and 5 modified, internally consistent
References & Assets	OK	greenfield, brownfield, run.md all updated symmetrically
Repo Hygiene	OK	All changes scoped to maestro-flow; no secrets; no cross-skill refs

Issues for Manual Review

Preamble vs. rule feat(CodedAgents): group skill by functionality #5 interaction in headless mode: The preamble says "take the best available option," while rule feat(CodedAgents): group skill by functionality #5 says "stop and report the open decision" when no dropdown option is marked recommended. The specificity ordering is correct (rule feat(CodedAgents): group skill by functionality #5 wins for dropdowns, rule feat: new coded agent skills #2 wins for debug), but the interplay across these three documents is subtle. Worth confirming in the next nightly that agents navigate it correctly — which is exactly what the PR description says is the real check.
No eval run: The PR correctly notes this changes agent behavior, so the nightly is the real verification. The pytest guards ensure structural correctness; behavioral correctness depends on that run.

Conclusion

Clean, well-motivated PR that solves a real problem (agents stopping at consent gates in headless runs). The approach — canonical text in the prompt, enforced by tests — is the right trade-off given coder_eval's lack of pattern-scoped defaults. The skill-side changes (rule #2 mandate, rule #5 carve-out, (default) removal) are internally consistent and correctly aligned.

No blocking issues. Approve.

github-actions · 2026-09-04T16:39:50Z

Claude finished @rockymadden's task in 11m 16s —— View job

Coder-eval task lint (advisory)

Read lint rubric
Identify changed task YAMLs
Check PR body for evidence of passing run
Apply rubric to sampled files (15 of 119)
Within-PR duplicate check
Compose final comment

Coder-eval task lint (advisory)

119 task YAMLs modified (0 added); all receive the same 10-line headless preamble. 15 files sampled across all directories for full rubric evaluation. Verdicts: 0 Critical, 0 High, 0 Medium, 0 Low introduced by this PR. Pre-existing findings below.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

❌ High — PR body explicitly states "No eval run." and explains: "This changes agent behavior on the same task set, so a skill-flow-* nightly is the real check." The 775 pytest passes cover the guard test (test_headless_preamble.py), not coder-eval task execution. The rationale is sound — a prompt-only change to 119 tasks is best verified by a nightly run, not 119 individual local runs — but per rubric convention, please consider adding a note after the nightly confirms results, e.g.: "Nightly skill-flow-* run on YYYY-MM-DD confirmed no regressions."

Per-task lint

All 119 files receive the identical 10-line preamble addition. The preamble itself is clean across all rubric axes:

Self-report: "Record every decision…in your final response" writes to agent monologue, not a graded file. ✅
Prompt over-specification: Preamble describes execution mode (headless/autonomous), not task procedure. ✅
Other axes: Unchanged — preamble doesn't affect success_criteria, sandbox config, or tags.

Some files also removed old autonomy variants ("Do NOT ask for approval…") — the preamble subsumes them. One file (drive_to_slack.yaml) was reformatted from inline string to block scalar (|); content is unchanged aside from the preamble.

Sampled files with unique findings (all pre-existing on main):

`connector_features/drive_to_slack.yaml` — verdict: Low (pre-existing; see Theme 2)

`evaluate/local_crud.yaml` — verdict: Low (pre-existing; see Theme 3)

`ixp/e2e_01_invoice_extraction_greenfield.yaml` — verdict: Low (pre-existing)

Issues:

[Low] Meaningful coverage: 5 grep-based run_command criteria check node-type presence but not wiring or configuration depth. Validate catches structural errors but not semantic correctness. Justified by CI limitations (IxP + SharePoint need tenant deployment).

Suggested fixes:

Consider adding a python checker that validates wiring between the IxP node output and the HTTP POST input, similar to escalation_slack_alert's outcome checker pattern.

Remaining 12 sampled files — verdict: OK

contractregistry_crud_filters.yaml, escalation_slack_alert.yaml (exemplary outcome-graded e2e), ceql_where.yaml, add_node.yaml, openmeteo_weather.yaml, voice_inbound_call.yaml (exemplary multi-dimensional smoke): all clean.

customer_escalation.yaml, idempotent_reconfigure.yaml, devcon_expense_approval.yaml, quality_01_schema_design.yaml, bellevue_weather.yaml, init_validate.yaml: clean except for Theme 1 (pre-existing missing mode:* tag).

Within-PR duplicates

No duplicate clusters detected. All 119 files are modifications of existing tasks (0 new task_ids), and the change is identical across all of them by design (enforced by test_headless_preamble.py).

Themes

All themes are pre-existing on main — none introduced by this PR.

Theme 1 (Low, pre-existing): 38 tasks missing required mode:* tag. Per test-writing rules, every task needs mode:build, mode:operate, or mode:diagnose. Affects files across bindings/, edit/, hitl/, multi_node/, single_node/, smoke/, ixp/, and e2e/devcon_expense_approval.yaml. Fix these →
Theme 2 (Medium, pre-existing): 1 task with dual tier tags. connector_features/drive_to_slack.yaml has both integration and e2e. Tier must be exactly one value.
Theme 3 (Medium, pre-existing): 3 tasks missing tier tag entirely. evaluate/local_crud.yaml, evaluate/no_auto_upload.yaml, and smoke/inline_agent_robust.yaml have no smoke/integration/e2e tag. (The last one is in the smoke/ folder but the tag is missing from its tag list.)

Conclusion

✅ All changes introduced by this PR pass the rubric. The headless preamble is clean on every axis. The mechanical consistency is enforced by test_headless_preamble.py (775 pytest passing). Pre-existing tag gaps (38 missing mode:*, 3 missing tier, 1 dual tier) are not introduced here. Evidence of passing coder-eval run is absent — author acknowledges this and defers to nightly; consider updating the PR body after the nightly confirms.

…e reads Self-review of the new guard. The module docstring was 24 lines restating the commit message and PR body — the nightly counts, the variant tally, the history. Two facts there are load-bearing for anyone editing the file: why the text is in the prompt rather than a config, and why this parses with regex instead of PyYAML. The rest went. Prose is now 14 of 83 lines, from 30 of 101. The presence check built a `needles` list from every line of the canonical block, then discarded it except for the first sentence, and ran `all()` over the resulting one-element list. It only ever checked "This run is headless." — which is the right cheap marker, since the wording test covers the full text, but the code said something else. Now it says what it does. Four tests each re-read every file in the suite; `_tasks()` is cached. All four guards re-proven after the rewrite: removing the preamble, rewording it, leaking it into a simulated task, and reintroducing a superseded variant each fail the expected test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Out of scope. The template still tells authors to hand-write autonomy language, which the canonical preamble supersedes, but changing it is a separate call from fixing the flow suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rockymadden · 2026-09-04T16:46:31Z

Dropped the tests/templates/test-task-template.yaml change — out of scope for this PR. Moved to the follow-up list, since the template still tells authors to hand-write autonomy language that the canonical preamble supersedes.

Scope is now skills/uipath-maestro-flow/ (4 files) and tests/tasks/uipath-maestro-flow/ (120 files). 775 pytest pass.

Several tasks say "Do NOT run or debug the flow — the grader executes it with the seeded inputs", and one (skill-flow-eval-no-auto-upload) deliberately asserts a refusal. The preamble says the actions a task implies are authorized, which sits against those without saying which wins. Added the precedence sentence to the canonical block and all 119 tasks. The test constant moved in the same commit, so the guard and the tasks cannot disagree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

🟡 Changes recommended

Debug authorization can trigger real side effects without explicit consent, and the completion workflow can still stop at a redundant headless decision gate.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a canonical headless-execution preamble to Maestro Flow evaluations and revises debug-consent guidance.

Changes:

Adds identical headless instructions to 119 zero-shot tasks.
Adds a regression guard for coverage, wording, and simulated tasks.
Revises debug authorization and removes automatic Studio Web publishing defaults.

File summaries

File	Description
`skills/uipath-maestro-flow/SKILL.md`	Revises debug consent rules.
`skills/uipath-maestro-flow/references/operate/run.md`	Updates debug authorization guidance.
`skills/uipath-maestro-flow/references/author/greenfield.md`	Removes publishing default.
`skills/uipath-maestro-flow/references/author/brownfield.md`	Removes publishing default.
`tests/tasks/uipath-maestro-flow/_shared/test_headless_preamble.py`	Guards canonical preamble usage.
`tests/tasks/uipath-maestro-flow/bindings/idempotent_reconfigure.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/bindings/multi_connector_independence.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/bindings/no_duplicate_connection_bindings.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/bindings/reconfigure_different_connection.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/ceql_where.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/complex_array.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/contractregistry_crud_filters.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/e2e_contract_intake_pipeline.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/integration_create_get.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_activation_negative.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_activation_positive.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_create_all_types.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_error.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_file_activities.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_query.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_update.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/smoke_update_existing_flow.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/datafabric_connector/trigger_lifecycle.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/drive_to_slack.yaml`	Converts prompt and adds preamble.
`tests/tasks/uipath-maestro-flow/connector_features/dtl_load_by_default_false.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/dtl_load_by_default_true.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/enhanced_enum.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/enum.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/generate_schema.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/generic_dynamic_node/generic_dynamic_node.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/jdbc_databricks_query/jdbc_databricks_query.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/multiselect.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/non-catalog-http-fallback/non_catalog_http_fallback.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/paginated_reference_lookup.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/path_params.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/query_params.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/searchable_joins.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/slack-http-fallback/slack_http_fallback.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_attachments/testmanager_attachments.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_crud_grounded/testmanager_crud_grounded.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_execution_results/testmanager_execution_results.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_generic_records/testmanager_generic_records.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_requirement_lifecycle/testmanager_requirement_lifecycle.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_testcase_lifecycle/testmanager_testcase_lifecycle.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_features/testmanager_testset_lifecycle/testmanager_testset_lifecycle.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_trigger/trigger_with_filter.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/connector_trigger/webhook_waitfor_parallel.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/context-grounding/batch_transform/batch_transform.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/context-grounding/summarize/summarize.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/devcon_expense_approval.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/escalation_jira_ticket/escalation_jira_ticket.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/escalation_orchestrator_paths/escalation_orchestrator_paths.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/escalation_slack_alert/escalation_slack_alert.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/jira_create_issue/jira_create_issue.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/jira_get_issue/jira_get_issue.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/jira_lifecycle/jira_lifecycle.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/e2e/jira_search_triage/jira_search_triage.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/add_node/add_node.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/add_output/add_output.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/group_to_subflow/group_to_subflow.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/move_node/move_node.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/remove_node/remove_node.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/edit/update_node/update_node.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/child_simulation/child_simulation_crud.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/evaluator_type_choice.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/inline_agent_eval/inline_agent_eval.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/local_crud.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/no_auto_upload.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/evaluate/simulation/simulation_crud.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/quality_01_schema_design.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/quality_02_result_downstream.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/quality_03_boolean_decision.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/quality_04_brownfield_insert.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/smoke_01_hitl_node_placed.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/smoke_02_completed_port_wired.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/hitl/smoke_03_multi_outcome_routing.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/e2e_01_invoice_extraction_greenfield.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/e2e_02_project_selection.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/e2e_03_project_creation_handoff/e2e_03_project_creation_handoff.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/integration_handle_routing.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/routing.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/routing_listing.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/routing_negative.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/scaffold_minimal.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/ixp/scaffold_multinode.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/bellevue_weather/bellevue_weather.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/billing_discrepancy_detector/billing_discrepancy_detector.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/billing_dispute_analyst/billing_dispute_analyst.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/billing_dispute_resolution/billing_dispute_resolution.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/billing_invoice_lookup/billing_invoice_lookup.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/billing_resolution_writer/billing_resolution_writer.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/calculator/calculator.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/customer_escalation/customer_escalation.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/dice_roller/dice_roller.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/feet_inches/feet_inches.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/loop_multiply/loop_multiply.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/multi_city_weather/multi_city_weather.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/reading_list/reading_list.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/slack_channel_description/slack_channel_description.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/slack_weather_pipeline/slack_weather_pipeline.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/multi_node/wiki_pageviews/wiki_pageviews.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/api_workflow/api_workflow.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/coded_agent/coded_agent.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/decision/decision.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/delay/delay.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/file_attachment/file_attachment.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/lowcode_agent/lowcode_agent.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/openmeteo_weather/openmeteo_weather.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/outlook_trigger_inbox/outlook_trigger_inbox.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/outlook_waitfor_email/outlook_waitfor_email.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/rpa/rpa.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/subflow/subflow.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/switch/switch.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/terminate/terminate.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/transform_filter/transform_filter.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/transform_group_by/transform_group_by.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/single_node/transform_map/transform_map.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/smoke/init_validate.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/smoke/inline_agent_robust.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/smoke/merge_parallel_sync.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/smoke/registry_discovery.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/smoke/scheduled_trigger.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/voice/voice_inbound_call.yaml`	Adds headless preamble.
`tests/tasks/uipath-maestro-flow/voice/voice_outbound_call.yaml`	Adds headless preamble.

Review details

Suppressed comments (1)

tests/tasks/uipath-maestro-flow/_shared/test_headless_preamble.py:60

The PR now documents nine superseded wording variants, but this test still says eight. Update the count so the guard explains the current baseline accurately.

    """8 variants is what made the old line unmaintainable. One wording, or none."""

Files reviewed: 124/124 changed files
Comments generated: 5
Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…unbounded consent A 10th autonomy variant survived the sweep: five billing tasks carried "Build the complete flow / without stopping to ask me." The guard advertised catching superseded variants and did not match that wording, so it passed with the duplication in place. Removed from the five, and the matcher now covers it — verified by reintroducing the phrase and watching the test fail. devcon_expense_approval's "Build the complete flow as a UiPath Flow project called ..." is task text and is untouched. Both What's next menus were unconditional, so a request that already named the next step ("run debug and iterate") still stopped for a redundant selection — the exact gate this work removes elsewhere. An instruction in the original request is now the selection; the menu shows only when the next step was unspecified. Rule #2's mandate said an ordinary build request authorizes a real run. That is too broad for side effects reaching someone who is not the user: a phone call, an email to a real person. Those now need the run asked for explicitly, whoever is watching, and rule #2 points at the outbound-call case in inline-voice-agent/impl.md, whose own gate this never touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…3090) On 2026-09-04 two flow tasks were scored FAILURE for reasons that had nothing to do with the agent: skill-flow-outlook-trigger-inbox AADSTS50173 — the Outlook grant was revoked on 2026-08-31 skill-flow-generic-dynamic-node the ServiceNow developer instance was hibernating, so every metadata call 403'd Both were root-caused as skill defects first. The 403 is several thousand characters into the checker output, and nothing above it says the tenant is down, so the reports read as "the agent wrote a display name where an ID belongs" — which it did, but only because the lookup it was told to use could not answer. A `pre_run` failure lands the run as FinalStatus.ERROR rather than FAILURE, and PreRunCommand's own docstring gives the reason: "the agent should not run against a broken environment." This uses that. preflight_connections.py takes connector keys, lists their connections with --all-folders (without it an empty result is a false negative), and exits non-zero unless at least one is Enabled. Wired into the two tasks above. Behaviour verified against six tenant states: all-enabled, hibernating, mixed with one up, no connections in any folder, CLI non-zero, and a Failure envelope. Only those two tasks are wired. 23 others name a connector, but that list comes from grepping prompts for `uipath-*` and it catches non-connectors like uipath-rpa and uipath-platform — worth doing deliberately, not from a crude match. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Resolves fifteen conflicts for #2756, all from main #3088 (one canonical headless preamble on every zero-shot flow task) meeting the campaign's loop-neutral prompt rewrites at the tail of `initial_prompt`. Resolution, identical in every file: keep the campaign's loop-neutral prompt text (no `uip` command hints, no .uiproj-only path), drop the superseded autonomy sentences ("Do NOT ask for approval...", "Do NOT pause between planning...", "...in a single pass") that #3088 replaced, and end the prompt with the canonical headless paragraph exactly as `_shared/test_headless_preamble.py` requires. Files: connector_features/{non-catalog,slack}-http-fallback, connector_trigger/webhook_waitfor_parallel, context-grounding/{batch_transform, summarize}, evaluate/{evaluator_type_choice,inline_agent_eval}, multi_node/slack_weather_pipeline, single_node/{delay,transform_filter, transform_group_by,transform_map}, smoke/{init_validate,merge_parallel_sync, scheduled_trigger}. ixp/routing.yaml and ixp/routing_negative.yaml auto-merged but the campaign-only scope sentence (09-01 routing RCA) still contained "Do NOT ask for approval, confirmation, or missing details", which the new guard rejects as a superseded variant. Reworded to "pick a sensible default for any missing detail and state it in your final report"; the scope statement itself is unchanged. Verified: uipath-maestro-flow 1159 passed (incl. test_headless_preamble and test_same_ground_corpus), uipath-maestro-case 124 passed / 14 skipped, tests/scripts 109 passed / 26 skipped, uipath-maestro-bpmn 12 passed; scripts/check-cli-verbs.py clean on all touched yaml. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JSsrfZwCh4tJeqpzjMeixP

rockymadden and others added 2 commits September 4, 2026 10:44

rockymadden requested a balanced review from Copilot September 4, 2026 16:48

Copilot started reviewing on behalf of rockymadden September 4, 2026 16:49 View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

jiyangzh approved these changes Sep 4, 2026

View reviewed changes

rockymadden merged commit 1ec2ba2 into main Sep 4, 2026
27 of 28 checks passed

rockymadden deleted the feat/flow-headless-task-prompts branch September 4, 2026 17:57

Navigation Menu

Uh oh!

feat(uipath-maestro-flow): one canonical headless preamble on every zero-shot flow task - #3088

feat(uipath-maestro-flow): one canonical headless preamble on every zero-shot flow task#3088
rockymadden merged 6 commits into
mainfrom
feat/flow-headless-task-prompts

rockymadden commented Sep 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

rockymadden commented Sep 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rockymadden commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Why the task prompt and not an experiment config

Changes

Skill side

Verification

What this costs

Follow-ups (separate)

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change-by-Change Review

1. tests/tasks/uipath-maestro-flow/_shared/test_headless_preamble.py (new)

2. skills/uipath-maestro-flow/SKILL.md — Critical Rules 2 and 5

3. skills/uipath-maestro-flow/references/author/greenfield.md and brownfield.md

4. skills/uipath-maestro-flow/references/operate/run.md

5. 119 task YAML files — headless preamble addition

6. tests/templates/test-task-template.yaml

What's Missing

Area Ratings

Issues for Manual Review

Conclusion

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coder-eval task lint (advisory)

Coder-eval task lint (advisory)

Evidence of passing run

Per-task lint

connector_features/drive_to_slack.yaml — verdict: Low (pre-existing; see Theme 2)

evaluate/local_crud.yaml — verdict: Low (pre-existing; see Theme 3)

ixp/e2e_01_invoice_extraction_greenfield.yaml — verdict: Low (pre-existing)

Remaining 12 sampled files — verdict: OK

Within-PR duplicates

Themes

Conclusion

Uh oh!

rockymadden commented Sep 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rockymadden commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

1. `tests/tasks/uipath-maestro-flow/_shared/test_headless_preamble.py` (new)

2. `skills/uipath-maestro-flow/SKILL.md` — Critical Rules 2 and 5

3. `skills/uipath-maestro-flow/references/author/greenfield.md` and `brownfield.md`

4. `skills/uipath-maestro-flow/references/operate/run.md`

6. `tests/templates/test-task-template.yaml`

github-actions Bot commented Sep 4, 2026 •

edited

Loading

`connector_features/drive_to_slack.yaml` — verdict: Low (pre-existing; see Theme 2)

`evaluate/local_crud.yaml` — verdict: Low (pre-existing; see Theme 3)

`ixp/e2e_01_invoice_extraction_greenfield.yaml` — verdict: Low (pre-existing)