fix(uipath-maestro-flow): derive flow debug consent from the mandate, unmark the publish default

[rockymadden]

Replaces the original scope of this PR. See the history-rewrite comment below for why.

Two defects, both consequences of rule #5 rather than gaps in it

1. flow debug is classed as a consent gate that is never auto-answered.

In the 2026-09-04 nightly, 5 of 8 skill-flow-* tasks validated their flow, reported success, and never executed it. The checker then ran debug and found a null End-node output mapping, a faulted script, and an empty result. flow validate passed on all of them.

Consent for debug is not a question that needs a live user. It comes from what was asked for: a request to build something that does X is a request for it to work, and building plus validating does not discharge that. Rule #2 now says so, rule #5 defers to it, run.md matches.

Added the one guardrail debug genuinely needs: it overwrites the Studio Web solution matching the local .uipx SolutionId, confirmed in 3 of 8 runs and by the eval suite's own cleanup_solutions.py ("Delete Studio Web solutions uploaded by uip maestro flow debug"). So never debug a solution this run did not scaffold.

2. The marked default is a tenant write.

Rule #5's non-interactive fallback takes the marked recommended option. The What's next dropdown marked Publish to Studio Web (default). A headless run would auto-publish to the tenant. Marking dropped in both greenfield and brownfield; nothing else in either table is marked, so the fallback now falls through to stop-and-report.

Scope

4 files, 5 insertions, 5 deletions. SKILL.md +596 bytes (+2.8%).

Modifies Critical Rules 2 and 5, called out per CONTRIBUTING.

Verification

• .maintenance/check-all.sh clean apart from one anchor that pre-exists on main.
• Both flavors compose.
• No eval run. This changes agent behavior on the same task set, so a skill-flow-* nightly is the real check.

Follow-ups (separate PRs)

1. 34 maestro-flow eval tasks have a checker that runs flow debug and a prompt that never asks the agent to. Align them.
2. greenfield.md Step 2: assert one .flow + one project.uiproj after scaffold. skill-flow-bindings-no-duplicates passed its actual regression and failed only because a stray duplicate project confused the checker glob.
3. script/impl.md: extend the existing node -e "new Function(...)" syntax check to a value test against real upstream shape. Would have caught the off-by-one in skill-flow-slack-weather-pipeline that shipped "Suite 2000" as a city name.
4. variables-and-expressions.md: document the =js: expression boundary. An End-node mapping using object spread returned null with a clean validate. Needs a repro first.
5. dice_roller picks a HITL quick-form to "display" a value, so the flow never terminates and debug times out. Node-selection guidance, not a consent problem.

🤖 Generated with Claude Code

[rockymadden]

Review fixes pushed

Self-review found the first cut both wrong and too expensive. All 9 findings addressed.

🔴 Correctness

C1 — the flow debug carve-out authorized the write the same PR forbids. Debug uploads the solution to Studio Web and overwrites whatever matches the local .uipx SolutionId, then writes the Studio Web id back. Confirmed in 3 of 8 nightly runs and by the eval suite's own cleanup script:

_shared/cleanup_solutions.py — "Delete Studio Web solutions uploaded by uip maestro flow debug during a task."

The old text called the run "disposable" and exempt from the no-tenant-writes line. Debug now requires two tests, either failing is a Stop:

1. Provenance — solution scaffolded by this run, so the overwrite can only hit what this run created. Pre-existing or solution download-ed → Stop, same rule as upload-safety.md.
2. Side-effect profile — every node reads.

C2 — the read test misread two node shapes. core.action.http* always carries operation: "create" (that describes the proxy activity, not the call), and in connector mode detail.method is POST to /http-request while the real verb is bodyParameters.method. Both made a read-only Slack list look like a write. Validated the corrected predicate against all 8 nightly flows:

Flow	Verdict
bellevue-weather, slack-channel-description, -simulated, slack-weather-pipeline	READ-ONLY → debug runs
cli-dice-roller (HITL quick-form)	WRITES → blocked
generic-dynamic-node (unconfigured connector)	WRITES → blocked

Exactly the right split: the 4 that needed a feedback loop get one, the HITL flow that hung the checker does not.

C3 — detection was not evaluable. It required knowing the run is single-turn. Absence of a structured-question tool is sufficient; the conjunct is gone.

🟠 Cost and structure

M2 — SKILL.md is always loaded. First cut: 21,655 → 28,376 bytes (+31%, ~1,700 tokens on every invocation, for a mode most runs are not in). Now +5% (22,745). The doctrine lives in references/shared/non-interactive.md behind numbered rule #13 plus a Capability router row. Depth check: 83 files, all within 2 hops.

M1 — violated .claude/rules/token-optimization.md §0 by name: "the same fact stated in N places compresses to ONE full statement + N pointers." Shipped a classification table and 17 clauses restating it. Table is now the single source of truth; clauses are pointers.

M3 — body order. New top-level ## violated skill-structure.mdc. Rule #13 + reference fits it.

🟡 Minor

m1 router row added · m2 precedence now adjacent to each prohibition · m3 one consistent test (the exemption language is gone) · m4 dropdowns now state "Never mark an option in this table as a default" positively, so a future editor cannot re-add it by accident · m5 advisory criterion on bellevue_weather asserting the agent runs debug itself · m6 unverified "disposable" claim removed

Verification

• .maintenance/check-all.sh: no new findings. 1 bad anchor pre-exists on main.
• Composed both flavors and re-checked links + anchors: default 1 pre-existing anchor + 1 pre-existing cross-tree link (from fix(skills): flatten nested references/ dirs, add a link checker, and drop a dead plugin pointer #2879), studioweb 2 pre-existing anchors. Zero introduced.
• Read-only predicate exercised against all 8 nightly .flow artifacts, results above.

Known gap

non-interactive.md carries solution upload and Orchestrator-deploy rows that do not apply to the studioweb flavor. Harmless noise rather than an error; adding flavor markers is more machinery than it is worth unless a reviewer disagrees.

[rockymadden]

Remove the never mark option comment.

[rockymadden]

Removed.

[rockymadden]

Remove the comment about never mark an option.

[rockymadden]

Removed.

[rockymadden]

Do we really need the studio web details? Probably not. Just say plainly without details. Also, the flags and examples you cite are coupling to things that can drift. Consider and revise for maintainability.

[rockymadden]

Rewritten plain. Dropped the Studio Web recovery mechanics and the --solution-id / --project-id flags. Now states the Stop, defers to the options already listed above it, and notes that flow debug performs the same write so it inherits the rule.

[rockymadden]

See if we can express this without the rule ref for future mutation concerns (numbers changing).

[rockymadden]

Now links the reference doc instead of naming a rule number.

[rockymadden]

See if we can express the same without citing the number which might change. Long term drift and maint concerns.

[rockymadden]

Same, links instead of numbering. All four rule-number references this PR introduced are gone. Left the pre-existing ones elsewhere alone as unrelated.

[rockymadden]

If you can do so without losing effectiveness, remove the specific examples that might drift and cause maint issues.

[rockymadden]

Dropped. Detection now reads "harness exposes no structured-question tool, the prompt forecloses asking, or the run is unattended". The phrasings stay in the reference doc, where drift is cheap and they are not loaded on every invocation.

[rockymadden]

Far too many comments. I think this should be removed as part of this PR.

[rockymadden]

Reverted the whole criterion, not just the comments. You were right to ask why it was in the PR at all: eval-suite work belongs in the follow-up that aligns prompts with checkers, and a weight-0 advisory cannot fail, so it was observability rather than a test. tests/ is untouched by this PR now.

[rockymadden]

Narration strip

Ran every added line through "does this change what the agent does?" and deleted what failed.

Cut

Line	Why
"A gate with no reachable answer is a deadlock, not a safeguard"	Argument aimed at a reviewer
"Owned by the no-user rule in SKILL.md… per-site notes point here rather than restating it"	Doc-structure narration
"A flow plus three stated assumptions beats a blocked turn with nothing built"	Justifies the rule above it, adds nothing to it
"the approval buys human review of an opaque diff and there is nobody to review it"	Same
"the overwrite can only hit what this run created" / "Someone else's Studio Web state may be behind that id"	Same
"flow validate checks structure, it does not parse script bodies…"	Same
"Silently taking an Assume is the defect, not the assumption"	Rhetoric; the Reporting list already mandates the behavior
"Two traps this table exists to defuse…" paragraph	Restated the table directly beneath it
uip is connections create "needs a browser"	The Stop is the instruction, the reason is not

Kept, because the reason is the instruction

• "A display name written where a resolved ID belongs passes flow validate and faults only at runtime, so a green validate is not evidence." Changes what the agent treats as proof.
• Which field to read per node type. Folded the two core.action.http* gotchas into the table cells that qualify, so the rule and its exception are one lookup.

Also

• Last two rule-number citations (rule #6, author rule #17) replaced with descriptions. Zero remain in the new file.
• run.md now points at the two tests instead of restating them.

Numbers

	before	after
non-interactive.md	7,705	6,396
SKILL.md vs main	21,655	22,541 (+4.1%)

Re-ran the embedded checker against all 8 nightly .flow artifacts: verdicts unchanged (4 read-only, HITL and unconfigured-connector blocked). check-all.sh clean apart from the anchor that pre-exists on main.

[rockymadden]

Rewrote the branch: 18 files → 4

The original premise was wrong, so I reset to main and reapplied only what survives. History rewritten rather than stacking a revert, since the intermediate commits describe a feature that no longer exists.

What was wrong. I built a "no user available" doctrine (new 6.4 KB reference doc, 17 per-site clauses, a Proceed/Assume/Stop vocabulary, a python side-effect checker) on the claim that the skill had no defined behavior when nobody can answer a question. It did. Rule #5 on main already reads:

Non-interactively (CI/headless, no user available) → take the marked recommended option... These fallbacks define "ask the user" / "confirm with the user" wherever this skill's references require it.

That last clause already applied rule #5 to every gate by default. My 17 clauses restated a rule that was already in force.

I had also conflated two different states under one label: genuinely headless (CI, cron, eval harness) and a present user who said "don't ask me." The second is not a deadlock. That user reads the report and answers next turn. The deadlock I was solving only exists in the first case, which rule #5 already covered.

What survives. Two real bugs, neither needing a new concept:

flow debug classed as a never-auto-answered consent gate	5 of 8 nightly tasks never executed their flow. Consent now derives from the mandate, which applies with or without a user watching.
(default) on a dropdown option that writes to the tenant	Rule #5's fallback takes the marked option, so a headless run would auto-publish. Marking dropped.

Dropped: references/shared/non-interactive.md, all 17 No user? clauses, the Proceed/Assume/Stop vocabulary, the read-only node checker, and the edits to planning gates, the scripting ladder, upload safety, voice trunks, and solution selection.

Known consequence. Without the Assume-over-Stop default, a headless run still stops on an unresolvable reference. That is what main already does. The 3 tasks that shipped a display name where an ID belongs ("Inbox", "Acr User") violated a rule that already existed as an anti-pattern; no wording fixes that, only an eval or a validator does. Tracked as follow-up.

[rockymadden]

Headless behavior is now a rule of its own, and self-sufficient

Two changes, both from reviewing where the instruction lived and what it said.

It was in the wrong place

The behavior was the tail of rule #5, whose headline is "Always present finite decisions as a dropdown with a final Something else escape hatch." That put it 61% into the file and 574 characters into the rule. An agent scanning rule headlines for "no user, what now?" does not match on a rule about dropdown menus.

Moving it up the file was the wrong fix: renumbering costs ~24 cross-references across 8 files, and a new top-level section violates the body order in skill-structure.mdc. Appending rule #13 renumbers nothing.

It was also wrong, because of this PR

Rule #5 said take the marked recommended option. Dropping the two (default) markings left exactly one marked option in the entire skill (impl-connector.md:36, manual mode). So every finite decision fell through to stop and report the open decision — the opposite of what a headless run should do with a choice it is perfectly capable of making. I introduced that by removing the markings without revisiting the rule that consumes them.

Rule #13

Covers the four cases the skill actually hits, and states detection as well as behavior, so it holds whether or not the harness ever announces the mode:

Situation	Do
Finite choice	Take the best option, not only a marked one
Missing value (required field with no default, an output whose meaning the request never states)	Supply the most defensible one, keep building
Irreversible or externally visible (destructive operations, tenant writes, a browser login)	Do not. Finish everything that does not depend on it first
A fact you could not resolve (a lookup that 401s, 403s, or returns nothing)	Stop on that field. Never a display name, alias, or remembered ID

Every decision, assumption and blocked step goes in the final report. Rule #5 keeps the ask mechanism and points at #13.

That last row is the pathology in 3 of the 8 nightly failures: parentFolderId: "Inbox" and objectName: "Acr User" written after a 403. Both pass flow validate and fault at runtime.

Also

Dropped the two rule #2 citations this PR had added, in SKILL.md and run.md, in favor of naming the rule. Zero rule-number references introduced by this PR now.

Scope

4 files, 6 insertions, 5 deletions. SKILL.md 21,655 → 22,817 bytes (+5.4%), which is the honest cost of the skill owning this behavior in prose rather than deferring it.

check-all.sh clean apart from the pre-existing anchor. Both flavors compose.

[rockymadden]

Headless rule moved above every link in the file

Was Critical Rule #13 at line 88 — past the capability list, the 13-row router, and 12 other rules. Now the first bullet of # Working style at line 14, fifteen lines ahead of the first link in the file, so no reference can be followed before it is read.

Why Working style and not Critical Rule #1

Rules 1–12 are .flow mechanics: node ownership, =js: prefixes, ID formats. Headless is an operating mode, which is exactly what that section holds ("Understand first, then decide", "Don't repeat work", "Keep outputs small").

Making it rule #1 would have renumbered the list and broken 29 rule #N citations across 7 files. Appending was free; prepending was not.

On the read-order argument

Worth recording that this did not fire in the nightly. All 8 runs read SKILL.md in full as their first action, before any reference:

sed -n 1,240p .../SKILL.md      # the file is 101 lines
SKILL.md@12-21    first-reference@25-48    (8/8: SKILL first)

Moved it anyway, for the ordering-of-meaning reason rather than the read-order one: a rule that changes how everything below it behaves should not be stated after all of it. The read-order case is a robustness argument for other harnesses and future readers, not an observed failure.

Links fixed

• Rule feat(CodedAgents): group skill by functionality #5 pointed at "the headless rule below" → now links up to [Working style](#working-style).
• The bullet said flow debug "is governed by its own rule above" → that rule is now below it, so it names the section instead of a direction.

Verification

• All 9 check-all.sh checkers: clean apart from the anchor that pre-exists on main. The new #working-style anchor validates.
• Composed both flavors and re-checked anchors: default 1, studioweb 2, identical to main. Zero introduced.
• The bullet survives the studioweb flavor (no markers, passes through).

4 files, 6 insertions, 5 deletions. SKILL.md 21,655 → 22,858 (+5.6%).

[rockymadden]

Context update: the companion harness PR (UiPath/coder_eval#156) is closed, per thread feedback.

We shouldn't be doing this as it directly impacts simulations. If you expect models to zero-shot it, we should either: 1/ update the skill to encourage zero-shot behavior, or 2/ create realistic simulations based test.

This PR is option 1, so it stands unchanged and is now load-bearing rather than complementary. The Working style bullet already states detection and behavior, so it does not depend on any harness announcement:

The run is headless when the harness says so, the prompt forecloses asking, or it is CI, a scheduled job, or an eval.

The declaration of the fact moves to tests/experiments/*.yaml under defaults.agent.system_prompt, which already exists and which experiment.variant > task > experiment.defaults makes overridable. That is a separate PR in this repo.

No change to this one.

[rockymadden]

Superseded by a single PR that carries these changes plus the eval-config work they depend on.

Splitting them was wrong. This PR removed the (default) marking that rule #5's fallback consumes, while the replacement instruction lived in a different PR — merged alone, it would have left headless runs stopping on decisions they should have made.

The surviving changes here (flow debug consent from the mandate, unmarking the publish default) move over intact. The headless behaviour that briefly lived in this PR as a Working style bullet moves to the flow experiment's system prompt, per thread feedback.

Review history worth keeping: the rule-number drift, the eval-suite change that did not belong, the narration cuts, and the flow debug carve-out that authorized the same Studio Web overwrite upload-safety.md forbids. All of that is folded into the replacement.