Spike: surface UIA events to MCP clients as notifications

[asklar]

Summary

Surface UI Automation events to MCP clients as notifications, so an agent can react to what an app does instead of polling for it.

This is a spike write-up, not a proposal to build it a particular way. Splitting it out of #45 because nothing there is blocked by it — see "Why this is separate" below.

Why it's worth doing

Today an agent driving an app through lvt mcp has two ways to know something happened:

• call wait_for, which polls a rebuilt UIA tree until a property matches, or
• re-read the tree and diff it themselves.

Both are polling, both cost a full walk per poll, and neither can tell you about something you didn't already know to wait for — a dialog appearing, focus moving, a background operation finishing. Events are how you notice the app doing something on its own.

What exists today

lvt does no UIA event subscription. watch (run_watch_loop, src/main.cpp) rebuilds the whole tree on an interval and diffs it against the previous snapshot. There is no AddAutomationEventHandler / IUIAutomationEventHandler anywhere in the codebase.

So this is new capability, not plumbing an existing one through.

The three questions a spike has to answer

1. How should events reach the client? (the real design decision)

MCP has no generic "something happened in the app" notification. The candidates:

shape	fit	cost
notifications/message (logging)	Easy — carries arbitrary JSON with a severity. But it is logging: hosts may filter it, and it isn't guaranteed to reach the model.	Small
resources + subscribe + notifications/resources/updated	Semantically right: the client subscribes to a window's tree and is told when it changes. This is MCP's native "tell me when this changes" pattern.	Adds a whole resources capability the server doesn't have
notifications/progress	Tied to an in-flight request token. Wrong shape for unsolicited events.	—

My reading is that resources + subscribe is the correct design and logging notifications are the cheap one, and that choosing between them is worth a discussion rather than a coin flip. It roughly doubles the server's protocol surface either way.

2. What does the C ABI need?

lvt_api_call(method, params, allow_input, char** result) is request/response only; events need a push channel. That would be additive — something like:

typedef void (*lvt_api_event_sink)(const char* event_json, void* context);
void lvt_api_set_event_sink(lvt_api_event_sink sink, void* context);

No existing signature changes. src/lvt_api.h is an installed public header, so additive matters.

3. What are the hard parts?

• Threading. UIA event callbacks arrive on UIA's own threads, not ours. They'd have to be marshalled onto the tokio runtime to be sent, without blocking the callback (the same "never block the provider" rule the TAP DLL already follows).
• Lifetime. Handlers must be removed on disconnect, on window close, and on server shutdown. RemoveAllEventHandlers is easy to forget and leaks into the target's process.
• Volume. A chatty app can raise thousands of property-changed events. Without filtering and coalescing this becomes a denial-of-service against the client's context window. Subscriptions almost certainly need to be scoped — by element, by event kind, or both.
• Which events. StructureChanged, AutomationPropertyChanged, AutomationFocusChanged, Window opened/closed, Invoke. Probably not all of them, and probably opt-in per subscription.
• Interaction with modes. A visual session has no UIA subscription available — the visual tree has no event model. Either events are UIA-only and say so, or visual sessions fall back to polling diffs like watch does.

Why this is separate from #45

• Nothing in Add MCP server mode, so lvt can be used as a computer-use tool #45 is blocked by it. The ABI extension is additive, so the event design cannot force a different shape on what's already there.
• Add MCP server mode, so lvt can be used as a computer-use tool #45 is +6736/-7 across 12 commits and has taken three review rounds, each of which found a critical bug. Adding a new capability to it now would put currently-green, mergeable work back at risk.
• The MCP modelling decision above deserves its own discussion, and the answer changes how much gets built.

Suggested first slice

If this proceeds, the smallest thing that proves the design:

1. lvt_api_set_event_sink + one UIA event kind (AutomationFocusChanged — low volume, easy to verify).
2. Emit as notifications/message first, since it needs no new capability, and see whether hosts surface it usefully.
3. Only then decide whether to model subscriptions as resources.