Add lvt Viewer: a graphical live visual tree GUI - #49
Merged
Conversation
A new WPF (.NET) desktop app under src/viewer/LvtViewer/ that gives lvt a Visual-Studio-Live-Visual-Tree / Inspect.exe style GUI: - Crosshair-drag target picking (Interop/CrosshairPicker.cs): press-drag a toolbar crosshair, a highlight overlay tracks the top-level window under the cursor, and on release it's resolved via WindowFromPoint + GetAncestor(GA_ROOT) + GetWindowThreadProcessId, exactly like Inspect.exe. - Two-pane view: a live TreeView of elements (color-coded by framework) and a property panel showing bounds/text/className/framework plus, for UIA targets, AutomationId/ControlType/SupportedPatterns/pattern state. - Real-time updates driven by lvt watch's JSON diff stream — chosen over polling lvt dump or driving lvt mcp (which has no push/subscribe tool, only wait_for). See src/viewer/README.md for the full reasoning. - Property editing for Toggle.ToggleState and Value.Value/RangeValue.Value, via lvt's existing oggle/set-value action verbs, addressed by lvt's durable element key rather than its positional eN id. The viewer never links lvt_core; it drives lvt.exe as a subprocess (Services/LvtCli.cs, Services/WatchSession.cs), located via Services/LvtLocator.cs (LVT_EXE env var, alongside the viewer, this repo's own build/ directory, or PATH). Verified end-to-end against a live Notepad instance: connect, live tree/ property population, a live text-value change via set-value, and a live bounds update from resizing the target window, all without manual refresh. See src/viewer/README.md for the one piece that could not be physically exercised in this sandbox (the mouse-drag gesture itself, blocked by a locked interactive desktop) and how its logic was verified instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Adds an opt-in LVT_BUILD_VIEWER option (default OFF, mirroring LVT_BUILD_MANAGED) that runs dotnet build on the viewer's csproj as a custom target, and copies lvt.exe alongside the viewer's output so a fresh cmake --build build produces a viewer that works with zero extra setup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
The physical crosshair-drag gesture was re-verified end-to-end on an unlocked desktop: a real mouse-down/drag/mouse-up correctly resolves and connects to the window under the cursor on release, cross-checked twice against an independent WindowFromPoint/GetAncestor/GetWindowThreadProcessId call at the same screen point (exact HWND/PID match both times). Replaces the prior locked-desktop caveat, which no longer applies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
… lvt.exe Manual testing against Notepad (with UI Automation off, using the visual tree) surfaced "lvt: TAP DLL not found" and a tree showing only bare Win32 bridge/window structure - none of the actual WinUI 3 content inside Notepad's XAML islands. Root cause: the LVT_BUILD_VIEWER wiring only copied lvt.exe next to the viewer's build output; get_tap_directory() resolves TAP DLLs relative to lvt.exe's own module directory, so that specific copy of lvt.exe could never find lvt_tap_x64.dll (or lvt_wpf_tap/lvt_winforms_tap, or any plugin). A partial copy is worse than none: it resolves silently and just stops enriching anything, indistinguishable from the target not using that framework at all. Fix mirrors the release packaging step in .github/workflows/release.yml and the install() rules already in this file: copy every TAP DLL, managed .NET assembly, and plugin (including the Avalonia TAP/managed walker and the Chromium extension/host) alongside the viewer's lvt.exe, guarded by if(TARGET ...) so it degrades gracefully when a framework is disabled. This logic had to move out of the LVT_BUILD_VIEWER block (right after the `lvt` executable) to after every target it references is actually defined further down the file - if(TARGET lvt_tap) and friends only see targets CMake has already processed by that point in a linear read of the file, so the first attempt silently no-op'd every check and only ever copied lvt.exe itself, which is exactly what caused the bug above. Verified with a clean build (build/viewer deleted, full rebuild): every TAP DLL, managed assembly, and plugin (Avalonia's TAP + tree walker, Chromium's plugin + native host + extension) now lands correctly alongside the viewer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
lvt watch re-injects XAML diagnostics and walks the entire visual tree from scratch on every poll tick (default 500ms), forever. For a rich production tree (hundreds to thousands of elements, e.g. File Explorer's or Settings' WinUI3 shell) that walk is dispatched via a blocking SendMessage onto the target's UI thread. Doing this unbounded, every tick, indefinitely, caused a real, reproduced hang: opening a second File Explorer window needed that same UI thread for its own startup handshake and got stuck behind our walk. Two independent bugs contributed: 1. Unbounded per-tick UI-thread occupancy (lvt_tap.cpp). Added a wall-clock time budget (kUiThreadBudgetMs) to CollectBounds/CollectPositionsAndText, tuned to 1500ms after measuring real per-node cost (~4ms on a richly styled WinUI3 app) — comfortably under the ~5s threshold Windows itself uses to mark a window as not responding, while covering far more of a real tree than an initial too-tight 200ms budget did (4% -> 32% node coverage on a 1104-node test case). 2. Cross-window content contamination (xaml_diag_common.cpp). InitializeXamlDiagnosticsEx targets a whole process, not a single HWND. When several top-level windows of the same app share one process (multiple Notepad or File Explorer windows are common, and were exactly how both bugs were found), every window's XAML content comes back in one combined stream, and the old bridge-matching heuristic had no rejection threshold: it always forced every root onto *some* bridge, including bridges belonging to a different window, silently misattaching one window's elements (and their Text/bounds) onto a sibling window's tree. Now a match is only accepted within a size-relative tolerance of its bridge, but only when more than one root is actually competing for attention — a single-root window (the overwhelmingly common case) keeps the exact legacy behavior, so no regression there. Verified: 111 unit tests and 74 integration tests (1 pre-existing skip) all pass, including the WinUI3-stitching suite that exercises this exact grafting logic. Manually reproduced the original hang against a live File Explorer instance, confirmed both processes stayed fully responsive after the fix, and confirmed WinUI3 content that had been dropped by an intermediate version of this fix (single-root, zero-measured-content case) is correctly restored.
asklar
force-pushed
the
lvt-visual-viewer
branch
from
August 21, 2026 22:54
45875e4 to
80f45fa
Compare
xaml_should_capture_property previously only captured a hand-picked list of "text" property names (Text, Content, Header, ...) plus a hand-picked list of "state" property names (Visibility, IsEnabled, Tag, ...) - every other property, however ordinary (FontSize, Opacity, a custom DP), was always rejected regardless of its value. The property panel should show a control's properties generally, not just the ones lvt happened to special-case for the earlier XAML/WPF property-interpretation bug fixes. The former "text property" branch and the desired general behavior turn out to be identical: accept a confirmed primitive ValueType (String, Boolean, Int32, Double, Enum, or empty/unconfirmed) unless the value's shape looks like an opaque XAML handle. So the property-name allowlist for that branch is gone; every non-state property now goes through the same check. State properties keep their existing, narrower exemption (bypass the handle heuristic entirely) since that protects specific known-string properties like AutomationId from being misjudged when ValueType doesn't come through confirmed - it is not something safe to extend to arbitrary properties. Verified via the existing xaml_should_capture_property unit tests (updated NonTextNonStatePropertiesAreIgnored, now ArbitraryPropertiesAreCapturedWithRecognizedValueTypes, plus a new test confirming reference-typed properties like Foreground/Brush are still excluded) and confirmed live against a running Settings window: property count captured across the tree increased substantially (734 properties across 1104 nodes, up from a much smaller curated-only baseline). Known remaining limit, not addressed here: IVisualTreeService:: GetPropertyValuesChain only reports properties that have some explicit value source in the chain (local/style/template/animation) - it does not enumerate every property a control type supports the way walking XAML's IXamlType metadata would. Truly exhaustive "every property, including ones nobody ever touched" needs that bigger, separate change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…lter Adds five requested viewer improvements on top of the existing crosshair targeting, live tree, and property panel: 1. Highlight selected node in the target app (MainWindow.xaml.cs). Reuses the same borderless/click-through/topmost HighlightOverlay the crosshair-drag gesture already draws, now also drawn around SelectedElement's bounds whenever "Highlight selection" is checked, and kept in sync live as a watch tick moves/resizes the selected element. 2. Point-to-select: drag a second crosshair (Interop/ElementPicker.cs) onto the target's rendered UI to navigate the tree selection to that element, Inspect.exe-style but at element granularity instead of window granularity. Hit-testing is done entirely client-side against the already-loaded live tree (lvt's bounds are already absolute screen pixels), so it needs no round-trip to lvt.exe. Programmatically selecting a TreeViewItem requires expanding every ancestor and resolving containers level by level (WPF only realizes a container once its parent is actually expanded) — ElementNodeViewModel gained a Parent back-reference, kept in sync by LiveTree.RebuildHierarchy, to make that walk possible. 3. Editable-properties hint: property editing has always been UIA-mode-only (toggle/set-value have no equivalent in visual-tree mode), but the UI never said so — a user testing in visual-tree mode would just find nothing editable with no explanation. Added an inline hint under the Properties header, visible only in visual-tree mode. 4. Richer tree labels (ElementNodeViewModel.DisplayName): structural containers (Grid, Border, StackPanel, ...) that never carry visible text used to show as a bare, indistinguishable repeated type name. When Text is empty, DisplayName now falls back through a priority list of identifying properties (AutomationProperties.Name, Name, AutomationId, x:Name) before giving up and showing just the type. 5. Framework/content-type filter (FrameworkFilterOption.cs, MainViewModel.FrameworkFilters/ApplyFrameworkFilter): a multiselect checkbox dropdown, populated lazily as new framework values are discovered in the live tree, visible only in visual-tree mode (UIA's tree has no equivalent per-node distinction). A node stays visible if its own framework passes the filter *or* any descendant's does, so excluding one type never hides an unrelated matching descendant deeper in the same branch — ElementNodeViewModel gained an IsVisible property bound to each TreeViewItem's Visibility for this. Verified: full CMake build (native + managed + viewer wiring) succeeds cleanly, all 127 unit tests and the WinUI3-stitching integration suite still pass, and the freshly built build/viewer/LvtViewer.exe launches and was handed off for live manual testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
The previous commit set _selectionHighlight.Owner = this in MainWindow's
constructor, before MainWindow itself had been shown. WPF requires a window
to have already been shown before it can be assigned as another window's
Owner, so this threw System.InvalidOperationException ("Cannot set Owner
property to a Window that has not been shown previously") wrapped in a
XamlParseException, crashing the app on every single launch with no window
ever appearing - this was the bug the user just hit trying to test the
other fixes in this PR.
Moved the assignment into the Loaded handler, alongside where the crosshair
and element pickers are already wired up - by the time Loaded fires,
MainWindow has been shown.
Verified: rebuilt via both `dotnet build` and the CMake `lvt_viewer` target,
launched the resulting build/viewer/LvtViewer.exe directly, and confirmed
the process stays running with its window visible (MainWindowTitle "lvt
Viewer", Responding: True) instead of exiting within seconds as it did
before this fix.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
If the target process closed or crashed while an element was selected and highlighted, the highlight overlay (item 1) stayed on screen forever, pointing at the last-known bounds of a window that no longer existed — nothing told MainWindow the target was gone. WatchSession already raises Exited for exactly this (process closed/ crashed), and MainViewModel already handled it by updating StatusText, but did not clear SelectedElement or reset the live tree, so nothing propagated to the view. Also fixed a related gap: MainWindow only refreshed the highlight in reaction to TreeView.SelectedItemChanged (a UI event) or the HighlightSelected toggle, not to SelectedElement changing programmatically from the view model — so even after fixing MainViewModel to clear it, the overlay would not have noticed. Now MainWindow's PropertyChanged handler also watches for SelectedElement itself. IsConnected (item 2's element-pick crosshair gate) is reset to false in the same Exited handler, and back to true in Reconnect(), so the gesture that needs a live tree to hit-test against is not left enabled with nothing to pick from either. Verified via full unit test suite (127 tests, unaffected — this is a viewer-only change) and by rebuilding via both `dotnet build` and the CMake `lvt_viewer` target; traced the fix logically end to end (watch exit -> SelectedElement cleared -> MainWindow's PropertyChanged handler fires -> UpdateSelectionHighlight sees a null SelectedElement -> overlay hidden). This is unrelated to the explorer.exe crash reported alongside it: that crash's faulting module is DUI70.dll, caused by lvt_dui_tap_x64.dll — a separate, pre-existing plugin in the user's ~/.lvt/plugins directory dated February 2026, not part of this repository or anything changed this session. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Investigated a report that watching the Microsoft Store app's visual tree "doesn't seem to work". It does — it is just very slow: a one-shot dump took ~8s, and `lvt watch`'s first tick (dump + injection + property/bounds collection, all before its first line of output) took ~20s, measured directly against a live Store window. That is long enough with the UI sitting on a plain "Connecting…"/"Watching…" status the whole time that it looks stuck or broken rather than just working slowly. (Root cause of the slowness itself — InitializeXamlDiagnosticsEx replaying the Store app's large/deep tree synchronously, on top of the property/ bounds walk this session already added a time budget to — is real but not addressed here: safely reducing it further would need more invasive changes to the XAML diagnostics pipeline than is worth risking right along the hang/contamination fixes already made today.) Added MainViewModel.ArmSlowConnectHint(): a delayed (4s) status update that only fires if no watch data has arrived yet, explaining that a rich UI tree can take 15-20+ seconds on first connect. Canceled the moment real data arrives (OnWatchEvent), so it never overwrites a status that has since moved on, and also canceled on watch.Exited/Dispose so a stale hint can't fire after the session has already ended. Verified: full unit test suite (127 tests) unaffected (this is a viewer-only status-text change), rebuilt via both `dotnet build` and the CMake `lvt_viewer` target, and confirmed the rebuilt build/viewer/ LvtViewer.exe launches normally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Investigated a report that watching Microsoft Store's visual tree "cannot connect within even 20s or longer" despite an earlier fix (a status hint) suggesting this was just slow. It was not just slow: `lvt watch` was stuck in a genuine runaway-output loop, verified directly against a live Store window (250MB+ of stdout within a minute, still growing, before being stopped). Root-caused to two compounding bugs, one a straight-up correctness bug that predates this session, the other a regression I introduced earlier today trying to fix a different problem (the UI-thread hang from the File Explorer session): 1. (This session's regression, now reverted.) lvt_tap.cpp's CollectBounds/CollectPositionsAndText had a wall-clock time budget (kUiThreadBudgetMs) added to bound how long a single per-tick property/ bounds walk could occupy the target's UI thread. Wall-clock cutoffs are inherently non-deterministic: measured against Store (~1100 nodes, ~4ms/node), ordinary timing jitter meant a *different* subset of nodes fit in the budget window each tick, so unchanged elements' bounds/ properties flip-flopped between "known" and "absent" every tick, purely as a budget artifact — and watch's diffing correctly (if unhelpfully) reported that as constant "changed" events forever. Reverted entirely: the actual protection against pathologically slow collection was already one layer up (see xaml_diag_common.cpp: the TAP DLL only connects to lvt.exe's pipe *after* this walk finishes, so lvt.exe's own existing 15-second "TAP DLL did not connect" timeout already bounds it — and, unlike a per-node budget, fails the whole tick cleanly with no partial data, which run_watch_loop already treats as a normal transient skip). 2. (Pre-existing correctness bug, unrelated to anything changed this session.) watch_diff.cpp's serialize_change_event called the same element_to_json used for `dump`'s full-tree output, which recursively serializes every element's *entire subtree*. For a per-node watch event describing one "added"/"removed"/"changed" node, that is enormous, redundant waste: the client (see LiveTree.ApplyAdded in the viewer) only ever reads an event's own scalar fields — it rebuilds tree structure from the flat stream of per-node events and their "path" field, never from an individual event's nested children. Serializing full subtrees per event meant a tree N nodes deep across D levels produced up to O(N*D) redundant data instead of O(N): measured on Store's ~1400-node, ~20-level-deep tree, a single node's "added" event line alone was 5.8MB. Added element_to_json_flat (no children) for watch events specifically, kept the recursive element_to_json for dump's own top-level serialization where nesting is correct, and replaced the equivalent deep-copy (`event.element = *indexed.element`, which via Element's value-type children also copied full subtrees) with element_without_children, built field-by-field so children are never touched. Verified directly against a live, freshly-relaunched Microsoft Store window (so no stale AppContainer-staged DLL copy could mask the fix): stdout grew ~530KB per 20s and CPU held around 27% average, versus the unfixed behavior's ~9MB/s sustained runaway growth and near-100%+ (multi-thread) CPU. Event composition changed from "changed" events dominated by AutomationId/HelpText/Tag/Visibility repeating 14-16x each with no bounds in between, to overwhelmingly "bounds" changes repeating only ~4x over the same window — consistent with Microsoft Store's own actually-animated home page content (promotional tiles), not tooling instability. Also verified: full unit test suite (127 tests) and full integration test suite (74 tests, 1 pre-existing skip) both pass, including the WinUI3- stitching suite that exercises the property/bounds collection path directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
1. Fix FindDeepestElementAt's tie-breaking bug (item 2, point-to-select). The previous logic let whichever sibling was checked *last* win a tie, on the theory that later-enumerated content is topmost/most specific. That is wrong when the last sibling is a bare utility window with no children of its own: UWP/ApplicationFrameHost windows carry a full- bounds "ApplicationFrameInputSinkWindow" HWND as a later sibling of the actual XAML-hosting bridge, so every point-to-select pick landed on that input sink instead of the real element underneath it, regardless of where in the UI you actually pointed. Fixed by tracking two tiers instead of one running "best": a match with real matching descendants (bestDeep) now always wins over a sibling that only matches itself with no deeper content (bestShallow), regardless of iteration order. Later-wins still applies as a tiebreaker, but only between siblings in the *same* tier. 2. Add a search bar (Find Next): a text box plus button in a new toolbar row, matching Text/ClassName/Type/any property name or value, case- insensitively. Cycles through matches in tree (depth-first) order on repeated presses (Enter in the box also works), wrapping around; a changed query resets the cursor to start over from the first match. Reuses the same SelectElementInTree path already built for point-to- select, so a found match gets the same expand-ancestors-and-select treatment (and, via the existing SelectedElement wiring, the same highlight-selection overlay). Verified: full unit test suite (127 tests, unaffected — both changes are viewer-only) and rebuilt via both `dotnet build` and the CMake `lvt_viewer` target; confirmed the rebuilt build/viewer/LvtViewer.exe launches normally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…'s UI thread
Reported live: after the viewer attached to an app, that app's own UI
became laggy — moving its window took a while to respond. Root cause: even
after removing the earlier (buggy) per-node time budget, a full
CollectBounds/CollectPositionsAndText pass over a rich tree is several
seconds of work, dispatched as one single blocking SendMessage call from a
worker thread to the target's UI thread. That call does not return until
the entire pass finishes, so the target's UI thread has no opportunity to
service its own pending messages in between — including the modal loop
DefWindowProc runs while the user drags the window (WM_NCLBUTTONDOWN/
SC_MOVE), which needs that exact same thread.
Fixed by dispatching both passes in small chunks (20 nodes per SendMessage
call, with a short Sleep between calls) instead of one unbroken call
covering every node. AdviseThreadProcImpl now flattens m_nodes into a
stable m_orderedHandles vector once per pass (a std::map has no efficient
random-access range) and loops sending one BatchRequest{self, start, count}
per chunk; CollectBounds/CollectPositionsAndText/CollectBoundsOnUIThread/
CollectPositionsOnUIThread all now take a (start, count) range into that
vector instead of iterating m_nodes directly. Every node still gets
collected, in the same order, every tick — only the dispatch is chunked —
so this does not reopen the flapping the earlier time budget caused: no
non-determinism is introduced, just gaps for the UI thread's own message
pump to run between chunks.
Verified: full unit test suite (127 tests) and the WinUI3-stitching
integration suite (28/29, the one skip a pre-existing, unrelated foreground-
window-contention flake) both pass, confirming the chunked collection still
produces correct, complete data end to end.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…shair 1. Find Previous: a second search button (and Shift+Enter in the search box) cycling backward through matches instead of only forward. Shares Find's match-computation and cursor-wrapping logic with Find Next, parameterized by step direction; a fresh search (cursor reset by a changed query) now starts at the natural end for the requested direction — first match for Next, last for Previous. 2. Hide the highlight overlay when the target window minimizes. A minimized target's bounds are meaningless to draw a rectangle around. MainViewModel now exposes the connected target's HWND (CurrentHwnd), and both ShowHighlightForCurrentNode (item 1's selection highlight) and PreviewElementAt (item 2's drag preview) check NativeMethods.IsIconic before showing. No separate polling needed: Windows moves a minimized window to a fixed off-screen "iconic" position, which is itself a real bounds change that already flows through the existing live-tree wiring and re-triggers these checks. 3. Investigated a report that "the blue crosshair doesn't work — cursor doesn't change". Traced with lvt itself, dogfooded against the running viewer's own UI: the element-pick crosshair's IsEnabled correctly reflects IsConnected (via lvt dump --pid <viewer>, its Border showed "enabled": "false" while nothing was connected) — working as designed, but a plain Border has no built-in disabled visual the way a Button does, so a legitimately-disabled crosshair looked identical to a broken one. Added an Opacity trigger (1.0 enabled, 0.35 disabled) and a tooltip that says to connect to a target first. Verified: full unit test suite (127 tests, unaffected — all three changes are viewer-only) and rebuilt via both `dotnet build` and the CMake `lvt_viewer` target; confirmed the rebuilt build/viewer/LvtViewer.exe launches normally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Reported: the element highlight rectangle appeared nowhere near the actual selected element (a Microsoft Store NavigationViewItem, bounds reported as 169,695,72x62, highlighted over by the Windows taskbar instead). Root-caused precisely, not guessed at: lvt.exe is a plain console app with no DPI-awareness declaration, so Windows silently virtualizes every Win32 coordinate query it makes (GetWindowRect and friends) down to a 96-DPI- equivalent space. Verified live on this system's 150% scaling: an unaware GetWindowRect call against the same Store window returned (161,319)- (702,826), while a properly Per-Monitor-V2-aware call against the identical HWND returned the true physical rect (242,479)-(1053,1240) - exactly a 1/1.5 scale-down, and 169 (the reported element's left edge) sits outside the *virtualized* window rect's own space consistently with that, not outside the *physical* one. This WPF viewer, on the other hand, is Per-Monitor-V2 DPI aware itself (the .NET default - no manifest overrides it), so positioning its own HWND via SetWindowPos needs true physical pixels. An earlier attempt at this fix assumed lvt's numbers already *were* physical pixels and used SetWindowPos directly with no conversion (correct only by accident on a single-DPI system, and still wrong in general) - actually testing that assumption against a real multi-provider run surfaced it was backwards (a WPF integration test broke: the injected WPF walker had its own matching DPI-compensation logic already, tuned to lvt.exe's *unaware* status, that a more invasive "make lvt.exe DPI aware everywhere" fix would have had to un-do consistently across every provider - reverted that broader change as too large a ripple to verify fully right now, in favor of this narrower, self-contained one). Fixed HighlightOverlay.MoveTo to scale lvt's rect up by the system DPI factor (GetDpiForSystem, called from this already-aware process, correctly returns the true system DPI - 144/96=1.5 here) before calling SetWindowPos. WPF's own Left/Top/Width/Height need no such conversion: device-independent units are the same 96-DPI-equivalent space lvt's numbers are already in. Verified: full unit test suite (127 tests) and full integration test suite (73 passed, 2 skipped on a known unrelated foreground-window-contention flake, 0 failed) both pass — confirming this narrower fix has no ripple effect on any provider, unlike the reverted broader attempt. Also independently verified the arithmetic: GetDpiForSystem() from a properly DPI-aware calling context on this system returns 144 (1.5x), and 161 (lvt's reported left edge) * 1.5 = 241.5 ≈ 242, matching the Store window's true physical left edge exactly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
The previous commit's fix put lvt-specific DPI scaling unconditionally inside HighlightOverlay.MoveTo. That broke a second, previously-working caller: reported live immediately after pushing — the crosshair-drag window-picker preview (CrosshairPicker.UpdateHighlight) started landing nowhere near the actual window, double-scaled and clearly wrong. Root cause: HighlightOverlay has two callers whose rects do not start out in the same coordinate space, and the previous fix conflated them. CrosshairPicker calls NativeMethods.GetVisibleFrame(hwnd) directly from this (Per-Monitor-V2 DPI aware) process, so that rect is already true physical pixels needing no conversion. MainWindow's selection/point-to- select highlight, on the other hand, builds its rect from ElementNodeViewModel's Bounds* fields, which came from lvt.exe - a plain console app with no DPI-awareness declaration, so its coordinates are virtualized to a 96-DPI-equivalent space by Windows. Scaling everything unconditionally fixed the second caller and double-scaled the first. Fixed by moving the DPI conversion out of HighlightOverlay entirely (back to a simple, unconditional "true physical pixels in, SetWindowPos" — no caller-awareness needed) and applying it only at the two call sites that actually need it: MainWindow.ToPhysicalRect (used by both ShowHighlightForCurrentNode and PreviewElementAt) scales an ElementNodeViewModel's lvt-sourced bounds up to physical pixels before constructing the rect passed to MoveTo. Also fixed a related, previously-unnoticed bug in the same family: point-to-select's hit-testing (FindDeepestElementAt) compared a screen point from GetCursorPos (already physical, from this process) directly against lvt-sourced element bounds (virtualized) — apples to oranges on any scaled display, which would have made point-to-select miss or mis-target elements even once the highlight rendering itself was correct. FindDeepestElementAtPhysicalPoint now converts the incoming physical point down to lvt's virtualized space first, so both sides of every comparison this file makes are now in one single, consistent space at each specific call site — never assumed globally. NativeMethods.LvtToPhysicalDpiScale centralizes the scale factor (GetDpiForSystem, called from this DPI-aware process) with one shared doc comment explaining the whole two-coordinate-space situation, so future callers have one obvious place to look rather than rediscovering this from scratch. Verified: full unit test suite (127 tests, unaffected) and rebuilt via both `dotnet build` and the CMake full build; confirmed the rebuilt build/viewer/LvtViewer.exe launches normally. Traced the arithmetic for both paths by hand: CrosshairPicker's already-physical rect now passes through MoveTo completely unchanged (matching its pre-regression behavior); the selection highlight's lvt-sourced rect is scaled by exactly the same verified factor (1.5x on this system) as the previous fix, which was independently confirmed correct against a real window's true physical position. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…ins focus Reported: when the target window moves, the highlight rectangle only catches up once the viewer app is refocused. Root cause: HighlightOverlay.MoveTo positions the overlay by calling SetWindowPos directly on the raw HWND (needed so the two callers' different coordinate spaces could be handled correctly at their own call sites rather than inside this class - see the last two commits). But WPF's own composition/render pipeline tracks a window's position and size through its Left/Top/Width/Height dependency properties, entirely independent of the HWND's actual Win32 position. SetWindowPos moves the real window fine, but leaves WPF's own understanding of where it is stale, so WPF keeps rendering the old frame until something else (refocusing, which forces a full redraw) resyncs it. Fixed by also updating Left/Top/Width/Height, in logical units via VisualTreeHelper.GetDpi(this), immediately after the SetWindowPos call — querying the DPI *after* the move specifically so it reflects whichever monitor the window is now actually on, rather than reintroducing the cross-monitor mismatch the physical-pixel-first SetWindowPos design was built to avoid in the first place. Verified: full unit test suite (127 tests, unaffected — viewer-only change) and rebuilt via both `dotnet build` and the CMake `lvt_viewer` target; confirmed the rebuilt build/viewer/LvtViewer.exe launches normally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Investigated a user report that lvt's initial tree enumeration was much slower than the UIA tree, with a hypothesis that the "key" field was a major contributor. Measured on a real ~1900-element WinUI3 tree (Microsoft Store): the "key" field alone was 3.98MB of a 9.78MB dump (40.7%), averaging 2372 chars per element. Root cause: base_identity_key() concatenated framework, type, AND className into every key segment, but for every provider that sets both (xaml_diag_common.cpp, wpf_inject.cpp), `type` is computed as the substring of `className` after its last '.' — so it never carries information className does not already have. Since a key is the full "/"-joined chain of every ancestor's segment, that redundant chunk was duplicated once per descendant, not just once per element. Fix: base_identity_key() now uses className alone (falling back to type only when className is empty, e.g. many UIA elements report no ClassName — see uia_provider.cpp) instead of concatenating both. Uniqueness is unaffected: assign_child_keys' existing sibling-collision counting already falls back to an index/hwnd/name discriminator whenever two children share a base identity, so this can only make that fallback trigger slightly more often in a narrow edge case, never silently collide. Measured after the fix on the same Store tree: key bytes dropped from 3.98MB to 3.11MB (-21.7%), total JSON from 9.78MB to 8.77MB (-10.3%). This changes the key's format from "framework|type|className" to "framework|className" (still an opaque, self-describing, "/"-joined durable key — nothing round-trips it by parsing individual pipe fields; verified via lvt_api.cpp's looks_like_visual_key/ref parsing, which only checks for the presence of '|', and via unit_tests.cpp's ElementKeys/ElementLookup suite, none of which assert on the literal key string). Updated the one README example that showed the old 3-part shape. Note: this key-bloat fix is a real, measured, but secondary contributor. The dominant cost on a rich XAML/WinUI3 tree is ~4.5ms/element in IVisualTreeService::GetPropertyValuesChain (property-chain collection) — inherent to how XAML diagnostics returns properties, and a separate, larger investigation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
… log Used this instrumentation to root-cause a "dump is much slower than the UIA tree" report: with only a thread-id prefix and no timestamps, the existing log couldn't show where time within a single AdviseVisualTreeChange cycle was actually going. Added GetTickCount64() to every log line, plus a few extra checkpoints (before AdviseVisualTreeChange, after each of the CollectBounds/CollectPositionsAndText batch-dispatch loops, and around SerializeAndSend's JSON-build and UTF-8-conversion steps) so a single capture can be split into per-phase durations after the fact. This is how the real bottleneck was found and measured: on real Calculator and Microsoft Store trees, CollectBounds (IVisualTreeService:: GetPropertyValuesChain per element) costs ~4.5ms/element and dominates total time; JSON building/writing is a few tens of milliseconds even for a multi-hundred-KB payload. Keeping the timestamps in permanently, since the existing thread-id-only log had no way to answer "which phase is slow" without re-instrumenting from scratch each time. Also caught and corrected a testing-methodology trap worth recording here: for AppContainer/UWP targets (Microsoft Store, Calculator, ...), this log does not live under %TEMP% — GetTempPathW() inside the injected, sandboxed target process resolves to that package's own virtualized temp directory instead (%LOCALAPPDATA%\Packages\<PackageFamilyName>\AC\Temp\ lvt_tap.log). That file is held open by the long-lived target process for as long as the TAP DLL stays loaded (it never unloads once injected), so it accumulates entries across every run against that process, not just the most recent one — reading it without filtering by these new timestamps looks exactly like a runaway "hundreds of repeated tree walks per dump" bug that briefly looked real during this investigation, until isolating just one run's actual time window showed a single, clean cycle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…er tree Root-caused why lvt's tree enumeration was much slower than the UIA tree on a rich XAML/WinUI3 app: IVisualTreeService::GetPropertyValuesChain (walking an element's entire property inheritance chain, potentially 50-150 properties, to read out ActualWidth/ActualHeight) costs ~4.5ms/element, measured live against Microsoft Store, Calculator, and a WinUI3 sample app. For a tree of a thousand-plus elements that is several seconds spent on a COM/cross-thread call almost entirely to answer "how big is this thing" — this is the dominant cost of a dump/watch tick, well ahead of the JSON size/key-bloat fix from the previous commit. The full per-element property set is a real, wanted feature (added recently to support showing every property a control has, not a curated subset) — this is not a bug to remove, so default behavior is unchanged. Added a second, cheaper collection path instead, opted into with --fast: - lvt_tap.cpp: CollectBounds now skips GetPropertyValuesChain entirely when fast mode is on. CollectPositionsAndText — which already reads Text and position via direct WinRT property access on an IInspectable it obtains anyway (no COM property-chain walk) — is extended to also read ActualWidth/ActualHeight (FrameworkElement) the same cheap way when bounds are not already known, and to read Content (ContentControl, unboxed via IPropertyValue only when it is actually a plain string, not a nested element subtree) in both modes — GetPropertyValuesChain's own filter treats a reference-typed Content as an opaque handle and drops it today, so this is new data either way, not a duplicate. - Repurposed the existing but entirely dead m_collectProps flag (parsed from a pipe-name suffix, never actually read by anything) into a real m_fastMode gate, renaming the suffix from the unused "|PROPS" to "|FAST". - Threaded a new fastProperties parameter through inject_and_collect_xaml_tree -> XamlProvider/WinUI3Provider::enrich -> build_tree, all defaulting to false so every existing caller is unaffected. - main.cpp: new --fast CLI flag, applies to both dump and watch (they share the same build_root_tree call). Trade-off, made explicit in the design and in docs: --fast reports bounds, Text, Content, and the existing curated state properties (IsEnabled, Visibility, IsChecked, ...) but not arbitrary custom properties outside that set — full mode remains the default specifically so nothing about today's behavior changes for an existing caller. Verified: 127 unit tests and 74 integration tests (2 pre-existing foreground-contention skips) pass, including a new WinUI3SampleFixture.FastModeStillFindsNamedControlsAndBounds that dumps a live WinUI3 app with --fast and checks a named control is still found with correct identity and that at least one element reports real (non-zero) bounds — the two things --fast exists to still guarantee. Measured live: a WinUI3 sample app dump went from 1943ms to 974ms (~2x) with identical bounds coverage between modes. Scope: XAML/WinUI3 only (the TAP DLL providers) — ComCtl/WPF/WinForms/ Avalonia/Chromium never had this cost, and WPF's separate injected walker (WpfTreeWalker.cs) does not use GetPropertyValuesChain's pattern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
MCP (mcp/src/server.rs, src/lvt_api.cpp): added an optional fast:bool
parameter to get_visual_tree, forwarded to build_tree_for/build_tree.
get_element_properties intentionally does NOT gain this parameter — its
own per-call cost (a full walk just to answer about one already-known
element) is a separate, lower-priority optimization for later, and it
should keep returning everything regardless of how the tree was built.
Viewer: the live tree (LvtViewer's sole data source, always a `lvt watch`
subprocess) now passes --fast, since it only needs bounds/Text/Content/
basic state to browse, search by, and highlight/hit-test elements — not
every custom property of every element, eagerly, on every tick. Since the
live tree therefore no longer carries a selected element's exhaustive
property set, MainViewModel.SelectedElement's setter now fires a one-shot
"lvt query <key>" call (deliberately without --fast, so it always returns
everything) whenever the selection changes, and merges the result into
that node's PropertyRows. "query" without a property name dumps every
property as flat top-level JSON fields (main.cpp's query_element_to_json)
— a different shape from watch's nested {properties: {...}}, so this is
parsed directly rather than reusing the ElementDto watch-event model.
Guards against a stale response landing on a since-changed selection.
Net effect for the viewer: connects roughly as fast as the underlying
--fast tree walk (see the previous commit's measurements) instead of
paying the full property-chain cost for the whole tree up front, and the
property panel still ends up showing every property — just fetched only
for the one element actually being looked at, and only when it is
selected.
Docs: documented --fast in README.md (CLI option table, watch section's
diffing implication), docs/mcp-server.md (get_visual_tree's tool
description and a dedicated explainer paragraph), and both copies of
skills/lvt/SKILL.md (kept in sync; verified identical after editing both).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
The viewer had no logging anywhere — a live-only bug (a tree rebuild that disrupted navigation, the crosshair picker going unexpectedly disabled) had no trail to diagnose from after the fact, only "reproduce it live under a debugger," which does not work for anything timing- or environment- dependent. This mirrors the TAP DLL's %TEMP%\lvt_tap.log (lvt_tap.cpp's LogMsg), which is what actually made several bugs in this same codebase root-causeable earlier this session. Services/Logger.cs: a minimal, thread-safe, append-only text logger, written to %TEMP%\lvt_viewer_<pid>.log — pid-suffixed, unlike the TAP DLL's single shared log, because several viewer instances (or the same instance across relaunches during development) can be live at once, and interleaving their output into one file would make any single run's story impossible to follow. Each line carries an elapsed-ms timestamp and a short category (watch/tree/viewmodel/app) so a log can be filtered by subsystem. App.xaml.cs: wired DispatcherUnhandledException, AppDomain. UnhandledException, and TaskScheduler.UnobservedTaskException to log through it too, so a crash leaves a record instead of just vanishing — relevant given this session's earlier "explorer crashed, the highlight overlay stayed" report, which had nothing to point at what actually happened. Instrumented the areas most relevant to the two bugs reported live in this session: - WatchSession.cs: process start/pid, each argument list, every exit (with exit code), every stdout event (type/key/path), every stderr line, and explicit Stop() calls — this is the direct trail for "the crosshair sometimes goes disabled for no reason", since ElementPickHandle.IsEnabled is bound to IsConnected, which only ever clears when this process exits. - LiveTree.cs: which key/event set the dirty flag that triggers a full RebuildHierarchy pass — the direct trail for "the tree refreshes as I navigate, resetting my place". - MainViewModel.cs: ConnectTo/Reconnect (target hwnd/pid/title), and every IsConnected transition tagged explicitly as "crosshair enabled/disabled" so grepping the log for that phrase finds every occurrence directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Root-caused "the tree refreshes as I navigate through it, which resets [expansion state]" by code review (git history shows this binding has been OneWay since it was first introduced, with no comment explaining why — an oversight, not a deliberate choice): MainWindow.xaml bound TreeViewItem.IsExpanded to ElementNodeViewModel. IsExpanded with Mode=OneWay. That means expanding a node by hand (clicking its expander arrow) only ever changed WPF's own TreeViewItem state — it never flowed back into the view model, since OneWay by definition never writes back. ElementNodeViewModel.IsExpanded stayed false for every node the user ever manually expanded. LiveTree.RebuildHierarchy runs on *any* add/remove/reorder anywhere in the tree, not just near whatever the user is currently looking at, and its SyncCollection can regenerate a node's TreeViewItem container as part of resyncing its parent's children. A regenerated container re-applies the OneWay binding from scratch — reading a view-model value that never recorded the user's manual expand — so it came back collapsed. On a target with any frequently-changing content (a clock, a spinner, virtualized list recycling), this could happen on nearly every tick, which is what looked like the tree resetting during ordinary navigation. Fix: Mode=TwoWay, so expanding/collapsing a node writes ElementNodeViewModel. IsExpanded, and a later container regeneration reads back the same state the user actually left it in. Verified: 127 unit tests and 75/76 integration tests (1 pre-existing foreground-contention skip) pass — this is a WPF-binding-only change with no C++ surface, so the existing suite is the correctness bar here; the live navigation behavior itself needs the user's own confirmation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…failure Root-caused "the element crosshair sometimes gets disabled for no reason": ElementPickHandle.IsEnabled is bound to MainViewModel.IsConnected, which only ever clears when the `lvt watch` subprocess exits (WatchSession. Exited). run_watch_loop's *first* tree build — the one done before entering the tick loop — had no retry tolerance at all: one failed injection attempt (this session already measured "InitializeXamlDiagnosticsEx failed" as a real, transient, occasionally-reproducing condition against XAML/WinUI3 targets, even moments before a retry would have succeeded) made the whole process return 1 and exit immediately. From the viewer's side, a process that exits one second after starting is indistinguishable from a deliberate disconnect, except that it was not one. This was already an inconsistency: every *later* tick already tolerates exactly this failure by design (see the existing comment on that code path) and just skips the tick rather than ending the session — only the initial connection lacked the same tolerance. Fix: wrap the initial build_output_tree call in the same kind of bounded retry (5 attempts, backing off 300ms per attempt) the tick loop already gets, rather than failing outright on the first attempt. Verified: 127 unit tests and 75/76 integration tests (1 pre-existing foreground-contention skip) pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
… per event Grepped the viewer's new log file (per the previous commit) from a live session against Microsoft Store and found the actual mechanism directly: LiveTree.Apply() called RebuildHierarchy() — a full walk of every node in the tree, resyncing every parent's Children collection — after processing *every single* watch event, not once per batch/tick. watch emits one JSON line per *element*, not one per tick, so this was never "one rebuild per tick": the initial connect alone (the whole tree arriving as "added" events) did as many full-hierarchy rebuilds as the tree has elements. Measured directly from the log: 5454 full-hierarchy rebuilds in one ~140s session against Microsoft Store, still happening at the very end (its animated home-page content keeps ticking, not just at initial connect). MainViewModel.ApplyFrameworkFilter() — a second, separate full-tree IsVisible walk — had the exact same per-event bug alongside it. This was never a correctness bug (SyncCollection's diffing is fine, and the previous commit's TreeViewItem.IsExpanded TwoWay fix is still correct and needed) — it was WPF re-laying-out the entire TreeView many times a second while the target's own live content kept changing anywhere in the tree, which reads exactly like the view resetting during ordinary navigation even though the underlying data model was fine throughout. Fix: LiveTree.Apply() now only mutates state and marks itself dirty; a new Flush() actually does the rebuild, and is meant to be called once per drained batch, not once per event. MainViewModel coalesces both Flush() and ApplyFrameworkFilter() behind one Dispatcher.BeginInvoke at Background priority per OnWatchEvent burst — Background priority is what does the batching: as long as more Normal-priority OnWatchEvent dispatches are already queued (which is true throughout a burst, whether from the initial connect or a live tick), this callback keeps getting pushed behind them and only actually runs once the queue drains. Verified: 127 unit tests unaffected (this is a viewer-only, watch-consumer change with no C++ surface). Live behavior needs the user's own confirmation, but this is root-caused directly from what the log showed happening, not guessed at. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
The previous fix (batching RebuildHierarchy behind a debounce) treated the symptom, not the disease. Per explicit direction: an event that only concerns already-enumerated elements must only ever touch those elements — add one node, remove one node, update one node's properties — never discard and recompute the tree's structure, no matter how that recompute is scheduled or how cheap any individual call to it is measured to be. Rewrote LiveTree.cs around that principle instead of layering another batching trick on top of the old design: - AttachToParent(node) inserts exactly one node into exactly the place its own Path says it belongs: Roots if it is the root, or its parent's Children (sorted-inserted by child index) if the parent is already known. If the parent has not attached yet — `watch`'s later ticks do not guarantee parent-before-child arrival order, only the very first burst does — the node queues in _pendingChildren keyed by the parent path it is waiting for, and resolves (transitively, for however many generations were queued) the moment that parent itself attaches. - DetachFromParent(node) removes exactly one node from wherever it currently sits — Roots, its parent's Children, or the pending queue it was waiting in if its own parent never attached — and nothing else. - A "changed" event whose path moved (a reorder/reparent) is exactly DetachFromParent + update Path + AttachToParent for that one node. - Property/bounds/scalar-field changes are unaffected — they already only ever touched the one node's own bound properties, never the hierarchy. MainViewModel.OnWatchEvent now calls LiveTree.Apply synchronously, on every event, with no deferral at all — there is no "whole tree" cost left to batch. ApplyFrameworkFilter (a separate, still-genuinely-O(tree-size) IsVisible walk that does not touch the hierarchy) keeps its own debounce timer, renamed to reflect that it is the only thing left needing one. Verified: 127 unit tests pass (viewer-only change, no C++ surface); built clean via both `dotnet build` and the CMake `lvt_viewer` target. Live navigation behavior needs the user's own confirmation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
… cancel
Reported: the window crosshair could select a minimized window, could
select a window that was actually occluded by another window on top of it
at that point, and the highlight rect it drew was sometimes wildly wrong
(a screenshot showed it spanning from partway up the desktop down through
an unrelated Store window, matching neither).
Root cause: ResolveWindowUnderCursor used WindowFromPoint + GetAncestor
(GA_ROOT) — Inspect.exe's classic approach, but incomplete here. It has two
real problems on modern Windows:
- It does not filter by IsIconic/IsWindowVisible at all, so a minimized
window (parked at Windows' fixed off-screen "iconic" position, which is
itself not where the window was ever visibly at) or an invisible one
was still a legal answer.
- It has no notion of DWM cloaking (DWMWA_CLOAKED) — a UWP app on another
virtual desktop, or one DWM is mid-transition on, is "cloaked" but
still a perfectly real HWND that WindowFromPoint can return, and its
rect can be stale garbage from whenever it last actually rendered. This
is almost certainly the direct cause of the wildly-wrong highlight rect
in the screenshot — a cloaked window's rect has no relationship to
anything currently on screen.
Fixed by replacing the WindowFromPoint-based resolution with an explicit
EnumWindows walk (top-level windows in true top-to-bottom Z-order) that
skips our own toolbar/overlay windows, anything minimized, invisible, or
DWM-cloaked, and hit-tests each remaining candidate's DWM extended-frame-
bounds rect against the cursor point in that Z-order — the first match is
therefore exactly the topmost *actually visible* window at that point.
A window fully occluded there is correctly never reached, no matter how
large its own rect is; a minimized window is never a candidate at all
rather than incidentally excluded by geometry.
Also added Escape-to-cancel to both CrosshairPicker (window picker) and
ElementPicker (item 2's point-to-select): mouse capture does not affect
keyboard focus, so both now hook PreviewKeyDown on the owning Window
(rather than the drag handle itself) and release capture on Escape, which
routes through the existing OnLostCapture cleanup (clears the dragging
flag, hides any preview highlight, restores the cursor) with no separate
cancel path to keep in sync. Status-bar hints during a drag now say
"(Esc to cancel)" so this is discoverable.
Verified: 127 unit tests pass (this is a Win32-interop/WPF-only change with
no C++ surface). Live picking behavior needs the user's own confirmation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…t key algorithm
Root-caused directly from the viewer's log (grepped for the exact bounds/
timing of a reported "reboot" while navigating to a ContentPresenter node)
combined with reading watch_diff.cpp and element_key.cpp side by side.
There were two completely separate durable-key algorithms in the codebase:
- assign_element_keys (element_key.cpp), used by dump/query/UIA output.
Disambiguates a child among just its own siblings — by native handle
first, then a stable name property, then a *local* sibling index only
as a last resort. A change anywhere outside a node's own immediate
parent can never affect that node's key.
- collect_index/assign_keys/index_tree/index_tree_pair (also in
element_key.cpp, but only ever called from watch_diff.cpp), used
exclusively by `watch`'s diffing. Disambiguated using a *global*,
whole-tree count of each identity (framework/className) plus the full
root-to-node path as the fallback discriminator when a collision
existed *anywhere* in the tree.
By the time watch_diff.cpp ran, every element already had a correct,
locally-stable `.key` from assign_element_keys — build_tree/build_uia_tree
both already call it. watch's own index_tree/index_tree_pair then silently
overwrote that with the inferior, globally-scoped one, discarding the good
key entirely. A real XAML tree is thick with duplicate-identity elements
(Grid, Border, TextBlock, ContentPresenter, Rectangle, ...), so almost any
live change anywhere in a large tree (an animation, a spinner, list
virtualization recycling an item several branches away) would flip the
"is there a collision anywhere in the whole tree" answer for one of these
identities, changing the full-path-derived key of a large, unrelated
portion of the tree on that tick. `diff_trees` matches purely by key, so
that showed up as those elements being reported as removed and different
ones added in their place — reproduced live as "the tree rebuilds while
navigating" against Microsoft Store's tree, and directly explaining why it
specifically happened once navigation reached a ContentPresenter (one of
the most common duplicate-identity element types in a WinUI3 tree).
Fix: deleted the divergent algorithm entirely (collect_index/assign_keys/
index_tree/index_tree_pair are gone from element_key.h/.cpp) and gave
watch_diff.cpp its own small, private indexing helper that calls
assign_element_keys — the one, single, already-correct algorithm — and
only computes structural "path" itself (which assign_element_keys was
never responsible for). diff_trees/snapshot_added_events now take a
mutable Element&, since assigning keys is a mutation; both call sites
(main.cpp's run_watch_loop, and the unit tests) already owned non-const
locals, so this needed no other changes.
Trade-off, made deliberately and documented in both code and tests:
assign_element_keys always threads an element's full parent-key chain
into its own key — that locality is exactly what stops an unrelated change
elsewhere in the tree from ever touching a node's key, but it also means
an actual reparent now changes the key regardless of whether the element
has a stable name/hwnd, so a genuine move is now reported as Removed+Added
rather than a single Changed/path event. Updated WatchDiff.MovedElement to
assert the new (still fully accurate, just less elegant) two-event shape,
and added WatchDiff.DuplicateIdentityElsewhereDoesNotDestabilizeUnrelatedSubtree
as a direct regression test for the actual reported bug: a same-identity
sibling inserted under one Pane must never touch an unrelated Pane's own
same-identity child.
Verified: 128 unit tests pass (127 existing + the 1 new regression test),
75/76 integration tests pass (1 pre-existing foreground-contention skip).
Also ran `lvt watch` live for ~30s against a real, animated Microsoft Store
window: 2066 elements arrived once as "added" (the initial snapshot), and
over the following ticks only 17 one-time "removed" events (each a
distinct key, no repeats — consistent with a few carousel items
legitimately scrolling out of a virtualized list) and 191 legitimate
"changed" events, with zero cascading remove/re-add churn.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Extends the same "connect once, reuse many times" mechanism from the XAML/WinUI3 redesign to the runtime-loaded plugin surface, per plan: any plugin can now optionally implement lvt_connection_open/get_tree/ poll_events/close (plugin.h) to let watch/MCP sessions reuse a connection across many refreshes instead of calling the one-shot lvt_enrich_tree fresh every time. - plugin.h: bump LVT_PLUGIN_API_VERSION to 2, add the new optional functions/struct. Backward compatible: every v2 function is probed independently via GetProcAddress, and a plugin's reported api_version now only needs to fall within [1, LVT_PLUGIN_API_VERSION] rather than match exactly, so an existing v1-only plugin binary keeps loading and working unchanged. - plugin_loader.cpp/h: relaxed the version check accordingly, probe the new optional exports, and add PluginConnection (an IFrameworkConnection adapter over the C ABI) plus open_plugin_connection() so the registry can treat a plugin connection identically to XamlDiagConnection. Extracted the JSON->Element grafting logic shared by the one-shot and connection paths into graft_plugin_tree_json (mirrors xaml_diag_common.cpp's graft_xaml_tree_json split, for the same reason: one implementation, not two that could drift). - tree_builder.cpp: the Plugin framework case now checks the ConnectionLookup (keyed by the plugin's own detected framework name) before falling back to the existing one-shot enrich_with_plugin. - docs: avalonia-plugin.md and chromium-plugin.md note the new optional capability; neither plugin implements it yet, so both continue to work exactly as before via the one-shot path. 129 unit + 76 integration tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Completes the last core-scope phase from the persistent-connection plan.
The TAP DLL's OnVisualTreeChange already tracks every Add/Remove for a
connection's lifetime (needed for correctness once the tree persists across
many refreshes - see the earlier commit); this makes it also push each
mutation to lvt.exe proactively, over the same pipe, instead of only ever
being visible via a GET_TREE poll.
- lvt_tap.cpp: OnVisualTreeChange now releases m_nodesMutex before calling
the new PushChangeEvent (writes one {"type":"CHANGE",...} line via the
existing WriteLine/m_pipeWriteMutex), keeping lock ordering simple and
avoiding any risk of a lock-order inversion. Pushing is best-effort:
WriteLine fails quietly (and is expected to) during the initial
synchronous tree replay, before the pipe is even connected.
- xaml_diag_common.cpp: get_tree() now loops over response lines, since a
pushed CHANGE event can legitimately arrive interleaved with the response
to a specific GET_TREE request - lines starting with '{' are parsed and
queued (queue_change_event), the tree response itself is the first line
starting with '['. poll_events() now returns the queued events instead of
always being empty. The queue is capped (drops oldest) so a caller that
never calls poll_events() at all cannot turn this into an unbounded leak.
- main.cpp/lvt_api.cpp: watch's tick loop and MCP's per-session build now
drain each connection's queue every refresh (logged under --debug for
watch), which is what actually bounds the queue in the common case - the
cap in xaml_diag_common.cpp is the defensive fallback for a caller that
doesn't.
Deliberately conservative in scope, per the plan: this lands the wire
mechanism and makes poll_events() real, but does not rearchitect watch's
tick loop to consume these events instead of polling GET_TREE - that is a
larger, separable data-flow change flagged as a further, optional
optimization, not required now that the core leak/reinjection fix is
already proven live.
Verified: PushChangeEvent's pre-connect skip path exercises correctly and
harmlessly (confirmed live via the TAP log - hundreds of legitimate
"sent=0" attempts during the initial tree replay, before the pipe
connects, exactly as designed). 129 unit + 76 integration tests pass (one
transient foreground-contention failure on a full-suite run reproduced
clean in isolation and on a subsequent full-suite rerun, consistent with
this suite's already-documented flakiness class, not a regression).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Root-causes and fixes the "WinUI3 bounds/text/property capture bug": every
WinUI3 element beneath a DesktopWindowXamlSource root (File Explorer's
modern address bar/breadcrumb/tab UI, confirmed live) reported zero bounds
and empty text, while system-XAML apps (Calculator) were unaffected.
The actual cause was environmental, not a code defect: LVT_WASDK_WINMD_DIR
could only ever be found for the older, unified "Microsoft.WindowsAppSDK"
NuGet package layout (winmds under lib/uap10.0). This machine only had the
newer split packaging cached (Microsoft.WindowsAppSDK.WinUI, .Foundation,
.Base, ... - each capability its own package, winmds under metadata/), so
detection silently found nothing, cppwinrt was never run, and
LVT_HAS_WINUI3_PROJECTION compiled to 0 - meaning CollectPositionsAndText's
WinUI3-specific (Microsoft.UI.Xaml) casts were compiled out entirely, and
every genuine WinUI3 element's TransformToVisual/Text/Content extraction
could only ever fail against the System XAML (Windows.UI.Xaml) fallback
that remained. GetPropertyValuesChain-based bounds looked like they
"worked" (ActualWidth/ActualHeight were found), but their values were "0" -
elements never measured via the code path that would have populated them
correctly.
- CMakeLists.txt: detect both NuGet layouts when searching for
LVT_WASDK_WINMD_DIR. For the split layout, also pull in the latest STABLE
version of every sibling Microsoft.WindowsAppSDK.* package (Xaml.winmd
references types they define) and of Microsoft.Web.WebView2 (a separate
package entirely, needed for WinUI3's WebView2 control type) - both
needed for cppwinrt to fully resolve the projection. Deliberately skips
experimental/preview/prerelease versions and never mixes two versions of
the same package: an earlier iteration of this fix picked a cached
2.0-experimental .winui package and fed cppwinrt multiple conflicting
versions of Microsoft.UI.Xaml.winmd simultaneously, which failed with
"Type ... could not be found" for a WebView2 type the experimental
version's Xaml.winmd referenced but the mismatched sibling set didn't
define.
Verified live: File Explorer's WinUI3 UI (address bar, tab strip, command
bar) now reports real bounds and text (e.g. a real button's icon glyph)
where every single node previously reported bounds={0,0,0,0} and empty
text. 129 unit + 76 integration tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Brings UIA onto the same persistent-connection architecture as XAML/WinUI3. Before this, every --uia watch tick and every MCP UIA tree read did a fresh CoCreateInstance(CUIAutomation[8]) + timeout setup + cache request + full walk. That was never the same severity as the old XAML reinjection leak (UIA injects nothing into the target), but it was still unnecessary per-call churn and it left UIA as the last major tree mode not participating in the ConnectionRegistry design this branch just introduced. - src/providers/uia_provider.h/.cpp: add UiaConnection as an IFrameworkConnection implementation for UIA; split the walk into create_automation + build_tree_with_automation so both the old one-shot UiaProvider path and the new persistent path share one tree-building implementation; re-apply per-call UIA timeouts on the reused automation object; and keep the process-wide MTA alive for the connection lifetime via CoIncrementMTAUsage/CoDecrementMTAUsage so a client created on one short- lived MTA worker stays valid for later short-lived MTA workers. - src/main.cpp: watch now acquires a persistent "uia" connection, and its --uia build path downcasts the generic IFrameworkConnection back to UiaConnection when it needs per-call UIA view/property/timeout options, falling back to the exact old one-shot path if no live connection is available. - src/lvt_api.cpp: MCP sessions now lazily acquire and hold a persistent UIA connection alongside the existing xaml/winui3 ones, use it for get_uia_tree / find_elements reads, and also retry missing visual-tree framework acquisitions even when the session already holds some other label (e.g. "uia") so mixed-mode sessions do not accidentally suppress later xaml/winui3 connects. - src/providers/uia_actions.h/.cpp: MCP action/wait paths can reuse the session's persistent UIA connection for the tree walks they already perform to resolve a reference, poll a wait, and refresh the post-action element, while leaving the live RuntimeId re-find / pattern invocation path itself as the existing one-shot automation object. Verified: `cmake --build build --target lvt lvt_core lvt_unit_tests lvt_integration_tests --config Release`; `build\lvt_unit_tests.exe` (129 passing); `build\lvt_integration_tests.exe` (76 passing, 1 expected skip: NotepadFixture.XamlBoundsIfDetected); the targeted WinUI3SampleFixture.UiaWatchEmitsAddedEvents test passed on a 3x repeat; and a live `lvt.exe watch --uia --pid <WinUI3Sample>` run stayed alive and emitted UIA added events continuously for 12 seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
…ck permanently Fix: once a persistent XAML/WinUI3 connection died (e.g. a timeout on an unusually large/slow tree), build_tree's ConnectionLookup correctly fell back to one-shot re-injection for that single tick, but nothing ever re-acquired a fresh persistent connection afterward. This permanently regressed watch/MCP sessions back to the old per-tick reinjection bug (repeated InitializeXamlDiagnosticsEx calls) for the remainder of the session, as soon as one connection death occurred. - main.cpp: add refresh_dead_watch_connections(), called every watch tick before build_output_tree. Detects dead connections via is_alive(), resets the dead handle *before* reacquiring (ConnectionRegistry::release is key-based only, so releasing after reacquiring would decrement the new entry's refcount instead of the dead one's), then re-derives fresh replacements via acquire_watch_connections. - lvt_api.cpp: apply the same dead-entry pruning to connection_lookup_for_session (XAML/WinUI3 MCP path), which only checked label presence, not liveness. This mirrors the pattern already correctly implemented in uia_connection_for_session. Verified live against Microsoft Store (the app that originally reproduced the bug): a multi-minute watch session under heavy load produced steady incremental added/changed/removed events with no full-tree-reset pattern, and a killed process produced clean teardown (message window destroyed, no leak). 129 unit + 76 integration tests pass.
Topmost placed the highlight rectangle in an independent always-on-top band, disconnected from the target window's actual z-order. This made the highlight remain visible even when the target was minimized or covered by an unrelated window on top of it. Fix: make HighlightOverlay a native Win32 owned window of the target (SetWindowLongPtr(GWLP_HWNDPARENT, ...)). Windows enforces that an owned window always stays directly above its owner in z-order, so when the owner is covered/minimized, the overlay is automatically covered/hidden right along with it - no separate occlusion-polling logic needed. - HighlightOverlay.xaml.cs: add SetWindowLongPtr P/Invoke (pointer-width correct, unlike the existing 32-bit SetWindowLong used for GWL_EXSTYLE) and SetOwner(IntPtr), which sets the native owner and immediately calls SetWindowPos to force the z-order change to take effect right away (changing GWLP_HWNDPARENT alone doesn't trigger an immediate reorder). - HighlightOverlay.xaml: remove Topmost="True". - CrosshairPicker.cs: call SetOwner(hwnd) before showing the drag-preview highlight, retargeting it to whatever window is currently under the cursor. - MainWindow.xaml.cs: replace the WPF-level Owner assignment with SetOwner(CurrentHwnd) in both the persistent selection highlight and the element-picker preview path; clarify in the doc comment that this is a separate HighlightOverlay instance from CrosshairPicker's, not the same one.
build_tree's XAML/WinUI3 branches shared a single else-branch between two different callers: one-shot CLI commands (dump/query/screenshot), which call build_tree with no ConnectionLookup at all and correctly want a single inject-collect-disconnect since they never hold a persistent connection by design, and watch/MCP, which always pass a real ConnectionLookup but could still land in that same branch whenever it returned null (no entry yet) or a dead connection. Both cases looked identical to the branch (`connection == nullptr`), so watch/MCP could silently do a full one-shot reinject - the exact per-tick cost this whole persistent-connection mechanism exists to eliminate - whenever their held connection died, without any way to distinguish that from the intentional one-shot CLI behavior. Combined with the previous gap (nothing ever refreshed a dead connection, fixed in 578748e), this is what let a single connection death permanently regress a watch session back to per-tick reinjection. Now the branch explicitly checks `!connectionLookup` (no lookup at all) before taking the one-shot path. When a lookup was supplied but has no alive connection right now, enrichment for that framework is skipped for this call instead - the existing refresh_dead_watch_connections (watch) / session reconnect logic (MCP) is responsible for re-acquiring a fresh persistent connection before the next tick/call, and does so on the very next tick rather than falling back to reinjection in the meantime. The Plugin branch is deliberately left unchanged: neither watch's acquire_watch_connections nor MCP's connection_lookup_for_session actually acquire a persistent connection for plugin frameworks yet (the plugin ABI v2 PluginConnection machinery exists in plugin_loader.cpp but isn't wired into either acquisition path), so a null lookup result there always means "not wired up," not "died" - applying the same change would silently break plugin enrichment in watch/MCP entirely rather than fix a bug. Tracked as a separate follow-up. 129 unit + 76 integration tests pass (1 known environmental skip, unrelated).
The viewer kept losing its tree/selection mid-session even after the lvt.exe core-side connection-death fixes (578748e, a2063a6) were verified solid — because this was never a core bug at all. WatchSession.Start() calls Stop() (kills the previous watch process) before starting a new one; .NET's Process.Exited event fires asynchronously on a threadpool thread, and can arrive *after* Start() has already assigned _process to the new, healthy process. The handler fired Exited unconditionally, so this stale notification for the just-replaced old process still reached MainViewModel, which wipes IsConnected and the entire tree in response — even though the new watch session was alive and streaming events at that exact moment. Confirmed live via the viewer's log: every occurrence logged "ExitCode=-1" immediately after a fresh ConnectTo, which is simply the ordinary exit code from our own Process.Kill(entireProcessTree: true) in Stop() - not a crash, and not related to the target app closing at all. Fix: the Exited handler now checks whether the process it fired for is still the one WatchSession is currently tracking (ReferenceEquals against _process) before forwarding the notification, discarding it otherwise. Stop() already nulls _process before killing the old process, and Start() assigns the new one immediately after, so this correctly distinguishes "a stale notification for an already-replaced session" from "the currently tracked session genuinely died".
The first attempt at fixing highlight occlusion (39caa4c: owned window + one-time SetWindowPos instead of Topmost) was insufficient - confirmed live, the highlight still showed through a covering app. Win32's owned-window z-order rule only guarantees "stays above its owner"; it does not guarantee "stays below whatever unrelated window already happens to be above the owner". Any subsequent z-order recalculation re-snaps an owned window directly above its owner regardless of what was on top a moment before, so ownership + a single SetWindowPos call cannot reliably keep the overlay hidden behind a covering app. Separately, the SetWindowPos call in SetOwner() was missing SWP_NOOWNERZORDER: repositioning an *owned* window is otherwise free to also reposition its *owner* in the z-order as a side effect (documented Win32 behavior) - this is what caused other apps' windows to visibly shuffle z-order every time the crosshair drag moved over a new candidate window, since each move called SetOwner + SetWindowPos again. Fixes: - HighlightOverlay.xaml.cs: add SWP_NOOWNERZORDER. Add Track(hwnd, rect), the new single entry point both callers use instead of calling SetOwner/MoveTo/Show individually; it owns + positions the window and starts a 200ms poll timer that re-checks (via Reevaluate) whether the target is still actually visible there - not minimized (NativeMethods.IsIconic) and not covered by some other real window (NativeMethods.IsOccludedAt) - hiding the overlay otherwise. Hide() is shadowed to also stop the poll timer so a deliberate hide does not get silently reversed by the next tick. - NativeMethods.cs: add IsOccludedAt(targetHwnd, point), an EnumWindows- based top-to-bottom occlusion check generalizing CrosshairPicker.ResolveWindowUnderCursor's existing technique - true if some other, real, visible top-level window not belonging to this process comes before targetHwnd in Z-order at that point. - CrosshairPicker.cs / MainWindow.xaml.cs: replace the separate SetOwner+MoveTo+Show call sequences with a single Track() call. Verified: 129 unit tests unaffected (viewer-only change); manual live retest pending against a real covering-window scenario.
refresh_dead_watch_connections (578748e) fixed a connection that died mid- session, but missed a related case: acquire_watch_connections previously only added a (label, handle) entry to `connections` when its acquire() call actually succeeded - so if the *very first* attempt to establish a "xaml"/"winui3" connection at watch startup failed (InitializeXamlDiagnosticsEx is known to fail transiently on a first try against a slow/busy target - observed live and repeatedly against Microsoft Store), no entry for that label ever existed at all. refresh_dead_watch_connections can only repair an existing (label, handle) pair that has gone dead; it had nothing to notice or retry for a label that was simply never added. Combined with tree_builder.cpp no longer silently falling back to one-shot reinjection when a supplied ConnectionLookup returns nothing (a2063a6), this meant a target whose first xaml/winui3 acquisition attempt failed would permanently show only bare Win32 windows with no framework enrichment for the rest of the watch session - confirmed live: Microsoft Store's tree showing three unlabeled "Window" nodes with empty class/text/bounds, "watching the visual tree live" but never recovering. Fixes: - acquire_watch_connections: always record a (label, handle) entry for each detected xaml/winui3 framework, even when this particular acquire() attempt failed (handle then empty/falsy) - the ordinary emplace_back is no longer conditional on success. - refresh_dead_watch_connections: treat a missing/empty handle exactly like a dead one (`!handle || !handle->is_alive()`) so it gets retried on the next tick, not just an entry that existed and later died. lvt_api.cpp's MCP path already re-evaluates from scratch on every call (has_label checks a truthy handle, and needXaml/needWinUI3 are recomputed every time), so it does not share this gap and needed no change. 129 unit + 76 integration tests pass (1 known environmental skip unrelated to this change: NotepadFixture.XamlBoundsIfDetected; a second skip seen once during a full run, WinUI3SampleFixture.ActionResultReports HowItWasPerformed, was confirmed environmental via isolated rerun - foreground contention from a concurrently running viewer instance, self- detected and gracefully skipped by the test itself).
ResolveWindowUnderCursor's Z-order walk unconditionally skipped past the
viewer's own windows (ownHwnd/overlayHwnd) without checking whether they
actually occupy the cursor's current point, then kept searching further
down the Z-order for the next window whose rect happened to also contain
that same screen point. Since the cursor is naturally still over the
viewer's own crosshair handle right at drag-start, this let the search
continue past our own (topmost, actually-visible-there) window and "find"
some entirely unrelated, far lower window instead - one that is not
actually visible at that point at all, being covered by our own window,
but whose rect (e.g. a maximized app spanning nearly the whole screen)
merely happens to also span those same coordinates.
Observed live: dragging the crosshair from directly over the viewer's own
button picked a large, unrelated, actually-hidden-behind-the-viewer window
(Teams) instead of correctly finding nothing (or whatever genuinely was
topmost) at that point - matching the user's own diagnosis ("preselected
... because it is large").
Fix: when the walk reaches one of our own windows, check whether it
actually contains the cursor point. If so, stop the search there (nothing
beneath is actually visible/selectable at this point); only continue
looking further down the Z-order if our own window does not occupy this
exact point.
The previous own-window occlusion fix treated the highlight overlay like the viewer window. Because the overlay covers the cursor once shown, every subsequent move and mouse-up resolved no target, making the rectangle flash and preventing selection. Always skip overlayHwnd while retaining the viewer-window occlusion check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Replace the verbose delayed still-connecting sentence with an explicit IsConnecting state and a compact indeterminate progress bar. Keep it active until the first tree event, restore the normal watching status then, and log measured process-start-to-first-event timing for diagnosing slow targets. Also wire the standard Ctrl+F Find command to focus and select the Find box from anywhere in the viewer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Bind F3 to Find Next and Shift+F3 to Find Previous, and expose the shortcuts in the button tooltips. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Build the CoreWindow connection probe from the Win32 skeleton only instead of running a complete one-shot framework enrichment immediately before opening the persistent connection. In fast mode, skip the no-op CollectBounds dispatch loop entirely. Tighten the persistent-watch integration test to require exactly one injection and exercise --fast. On Microsoft Store, the redundant probe measured about 25 seconds and the no-op fast bounds dispatch about 1.9 seconds. After removal, manual connection dropped to about five seconds and first tree output measured about three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Preserve IXamlDiagnostics InstanceHandle on grafted XAML/WinUI3 elements and use xaml:0x... / winui3:0x... as their globally compact keys, retaining structural paths as fallback for providers without native identity. Store key payload drops from 4,210,498 to 37,757 characters (average 2,018 to 18 per node), and total JSON drops from 9.91 MB to 5.74 MB. Keys remain stable across independent dumps and resolve in a later query process. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Explicitly set StandardOutputEncoding and StandardErrorEncoding to UTF-8 for both the long-running watch process and one-shot CLI calls. Native lvt output already contains the correct Unicode private-use glyph; mojibake was introduced only by the viewer's implicit process-stream decoder. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Use the new multi-frame ICO for the executable resource, main window, and taskbar representation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Clear per-request properties and geometry before collecting a persistent tree snapshot. This prevents Text/Content entries from accumulating on every watch tick and forces fast mode to refresh ActualWidth/ActualHeight after resizes. Upsert directly-read Text/Content values within each snapshot instead of producing duplicate JSON properties. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Private-use characters depend on app-specific fonts that may not exist in the viewer. Render them deterministically as U+XXXX in element labels instead of displaying a missing-glyph rectangle, while leaving printable Unicode unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Install the public framework_connection header under include/lvt/providers so tree_builder.h works for package consumers. Classify compact xaml:0x... and winui3:0x... durable keys as visual references so UIA MCP sessions reject them with the correct mode guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Distinguish WinUI 3 from WinUI 2 by requiring Microsoft.Internal.FrameworkUdk.dll instead of treating Microsoft.UI.Xaml.dll alone as WinUI 3. Add Windows.UI.Composition.DesktopWindowContentBridge hosting to XamlProvider and graft system DesktopWindowXamlSource roots beneath it. Scan up to 10,000 monotonically allocated diagnostics connection identifiers, matching UWPSpy behavior for long-lived processes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
There are correctness issues in the new persistent-connection plumbing (teardown/concurrency and plugin option handling) that can cause unsafe behavior or silent behavior changes.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
 |
src/lvt_api.cpp — method_disconnect erases g_sessionConnections (and can drop the per-target lock entry) without… |
 |
src/tree_builder.cpp — When a plugin has a persistent connection (IFrameworkConnection) the code calls… |
|
src/plugin_loader.cpp — open_plugin_connection can create a persistent connection even if the plugin doesn't export… |
What changed in this PR
This PR adds a new lvt Viewer WPF desktop GUI (under src/viewer/LvtViewer/) that drives lvt.exe as a subprocess (primarily via lvt watch) to provide a Live Visual Tree / Inspect.exe-style interactive front-end, while also extending lvt_core to support more robust/efficient live updates (persistent framework connections, improved durable keys, and a --fast mode) needed to make long-running watch/viewer scenarios performant and stable.
Changes:
- Add the WPF Viewer app (UI, interop crosshair pickers/overlays, view models, and subprocess-driven services).
- Introduce persistent per-framework connections (
IFrameworkConnection+ConnectionRegistry) and wire them intowatchand MCP visual-tree reads. - Update durable key logic (compact XAML/WinUI3 instance-handle keys), expand XAML property capture rules, add
--fastplumbing, and add/extend unit + integration tests + docs.
| File | Description |
|---|---|
| tests/unit_tests.cpp | Adds regression/unit tests for compact keys, diff semantics, reconciliation, and XAML property capture. |
| tests/integration_tests.cpp | Adds integration coverage for --fast and persistent connection reuse across watch ticks. |
| src/watch_diff.h | Updates diff APIs to take mutable trees (internal key assignment). |
| src/viewer/README.md | Documents viewer architecture, watch-based data source decision, and build/run steps. |
| src/viewer/LvtViewer/ViewModels/RelayCommand.cs | Adds minimal ICommand helper for MVVM commands. |
| src/viewer/LvtViewer/ViewModels/PropertyRowViewModel.cs | Models property rows and editability classification (toggle/value). |
| src/viewer/LvtViewer/ViewModels/ObservableObject.cs | Adds minimal INotifyPropertyChanged base for view models. |
| src/viewer/LvtViewer/ViewModels/FrameworkFilterOption.cs | Adds framework filter option model for visual-tree filtering UI. |
| src/viewer/LvtViewer/ViewModels/ElementNodeViewModel.cs | Adds long-lived element node VM with live updates + property rows + display formatting. |
| src/viewer/LvtViewer/Services/WatchSession.cs | Implements long-running lvt watch subprocess session + stdout/stderr pumps. |
| src/viewer/LvtViewer/Services/LvtLocator.cs | Implements lvt.exe discovery logic (env/sibling/build/userprofile/PATH). |
| src/viewer/LvtViewer/Services/LvtCli.cs | Implements one-shot lvt.exe verb invocation for edits (toggle/set-value). |
| src/viewer/LvtViewer/Services/Logger.cs | Adds viewer diagnostic log to %TEMP% for post-mortem debugging. |
| src/viewer/LvtViewer/Services/LiveTree.cs | Maintains incremental in-memory tree keyed by durable key, applying watch events. |
| src/viewer/LvtViewer/Services/JsonDefaults.cs | Centralizes System.Text.Json defaults for parsing lvt JSON. |
| src/viewer/LvtViewer/Models/WatchEventDto.cs | Defines DTOs for watch event stream and field changes. |
| src/viewer/LvtViewer/Models/ElementDto.cs | Defines DTOs matching lvt JSON dump/watch element payloads. |
| src/viewer/LvtViewer/MainWindow.xaml.cs | Wires pickers, TreeView selection, and highlight overlay behavior. |
| src/viewer/LvtViewer/MainWindow.xaml | Defines the viewer UI (toolbar, tree, property panel, search, filter popup). |
| src/viewer/LvtViewer/LvtViewer.csproj | Adds WPF project targeting net10.0-windows with icon resource. |
| src/viewer/LvtViewer/Interop/NativeMethods.cs | Adds P/Invoke helpers and occlusion/DPI utilities for pickers/overlays. |
| src/viewer/LvtViewer/Interop/HighlightOverlay.xaml.cs | Implements click-through owned overlay window with occlusion polling. |
| src/viewer/LvtViewer/Interop/HighlightOverlay.xaml | Defines overlay window chrome and rectangle visuals. |
| src/viewer/LvtViewer/Interop/ElementPicker.cs | Adds press-drag point-to-select element picker (screen point emitter). |
| src/viewer/LvtViewer/Interop/CrosshairPicker.cs | Adds Inspect.exe-style window picker with live highlight overlay. |
| src/viewer/LvtViewer/Converters/InverseBooleanToVisibilityConverter.cs | Adds inverse boolean visibility converter used by UI hints. |
| src/viewer/LvtViewer/Converters/FrameworkToBrushConverter.cs | Maps framework labels to colors for tree dot indicators. |
| src/viewer/LvtViewer/Converters/EditKindToVisibilityConverter.cs | Template visibility converter for property row edit UI. |
| src/viewer/LvtViewer/Assets/LvtViewer.svg | Adds vector icon asset (design source). |
| src/viewer/LvtViewer/AssemblyInfo.cs | Adds WPF ThemeInfo assembly attributes. |
| src/viewer/LvtViewer/App.xaml.cs | Adds app-level exception logging hooks and startup log. |
| src/viewer/LvtViewer/App.xaml | Defines WPF App startup URI. |
| src/tree_builder.h | Extends build_tree API to support fastProperties + persistent ConnectionLookup. |
| src/tree_builder.cpp | Uses ConnectionLookup to reuse XAML/WinUI3/plugin persistent connections. |
| src/tap/xaml_property_filter.h | Broadens property capture to arbitrary primitive-typed properties (with handle heuristic). |
| src/providers/xaml_provider.h | Adds fastProperties + connection open/reuse APIs. |
| src/providers/xaml_provider.cpp | Implements persistent connection support and desktop bridge handling. |
| src/providers/xaml_diag_common.h | Documents fastProperties and declares persistent connection factory. |
| src/providers/winui3_provider.h | Adds fastProperties + connection open/reuse APIs. |
| src/providers/winui3_provider.cpp | Requires FrameworkUdk for WinUI3, adds persistent connection support. |
| src/providers/uia_provider.h | Adds reusable UIA client connection (UiaConnection). |
| src/providers/uia_provider.cpp | Implements UiaConnection with MTA usage cookie + timeout reapplication. |
| src/providers/uia_actions.h | Allows reusing UiaConnection for action resolve/readback walks. |
| src/providers/uia_actions.cpp | Reuses UiaConnection for tree walks around actions when available. |
| src/providers/framework_connection.h | Introduces generic persistent-connection interface + event shape. |
| src/providers/connection_registry.h | Introduces refcounted per-process registry + move-only ConnectionHandle. |
| src/providers/connection_registry.cpp | Implements ConnectionRegistry and ConnectionHandle RAII behavior. |
| src/plugin.h | Bumps plugin API to v2 and defines optional persistent-connection ABI exports. |
| src/plugin_loader.h | Extends LoadedPlugin with optional v2 connection function pointers and declares open_plugin_connection. |
| src/plugin_loader.cpp | Adds v2 probing, plugin connection adapter, and refactors grafting helper. |
| src/lvt_api.cpp | Adds per-session connection reuse for MCP visual/UIA reads and actions. |
| src/framework_detector.cpp | Tightens WinUI3 detection to require FrameworkUdk signal in addition to Microsoft.UI.Xaml.dll. |
| src/element_key.h | Documents key algorithm and exposes stable_name_key for reconciliation use. |
| src/element_key.cpp | Implements compact xaml:0x… / winui3:0x… keys and reduces base key size. |
| skills/lvt/SKILL.md | Updates guidance to mention --fast for rich XAML/WinUI3 apps. |
| README.md | Documents viewer and --fast behavior in CLI usage and watch semantics. |
| mcp/src/server.rs | Adds fast option to get_visual_tree tool args and params. |
| docs/tap-dll-design.md | Updates TAP DLL design docs to reflect persistent connection lifecycle + protocol. |
| docs/mcp-server.md | Documents fast:true and clarifies durable key forms (compact handles vs structural). |
| docs/chromium-plugin.md | Notes plugin ABI v2 persistent connections (not yet implemented by plugin). |
| docs/avalonia-plugin.md | Notes plugin ABI v2 persistent connections (not yet implemented by plugin). |
| docs/architecture.md | Documents reusable connections and registry in overall architecture. |
| CMakeLists.txt | Adds LVT_BUILD_VIEWER and build/copy wiring for viewer and required artifacts. |
| .github/skills/lvt/SKILL.md | Mirrors --fast guidance update for GitHub skill copy. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Keep shared connection snapshots alive across concurrent MCP disconnect, serialize session teardown with the target guard, and reject requests whose session was removed while waiting. Add a regression that races visual reads against disconnect. Forward pluginOption through persistent plugin get_tree calls so filtering matches the one-shot path. Enable plugin persistence only when open/get_tree/close and lvt_plugin_free are all present, and require the poll/free pair before draining plugin events. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
Build the framework-dependent x64 WPF viewer only on the x64 release leg and publish lvt-viewer-vX.Y.Z-x64.zip separately from the lean CLI archives. Include the complete matching x64 lvt runtime, TAP DLLs, managed walkers, plugins, license, and runtime instructions while excluding debug symbols. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 624b695e-57bc-426d-b037-4c0d6bd3db46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
lvt Viewer: a WPF (.NET) desktop GUI under
src/viewer/LvtViewer/thatgives lvt a Visual-Studio-Live-Visual-Tree / Inspect.exe-style graphical
front end:
toolbar crosshair; a highlight overlay tracks the top-level window under
the cursor; on release,
WindowFromPoint+GetAncestor(GA_ROOT)+GetWindowThreadProcessIdresolve the exact window and it becomes thetarget (by HWND, not by process name/PID — precise even for multi-window
processes).
property panel showing bounds/text/className/framework, plus for UIA
targets
AutomationId/ControlType/SupportedPatterns/pattern state(e.g.
Toggle.ToggleState).lvt watch's JSON diff stream — no manualre-dump needed when the target's UI changes.
Toggle.ToggleStateandValue.Value/RangeValue.Value, via lvt's existingtoggle/set-valueaction verbs (never an arbitrary property), addressed by lvt's durable
element key rather than its positional
eNid.Architecture decision: data source
Three integration points were on the table: polling
lvt dump, drivinglvt mcpover stdio, or readinglvt watch's live diff stream. I usedlvt watch, and it's the sole data source (notdump+watchlayeredtogether). Full reasoning is in
src/viewer/README.md, short version:docs/mcp-server.mdandmcp/src/server.rsare explicit:get_uia_tree/get_visual_tree/find_elementsare pull, andwait_forblocks on one condition andreturns once. Live updates over MCP would mean polling and diffing
client-side — strictly worse than
watch, which already does thatdiffing inside lvt and streams the result.
dumponce +watchon top would race two independent walks of thetarget (
watch's own internal starting snapshot vs. the separatedumpcall), and a node that changed in that gap could go uncorrected forever.
A freshly started
watchprocess's first burst ofaddedevents alreadyis a complete, self-consistent snapshot (see
run_watch_loop/snapshot_added_events), and everything after is a true diff against thatsame walk — so
watchalone has no race to have.The viewer never links
lvt_core; it only shells out tolvt.exe(
Services/LvtCli.csfor one-shot verbs,Services/WatchSession.csfor thelong-running
watchprocess) and parses its JSON.lvt.exeis located viaServices/LvtLocator.cs:LVT_EXEenv var → nextto
LvtViewer.exe→ this repo's ownbuild/lvt.exe(found by walking upfrom the viewer's build output — makes
dotnet build+run work immediatelyin a normal dev checkout) →
%USERPROFILE%\.lvt\lvt.exe→ PATH.Build wiring
-DLVT_BUILD_VIEWER=ON(defaultOFF, mirroringLVT_BUILD_MANAGED) addsan opt-in CMake custom target that runs
dotnet buildon the viewer andcopies
lvt.exealongside its output, socmake --build buildproduces aviewer that works out of the box. Building it standalone
(
dotnet build src/viewer/LvtViewer -c Release) also works and is documentedin
src/viewer/README.md.What I verified manually
Built
lvt.exefrom the main CMake preset and the viewer from bothdotnet buildand theLVT_BUILD_VIEWERCMake target; both run. Against a real,live Notepad instance I verified, end-to-end:
watchsession.Window,Document/text-editorchild, etc.), and selecting one populates the property panel with real
data (
Id,Key,Framework,Class,Text,Bounds, and the full UIAproperty set including
SupportedPatterns).lvt set-valueon Notepad's text element tochange its content — the viewer's property panel picked up the new
Value.Valuewith no manual refresh, purely from the livewatchstream.SetWindowPos— theviewer's
Boundsreflected the new size live, same mechanism.dotnet buildsucceeds in both Debug and Release.Update: the crosshair-drag gesture has now been verified live, with a real
mouse, on an unlocked desktop (the sandbox this PR was originally built in
had a locked interactive desktop, which blocked all real input — see below
for what that looked like). With a genuine mouse-down on the crosshair,
drag, and mouse-up:
tracks the window under the cursor while held, and resolves + connects to
the released-over window on mouse-up — done twice, against two different
real Notepad windows.
TargetText,read back via
lvt dump --uia) were cross-checked against an independent,ground-truth
WindowFromPoint/GetAncestor(GA_ROOT)/GetWindowThreadProcessIdcall made directly against the same screenpoint — an exact match both times (same HWND, same PID).
happened to be positioned so it covered the viewer's own crosshair, so the
mouse-down landed on Notepad instead of the crosshair — confirmed by
checking what
WindowFromPointreturned for the crosshair's own screencoordinates at that moment. That's correct, expected window z-order
behavior (you can't click through an opaque window on top), not a bug;
bringing the viewer to the front before dragging fixed it. Worth being
aware of as a usability note — Inspect.exe has the identical constraint —
but not something to change in the implementation.
src/viewer/README.md's "sandbox caveat" section has been removed/updatedto reflect this.
Stretch / incomplete
toggleandset-value; the rest ofuia_actions.cpp(click,select,expand,scroll, ...) isn't wiredinto the property panel yet. The plumbing generalizes easily.
identically, but property editing is UIA-only, matching lvt's own
mode restrictions (see "Modes" in
docs/mcp-server.md).How to build and run
or standalone once
lvt.exeis built:Full details in
src/viewer/README.md.Release packaging
The existing per-architecture CLI archives remain lean. The x64 release leg also publishes a separate
lvt-viewer-vX.Y.Z-x64.zipcontaining the framework-dependent viewer plus the complete matching x64 CLI, TAP DLLs, managed walkers, and plugins. It requires the .NET 10 Desktop Runtime; users do not need to combine it with a CLI archive.