feat: analytics auto-tracking helpers for Web and Flutter - #1622
feat: analytics auto-tracking helpers for Web and Flutter#1622lohanidamodar wants to merge 8 commits into
Conversation
Adds a hand-written companion to the generated `Analytics` service that wires up common auto-tracking behaviours (pageviews, outbound links, downloads, scroll depth, active engagement) on top of the standard `event()` endpoint. The helper is composition-based and takes a plain event-emitter callback so it does not hard-couple to the generated service class. All helpers respect `navigator.doNotTrack` by default and can be opted out. Public surface: - `AnalyticsTracking` — main entry point, wraps an emitter callback - `enableAutoPageviews()` / `disableAutoPageviews()` — SPA-aware pageview tracking that hooks `pushState`, `replaceState`, `popstate` and hashchange - `enableAutoOutboundTracking(options?)` — anchor-click detection for external links with optional subdomain handling - `enableAutoDownloadTracking(options?)` — anchor-click detection by file extension list - `enableAutoScrollDepth()` — max scroll % per page, flushed on unload and SPA route change - `enableAutoEngagementTime()` — active-time counter with visibility gating - `enableAllAutoTracking()` — opinionated default set (pageviews, outbound, scroll, engagement) - `track(name, options)` / `pageview(url, referrer)` — manual passthroughs The new template file is registered in `Web.getFiles()` and the class is re-exported from the SDK entry point.
Adds hand-written companions to the generated `Analytics` service for the
Flutter SDK, following the same composition-based pattern as the Web helpers.
The observer plugs into `MaterialApp.navigatorObservers` and fires an event
for every route push/pop/replace/remove; the tracking class listens to
`WidgetsBindingObserver` for app-backgrounded / app-foregrounded events and
exposes a mobile-idiomatic `screenView(name, {className, props})` shortcut.
Public surface:
- `AnalyticsEventEmit` typedef — the callback used to send events, keeping
the helpers decoupled from the generated service class
- `AnalyticsObserver extends NavigatorObserver` — auto screen-view tracking
with an optional `ScreenNameExtractor`
- `AnalyticsTracking` — `WidgetsBindingObserver`-driven lifecycle events,
plus `screenView()` and `event()` shortcuts and an `enableAllAutoTracking()`
composite
The new templates are registered in `Flutter.getFiles()` and re-exported from
the package entry point.
Greptile SummaryThis PR adds hand-written auto-tracking helpers for the Analytics service to the Web and Flutter SDKs, following the same composition pattern used by the Realtime helpers. The helpers accept an event-emitter callback rather than coupling to the generated
Confidence Score: 4/5Safe to merge with one logic fix: the engagement timer starts unconditionally without checking initial tab visibility, which corrupts engagement metrics for background-tab opens. The Web engagement tracker calls templates/web/src/services/analytics-tracking.ts.twig — specifically the Important Files Changed
Reviews (3): Last reviewed commit: "fix(analytics): stop engagement timer on..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| private rootDomain(hostname: string): string { | ||
| const parts = hostname.split('.'); | ||
| if (parts.length <= 2) { | ||
| return hostname; | ||
| } | ||
| return parts.slice(-2).join('.'); | ||
| } |
There was a problem hiding this comment.
rootDomain breaks for multi-part TLDs
parts.slice(-2).join('.') returns co.uk for both myapp.co.uk and google.co.uk, so any click from a .co.uk site (or .com.au, .com.br, .co.jp, etc.) to any other domain on the same TLD registers as internal and is silently dropped from outbound tracking. Concretely, from shop.example.co.uk, a click on https://google.co.uk returns rootDomain = co.uk on both sides, isExternal returns false, and no event is fired.
There was a problem hiding this comment.
Real, and fixed in 3f319cea6.
Confirmed the exact case: from shop.example.co.uk a click to https://google.co.uk reduced both sides to co.uk, so isExternal() returned false and no event fired. Same on .com.au, .com.br, .co.jp and the rest — the outbound events simply never existed for those sites.
rootDomain() now takes three labels when the last two are a known two-part public suffix. That's a heuristic rather than the Public Suffix List, which is far larger than this entire file and can't ship in a tracking snippet; sites on a suffix that isn't listed can still pass includeSubdomains: true.
Checked the behaviour rather than assuming it:
PASS two different .co.uk sites -> external [example.co.uk vs google.co.uk]
PASS same site, parent domain -> internal [example.co.uk vs example.co.uk]
PASS sibling subdomains -> internal [example.co.uk vs example.co.uk]
PASS www vs apex -> internal [example.com vs example.com]
PASS .com.au across sites -> external [example.com.au vs other.com.au]
PASS lookalike host -> external [example.com vs evil-example.com]
The last two are the ones a naive suffix-match fix gets wrong, so they're worth pinning.
| this.pageviewCleanups.push(() => { | ||
| history.pushState = originalPushState; | ||
| history.replaceState = originalReplaceState; | ||
| window.removeEventListener('popstate', popstateHandler); | ||
| window.removeEventListener('hashchange', hashchangeHandler); | ||
| }); |
There was a problem hiding this comment.
history methods can clobber third-party router patches
disableAutoPageviews() restores history.pushState and history.replaceState to the references captured at enableAutoPageviews() call time. If a router (Next.js App Router, React Router, etc.) patched the same methods after this tracker initialised, the restore will silently discard that library's wrapper. The typical pattern for chaining patches is to keep the wrapped function and call through to it; restoring to a saved reference is safer only when no other library ever patches these globals.
There was a problem hiding this comment.
Fair point, fixed in 3f319cea6 — and fixing it properly needed one more step than "don't restore".
disableAutoPageviews() now only unwinds its own patch: it assigns the originals back only when history.pushState/replaceState are still the exact wrappers we installed. If a router patched on top of us, the current value is theirs and we leave it alone.
That alone would have left a hole, though. When a foreign patch stays in the chain it keeps calling our wrapper, which called handleNavigation() unconditionally — so "disable" wouldn't actually have stopped pageviews in precisely the scenario you describe. The wrapper now carries a patchActive flag that the cleanup clears, so it keeps delegating to the original but stops reporting.
Chaining rather than restoring is the other option, but it leaves our frame on the stack forever for callers who enable/disable repeatedly. Making the wrapper inert gets the same outcome without growing the chain.
Auto-emitted event names and prop keys now align with the analytics endpoint's canonical patterns: - Event names are snake_case + lowercase (`pageview`, `outbound_link`, `file_download`, `scroll_depth`, `engagement_time`, `screen_view`, `app_backgrounded`, `app_foregrounded`) rather than PascalCase strings with parametric suffixes. Data that was previously encoded into the event name (hostname for outbound links, filename for downloads) now moves into camelCase prop fields. - Web: add optional `scrollDepth` and `engagementTime` fields on `AnalyticsEventOptions` so the scroll-depth and engagement-time auto-trackers forward those values as top-level endpoint params rather than folding them into `props`. - Flutter: rename `AnalyticsEventEmit` -> `AnalyticsEventEmitter` for cross-platform typedef consistency with the Web SDK; rename the `screenView` `class` prop to `screenClass` so it follows the camelCase convention. - Docblocks on both platforms document the naming convention.
…ng before observer Address two bugs surfaced by review. Web: `flushEngagement` called `startEngagement()` at the end of every flush, including the pagehide flush that precedes back-forward-cache entry. When the tab was restored via bfcache the timer resumed with an engagementStart snapshotted before the freeze, so the next flush billed the entire bfcache dwell time as engagement. flushEngagement no longer auto-restarts; restart is now driven by explicit resume signals — pageshow with persisted === true, visibilitychange back to visible, or the SPA navigation flow. Flutter: `enableAutoLifecycleEvents()` touched `WidgetsBinding.instance` without ensuring the binding, so callers who followed the docstring example (construct + `enableAllAutoTracking()` before `runApp`) hit `StateError: Binding has not yet been initialized`. Call `WidgetsFlutterBinding.ensureInitialized()` before attaching the observer (idempotent). Also fold in a few smaller review findings: - Flutter: guard `app_backgrounded` on `_foregroundSince` so it emits once per background transition instead of firing a second empty event when the desktop/web lifecycle passes through both `hidden` and `paused`. - Web: drop image extensions (jpg/jpeg/png/gif/svg/webp) from the default download list — anchor-wrapped inline images (lightboxes, galleries) were being counted as file downloads. - Web: add `disableAutoOutboundTracking`, `disableAutoDownloadTracking`, `disableAutoScrollDepth`, `disableAutoEngagementTime` for parity with `disableAutoPageviews`, so consent flows can tear down individual trackers without discarding the whole `AnalyticsTracking` instance.
|
Thanks for the careful review — both P1s addressed, plus a few smaller items. Required fixes
Smaller cleanups folded in
Not tackled in this pass
Verified: PHPUnit unit suite passes, Rector dry-run clean, djLint clean, phpcs clean, regenerated Web SDK compiles with no |
…ltas Engagement time was accumulated in memory and flushed exactly once, on `beforeunload`/`pagehide`. That is the least reliable moment in the whole visit to send anything: browsers routinely cancel in-flight `fetch()` at unload, and `navigator.sendBeacon` is not an option here because the ingestion endpoint requires the `X-Appwrite-Project` header — beacons cannot set custom headers, and neither a `?project=` query param nor a `text/plain` body is accepted. A single unload flush is therefore exactly the delivery most likely to be lost. Switch to Plausible's model and emit engagement during the visit instead: - Web: `visibilitychange` -> hidden now flushes rather than only pausing, joining the existing SPA route-change flush. Unload stays as a best-effort final flush but is no longer the only delivery. - Each event carries a delta (seconds since the previous flush), not a running total, so values are additive per session and a lost event costs only its own slice. - `flushEngagement` deducts only the whole seconds it actually emitted and keeps the sub-second remainder in the accumulator, so repeated flushes can neither double-count nor round time away. A route change immediately followed by unload emits the seconds once. - The engagement clock no longer starts for a tab that is already hidden when tracking is enabled (prerender, background restore). bfcache handling is unchanged: `pagehide` still flushes without restarting, and `pageshow`/`persisted` restarts, so dormant time is never billed. - No periodic ping is added. Cadence stays navigation- and tab-switch driven, matching the unbatched one-request-per-event ingestion path. Flutter gets the equivalent treatment. `app_backgrounded` already fired on every background transition — the mobile equivalent of a tab going hidden — but it reported the time as a `foregroundSeconds` prop, which never reached the `engagement_time` column. It now carries `engagementTime` as a top-level param, as a delta since the last resume, with the same sub-second carry-forward. The seconds ride on the lifecycle event already being sent at that moment rather than adding a second request. Also add `propertyId` to the tracking options on both platforms. It was previously absent from the helpers entirely, forcing every consumer to hand-roll it into their emitter closure and failing at runtime when they forgot. The emitter-function design is preserved: when no default is configured the emitter receives exactly what it received before, and a per-event `propertyId` still wins.
…ory unpatch Two of the seven review comments were still live; the other five were fixed in 5a5939e / 0b5c5af after the review ran. rootDomain() reduced every host to its last two labels, so on registries that sell names one level down it returned the public suffix itself: from shop.example.co.uk a click to google.co.uk compared co.uk against co.uk, read as internal, and was dropped. Same for .com.au, .com.br, .co.jp and friends - the outbound events simply never fired. It now takes three labels after a known two-part suffix. That is a heuristic, not the Public Suffix List, which is far larger than this whole file; sites on a suffix not listed can still pass includeSubdomains: true. Verified against the cases that matter, including that sibling subdomains stay internal and a lookalike host (evil-example.com) stays external. disableAutoPageviews() restored history.pushState/replaceState from saved references, which throws away the wrapper of any router that patched after us. It now only unwinds its own patch, and the wrapper carries a patchActive flag so that when it does stay in a foreign chain it keeps delegating but stops reporting - otherwise disabling would not actually have stopped pageviews in exactly the case the comment describes. Typechecked by compiling the template as TypeScript (it carries no Twig placeholders): tsc clean.
1 participant
Summary
Adds hand-written auto-tracking helpers for the Analytics service across the Web and Flutter SDKs, following the same pattern as Realtime (fully hand-written per platform, not generated from OpenAPI spec).
The helpers are composition-based: each takes a plain event-emitter callback rather than importing the generated service class directly, so they render and compile independently of any given spec revision. Users wire them up in one line:
What's included
Web (
templates/web/src/services/analytics-tracking.ts.twig)enableAutoPageviews()/disableAutoPageviews()— SPA-aware: initial load +pushState/replaceState/popstate/hashchangeenableAutoOutboundTracking(options?)— click detection on external anchor links, optional subdomain handlingenableAutoDownloadTracking(options?)— click detection on file-download links, extension list configurableenableAutoScrollDepth()— max scroll % per page, flushed on unload and on SPA route changeenableAutoEngagementTime()— incremental, Plausible-style engagement reporting (see below)enableAllAutoTracking()— opinionated default set (pageviews, outbound, scroll, engagement)track(name, options)/pageview(url, referrer)— manual passthroughsnavigator.doNotTrackby default (opt-out viarespectDoNotTrack: false)Flutter (
templates/flutter/lib/src/analytics_observer.dart.twig,analytics_tracking.dart.twig)AnalyticsObserver extends NavigatorObserver— auto screen-view tracking on route push / pop / replace / remove, with an optionalScreenNameExtractorfor filteringAnalyticsTracking—WidgetsBindingObserver-driven lifecycle events (app_backgrounded/app_foregrounded), with engagement time attached to the backgrounding eventscreenView(name, {className, props})— mobile-idiomatic convenience for the pageview equivalentenableAutoLifecycleEvents()/disableAutoLifecycleEvents()/enableAllAutoTracking()Registrations
src/SDK/Language/Web.php— new template listed ingetFiles()src/SDK/Language/Flutter.php— both new templates listed ingetFiles()templates/web/src/index.ts.twig—AnalyticsTrackingand its supporting types re-exportedtemplates/flutter/lib/package.dart.twig— observer and tracking modules re-exportedEngagement time model
Engagement is reported incrementally during the visit, matching Plausible, rather than as a single total flushed at unload.
Why not a single unload flush. Browsers routinely cancel in-flight
fetch()atbeforeunload/pagehide.navigator.sendBeaconis not an available fallback: the ingestion endpoint requires theX-Appwrite-Projectheader, beacons cannot set custom headers, and neither a?project=query param nor atext/plainbody is accepted (both 400, verified against a live server). A single unload flush is therefore precisely the delivery most likely to be lost.Cadence. An
engagement_timeevent is emitted at every natural checkpoint of the visit:visibilitychange→ hiddenAppLifecycleState.paused/hiddenpushState/replaceState/popstate/hashchangebeforeunload/pagehide, best-effortThere is deliberately no periodic ping. Cadence is navigation- and tab-switch driven, so request volume tracks what the visitor actually does — the ingestion path is one request per event with no batching, so a timer would multiply volume for no extra signal.
Payload: deltas, not cumulative totals. Each event carries only the seconds accrued since the previous flush. This is what Plausible does, and it is robust to loss: a dropped event costs only its own slice instead of the whole visit or the whole running total. The alternative — each event carrying a running total that supersedes its predecessors — would require the backend to de-duplicate with a
max()per page-visit anyway, so it is not actually the "no backend change" option it looks like.Anti-double-counting.
flushEngagementdeducts only the whole seconds it actually emitted and keeps the sub-second remainder in the accumulator, so:visibilitychange→ hidden followed bypagehide(which is the normal ordering on tab close) likewise emits once;bfcache. Unchanged and still correct:
pagehideflushes without restarting the clock, and onlypageshowwithpersisted === truerestarts it, so bfcache dwell time is never billed as engagement. Additionally, the clock no longer starts at all for a tab that is already hidden when tracking is enabled (prerender, background tab restore).Concrete sequence — load → scroll → switch tab (30s away) → return → SPA navigate → close:
pageviewurl,referrerengagement_timeengagementTime: 20(400 ms carried forward)scroll_depthscrollDepthfor page Aengagement_timeengagementTime: 15(600 ms carried), attributed to page A's URLpageviewscroll_depthengagement_timeengagementTime: 3, best-effortTotal reported engagement: 20 + 15 + 3 = 38 s, against 38.5 s actually engaged, with the 30 s of background time correctly excluded. If step 10 is lost at unload — the likely failure — 35 of the 38 s still arrived.
Backend:
engagementTimeaggregation must changeThe current cloud aggregation in
Appwrite\Cloud\Analytics\Adapter\ClickHouse::compileEventMetricExpression()is an average per event:That was correct while exactly one engagement event existed per page-visit. Under delta reporting it is not correct — three flushes of 10 s each would average to 10 rather than summing to 30. The expression needs to sum the deltas per session and then average across sessions:
Single-level, so it drops straight into the existing
matcharm and composes with the other metrics on the sameSELECT.nullIfkeeps the expression null (rather than dividing by zero) for windows with no engaged sessions;castMetricValue()already castsengagementTimetoint.If per-page-visit semantics are preferred instead — closest to what the old
avgIfactually meant, "average engagement per pageview" — the same shape works with a tuple key:This PR does not make that change — it belongs to appwrite-labs/cloud#3965. Until it lands,
engagementTimewill read low.tests/unit/Analytics/Aggregate/SessionsMetricsTest.phpasserts the current fragment and will need updating with it.Other changes in this revision
propertyIdon the tracking options. It previously appeared nowhere in either template, so every consumer had to hand-roll it into their emitter closure and forgetting failed at runtime rather than at compile time.AnalyticsTracking(both platforms) andAnalyticsObservernow accept an optionalpropertyIdand forward it with every event. The emitter-function design is untouched: with no default configured the emitter receives exactly what it received before, and a per-eventpropertyIdstill wins over the configured one.propertyIdandengagementTimenamed params, so engagement lands in theengagement_timecolumn instead of aforegroundSecondsprop that the metric never read. Existing(name, {props}) => ...closures need widening to(name, {props, propertyId, engagementTime}) => ...; the helpers are unreleased, so there is no compatibility surface to preserve.Naming conventions
Auto-emitted event names and prop keys follow a single convention on both platforms so they compose cleanly with the analytics endpoint's parameter naming:
snake_case+ lowercase. The helpers own the names of the events they emit themselves:pageview,outbound_link,file_download,scroll_depth,engagement_time(Web),screen_view,app_backgrounded,app_foregrounded(Flutter). Callers can still pass any custom event name to the manualtrack()/event()methods; those strings are forwarded verbatim.camelCase, matching the endpoint's top-level parameter naming (hostname,href,filename,extension,screenClass, etc.).AnalyticsEventOptionsexposespropertyId,scrollDepthandengagementTimeas first-class option fields so the trackers forward them as endpoint params rather than folding them intoprops.AnalyticsEventEmitter(wasAnalyticsEventEmit), matching the Web SDK's exported name for cross-platform consistency.Notes
Analyticsservice only through user-supplied callbacks, so this PR is safe to merge before / after the spec adds the ingestion endpoint. Once the spec exposesanalytics.event(...), the wiring shown above is the intended usage.analytics.event(...)from the standard generated code path, so no hand-written helper is added there.AnalyticsObserveris a separate object fromAnalyticsTracking, so engagement is flushed on backgrounding only. Under delta semantics the session total is still exact, just attributed at coarser granularity.app_backgroundedevent rather than a separateengagement_timeevent, to avoid doubling request volume for a single number. The aggregation SQL above is event-name agnostic, so both platforms feed the same metric.Verification
composer test— 33/33 unit tests pass (vendor/bin/phpunit --testsuite Unit)composer refactor:check— clean (rector, dry run)composer lint-twig— clean (djlint over all 565 template files)composer lint— clean (phpcs)tsc --noEmit --strictreports zero errors in the newanalytics-tracking.ts(pre-existing errors in other generated services are unrelated)flutter analyze lib/src/analytics_observer.dart lib/src/analytics_tracking.dartreports zero issues, including the widened emitter typedef against real caller closures