feat(world/anchors): persistent spatial anchors - #494
Open
salmanmkc wants to merge 84 commits into
Open
Conversation
the shapes everything else is written against: what the platform can do, what a tracked anchor is, and what survives to the next session.
an interface rather than local storage directly, so an app can keep handles on a server and share a room between people.
off by default. persistence is a separate opt-in from anchoring, since a platform can track anchors without being able to save them.
every anchor api is optional in webxr and they are independent, so each one is probed rather than inferred from the presence of a session.
covers a disabled or full storage, malformed json, and evicting the oldest record once the cap is reached.
these fail until the manager exists. restoring is deliberately allowed to come back empty, since re-localisation is probabilistic and being somewhere else is not an error.
Storage takes null to mean persistence is disabled, distinct from omitting the argument, because a default parameter cannot tell an explicit undefined from an absent one and would silently fall back to real storage when a caller meant to opt out.
Restore failures are per-record rather than fatal, because a handle that cannot be re-localised here is the expected outcome when the user is in a different room, and one such handle must not stop the rest of the batch. An absent trackedAnchors set is treated differently from an empty one: only the latter means the platform has released everything.
Requests the anchors WebXR feature as optional, matching how plane and mesh detection are requested, so a browser without anchor support still enters the session and the subsystem reports itself unsupported instead of blocking entry.
createAnchor takes an XRSpace, and an XRSession is not one. It type checked only because XRSpace is declared as an empty interface, so the session structurally satisfied it and every anchor would have been placed against the wrong space on a real device. Also makes restoreAll idempotent, since calling it on each session start previously minted a duplicate anchor per stored record, and stops getPose throwing when called outside the frame it was given.
The desktop simulator has no tracking system to anchor against, so an app could not be developed or demoed without a headset. This holds poses directly instead, behind an opt-in flag and a distinct 'simulated' capability, so nothing can mistake it for real anchoring on a device that simply lacks support.
there is no XRFrame at all outside an immersive session, so update() returned early and the desktop fallback never activated. getPose also reads a simulated anchor's own pose now, since it has no tracked space to resolve against.
importmap and the controls that only exist outside an immersive session.
restores once per capability rather than once at startup: before entering XR the capability is simulated, and records saved by a headset have no pose to rebuild from, so a single attempt would lose them.
Spells out that a not-found restore is a normal outcome rather than a failure, and that the desktop fallback proves app wiring only, since both are easy to misread as the feature working or being broken.
Every app on the raw manager repeats the same work: hold a map from anchor to object and copy poses across each frame. This owns that so an app anchors an object and then forgets about it. Poses are built from the platform type only when it exists, since XRRigidTransform is a browser global and constructing it unconditionally made the helper unusable under test and in the simulator.
its own storage key so this demo and the marker demo, which share an origin, never restore each other's content.
the note text is the anchor label, so what persists is real user content rather than a placeholder.
Two apps served from one origin shared the anchor store, so a demo would restore another demo's anchors and look like it had invented content on first run. The default key is now page-scoped; set it explicitly to share. getPose also falls back to the renderer's reference space, since the manager already holds the renderer and every caller was otherwise branching on whether a frame existed.
Pinning twice without moving placed both items at the same point, which rendered as one unreadable blob of overlapping text.
createAnchor must run on an active frame, but apps create anchors from input handlers, which fire after the cached frame has gone inactive and the call starts rejecting. A rejected attempt now retries once on the next live frame, while permanent failures still fail immediately. Anchors do not outlive their session, so an ended session now drops them rather than leaving dead handles that a later restore would treat as already restored. Simulated anchors are also exempt from platform pruning, since an empty trackedAnchors set says nothing about anchors the platform never knew about. The demos restore once per backing rather than once ever: before entering XR the capability is simulated, and records saved by a headset have no pose to rebuild from, so a single attempt at startup lost them for good.
persist() returned void, so an app could tell a user their anchor was saved when storage was disabled or the quota was full. save() now returns a boolean the manager forwards. also validates the pose shape on load, so one malformed record can no longer take out the whole restore.
delete() and forgetAll() dropped our own records but never told the platform, so persistent handles piled up against the quota with no app left able to name them. both now call deletePersistentAnchor. it is optional in WebXR and absent for simulated anchors, so a missing or rejecting implementation still forgets the record.
tsdoc rejects dotted parameter names, and the restore factory does not read its label argument.
restoring twice built a second object for the same anchor and left the first one in the scene, untracked and never moved again. demos restore once per capability change, so this happens on the way into XR.
the store drops the oldest record silently once maxStoredAnchors is reached, so its platform handle stayed allocated with nothing left able to name it. persist now diffs the store around the write.
three.js keeps buffers and programs alive after an object leaves the scene, so deleting markers or notes in a long session grew VRAM use without bound. both demos now dispose what they drop.
whether an anchor can hand back a handle is only knowable once that anchor exists, so the session-level probe cannot speak for it. persist() already reports the per-anchor answer.
every control was a DOM button, and xrblocks never requests dom-overlay, so entering an immersive session on a phone or headset left no way to drop or forget a marker. adds a spatial panel carrying the same two actions and the status line, and turns on controller rays so it is clear what you are pointing at.
the note text came from a DOM input, which is unreachable once an immersive session starts, so the demo could not be used on a device at all. adds a spatial panel and the virtual keyboard addon. either surface can supply the text now, and pinning clears both.
markers were fanned across a row so repeated drops stayed visible, which meant a drop landed up to 32cm to the side of where you were facing and made placement feel arbitrary. goes exactly along the look direction now. two drops from the same spot still separate, but only when they would otherwise overlap.
the simulator drives the camera from wasd, so writing anything with those letters in it moved you while you typed. typing "wasdwasd" into the note box carried the camera two metres forward. releases the simulator's keyboard while the box has focus and gives it back on blur, same as the netblocks samples.
it was hard to tell apart from the marker demo, since both put a thing where you look and bring it back. the difference is that this one restores the angle too, which the helper does on the app's behalf. drops the sphere, the one shape that cannot show which way it faces, and says in the status that pieces come back at the angle they were left.
the home page described them almost identically, so there was no reason to open the second one.
a cone and a torus are symmetric about their own axis, so spinning one looks identical and the demo could not actually show the angle being restored. an arrow with a fin, a knot and a cube all read differently from every side.
the key came from the raw pathname, so reaching a demo as a directory and as its index file produced two separate stores. anchors saved through one route were invisible through the other while the device still held their handles, which looked like anchors going missing. keys on the directory now, so both routes are the same page.
both work, but only one of them matches how you'd type the url.
the platform lists handles per origin, but a store is per page, so anything another page on the same origin owned was reported as leaked. acting on that would have deleted a working app's anchors. drops the call that could not tell those apart and reports the total the device holds instead, which is true.
releasing silently did nothing when the platform had no deletePersistentAnchor, which looks exactly like a release that worked while the handle keeps its slot. every outcome is logged now. also drops the blank entries the platform returns from its handle list, including on an origin that has never saved anything. they are not handles and it refuses to delete them.
the budget is small, five or six on a quest 3, and shared by every site in the browser. once a record is lost nothing names its handle, so forgetAll cannot reach it and the slot is held until the browser's storage is cleared at the device level. releaseAllPlatformHandles reads the platform's own list instead, so an app can offer a way out. origin wide and destructive by nature, so it is documented as an explicit user action rather than cleanup.
blaming storage was wrong and sent me looking in the wrong place for an afternoon. the platform says why it refused, and that reason is already on lastError, so show it.
Clear only reaches handles this demo recorded. once the budget is full of handles nothing names, there was no way out from inside the app. sits next to Clear rather than replacing it, since it is origin wide and takes the other anchor demos' anchors with it.
the other two already had it, and without it a refused save says nothing about why.
hit this on a quest 3: the sixth anchor was refused, and the budget belongs to the browser rather than the page, so another site can be holding the slot you need. covers which cleanup call reaches what, and that clearing browsing data empties the handle list while leaving the anchors themselves, which looks like a full budget with nothing in it.
the budget belongs to the browser, so it fills up regardless of which demo you are in. the way out has to be reachable from each of them, not just the marker demo.
nothing reads the platform's list back any more, since it does not shrink while the session is running.
dli7319
self-requested a review
August 6, 2026 15:03
the demo compensated for the keyboard applying its own offset after the app had positioned it. google#495 fixed that upstream, so the compensation comes out and the position says where the keys go.
Contributor
Author
|
above commit is since #495 merged in it would be interesting to know the limits of android XR since meta quest has a low limit for anchors, at only 8 |
Collaborator
Wow so a malicious page could essentially render anchors unusable by all other pages |
Contributor
Author
Yeah, in building this I actually made a page as part of debugging that was meant to create an anchor and then delete it immediately after, I couldn't get that fully working since I had too many anchors already. My undestanding is, that what makes it stick is that the page on the receiving end has nothing it can do about it. The list of persistent anchors you get off the session is scoped to your origin, but the budget is device wide, so when the slots are held elsewhere your list comes back empty and there's nothing to delete. I hit that state on my own origin a few times while testing, with I should also correct what I wrote above: browsers are allowed to evict now. I did try but maybe it's some issue on the quest. The spec was updated after that issue was filed and says the UA may free up an entry once the maximum is reached across all origins. It doesn't look like anything implements it yet, so it's an implementation gap rather than something the spec forbids. |
The spec was updated after immersive-web/anchors#79 and now lets a user agent evict an entry to make room, so saying browsers are not allowed to is wrong. Nothing implements it yet, which is the part that still bites. Device logs show the platform refusing a new anchor while reporting no handles at all for the origin, so drop the unverified explanation of how handles get stranded and warn that a release button cannot always free a full budget.
Contributor
Author
|
Tried to share some of my findings there too: immersive-web/anchors#79 (comment) Hopefully there will be some better fix for it in the future on the quest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Would like to record an on device demo but here's a 'simulation' of it on a laptop:
anchors.demo.mp4
src/world/anchors/. content you place in a room comes back in the same physical spot after you leave and re-enter the session.been wanting to build this for xrblocks for a while. i did some research with azure spatial anchors about 5 years ago and persistence was something I used to build my shared supermarket experience, so getting it onto the open web stack was something i was pretty excited to try. kept stalling because i was abroad without my headset, and chrome on a phone implements the anchors module but not the persistence half of it, so all i could prove was that anchors got created. finally got back to my quest 3 and could actually test the thing properly.
anchors matter because a saved position drifts. the platform keeps refining its map of the room, so a coordinate you wrote down last week points somewhere slightly wrong today. an anchor gets corrected as that understanding improves, which is why
agenthandscurrently saving raw world coordinates doesn't survive a session.xb.core.world.anchorsgives youcreate/persist/restoreAll/getPose/delete. every anchor api in webxr is optional and they're independent of each other, so capability is probed rather than assumed and reports one ofpersistent,session-only,simulatedorunsupported. nothing throws on a platform without support, creation just returns null.worth knowing: only the headset runtimes implement persistence. per mdn,
requestPersistentHandle/restorePersistentAnchor/deletePersistentAnchor/persistentAnchorsare recorded against oculus browser 31.2 and nothing else, and chromium's ownxr_anchor.idldeclares justanchorSpaceanddelete(). so on a phone you getsession-onlyand the demo says so instead of silently failing. there's no separatepersistent-anchorsfeature descriptor to request, the module defines exactly one,anchors.what actually gets stored is worth being explicit about, since it differs by platform. on a headset a saved record is
{uuid, label, createdAt}and nothing else: the uuid is the opaque handle fromrequestPersistentHandle, and there are no coordinates in it at all. the spatial part lives in the platform's own map, which is why a restore can come back correctly placed after the room has been re-scanned, and why it can honestly report not-found somewhere else. records go inlocalStorageunder a key scoped to the page's directory so two demos on one origin don't rebuild each other's content, and keyed on the directory rather than the raw path because a demo reached as/demos/anchors/and as/demos/anchors/index.htmlwould otherwise get two separate stores, which on a headset looks exactly like anchors going missing, and the store is behind anAnchorStoreinterface so an app that wants handles on a server can swap it.the simulator record is the one that carries a pose, because nothing else is going to remember it. that's the tell that it isn't real anchoring, along with the
sim-prefix on its handles.desktop has no tracking system to anchor against at all, and there's no
XRFrameoutside an immersive session, so there's an opt-insimulatorFallbackthat holds poses locally. it proves app wiring only, never re-localisation, and the status line always says which backing is in use so that distinction can't be missed.three demos:
demos/anchors/drops markers,demos/anchors_notes/pins a typed note where you're looking, with the note text as the anchor label so what persists is real content, anddemos/anchors_gallery/leaves shapes around the room through theAnchoredObjectshelper, which owns the anchor-to-object map and copies poses each frame so an app doesn't have to. all three use a spatial panel rather than dom buttons since xrblocks doesn't requestdom-overlay, which means the page is gone the moment you enter xr.also exports
src/utils/ThreeDisposalfrom the barrel. six subsystems already use it but it wasn't reachable from an app, so the demos had hand-rolled the same traversal.to try it,
npm run devand opendemos/anchors_home/on a headset, which lists the three and links between them the same way the netblocks samples index does. each one has an in-scene panel, so the controls are still there once you're in xr, and the status line on the panel tells you which backing you got. you want it to say "real anchors, saved across sessions" before any of this means anything, on a phone it'll say the platform can't save them and on desktop it'll say simulated.demos/anchors/is the one to start with. Drop puts a marker 80cm in front of wherever you're facing. it doesn't hit-test against the room, so it floats there rather than landing on a surface, which is worth knowing when you go looking for it again. drop two or three next to something you'll recognise, then leave the session and come back in. they should be exactly where you left them. walk to another room and re-enter and they should report as not found rather than turning up in the wrong place. Clear removes them and forgets the saved handles, and it'll tell you whether the device actually released them.demos/anchors_notes/is the same idea with real content. type on the spatial keyboard, hit Pin, and the note is anchored where you were looking with the text as its label. useful for leaving yourself something on the kettle or a door.demos/anchors_gallery/is the same idea one layer up. pieces are placed at the angle you were looking and come back at that angle, so it's the whole pose being restored rather than a point, andAnchoredObjectsis doing that per-frame work instead of the app. shapes that show which way they face, for that reason.colour means the same thing in all three: something you just placed this session is purple in the marker and notes demos and orange in the gallery, and anything green came back out of storage. so after a reload everything should be green, and if you then place one more it stands out as the only one that isn't. that's the quickest way to tell a genuine restore from content that was simply never cleared.
all three keep their anchors in separate stores, so you can go between them without one rebuilding another's content.
in an app it's three lines:
then read
getPose(id)each frame and copy it onto your object, or hand that job toAnchoredObjectsand skip the bookkeeping entirely, which is what the gallery demo does.testing this on a quest turned up something worth writing down, since nothing in the spec prepares you for it.
persist()can be refused, and it happens sooner than you'd think:the budget belongs to the browser and is shared by every site in it, not per page, so another origin can be holding the slot you want. it's also small: mine refused the sixth anchor. the spec lets a user agent cap it, and lets it evict an entry to make room when every origin's anchors together hit the system maximum (immersive-web/anchors#79), but nothing appears to do that evicting yet, so in practice once it's full it stays full until something explicitly releases handles. worth knowing that the platform can refuse a new anchor while reporting no handles at all for your origin, which leaves the page with nothing it can name and nothing it can free.
that makes a refused save an ordinary state rather than an edge case, and there's no way to check remaining capacity first. so the demos show the platform's own words instead of a generic failure, which is the difference between "it didn't work" and knowing exactly why. i had this blaming storage at first and lost an afternoon to it.
it also means a handle you lose track of is worse than useless: it holds its slot and nothing can name it.
forgetAll()only reaches handles it recorded, soreleaseAllPlatformHandles()reads the platform's own list instead, and each demo has a Release next to Clear for exactly that. it's origin wide, so it's an explicit action rather than anything automatic.two quirks worth knowing if you go near
persistentAnchors: it contains a blank entry even on an origin that has never saved anything, and it doesn't shrink as handles are released during a session, so reading it back to confirm a release tells you nothing. both are handled, and both are in the skill doc along with what to do when the budget is full.open question on placement: i put this in
src/world/anchors/next toplanes/andmesh/, since those are also thin wrappers over a native webxr world-sensing feature and it ends up as one line inCore.tsalongside them. but the usual route here is demo first and promote later, which is whatobjects_3ddid before it became theobjects3daddon, and a demo can already pushanchorsontooptions.webxrOptionalFeaturesitself, so nothing forces it to live in the sdk. happy to pull the subsystem out and land the demos alone first if you'd rather see it proven that way, the demos are the part worth playing with anyway.sat on pushing more while #417 was open, didn't want to put too much in front of you to review at once.
Type of Change
Media / Screen Recordings & Screenshots (If Applicable)
Checklist
virtualkeyboardaddon.