Skip to content

Team spaces: the agent syncs several spaces (after P1-4) - #61

Open
Skyfuller1303 wants to merge 224 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:team-5-agent-spaces
Open

Skyfuller1303 wants to merge 224 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:team-5-agent-spaces

Conversation

@Skyfuller1303

Copy link
Copy Markdown

Merge after #60 (P1-4). The diff of this PR alone: team-4-readonly-sync...team-5-agent-spaces.

P1-3 gave the server the routes for team spaces and P1-4 taught the engine what a viewer is. This makes the agent use them: it finds out which spaces the account is in, and syncs the ones the person turns on, each to a folder of its own. The personal space, its folder, its state files and its tests are untouched. A server from before team spaces (HTTP protocol 1) is never sent a craft-space header and shows no team space.

What

  • Discovery. GET /spaces is asked at the start, every two long-poll waits, and at once after an invitation is accepted or declined here or a space answers 404. It runs apart from the agent's loop (a server that doesn't answer never holds the loop up) and a failed look changes nothing. /api/info is asked first: protocol 1, or a 404 for /spaces, means no team spaces, and the agent then asks ten times less often.
  • Folders (the layout the spec recommends, Q5). A team space syncs to Craft Cloud Shared/<name>/, beside the personal folder, so synced folders never nest and the personal engine needs no exclusion. The name goes through the node name rules (/ \ : * ? " < > | and control characters become _, no dot or space at the end, no leading dot, at most 100 characters, reserved names get " (team space)"). A clash, with another space's folder or one already on disk, takes (2), (3), compared the way node names are. The folder is chosen when the space is first seen and kept when the space is renamed on the server; GET /v1/spaces shows the new name.
  • Off by default (spec 5.3). An accepted space does not sync until POST /v1/spaces/{id}/sync {on: true}. That makes the folder and its root marker. A folder that already has files in it needs adopt: true (otherwise folder_has_files); another space's marker refuses it; a folder inside the personal folder, another synced folder or the data folder is refused. A fresh folder starts from a fresh state, so a state left from an older folder never decides what the new one is missing.
  • One engine per synced space. Each has its own session (state in spaces/<id>/state.json, root, marker, guards), scheduler, folder watcher and long-poll, over HttpTransport::bound_to(space). Passes run one at a time across all spaces, the personal one included (a gate both pass threads take), round-robin. pause, resume and quit apply to every space; a problem in one space never pauses another.
  • The echo check (spec 4.1). A bound transport refuses an answer without the space's echo, or with another one, on success and on 403, 404, 409, 410 (a 400, 401 or 429 can come from before the space layer, or from a proxy): "the server is too old for team spaces; ask the administrator to update it". The space's passes stop on it and retry slowly. Session::pass also checks the protocol before it sends anything for a team space.
  • Roles. viewer, or an archived space, from GET /spaces puts the engine in read-only mode before its first write (P1-4); a role that changes either way asks for a pass at once, so a viewer made an editor uploads what it kept without waiting for the next change.
  • Losing access (spec 5.6). A space the server no longer lists stops syncing and shows run_state: no_access with "You no longer have access to . Your local copy is kept and no longer syncs." Its folder and marker are not touched, and nothing is deleted anywhere. Its saved sync state is dropped (after its session is closed), so if the account is added again the folder starts afresh and is merged with the space like a first sync: no local file is deleted or replaced, a file deleted here meanwhile is not deleted on the server, one deleted on the server meanwhile is not deleted here, and a file that differs on both sides is kept twice (one as a conflict copy). A 404 to a pass or to the long-poll only makes the agent look at the list at once; it never stops a space by itself. An archived space is not lost: the server lists it to its members (archived: true, access capped at viewer) and the agent syncs it read-only.
  • A state belongs to its folder. spaces/<id>/state.json records the space and the canonical folder it was made for and is used for that pair only; for any other folder the space starts afresh as a first sync (a merge, nothing deleted on either side). Moving a space that syncs to another folder, or turning it off, ends its session and waits for the pass thread to let go of it before the old state is removed or the new folder is set up, so no pass for the old folder can write a state afterwards; a result of a pass of a stopped sync is ignored. Signing out removes the team spaces' states as it does the personal one (folders, markers and files stay).
  • Second review round. Turning a space off forgets its state, so turning it on again is a first sync (nothing deleted on either side). GET /spaces gives an accepted member joined_ms (new when they are removed and invited again); the agent keeps it in the space's state and does not use a state made under another one, which covers an agent that was stopped while access was lost and regained. Waiting for a session to close no longer holds up the spaces task: other spaces, the local API and a second Ctrl-C go on, a pass queued before the space was closed does not run, and a second change of the same space meanwhile is refused with 409. A discovery that fails while something waits for the list is retried after three seconds (at most fifteen). Known and documented: a first sync uploads again files teammates deleted meanwhile, a file that differs on both sides becomes a conflict copy, and an archived space is listed with the member's stored role (read archived).
  • Third review round. Turning a space on no longer waits behind another space's long pass (it waits only for a space that has a session or a pass here), and a stop while a space is being turned off saves the turn-off, so the space stays off.
  • Fourth review round. A session notices a link or junction that was pointed at another folder (canonical path and folder ID are compared before every pass) and starts afresh there; each space's setting has a generation that is raised and saved before a state is retired or a new folder's marker is written, and a state under another generation is not used, so a state that survives a failed removal or a crash is never trusted; a turn-off whose state cannot be removed fails with the error.
  • Fifth review round. A saved state also records the folder's ID and is used only for a folder with the same ID, so a folder restored from a backup at the same path starts afresh; a session takes one look at the folder for both choices; a folder found moved raises and saves the space's generation; the checkpoint just before deletions stops a pass whose folder was pointed elsewhere; a generation that could not be saved is kept, retried, and holds new syncs until saved.
  • Sixth review round. A session whose folder was found to lead elsewhere during a pass is poisoned and never used again (also if the link is pointed back); the engine asks the folder check before every commit; a team folder carries a nonce rewritten at every checkpoint and recorded in the state, so a folder restored from an older copy is told apart even where folder IDs are reused; a generation that could not be saved holds back only its own space, and the space's state says so.
  • Mass-delete approval for a team space. craft-cloud-agent sync --approve-deletes[=<set>] --space <id or name> and POST /v1/spaces/{id}/approve-deletes ({set?} → {approved, message}; craft-cloud-app has approve_space_deletes) write spaces/<id>/approval.json under the personal space's rules (the set last shown, or the named set if it is the one waiting). The command needs the running agent.
  • Stopping and pausing. A second Ctrl-C ends the team spaces at once too: no queued team pass starts and the team pass thread is not waited for (as the personal one is not). Pausing ends the team spaces' long-polls and resuming starts them again.
  • Discovery. One 404 for /spaces from a server that said protocol 2 is a failed look, not the end of team spaces (the version is asked again first).
  • Local API. GET /v1/spaces (the personal space first, then team spaces and invitations with folder, sync, run_state, read_only), POST /v1/spaces/{id}/accept and /decline (passed on to the server, which decides; the agent looks at its list again before it answers), POST /v1/spaces/{id}/sync, and spaces in GET /v1/status. craft-cloud-app has the matching calls (spaces, accept_space, decline_space, set_space_sync); an older agent's "no such call" is agent_too_old. The types' Debug output leaves out names and folders (also the words of no_access), and the agent's log names a space by its ID only.
  • account.json gains spaces: [{space, folder, sync, name?, root?}], left out while there are none: a data folder from before team spaces is not rewritten (a test compares the bytes).
  • Server, a small change to P1-3. GET /spaces lists an archived space to every accepted member, marked archived: true, not only to its owners (an invitation to an archived space is still not listed). An archived space gives its members read access (their role is capped at viewer), so the agent syncs it read-only instead of telling them they lost access. This is the first commit after P1-4; the spec text for it is in the corrections list.
  • Docs: a "Team spaces" section in docs/agent.md and the workspace map in AGENTS.md.

Checks

On Windows, through a memory-limited runner (highest peak 2.3 GiB):

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test -p craft-cloud-meta -p craft-cloud-api -p craft-cloud-server -p craft-cloud-proto -p craft-cloud-sync -p craft-cloud-agent -p craft-cloud-app --locked against Postgres 17 and SeaweedFS 4.48, with CRAFT_CLOUD_REQUIRE_BACKENDS=1
  • cargo +1.95 check --workspace --all-targets --locked (MSRV)

The listed commands passed (1599 tests; the tests against Postgres and SeaweedFS were run with CRAFT_CLOUD_REQUIRE_BACKENDS=1). New tests, in apps/craft-cloud-agent/tests/team_spaces.rs, with agents running their real loop (real folder watchers) against a real server on a loopback port:

  • An invitation is listed with no folder, accepted through the local API, and the folder is chosen but not made; turned on, the folder appears with its marker and syncs both ways between two accounts (new files, an edit, a subfolder); the status names the space; the personal folder is untouched; the state is in its own place; and a restarted agent still syncs the folder without being told again.
  • Declining removes the invitation; declining again is the server's not_found, not an old agent's.
  • A role lowered to viewer makes the folder read-only (the viewer's edit and new file never reach the server, the server's changes still arrive, nothing is undone here); raised again, what was kept goes up.
  • A removed member keeps the local copy, which stops syncing and is reported as no_access; nothing moves in either direction afterwards, the server keeps the member's earlier work, and on being invited again the sync catches up without losing a local file.
  • A renamed space keeps its folder and shows the new name, and the sync goes on.
  • Names that make the same folder get numbered, and a folder already there is not taken; turning a space on in a folder with files needs consent, and a folder inside the personal folder is refused.
  • A server that answers without the echo is refused (a front that strips the header), an unbound transport is not checked, a 404 of the real server passes through.
  • An old server (protocol 1, no /spaces, behind the same kind of front): the personal folder syncs, only the personal space is listed, no request names a space, /spaces is never asked, the version checks are few (no loop), and turning a team space on is refused.
  • A Phase 1 data folder starts unchanged: account.json is byte for byte what it was and no spaces/ folder appears.
  • Review round, same file: a state made for one folder is not used for another (a space moved to an empty folder and back, files added on the server meanwhile, then signing out); moving a syncing space syncs the new folder and deletes nothing, also after a restart; turning a space off waits for its running pass and the old state is not written afterwards; regained access merges (deletes on both sides, an edit on both, a new file) and never deletes; held deletions of a team space are approved from the command line (bare flag, wrong set, the set, by name and by ID) and the local API; a second stop ends the loop at once with a silent server; an archived space stays a read-only copy; pausing rests the long-polls; one 404 for the list does not end team spaces.
  • Unit tests: folder names, numbering, and what a discovery changes in the settings (new spaces with syncing off, renames keep the folder, a space no longer listed keeps its setting); the local API's types; the app client's calls.

Latest check of this head

Checked on Windows with Postgres 17, SeaweedFS 4.48 (S3) and CRAFT_CLOUD_REQUIRE_BACKENDS=1 under a 16 GiB memory limit at 3622a97: cargo fmt --all -- --check and cargo clippy --workspace --all-targets --locked -- -D warnings pass; cargo test --workspace --locked --no-fail-fast: 1885 passed, 0 failed, 2 ignored. This PR changes the desktop audit-filter test to count the choices from AUDIT_ACTIONS, which is why it passes here and fails on P1-1 to P1-4.

Not in this PR

  • The turmoil network simulation with two accounts: the agent tests above run against a real server instead; the simulation still syncs the personal space only.
  • leave, the member, usage and admin pass-throughs of the local API, and the app's "Remove local copy" and "Keep as a plain folder": P1-6.
  • Live documents and recents of team spaces: P2 (documents of files in a team folder are ordinary files until then).
  • The default for a newly accepted space is syncing off, as the spec decides; it is the single sync: false in spaces::reconcile if you would rather have it on.

Skyfuller1303 and others added 30 commits November 3, 2026 22:55
`craft-cloud-server healthcheck [--addr 127.0.0.1:8080]` sends GET /api/info
over a plain TcpStream and exits 0 only on a 200 status line, within 3 s.
Otherwise it exits 1 with one line on stderr. It prints nothing on stdout and
adds no dependency. A container image without a shell or curl can use it as
its health check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ment

packaging/docker holds a Dockerfile (Debian trixie build, distroless cc
runtime as uid 65532, base images pinned by digest) and a Compose deployment:
the server behind Caddy for TLS (acme, internal or custom certificates), with
SQLite and the local disk by default and overlays for Postgres and S3.

Secrets never reach a service's environment. A one-shot secrets-init service
renders config.toml from a template and writes the secret files into a volume;
every other service mounts only its own directory read-only. Every service has
a read-only root, no capabilities and no-new-privileges. Caddy has a fixed
address that the config lists in trusted_proxies.

install.sh writes .env with random passwords, builds the image, starts the
stack and creates the first admin from a password read on stdin. The Docker
section in docs/server.md covers use, TLS, Postgres, S3, upgrading and what is
not covered. The Dockerfile uses the healthcheck subcommand.

Derived from the deployment in Skyfuller1303/craftcloud (deploy/compose),
written by the same author, MIT OR Apache-2.0, adapted to this repository's
TOML-only configuration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Migration 0007 gives a device its client kind, platform and app version
and, once revoked, the time and the reason, and gives a user the time of
the last password change (plus a disabled time nothing reads yet). All
of it is additive, BIGINT on Postgres, and the flag stays the check.

Meta gets create_device_with and redeem_with, rename_device,
revoke_device_as, revoke_other_devices, revoke_all_devices, account,
change_password and reset_password. Flag, time and reason are set
together and only by the first revoke. A password change or reset sets
the hash, revokes the devices and deletes the account's invite and
pairings in one transaction.

Tests run on both databases, including the migration from 0006.

Co-Authored-By: Claude <noreply@anthropic.com>
New: GET /account, POST /account/password, PATCH /devices/{id} and
POST /devices/revoke-others. Sign-in and join take an optional client
kind, platform and app version, checked before a try is counted. The
device list shows them with the time and reason of a revoke; all new
fields are optional on the wire.

Changing the password answers a wrong current password with 403
wrong_password, never 401, since the agent reads every 401 on a
signed-in route as a revoked device. It is limited to five tries per
account in 15 minutes, takes a password-check slot, and ends the revoked
devices' live connections at once. Password and name bodies are read up
to the new TINY_JSON_BYTES (4 KiB).

wrong_password and forbidden move into the shared error codes; the local
API refers to them. The agent, app and desktop only change where they
build the extended types.

Co-Authored-By: Claude <noreply@anthropic.com>
user reset-password now revokes every device of the account (reason
reset) and deletes its open invite and pairings, so a stolen token does
not outlive the old password. --keep-devices restores the old behaviour
for the devices. device revoke-all signs out an account's devices
without touching the password. Both print how many devices they ended;
device revoke records the admin as the reason.

Co-Authored-By: Claude <noreply@anthropic.com>
The API table and error codes in the design document, the password and
sign-out commands, the live-connection delay of the command line and the
rollout note (an older server refuses the newer database) in the
server guide.

Co-Authored-By: Claude <noreply@anthropic.com>
…evice

A /changes long-poll that was waiting when its device was signed out
through the API now ends at once, and one that wakes for any other reason
looks the device up again before it answers, so a device signed out by
the command line gets 401 instead of the next change set.

The live endpoint listens for revocations and looks the device up before
the upgrade, so a revoke between the sign-in check and the subscription
is not missed.

Co-Authored-By: Claude <noreply@anthropic.com>
change_password checks, inside its transaction, that the calling device
is still signed in (it may have been revoked while the password was
hashed) and answers 401 if not. A password that changed meanwhile is now
409 conflict instead of 400 invalid. Both are new MetaError variants
mapped in the server.

set_password_hash had no caller outside a test and is removed. A new test
breaks the pairing deletes with a trigger and checks that a password
change, an admin reset and a revoke-others undo everything on SQLite and
Postgres.

Co-Authored-By: Claude <noreply@anthropic.com>
The request used bare line feeds instead of CRLF, and an answer that is not
HTTP at all was reported as a bad status. Send CRLF line endings and report
"not an HTTP answer" unless the first word is an HTTP version.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The long-poll now listens for revocations and looks its device up before
it waits, so a revoke between the sign-in check and the wait is not
missed, and looks it up once more after the change set is fetched, as
the last step before the answer, so a revoke during the fetch is not
answered with changes.

Co-Authored-By: Claude <noreply@anthropic.com>
…changes a password

change_password locked the caller's device row and then the user's row,
while revoke_other_devices from another device locked its own device and
then the user's, and reset_password locked the user before the devices.
On Postgres two of them at once could deadlock and fail with a database
error. All of them now take the user's row first (a no-op update), then
the devices, then the pairings.

A new scenario runs password changes, sign-outs and an admin reset at the
same moment, over and over, on both databases and expects only successes,
conflicts and signed-out callers. The test trigger helper also drops its
Postgres function.

Co-Authored-By: Claude <noreply@anthropic.com>
An answer that stopped inside the status line ("HTTP/1.1 200" and then EOF) or
filled the 64 byte buffer without a line end counted as healthy or as a bad
status. Now only a status line that ends in CRLF is read, and the other cases
are errors. Tests cover a truncated line, a line that does not end and a bare
line feed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
When the server is down, Caddy's error log entry for a failed request carried
the full request URI and headers, so an invite code (/join/<code>), a query
or a bearer token could end up in the log. The default log now drops
request>uri and request>headers. Caddy also skips installing its internal CA
into the container, which only produced an error line.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
db-init exported PGPASSWORD, which put the superuser password in the
environment of the shell and of psql. It now writes a mode 0600 password file
on a tmpfs, passes only PGPASSFILE and removes the file on exit. The
db-init service gets a small noexec tmpfs for it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…docs

The server started before Caddy and was given 172.29.215.2, the address
reserved for Caddy, so Caddy failed to start. The frontend network now hands
out dynamic addresses from CC_FRONTEND_POOL (the upper half by default). The
docs name the new setting, the error of a data directory with the wrong owner
and what was tried with S3.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
render-config.sh read only the first line of a secret file, so a database
password with a second line passed the check while db-init, which reads the
whole file, set a different password than the one in config.toml. The whole
file is read now, one trailing line feed is allowed and any other line feed is
refused. The tests cover a second line, an empty second line and the allowed
trailing line feed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Compose gives a container its secrets from .env when it is created. After
`docker compose run --rm secrets-init`, a later `up -d db-init` started the
original secrets-init container again, which wrote the old values back. The
docs and the script comment now say to recreate it (`up -d --force-recreate
secrets-init`) before restarting the services that read the files.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…-in adds a device

redeem consumed the invite and then updated the user, while a password
reset locks the user and then deletes the invite, so the two could
deadlock on Postgres. redeem now finds the invite or pairing with a plain
read, locks the account, and consumes the code again under the lock; a
code that is gone by then is invalid_code. create_pairing locks the
account first too.

A password sign-in verified the password outside any transaction, so a
reset or change that committed meanwhile could be followed by a new
device. sign_in_device locks the account and compares the stored hash
again before it creates the device; if it differs, login answers as for
a wrong password.

New scenarios run redeems, a sign-in, a pairing and a reset at the same
moment on both databases and expect no database error and no live device
once the reset is done.

Co-Authored-By: Claude <noreply@anthropic.com>
The invite delete and the pairing update now name the account the plain
read found, so a code hash removed and made again for another account
between that read and the lock is refused as invalid_code instead of
adding a device to the wrong account.

Co-Authored-By: Claude <noreply@anthropic.com>
The admin API's server overview shows whether blobs are on disk, in S3 or in memory. A backend that does not say is reported as other, so nothing outside the store crate has to change.

Co-Authored-By: Claude <noreply@anthropic.com>
New request and answer types for the routes under /api/v1/admin, with a redacting Debug for everything that holds a password or a code, and the error codes account_disabled and last_admin. The confirmation is a body with an optional password today, so an optional totp_code can join it later without breaking a client.

Co-Authored-By: Claude <noreply@anthropic.com>
Migration 0008 adds the purpose of an invite (invite or reset), a quota per space and a removal mark per user. The new Meta methods for the admin API each re-check the acting admin inside their own transaction and take the locks in one order, with a transaction-wide lock before the last-admin count. Disabled accounts are refused by the token check, sign-in, redeem, pairing and device creation, decided under the account's row lock. A reset code always sets a new password, signs every device out and then signs the redeeming computer in. A space's quota replaces the server-wide cap in the preflight check and in the count that follows.

Co-Authored-By: Claude <noreply@anthropic.com>
The require_admin layer and the routes under /api/v1/admin: counters, a paged and searchable user list, creating accounts, changing role and quota, disabling, enabling and removing, invites, password reset codes and a user's devices. Every change asks for the acting admin's password in its body. Sign-in and join answer account_disabled for a disabled account, and a redeemed reset code ends the live connections of the devices it signed out. Documents the API, the confirmation and the rollout in the design document and the server guide.

Co-Authored-By: Claude <noreply@anthropic.com>
Limiter::count added the attempt before comparing it with the maximum, so tries refused at the limit still raised the count. A refund of an attempt that had been accepted at the same moment then left the key above its limit with fewer wrong passwords than allowed. A refused try now leaves the count alone.

Co-Authored-By: Claude <noreply@anthropic.com>
Disabling and enabling an account leaves the password hash as it was, so a sign-in that verified the password before the disable could still create a usable device after the enable. Users get an auth_gen counter (migration 0008) that a disable, an enable, a removal and every password change or reset bump. The sign-in reads it with the hash and sign_in_device compares both under the account's lock. The server's refunds test covers the limiter change.

Co-Authored-By: Claude <noreply@anthropic.com>
Compaction asks a replica for a snapshot of a live document at an older
version. Loro's SnapshotAt export checks the document out at that version,
and on a long text history that needs many GiB: the live_documents test
an_app_that_misbehaves_is_refused_or_dropped_and_the_rest_go_on, whose app
sends 60 updates of 200,000 characters, ran out of a 16 GiB limit (itrycodinganditsbad#15).

snapshot_at now exports the operations up to the version, imports them into
a fresh document and takes that document's snapshot: the same history and
state, with memory linear in the history. A version whose operations need
others past it is refused instead of silently widened, a shallow document
is refused at once (Loro can't fork one at a version), and a scratch
document that Loro panicked in is let go of without being dropped.

Co-Authored-By: Claude <noreply@anthropic.com>
The typed details of a security event, an event, a page of them and the last line of the NDJSON export. Every
field of the details is a closed list of words or a number, and fields a later release adds are ignored.

Co-Authored-By: Claude <noreply@anthropic.com>
Migration 0009 adds audit_events, with no foreign keys so that an event outlives its account. Every change that the
log names writes its event in its own transaction before the commit, so a change whose event cannot be written is
not made. A change that changes nothing writes none. A failed sign-in is one row per account and minute whose
counter goes up, written in a transaction of its own. The values of an event are typed and come from closed
lists, never from what a client typed. Old events are pruned in batches outside those transactions.

Meta methods that had no way to say who asked get one: revoke_device_by takes the origin, and admin_enable_user and
admin_revoke_invite take the time.

Co-Authored-By: Claude <noreply@anthropic.com>
GET /account/activity, GET /admin/audit and the streamed GET /admin/audit/export; a failed sign-in is written
after the answer is decided and never changes it. [audit] retention_days (1 to 3650, default 365) and
audit list and audit prune on the command line. The export reads fixed pages through a bounded channel up to a
fixed end of the log and ends with a line that says complete or truncated, or breaks the connection.

Co-Authored-By: Claude <noreply@anthropic.com>
…he review findings of the space routes

A member whose role is too low inside the change's transaction is read_only, not forbidden. An invitation to an archived space can be neither accepted nor declined. One PATCH checks its name and limit before it writes. The invitation limit's Retry-After rounds up, a space name shaped like a space ID is refused, the command line accepts a pending invitation when it sets a role, a confirming administrator who owns a space acts as an administrator, and the administrator's search is folded text instead of a validated name. Disabling or removing an account over the admin pages wakes its spaces' long-polls.
A viewer's engine applies the server's changes and sends nothing. A local edit is kept and listed, never overwritten (a changed server version lands under the original name and the edit stays as a conflict copy); a new local file stays local; a file or folder deleted here that the server still has is downloaded again and reported. The first read_only refusal switches the engine over within the same pass, with no retry; while read-only it asks for the role once per pass and resumes uploads when the role reaches editor. The agent's HTTP transport maps 403 read_only, can be bound to a team space with the craft-space header, and the pass summary says when a space is read-only.
…ed pushes as they happened

Review fixes of the read-only mode.

A tracked folder whose place holds a file or a link (or whose parent does) is
the user's: nothing that was in it is put back through it, and nothing is
moved aside to make room. A folder put back is reported once, as restored.

A read-only pass no longer reads every edited file each time: a different size
says "edited" without reading, and the answer for the same size is kept per
file while its size, time and ID and the agreed content stay the same.

A refusal in a later batch keeps what the committed batches did in the
report; content sent for files no commit took is not reported as uploaded.
The two limits that stay (a delete held after the role rises, a swapped
tracked file not listed) are written down in the code.
…r each

The agent looks at GET /spaces (at the start, every two long-poll waits, and
after an invitation is answered here or a space answers 404) and syncs the team
spaces the person turns on. A team space syncs to Craft Cloud Shared/<name>/
beside the personal folder; the name goes through the node name rules and a
clash is numbered. Syncing is off for an accepted space until
POST /v1/spaces/{id}/sync.

Each synced space has its own session (state in spaces/<id>/state.json, root,
marker, guards), scheduler, folder watcher and long-poll, over a transport
bound to the space that refuses an answer without the space's echo. Passes run
one at a time across the spaces and the personal space. A viewer or archived
space runs read-only before its first write. A space the server no longer lists
stops syncing and is reported as no access, with its folder, marker and state
left alone.

Local API: GET /v1/spaces, accept and decline passed on to the server, sync on
or off, and spaces in GET /v1/status; craft-cloud-app has the calls. A server
of protocol 1 is asked about its version only and never sent craft-space.
account.json keeps its bytes until there is a team space.
Review fix of the agent's team spaces (M2).

GET /spaces listed an archived space to its owners only, so an editor or viewer
saw it vanish and the agent told them they had lost access, although an
archived space still gives its members read access (the role is capped at
viewer). It now stays in the list of every accepted member, marked archived,
so the agent syncs it read-only. An invitation to an archived space is still not
listed. The route test checks the member's view of an archived space.
…ess, approve its deletes, stop and pause cleanly

Review fixes of the agent's team spaces.

state.json of a team space records the space and the canonical folder it was
made for and is used for that pair only; for another folder the space starts
afresh as a first sync, which merges and deletes nothing on either side. Moving
a syncing space to another folder, or turning it off, now closes its session
and waits until the pass thread has let go of it before the state is removed or
the new folder is set up, and the thread reopens a session whose folder
changed. Signing out removes the team spaces' states as it removes the personal
one.

A space that was lost drops its state once its session is closed, so regained
access merges the folder with the space instead of resuming: a delete made here
meanwhile is not sent, a delete made on the server does not remove a local
file, and differing content becomes a conflict copy.

sync --approve-deletes[=SET] --space ID_OR_NAME and POST
/v1/spaces/{id}/approve-deletes write spaces/<id>/approval.json under the
personal space's rules; craft-cloud-app has approve_space_deletes.

A second stop no longer waits for the team passes: no queued team pass starts
and their thread is not joined. Pausing ends the team long-polls and resuming
starts them. One 404 for /spaces from a server that said protocol 2 is a
failed look, not the end of team spaces. The Debug output of SpaceRun leaves out
the words of no_access, and the log names a renamed space by its ID.
GET /spaces gives an accepted member of a team space joined_ms, the time they
accepted their membership. It stays while they are a member (a role change does
not move it) and is new when they are removed and invited again, so a client
can tell that its access was interrupted. A personal space and an invitation
have none, and the field is optional for older servers and clients.
…access, and never block the spaces task

Turning a team space's sync off now forgets its saved state, so turning it on
again, in the same folder or another, is a first sync: nothing is deleted on
either side and differing files become conflict copies.

The agent records the server's joined_ms of the space in its state and does not
use a state made under another stamp (or without one while the server gives
one). That covers an agent that was not running while access was lost and
regained. A server without the field keeps the earlier behaviour, and a running
session whose stamp changes is reopened.

Closing a space's session no longer holds the spaces task for up to 50 seconds:
the change is answered when the session is closed, other spaces, the local API
and a second Ctrl-C go on meanwhile, a pass queued before the space was closed
does not run, and a second change of the same space meanwhile is refused with
409. Someone waiting for the list after an accept or decline is kept waiting
through a failed discovery, which is tried again after three seconds (fifteen
at most). A second stop is told apart from a pass thread that is gone.

The module doc and docs/agent.md say what a first sync does: files deleted on
the server meanwhile are uploaded again, files that differ become conflict
copies, and an archived space is listed with the member's stored role.

The hunks of these changes share spaces.rs, cli.rs and the test file, so they
are one commit. This commit also adds joined_ms to the agent's test helper that
builds a list item.
…keep a space off when the agent stops during its turn-off

Turning a team space on waited for the pass thread to close its session even
when the space had none. The thread works through its queue in order, so the
wait sat behind whatever it did for another space, a first sync of minutes or a
pass parked behind the personal one, and ran out after 50 seconds with a
message about a pass of a space that never ran. The wait now only happens for a
space that has a session here or a pass under way: nothing else can write its
state, and an older Close of the space is ahead of any new pass of it in the
queue. The message for a pass that has not ended in time no longer claims more
than that.

A stop while a space waited to be turned off dropped the change but not the
queued forgetting of its state, so the space synced again as a first sync at the
next start. The spaces task now saves the turn-off before it ends.
…ession opened

A session compared the space's folder with its saved state only when it opened.
If the synced folder was a link or junction and was pointed at another folder
of the same space while the session was open, the next pass followed the link
with the first folder's state in memory, took the files the other folder lacked
for deleted and sent the deletions to the server.

The session now remembers the canonical path and the folder ID it opened on,
and the pass thread compares them before every pass. When they differ the
session is dropped, the saved state is removed (a failure to remove it fails the
pass), and the space starts afresh in the folder as a first sync, which merges
and deletes nothing on either side. A folder that cannot be looked at (gone,
drive unmounted) is not treated as moved; the pass reports what is wrong with it.
…emoved

Turning a team space off removes its saved state once the session is closed. The
pass thread logged a failure to remove it, such as a Windows reader holding the
file without delete sharing, and acknowledged the close anyway, so the turn-off
succeeded with the old state still on disk. Turning the space on again in the
same marked folder then trusted that state, and files the user had deleted
meanwhile were sent to the server as deletions.

Each team space's setting now has a generation, saved in account.json, and a
saved state records the generation it was made under; a state under another
generation is not used. A turn-off raises and saves the generation before the
state is retired, so a state that survives its removal is never trusted again,
and a failure to remove it is carried back to the caller: the change fails with
the error instead of reporting success. Losing access raises the generation too.
…stalled

Setting up a team folder that has no marker writes the marker and then removes
the space's saved state. If the agent stopped between the two, for a folder that
was wiped and made again at the same path, the old state survived next to a
valid marker; the account still pointed at the folder with syncing on, the
canonical path and membership matched, and at the next start the files the old
state knew were taken for deleted and sent to the server.

The set-up now raises and saves the space's generation before it writes the
marker, so a state made under the earlier generation is never trusted for the
folder, however the set-up ends. A test hook stops the set-up right after the
marker, as a crash would.
…e path

The folder identity a session compares with (canonical path and folder ID) lived
only in memory. A saved state recorded the path alone, so once the session was
gone, from a restart, a pass that stopped because the folder was missing, or a
failed cleanup of a moved folder, a folder restored from a backup at the same
path (with its marker but without the newer files) got the old state back and the
files it lacked were sent to the server as deletions.

state.json now records the folder ID next to the path, and a state is used only
for a folder with the same ID; a state without an ID is not trusted when the
folder has one. A session takes one look at the folder for both the state it
chooses and what it compares with later. When the pass thread finds that the
folder leads somewhere else than the session opened on, it tells the spaces
task, which raises and saves the space's generation, so a state that cannot be
removed is not trusted again either; that pass is not run and the next one
starts afresh.
…sends deletions

The folder was compared with the one the session opened on only before a pass. A
link or junction pointed at another folder while the pass was already under way
made it read the new folder with the old state in memory, and the files the new
folder lacked were taken for local deletions and sent to the server.

The engine saves the state just before it sends a batch with deletions, and the
session's checkpoint now compares the folder with the one it opened on first: if
it differs the pass stops there, nothing is sent or saved of what it saw, and the
next pass starts afresh in the folder.
…syncs until it is

Raising a space's generation (on access lost, on a replaced folder, on a turn-off)
changed it in memory first and saved account.json after. When the save failed,
which was only logged for a lost space, a restart before any later save found the
old generation on disk and trusted a state that was meant to be dropped if the
removal of that state had failed too.

A raise that could not be saved is now remembered and saved again at every look at
the list, and no space is started until it is saved; running syncs are not
stopped, their state is valid. The module doc and docs/agent.md say so, and the
docs also describe the folder check before deletions and the folder ID.
…very commit

When the folder check stopped a pass, the session stayed open with an engine
state that had taken in part of the other folder (a server file written into it,
deletions noted after its scan), and the pass's end saved that state under the
original folder's identity. If the link was pointed back before the next pass,
the check passed again and the file the first folder never received was taken
for deleted and removed on the server.

The check now poisons the session when it trips, and the end of a pass compares
the folder once more and saves nothing if it moved; the pass thread then drops
the session, removes the state and raises the generation. The engine also asks
the same check before every commit of a push, not only before one with
deletions, so no batch of any kind is sent after a retarget (Engine::on_guard).
A folder restored from a backup at the same path got the old state back whenever
the file system gave it the same ID, which ext4 does when it reuses an inode
number. The folder's .craft-cloud now carries a nonce that is rewritten with a
new random value at every checkpoint and at the end of every pass, and the state
saved with it records the same value. A state is used only for a folder that
carries the nonce it recorded; a folder without one suits a state without one
only, so states from before nonces are refused by a folder that has one, as with
the folder ID. A crash between writing the nonce and the state costs a first-sync
merge, which deletes nothing.
…nd say so

When account.json could not be written after a generation was raised, no space
was started until the save worked, and the only trace was a line in the run log.
Only the affected spaces are held back now; other spaces start and sync as usual.
A held-back space shows as stopped in its state, with the reason in words, in the
local API's list and in the status.
The test expected 36 choices, but the page offers six groups and every name of the closed list in craft_cloud_app::AUDIT_ACTIONS, now 44. It counts them from the list, so adding an action no longer breaks it.
With links not followed, inotify watches a link given as the synced folder and not the folder behind it, so a local change
in it raised no event and waited for the next periodic pass. The folder the link leads to is watched instead.
The watcher and the folder's roots were made once, when the space started. When a pass found that the folder leads somewhere
else (a link pointed at another folder), only the generation was raised, so changes in the new folder raised no event. The
space now makes its watcher and roots again on that event, under a new watcher generation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant