Repository navigation
Docs: README and admin pages for the browser admin, team spaces and Docker updates - #76
Open
Skyfuller1303 wants to merge 557 commits into
Open
Skyfuller1303 wants to merge 557 commits into
Skyfuller1303 wants to merge 557 commits into
Conversation
…r each
The agent looks at GET /spaces (at the start, every two long-poll waits, and
after an invitation is answered here or a space answers 404) and syncs the team
spaces the person turns on. A team space syncs to Craft Cloud Shared/<name>/
beside the personal folder; the name goes through the node name rules and a
clash is numbered. Syncing is off for an accepted space until
POST /v1/spaces/{id}/sync.
Each synced space has its own session (state in spaces/<id>/state.json, root,
marker, guards), scheduler, folder watcher and long-poll, over a transport
bound to the space that refuses an answer without the space's echo. Passes run
one at a time across the spaces and the personal space. A viewer or archived
space runs read-only before its first write. A space the server no longer lists
stops syncing and is reported as no access, with its folder, marker and state
left alone.
Local API: GET /v1/spaces, accept and decline passed on to the server, sync on
or off, and spaces in GET /v1/status; craft-cloud-app has the calls. A server
of protocol 1 is asked about its version only and never sent craft-space.
account.json keeps its bytes until there is a team space.
Review fix of the agent's team spaces (M2). GET /spaces listed an archived space to its owners only, so an editor or viewer saw it vanish and the agent told them they had lost access, although an archived space still gives its members read access (the role is capped at viewer). It now stays in the list of every accepted member, marked archived, so the agent syncs it read-only. An invitation to an archived space is still not listed. The route test checks the member's view of an archived space.
…ess, approve its deletes, stop and pause cleanly
Review fixes of the agent's team spaces.
state.json of a team space records the space and the canonical folder it was
made for and is used for that pair only; for another folder the space starts
afresh as a first sync, which merges and deletes nothing on either side. Moving
a syncing space to another folder, or turning it off, now closes its session
and waits until the pass thread has let go of it before the state is removed or
the new folder is set up, and the thread reopens a session whose folder
changed. Signing out removes the team spaces' states as it removes the personal
one.
A space that was lost drops its state once its session is closed, so regained
access merges the folder with the space instead of resuming: a delete made here
meanwhile is not sent, a delete made on the server does not remove a local
file, and differing content becomes a conflict copy.
sync --approve-deletes[=SET] --space ID_OR_NAME and POST
/v1/spaces/{id}/approve-deletes write spaces/<id>/approval.json under the
personal space's rules; craft-cloud-app has approve_space_deletes.
A second stop no longer waits for the team passes: no queued team pass starts
and their thread is not joined. Pausing ends the team long-polls and resuming
starts them. One 404 for /spaces from a server that said protocol 2 is a
failed look, not the end of team spaces. The Debug output of SpaceRun leaves out
the words of no_access, and the log names a renamed space by its ID.
Conflicts kept from both sides. The storage figures module of the desktop stack (`space.rs`) is renamed `storage_info.rs` because the team stack's space layer owns that name.
The audit names of team spaces (space.* and the cli.space_* ones) have labels, a Team spaces group in the filters, and a kind in the activity filters.
…t "Show in folder" At the smallest width the table's last column cut the labelled Folder button off. The icon fits, and the columns now fit the smallest window without scrolling sideways.
The poll every two seconds took any screen it didn't list for a first sign-in and showed the overview; the Files page was not listed.
GET /spaces gives an accepted member of a team space joined_ms, the time they accepted their membership. It stays while they are a member (a role change does not move it) and is new when they are removed and invited again, so a client can tell that its access was interrupted. A personal space and an invitation have none, and the field is optional for older servers and clients.
…access, and never block the spaces task Turning a team space's sync off now forgets its saved state, so turning it on again, in the same folder or another, is a first sync: nothing is deleted on either side and differing files become conflict copies. The agent records the server's joined_ms of the space in its state and does not use a state made under another stamp (or without one while the server gives one). That covers an agent that was not running while access was lost and regained. A server without the field keeps the earlier behaviour, and a running session whose stamp changes is reopened. Closing a space's session no longer holds the spaces task for up to 50 seconds: the change is answered when the session is closed, other spaces, the local API and a second Ctrl-C go on meanwhile, a pass queued before the space was closed does not run, and a second change of the same space meanwhile is refused with 409. Someone waiting for the list after an accept or decline is kept waiting through a failed discovery, which is tried again after three seconds (fifteen at most). A second stop is told apart from a pass thread that is gone. The module doc and docs/agent.md say what a first sync does: files deleted on the server meanwhile are uploaded again, files that differ become conflict copies, and an archived space is listed with the member's stored role. The hunks of these changes share spaces.rs, cli.rs and the test file, so they are one commit. This commit also adds joined_ms to the agent's test helper that builds a list item.
Only the import lists and the module order of the merge resolution.
…m space calls craft-cloud-app gets the calls the Shared tab and the admin pages need (members: list, invite, change role, remove; usage; rename or limit; leave; the administrator's list, create, archive, unarchive and owner), and the agent passes them on to the server through its local API as it does for the admin calls. Leaving turns the space's sync off first, then leaves on the server; the folder and its files stay. An agent without a route answers "no such call", which the client reports as agent_too_old.
…e admin's Team spaces part The Shared tab (after Files) lists the account's team spaces with role, sync here, folder, state and what each stores; invitations are at the top with Accept and Decline, and the header's tab and the overview say how many wait. Accepting offers to sync the space to this computer, with the folder under Craft Cloud Shared and a choice of another; the sync stays off until the person says so, and a folder with files asks again. A space's page has the folder and Open folder, the sync switch, banners for a read-only role, an archived space and lost access (the local copy is kept), held deletes with Approve deletes (it lists the held set and the count, asks, and approves exactly that set), the members with roles (an owner invites by account name, changes roles and removes, with a confirmation; every role can leave, with a confirmation) and Rename. The admin pages gain a Team spaces part: the list with search and pages, a form for a new space (the owner is looked up before the password is asked for), and a page per space with its limit, archive and owner, all with the admin confirmation. Every team call that an agent from before them answers with agent_too_old ends in one message.
…the header with its new tab Every screen with the header is redrawn (it has one more tab). New: shared-* and admin-spaces-*, light, dark and at the smallest width.
The server's joined_ms in the space list and the agent's 409 while a space is still being changed. Only the tests of both sides (appended at the same place) conflicted; both are kept.
…same click The agent answers 409 while a space is still being closed after an earlier change. The offer or the page says so and offers Try again.
…fused sync is not recorded by the fake
…15) now that compression comes first
…keep a space off when the agent stops during its turn-off Turning a team space on waited for the pass thread to close its session even when the space had none. The thread works through its queue in order, so the wait sat behind whatever it did for another space, a first sync of minutes or a pass parked behind the personal one, and ran out after 50 seconds with a message about a pass of a space that never ran. The wait now only happens for a space that has a session here or a pass under way: nothing else can write its state, and an older Close of the space is ahead of any new pass of it in the queue. The message for a pass that has not ended in time no longer claims more than that. A stop while a space waited to be turned off dropped the change but not the queued forgetting of its state, so the space synced again as a first sync at the next start. The spaces task now saves the turn-off before it ends.
…r_too_old, a reason code for a paused space - approve-deletes answers the waiting set as data (set, total, truncated, up to 1000 entries, deletes held behind a link) besides the command line's words; an app reads the fields and approves by the set's name. - leave goes to the server first; only a successful leave lets the lost-access path end the sync, so a refusal (the last owner, a rate limit, no connection) changes nothing here. - a route the server doesn't have (404 no such endpoint) is 409 server_too_old, not not_found. - SpaceRun::Paused carries an optional reason_code (server_full, quota_exceeded, read_only).
- Try again belongs to the refusal it came with (space and action); any other click, navigation or error drops it. - Leave is refused for the last owner before asking, and the page waits for the server. - Held deletes come as data: the count, a scrollable list of up to 1000 paths (Showing N of M), deletes held behind a link counted; approval by the set's name, a changed set is asked again. - The account's own row has no role picker and no Remove; rename is the owners' and not for an archived space, in the model too. - A change of the agent's account resets the tab and drops answers in flight; usage is asked once per look on one thread; the server that needs an update and the paused-by-code states are said.
Turning a space on no longer waits for another space's pass, and a pending OFF is saved at stop. Only the tests appended at the end of team_spaces.rs conflicted; both are kept.
…ers, so the client sees the answer A refusal decided before the body is read (an oversized chunk or manifest, a full server) was answered while the client was still sending; the server then closed and the client, on Windows before it could read the answer, saw a reset connection. The agent's transport showed it as 'server unavailable' instead of the server's 413. The rest of the body is now read and dropped first, up to the biggest manifest and a megabyte, for at most ten seconds. bad_uploads_are_refused and oversized_manifests_are_refused failed in 5 of 20 runs of the server api package before, in none after; an agent test sends 200 oversized chunks, 8 at a time.
…e, since JSON may read them back one unit off estimating_is_read_only_and_remembered compared the estimate read back from the server value with assert_eq. serde_json reads a float back one unit in the last place off at times (1.759115717544134 came back as 1.7591157175441339 in a full workspace run), so the test failed now and then. The floats are now compared within a relative tolerance of 1e-9 and everything else exactly.
…ession opened A session compared the space's folder with its saved state only when it opened. If the synced folder was a link or junction and was pointed at another folder of the same space while the session was open, the next pass followed the link with the first folder's state in memory, took the files the other folder lacked for deleted and sent the deletions to the server. The session now remembers the canonical path and the folder ID it opened on, and the pass thread compares them before every pass. When they differ the session is dropped, the saved state is removed (a failure to remove it fails the pass), and the space starts afresh in the folder as a first sync, which merges and deletes nothing on either side. A folder that cannot be looked at (gone, drive unmounted) is not treated as moved; the pass reports what is wrong with it.
…emoved Turning a team space off removes its saved state once the session is closed. The pass thread logged a failure to remove it, such as a Windows reader holding the file without delete sharing, and acknowledged the close anyway, so the turn-off succeeded with the old state still on disk. Turning the space on again in the same marked folder then trusted that state, and files the user had deleted meanwhile were sent to the server as deletions. Each team space's setting now has a generation, saved in account.json, and a saved state records the generation it was made under; a state under another generation is not used. A turn-off raises and saves the generation before the state is retired, so a state that survives its removal is never trusted again, and a failure to remove it is carried back to the caller: the change fails with the error instead of reporting success. Losing access raises the generation too.
…stalled Setting up a team folder that has no marker writes the marker and then removes the space's saved state. If the agent stopped between the two, for a folder that was wiped and made again at the same path, the old state survived next to a valid marker; the account still pointed at the folder with syncing on, the canonical path and membership matched, and at the next start the files the old state knew were taken for deleted and sent to the server. The set-up now raises and saves the space's generation before it writes the marker, so a state made under the earlier generation is never trusted for the folder, however the set-up ends. A test hook stops the set-up right after the marker, as a crash would.
…e path The folder identity a session compares with (canonical path and folder ID) lived only in memory. A saved state recorded the path alone, so once the session was gone, from a restart, a pass that stopped because the folder was missing, or a failed cleanup of a moved folder, a folder restored from a backup at the same path (with its marker but without the newer files) got the old state back and the files it lacked were sent to the server as deletions. state.json now records the folder ID next to the path, and a state is used only for a folder with the same ID; a state without an ID is not trusted when the folder has one. A session takes one look at the folder for both the state it chooses and what it compares with later. When the pass thread finds that the folder leads somewhere else than the session opened on, it tells the spaces task, which raises and saves the space's generation, so a state that cannot be removed is not trusted again either; that pass is not run and the next one starts afresh.
…atever kind of entry it is An end-to-end run reported `store clean-temp` leaving some two-day-old `.tmp-<UUIDv7>` files in place while it printed "removed 0". The same layout could not be reproduced: planted by hand in a store directory next to a real chunk and two younger temporary files, in odd shard names (`g1`, `g2`), in a root-owned or a craft-owned directory, among 600 other files, the sweep removed every old file (a new fault test plants exactly that, with the e2e's names, in five directories; --ignored, needs the container). A root-owned directory is a loud permission error, not a silent zero. What the sweep could not do before is show why a file is left. `remove_stale_temp` now returns a report: how many it removed, how many of the backend's own names were younger than the limit, and how many other `.tmp-…` names were not exactly ours. The command prints the removed count as before and, when there are any, the other two, so a file that is still there can be told from one the command never matched. A leftover with our exact name and an old stamp is removed when it is not a plain file as well (a link made by hand, or an entry the server describes without a file type); a directory is only walked. Tested for a link in the container.
The temporary file of an upload is `.tmp-<uuid>`, as on disk, not `<name>.tmp-<id>`: corrected in the SFTP module's comment and the guide. The guide now says plainly that a restore always writes `meta.db` and a local `blobs/` tree, whatever the backed-up server stored its blobs in, so the restored server runs with `kind = "disk"`; to go back to S3 or SFTP, copy `blobs/` into the bucket or root with the same layout first (rclone copy or aws s3 sync as examples) and only then change `[storage]`. `store clean-temp`'s output is described.
…ges off, and secrets-init makes the two-factor key once The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
…nt's address The admin API listened on loopback port 2019 in the Caddy container and was used only by the health check. It is off now (admin off), and the health check asks a small loopback site that answers 204. The error log filter also deletes remote_ip, remote_port and client_ip. The server guide says that the deployment is not IP-free: some messages of Go's HTTP library carry the address in their text. A test script checks the Caddyfile and the compose file.
…limit what a restore may use A hostile backup server could replace the database dump and rewrite the hash beside it, and a restore would accept it. The first backup now makes a backup key (backup.key, mode 0600, never overwritten). The dump is sealed with AES-256-GCM, the manifest carries a keyed BLAKE3 MAC over every field, and the blobs are listed in a hashed inventory. A restore checks the MAC before it believes any field, fetches only the listed blobs, and publishes only a complete inventory. Generations from before are refused unless --allow-unauthenticated is given. SFTP listings stop at 1,000,000 entries or 64 MiB of names per directory, at a repeated name, and at 16 levels of depth. A restore has operator limits (--max-db-bytes, --max-total-bytes, --max-objects, --min-free-bytes) that are enforced before each blob.
… and why the store does not live there
…them, redact the whole database URL An existing data folder was used as it was: another user who could write to it (or to a folder above it without the sticky bit, or who had put a link there) could replace local-api.json and point apps at a listener of their own. On Unix the agent now refuses a data folder that is a link, belongs to someone else, or can be written by group or others, and apps do not read local-api.json, or send its token anywhere, unless the folder and the file pass the same checks. Windows keeps the %LOCALAPPDATA% rule; std has no owner check there. DatabaseConfig's Debug output hid only the credentials before the last @; the query string (which can carry user and password) and fragment are now hidden too. Cargo.lock: patch updates of rustls 0.23.46, rustls-webpki 0.103.16, webpki-root-certs 1.0.10, tokio 1.53.3 and ctutils 0.4.4 for RUSTSEC-2026-0346 to 0349, 0382, 0383, 0385 and 0386.
… whose password holds ? or #, keep the old error for a file at the data path on Windows old_agent_blobs and recents wrote local-api.json with the umask's mode, which the client now refuses on Unix; they write it 0600. redact_url cut at the first ? or # before it looked for the @, so postgres://cc:pa#ss@db/cc showed the user and the start of the password; the credentials now end at the last @ of the authority, wherever a ? or # sits. DataDir::create treats a file or a dangling link at the data path as an error again on Windows.
None of them ran in CI. A new job runs every packaging/docker/tests/*.sh: the rendered config, the two-factor key files, the Caddyfile and compose checks and the version script. They need no Docker.
… the key as text, honest limits and docs SFTP listings: 100,000 entries and 8 MiB of names per directory, depth 6 (the layout is three levels), and the levels of one walk or sweep together may hold about 48 MiB, so nested large directories no longer add up to gigabytes. Restore: a generation from before authentication is restored only when --generation names it and --allow-unauthenticated is given; when no generation is named it is passed over, so a plain dump the backup server plants as newest is never taken. --max-total-bytes counts what is written to the local disk, and the legacy path no longer trusts the sizes the backup server reports. The backup key file is 64 hexadecimal digits and a line break, readable and writable by hand (white space and capitals ignored); anything else is refused with a message. The rclone section names only targets the backup accepts (sftp://).
# Conflicts: # docs/server.md
# Conflicts: # Cargo.lock # crates/app/src/lib.rs
…in-tests applies on Unix The test module was gated by cfg(all(test, unix)); clippy only treats a plain cfg(test) as test code, so Linux and macOS clippy rejected its unwrap() calls (the Windows build never compiles the module). Mechanical repair of the merge of fix/local-security-hardening.
The browser admin gains a Storage page with sizes, capacity, the filesystem and the largest accounts, and a form to set the compression level. The form uses the session's CSRF token, asks for the password or second factor again, and goes through the same operation as the JSON route, so it writes the same audit entry. The overview now shows server health and whether the server refuses uploads because it is full; an account's page shows its usage. The server and storage operations that both admin surfaces use moved into shared functions in admin.rs. Fields the server does not measure per account are labelled as such instead of being left out.
Cover who can open the storage page and its security headers, the compression form's CSRF check, reconfirmation (password and second factor), level validation and its single audit entry, and the health and usage rows on the overview and account pages.
# Conflicts: # apps/craft-cloud-server/src/web/html.rs
The stylesheet takes the desktop kit's tokens (colours for light and dark, type scale, spacing) and its components: the header bar with the inverted current navigation item, joined stat cells, tables with mono headers, status badges, form panels, notices and focus rings. Tables scroll inside their own frame, the header and navigation wrap, and definition lists run in one column at phone width. New helpers emit stat cells and status badges with escaped text. The overview, storage page, account page and team-space pages use them; behaviour is unchanged. The server health section is a second-level heading under the overview's.
A small update-check service in the Docker stack asks GitHub every 12 hours for the newest release (or the newest commit of a branch) and writes a tiny status file into a volume of its own. The server mounts that volume read-only and never contacts the network itself; it stays on its internal network. The status file is treated as untrusted: it is size-capped, strictly parsed, and only a validated version or commit hash is ever shown. Links are built from the configured repository. When a newer version exists, the overview shows a banner with a link to the release notes or the comparison and the command to run on the host, and GET /admin/server reports it in a new update field (null when off). Branch checks compare against the revision now baked into the binary at build time; an unknown revision shows nothing. packaging/docker/update.sh refuses a dirty checkout, asks first, pulls fast-forward only and builds while the old server keeps running. Only then does it stop the server, back up the data volume (and dump Postgres), and start the new server; on failure it keeps the backup and says how to go back. The server itself can never update itself. CC_UPDATE_CHECK=off turns the check off. Shell tests cover the response parsing, the refusal of a dirty tree and the rendered config; web tests cover the banner for newer, equal, malformed, missing and disabled.
…sbad#74, itrycodinganditsbad#75) into integration/all Conflicts: the navigation (integration/all already had the current-page marking from itrycodinganditsbad#43; the new branch adds Storage), one navigation test message, and the backup paragraph in docs/server.md (kept the SFTP backup sentence from integration/all after the new update section).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge last, after all PRs in the #37 checklist. The branch is the combined state of those PRs plus this commit; test/integration-all...docs-refresh shows only this change. If the stack is merged in a different shape, I will rebase it.
The README and the docs describe what the open PRs add together, with screenshots of the admin pages.
What
docs/.docs/install.md: how to add a computer from one that is signed in, and what the Computers tab lists, with two screenshots of the app.docs/web-admin.md(new): what each admin page does, with screenshots: Overview and the update banner, Users and an account, Team spaces, Storage, Audit log, phone width.docs/server.md: only where it had become wrong or incomplete: the storage page in the list of admin pages, a paragraph on the update banner with a link toweb-admin.md,update-checkin the Docker services table and on theedgenetwork, and[updates]among the sections an older server refuses.Screenshots
The images are not in this branch. They live on the branch
screenshotsof my fork (24 browser-admin PNGs, 11-20 KB each, made-up accounts on a local test server, and 8 PNGs of the desktop app underapp/, 35-129 KB, taken from the app's fixture-driven snapshot tests) and the docs link them by raw URL, so the repository history gets no binary files. If you would rather keep them in the repository, I can move them todocs/images/and switch the links.The storage page is described but not shown: on a local server it shows the host's file systems.
Checks
Every relative link and anchor in the README and
docs/resolves (38), and every screenshot URL (16) points to an existing file (checked by script and over HTTP). No code changed.