Skip to content

Docs: README and admin pages for the browser admin, team spaces and Docker updates - #76

Open
Skyfuller1303 wants to merge 557 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:docs-refresh
Open

Skyfuller1303 wants to merge 557 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:docs-refresh

Conversation

@Skyfuller1303

@Skyfuller1303 Skyfuller1303 commented Nov 11, 2026 •

Copy link
Copy Markdown

Merge last, after all PRs in the #37 checklist. The branch is the combined state of those PRs plus this commit; test/integration-all...docs-refresh shows only this change. If the stack is merged in a different shape, I will rebase it.

The README and the docs describe what the open PRs add together, with screenshots of the admin pages.

What

  • README: the status note and the feature list now name team spaces, the admin pages in the browser, blobs on disk, S3 or SFTP, backups to SFTP, and the Docker install with its update notice. A one-line Docker quick start next to the existing one, three screenshots (overview with the update notice, a team space in dark, team spaces on a phone), a small "The app" section with three screenshots of the desktop app (overview, recent files, shared), and the documentation list with every file in docs/.
  • docs/install.md: how to add a computer from one that is signed in, and what the Computers tab lists, with two screenshots of the app.
  • docs/web-admin.md (new): what each admin page does, with screenshots: Overview and the update banner, Users and an account, Team spaces, Storage, Audit log, phone width.
  • docs/server.md: only where it had become wrong or incomplete: the storage page in the list of admin pages, a paragraph on the update banner with a link to web-admin.md, update-check in the Docker services table and on the edge network, and [updates] among the sections an older server refuses.

Screenshots

The images are not in this branch. They live on the branch screenshots of my fork (24 browser-admin PNGs, 11-20 KB each, made-up accounts on a local test server, and 8 PNGs of the desktop app under app/, 35-129 KB, taken from the app's fixture-driven snapshot tests) and the docs link them by raw URL, so the repository history gets no binary files. If you would rather keep them in the repository, I can move them to docs/images/ and switch the links.

The storage page is described but not shown: on a local server it shows the host's file systems.

Checks

Every relative link and anchor in the README and docs/ resolves (38), and every screenshot URL (16) points to an existing file (checked by script and over HTTP). No code changed.

…r each

The agent looks at GET /spaces (at the start, every two long-poll waits, and
after an invitation is answered here or a space answers 404) and syncs the team
spaces the person turns on. A team space syncs to Craft Cloud Shared/<name>/
beside the personal folder; the name goes through the node name rules and a
clash is numbered. Syncing is off for an accepted space until
POST /v1/spaces/{id}/sync.

Each synced space has its own session (state in spaces/<id>/state.json, root,
marker, guards), scheduler, folder watcher and long-poll, over a transport
bound to the space that refuses an answer without the space's echo. Passes run
one at a time across the spaces and the personal space. A viewer or archived
space runs read-only before its first write. A space the server no longer lists
stops syncing and is reported as no access, with its folder, marker and state
left alone.

Local API: GET /v1/spaces, accept and decline passed on to the server, sync on
or off, and spaces in GET /v1/status; craft-cloud-app has the calls. A server
of protocol 1 is asked about its version only and never sent craft-space.
account.json keeps its bytes until there is a team space.
Review fix of the agent's team spaces (M2).

GET /spaces listed an archived space to its owners only, so an editor or viewer
saw it vanish and the agent told them they had lost access, although an
archived space still gives its members read access (the role is capped at
viewer). It now stays in the list of every accepted member, marked archived,
so the agent syncs it read-only. An invitation to an archived space is still not
listed. The route test checks the member's view of an archived space.
…ess, approve its deletes, stop and pause cleanly

Review fixes of the agent's team spaces.

state.json of a team space records the space and the canonical folder it was
made for and is used for that pair only; for another folder the space starts
afresh as a first sync, which merges and deletes nothing on either side. Moving
a syncing space to another folder, or turning it off, now closes its session
and waits until the pass thread has let go of it before the state is removed or
the new folder is set up, and the thread reopens a session whose folder
changed. Signing out removes the team spaces' states as it removes the personal
one.

A space that was lost drops its state once its session is closed, so regained
access merges the folder with the space instead of resuming: a delete made here
meanwhile is not sent, a delete made on the server does not remove a local
file, and differing content becomes a conflict copy.

sync --approve-deletes[=SET] --space ID_OR_NAME and POST
/v1/spaces/{id}/approve-deletes write spaces/<id>/approval.json under the
personal space's rules; craft-cloud-app has approve_space_deletes.

A second stop no longer waits for the team passes: no queued team pass starts
and their thread is not joined. Pausing ends the team long-polls and resuming
starts them. One 404 for /spaces from a server that said protocol 2 is a
failed look, not the end of team spaces. The Debug output of SpaceRun leaves out
the words of no_access, and the log names a renamed space by its ID.
Conflicts kept from both sides. The storage figures module of the desktop
stack (`space.rs`) is renamed `storage_info.rs` because the team stack's
space layer owns that name.
The audit names of team spaces (space.* and the cli.space_* ones) have labels,
a Team spaces group in the filters, and a kind in the activity filters.
…t "Show in folder"

At the smallest width the table's last column cut the labelled Folder button
off. The icon fits, and the columns now fit the smallest window without
scrolling sideways.
The poll every two seconds took any screen it didn't list for a first sign-in
and showed the overview; the Files page was not listed.
GET /spaces gives an accepted member of a team space joined_ms, the time they
accepted their membership. It stays while they are a member (a role change does
not move it) and is new when they are removed and invited again, so a client
can tell that its access was interrupted. A personal space and an invitation
have none, and the field is optional for older servers and clients.
…access, and never block the spaces task

Turning a team space's sync off now forgets its saved state, so turning it on
again, in the same folder or another, is a first sync: nothing is deleted on
either side and differing files become conflict copies.

The agent records the server's joined_ms of the space in its state and does not
use a state made under another stamp (or without one while the server gives
one). That covers an agent that was not running while access was lost and
regained. A server without the field keeps the earlier behaviour, and a running
session whose stamp changes is reopened.

Closing a space's session no longer holds the spaces task for up to 50 seconds:
the change is answered when the session is closed, other spaces, the local API
and a second Ctrl-C go on meanwhile, a pass queued before the space was closed
does not run, and a second change of the same space meanwhile is refused with
409. Someone waiting for the list after an accept or decline is kept waiting
through a failed discovery, which is tried again after three seconds (fifteen
at most). A second stop is told apart from a pass thread that is gone.

The module doc and docs/agent.md say what a first sync does: files deleted on
the server meanwhile are uploaded again, files that differ become conflict
copies, and an archived space is listed with the member's stored role.

The hunks of these changes share spaces.rs, cli.rs and the test file, so they
are one commit. This commit also adds joined_ms to the agent's test helper that
builds a list item.
Only the import lists and the module order of the merge resolution.
…m space calls

craft-cloud-app gets the calls the Shared tab and the admin pages need (members: list, invite, change role, remove; usage; rename or limit; leave;
the administrator's list, create, archive, unarchive and owner), and the agent passes them on to the server through its local API as it does for
the admin calls. Leaving turns the space's sync off first, then leaves on the server; the folder and its files stay. An agent without a route
answers "no such call", which the client reports as agent_too_old.
…e admin's Team spaces part

The Shared tab (after Files) lists the account's team spaces with role, sync here, folder, state and what each stores; invitations are at the
top with Accept and Decline, and the header's tab and the overview say how many wait. Accepting offers to sync the space to this computer, with
the folder under Craft Cloud Shared and a choice of another; the sync stays off until the person says so, and a folder with files asks again.
A space's page has the folder and Open folder, the sync switch, banners for a read-only role, an archived space and lost access (the local copy
is kept), held deletes with Approve deletes (it lists the held set and the count, asks, and approves exactly that set), the members with roles
(an owner invites by account name, changes roles and removes, with a confirmation; every role can leave, with a confirmation) and Rename.
The admin pages gain a Team spaces part: the list with search and pages, a form for a new space (the owner is looked up before the
password is asked for), and a page per space with its limit, archive and owner, all with the admin confirmation.
Every team call that an agent from before them answers with agent_too_old ends in one message.
…the header with its new tab

Every screen with the header is redrawn (it has one more tab). New: shared-* and admin-spaces-*, light, dark and at the smallest width.
The server's joined_ms in the space list and the agent's 409 while a space is still being changed. Only the tests of both sides (appended at the same place) conflicted; both are kept.
…same click

The agent answers 409 while a space is still being closed after an earlier change. The offer or the page says so and offers Try again.
…keep a space off when the agent stops during its turn-off

Turning a team space on waited for the pass thread to close its session even
when the space had none. The thread works through its queue in order, so the
wait sat behind whatever it did for another space, a first sync of minutes or a
pass parked behind the personal one, and ran out after 50 seconds with a
message about a pass of a space that never ran. The wait now only happens for a
space that has a session here or a pass under way: nothing else can write its
state, and an older Close of the space is ahead of any new pass of it in the
queue. The message for a pass that has not ended in time no longer claims more
than that.

A stop while a space waited to be turned off dropped the change but not the
queued forgetting of its state, so the space synced again as a first sync at the
next start. The spaces task now saves the turn-off before it ends.
…r_too_old, a reason code for a paused space

- approve-deletes answers the waiting set as data (set, total, truncated, up to 1000 entries, deletes held behind a link) besides the command line's words; an app reads the fields and approves by the set's name.
- leave goes to the server first; only a successful leave lets the lost-access path end the sync, so a refusal (the last owner, a rate limit, no connection) changes nothing here.
- a route the server doesn't have (404 no such endpoint) is 409 server_too_old, not not_found.
- SpaceRun::Paused carries an optional reason_code (server_full, quota_exceeded, read_only).
- Try again belongs to the refusal it came with (space and action); any other click, navigation or error drops it.
- Leave is refused for the last owner before asking, and the page waits for the server.
- Held deletes come as data: the count, a scrollable list of up to 1000 paths (Showing N of M), deletes held behind a link counted; approval by the set's name, a changed set is asked again.
- The account's own row has no role picker and no Remove; rename is the owners' and not for an archived space, in the model too.
- A change of the agent's account resets the tab and drops answers in flight; usage is asked once per look on one thread; the server that needs an update and the paused-by-code states are said.
Turning a space on no longer waits for another space's pass, and a pending OFF is saved at stop. Only the tests appended at the end of team_spaces.rs conflicted; both are kept.
…ers, so the client sees the answer

A refusal decided before the body is read (an oversized chunk or manifest, a full server) was answered while the client was still
sending; the server then closed and the client, on Windows before it could read the answer, saw a reset connection. The agent's
transport showed it as 'server unavailable' instead of the server's 413. The rest of the body is now read and dropped first, up to the
biggest manifest and a megabyte, for at most ten seconds. bad_uploads_are_refused and oversized_manifests_are_refused failed in 5 of
20 runs of the server api package before, in none after; an agent test sends 200 oversized chunks, 8 at a time.
…e, since JSON may read them back one unit off

estimating_is_read_only_and_remembered compared the estimate read back from the server value with assert_eq. serde_json reads a float back one unit in the last place off at times (1.759115717544134 came back as 1.7591157175441339 in a full workspace run), so the test failed now and then. The floats are now compared within a relative tolerance of 1e-9 and everything else exactly.
…ession opened

A session compared the space's folder with its saved state only when it opened.
If the synced folder was a link or junction and was pointed at another folder
of the same space while the session was open, the next pass followed the link
with the first folder's state in memory, took the files the other folder lacked
for deleted and sent the deletions to the server.

The session now remembers the canonical path and the folder ID it opened on,
and the pass thread compares them before every pass. When they differ the
session is dropped, the saved state is removed (a failure to remove it fails the
pass), and the space starts afresh in the folder as a first sync, which merges
and deletes nothing on either side. A folder that cannot be looked at (gone,
drive unmounted) is not treated as moved; the pass reports what is wrong with it.
…emoved

Turning a team space off removes its saved state once the session is closed. The
pass thread logged a failure to remove it, such as a Windows reader holding the
file without delete sharing, and acknowledged the close anyway, so the turn-off
succeeded with the old state still on disk. Turning the space on again in the
same marked folder then trusted that state, and files the user had deleted
meanwhile were sent to the server as deletions.

Each team space's setting now has a generation, saved in account.json, and a
saved state records the generation it was made under; a state under another
generation is not used. A turn-off raises and saves the generation before the
state is retired, so a state that survives its removal is never trusted again,
and a failure to remove it is carried back to the caller: the change fails with
the error instead of reporting success. Losing access raises the generation too.
…stalled

Setting up a team folder that has no marker writes the marker and then removes
the space's saved state. If the agent stopped between the two, for a folder that
was wiped and made again at the same path, the old state survived next to a
valid marker; the account still pointed at the folder with syncing on, the
canonical path and membership matched, and at the next start the files the old
state knew were taken for deleted and sent to the server.

The set-up now raises and saves the space's generation before it writes the
marker, so a state made under the earlier generation is never trusted for the
folder, however the set-up ends. A test hook stops the set-up right after the
marker, as a crash would.
…e path

The folder identity a session compares with (canonical path and folder ID) lived
only in memory. A saved state recorded the path alone, so once the session was
gone, from a restart, a pass that stopped because the folder was missing, or a
failed cleanup of a moved folder, a folder restored from a backup at the same
path (with its marker but without the newer files) got the old state back and the
files it lacked were sent to the server as deletions.

state.json now records the folder ID next to the path, and a state is used only
for a folder with the same ID; a state without an ID is not trusted when the
folder has one. A session takes one look at the folder for both the state it
chooses and what it compares with later. When the pass thread finds that the
folder leads somewhere else than the session opened on, it tells the spaces
task, which raises and saves the space's generation, so a state that cannot be
removed is not trusted again either; that pass is not run and the next one
starts afresh.
…atever kind of entry it is

An end-to-end run reported `store clean-temp` leaving some two-day-old `.tmp-<UUIDv7>` files in place while it printed
"removed 0". The same layout could not be reproduced: planted by hand in a store directory next to a real chunk and
two younger temporary files, in odd shard names (`g1`, `g2`), in a root-owned or a craft-owned directory, among 600
other files, the sweep removed every old file (a new fault test plants exactly that, with the e2e's names, in five
directories; --ignored, needs the container). A root-owned directory is a loud permission error, not a silent zero.

What the sweep could not do before is show why a file is left. `remove_stale_temp` now returns a report: how many it
removed, how many of the backend's own names were younger than the limit, and how many other `.tmp-…` names were
not exactly ours. The command prints the removed count as before and, when there are any, the other two, so a file that
is still there can be told from one the command never matched. A leftover with our exact name and an old stamp is
removed when it is not a plain file as well (a link made by hand, or an entry the server describes without a file
type); a directory is only walked. Tested for a link in the container.
The temporary file of an upload is `.tmp-<uuid>`, as on disk, not `<name>.tmp-<id>`: corrected in the SFTP module's
comment and the guide. The guide now says plainly that a restore always writes `meta.db` and a local `blobs/` tree,
whatever the backed-up server stored its blobs in, so the restored server runs with `kind = "disk"`; to go back to S3
or SFTP, copy `blobs/` into the bucket or root with the same layout first (rclone copy or aws s3 sync as examples) and
only then change `[storage]`. `store clean-temp`'s output is described.
…ges off, and secrets-init makes the two-factor key once

The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists

The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could
write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made
exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code
is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
…nt's address

The admin API listened on loopback port 2019 in the Caddy container and was used only by the health check. It is off now (admin off), and the health check asks a small loopback site that answers 204. The error log filter also deletes remote_ip, remote_port and client_ip. The server guide says that the deployment is not IP-free: some messages of Go's HTTP library carry the address in their text. A test script checks the Caddyfile and the compose file.
…limit what a restore may use

A hostile backup server could replace the database dump and rewrite the hash beside it, and a restore would accept it. The first backup now makes a backup key (backup.key, mode 0600, never overwritten). The dump is sealed with AES-256-GCM, the manifest carries a keyed BLAKE3 MAC over every field, and the blobs are listed in a hashed inventory. A restore checks the MAC before it believes any field, fetches only the listed blobs, and publishes only a complete inventory. Generations from before are refused unless --allow-unauthenticated is given.

SFTP listings stop at 1,000,000 entries or 64 MiB of names per directory, at a repeated name, and at 16 levels of depth. A restore has operator limits (--max-db-bytes, --max-total-bytes, --max-objects, --min-free-bytes) that are enforced before each blob.
…them, redact the whole database URL

An existing data folder was used as it was: another user who could write to it (or to a folder above it without the sticky bit, or who had put a link there) could replace local-api.json and point apps at a listener of their own. On Unix the agent now refuses a data folder that is a link, belongs to someone else, or can be written by group or others, and apps do not read local-api.json, or send its token anywhere, unless the folder and the file pass the same checks. Windows keeps the %LOCALAPPDATA% rule; std has no owner check there.

DatabaseConfig's Debug output hid only the credentials before the last @; the query string (which can carry user and password) and fragment are now hidden too.

Cargo.lock: patch updates of rustls 0.23.46, rustls-webpki 0.103.16, webpki-root-certs 1.0.10, tokio 1.53.3 and ctutils 0.4.4 for RUSTSEC-2026-0346 to 0349, 0382, 0383, 0385 and 0386.
… whose password holds ? or #, keep the old error for a file at the data path on Windows

old_agent_blobs and recents wrote local-api.json with the umask's mode, which the client now refuses on Unix; they write it 0600. redact_url cut at the first ? or # before it looked for the @, so postgres://cc:pa#ss@db/cc showed the user and the start of the password; the credentials now end at the last @ of the authority, wherever a ? or # sits. DataDir::create treats a file or a dangling link at the data path as an error again on Windows.
None of them ran in CI. A new job runs every packaging/docker/tests/*.sh: the rendered config, the two-factor key files, the Caddyfile and compose checks and the version script. They need no Docker.
… the key as text, honest limits and docs

SFTP listings: 100,000 entries and 8 MiB of names per directory, depth 6 (the layout is three levels), and the levels of one walk or sweep together may hold about 48 MiB, so nested large directories no longer add up to gigabytes.

Restore: a generation from before authentication is restored only when --generation names it and --allow-unauthenticated is given; when no generation is named it is passed over, so a plain dump the backup server plants as newest is never taken. --max-total-bytes counts what is written to the local disk, and the legacy path no longer trusts the sizes the backup server reports.

The backup key file is 64 hexadecimal digits and a line break, readable and writable by hand (white space and capitals ignored); anything else is refused with a message. The rclone section names only targets the backup accepts (sftp://).
# Conflicts:
#	Cargo.lock
#	crates/app/src/lib.rs
…in-tests applies on Unix

The test module was gated by cfg(all(test, unix)); clippy only treats a plain cfg(test) as test code, so Linux and macOS
clippy rejected its unwrap() calls (the Windows build never compiles the module). Mechanical repair of the merge of
fix/local-security-hardening.
The browser admin gains a Storage page with sizes, capacity, the
filesystem and the largest accounts, and a form to set the compression
level. The form uses the session's CSRF token, asks for the password or
second factor again, and goes through the same operation as the JSON
route, so it writes the same audit entry.

The overview now shows server health and whether the server refuses
uploads because it is full; an account's page shows its usage. The
server and storage operations that both admin surfaces use moved into
shared functions in admin.rs. Fields the server does not measure per
account are labelled as such instead of being left out.
Cover who can open the storage page and its security headers, the
compression form's CSRF check, reconfirmation (password and second
factor), level validation and its single audit entry, and the health
and usage rows on the overview and account pages.
# Conflicts:
#	apps/craft-cloud-server/src/web/html.rs
The stylesheet takes the desktop kit's tokens (colours for light and
dark, type scale, spacing) and its components: the header bar with the
inverted current navigation item, joined stat cells, tables with mono
headers, status badges, form panels, notices and focus rings. Tables
scroll inside their own frame, the header and navigation wrap, and
definition lists run in one column at phone width.

New helpers emit stat cells and status badges with escaped text. The
overview, storage page, account page and team-space pages use them;
behaviour is unchanged. The server health section is a second-level
heading under the overview's.
A small update-check service in the Docker stack asks GitHub every 12
hours for the newest release (or the newest commit of a branch) and
writes a tiny status file into a volume of its own. The server mounts
that volume read-only and never contacts the network itself; it stays
on its internal network. The status file is treated as untrusted: it is
size-capped, strictly parsed, and only a validated version or commit
hash is ever shown. Links are built from the configured repository.

When a newer version exists, the overview shows a banner with a link to
the release notes or the comparison and the command to run on the host,
and GET /admin/server reports it in a new update field (null when off).
Branch checks compare against the revision now baked into the binary at
build time; an unknown revision shows nothing.

packaging/docker/update.sh refuses a dirty checkout, asks first, pulls
fast-forward only and builds while the old server keeps running. Only
then does it stop the server, back up the data volume (and dump
Postgres), and start the new server; on failure it keeps the backup and
says how to go back. The server itself can never update itself.

CC_UPDATE_CHECK=off turns the check off. Shell tests cover the response
parsing, the refusal of a dirty tree and the rendered config; web tests
cover the banner for newer, equal, malformed, missing and disabled.
…sbad#74, itrycodinganditsbad#75) into integration/all

Conflicts: the navigation (integration/all already had the current-page
marking from itrycodinganditsbad#43; the new branch adds Storage), one navigation test
message, and the backup paragraph in docs/server.md (kept the SFTP
backup sentence from integration/all after the new update section).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant