Skip to content

Web admin: storage page, compression level, server health and account usage (after #72) - #73

Open
Skyfuller1303 wants to merge 502 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:web-admin-parity
Open

Skyfuller1303 wants to merge 502 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:web-admin-parity

Conversation

@Skyfuller1303

Copy link
Copy Markdown

Merge after #72 (P2-5). The branch is #72's head plus two commits; team-p2-5-desktop...web-admin-parity shows only them.

The browser admin catches up with what the launcher's admin already shows, so an administrator without the desktop app sees the same server state.

What

  • Storage page (/web/storage), a new navigation entry: stored bytes, configured capacity and how much of it is used, the filesystem of the store and of the data directory, the manifest index, and the ten largest accounts. The storage summary on the overview moved here, and the overview links to it.
  • Compression level form on that page: the level (1 to 19) goes through the same operation as PUT /admin/storage/compression, so it writes the same audit entry. Like every other admin change in the browser, it needs the session's CSRF token and the password again (or the second factor when the session signed in with one).
  • Overview: server health (database, store, refusing uploads because the server is full), shown as "Healthy" only when nothing reports a fault.
  • Account page: the account's usage rows (stored, effective quota, trash and history estimates). The server does not measure physical size and file count per account today; the page says "not measured" instead of leaving them out.
  • The server and storage operations both admin surfaces use moved into shared functions in admin.rs.

No JavaScript and no change to the CSP (script-src 'none'; style-src 'self').

Tests

  • New integration tests in tests/web.rs (SQLite and Postgres): who may open /web/storage, and its security headers; the compression form refuses a missing or wrong CSRF token, a missing or wrong password, and levels 0 and 20, changing nothing and writing no audit entry then, and writes exactly one entry for a valid change; reconfirmation with the second factor; the health and usage rows. The escaping test also fetches the storage page.
  • Local, Windows: cargo fmt --check, cargo clippy -p craft-cloud-server --all-targets -- -D warnings, cargo test -p craft-cloud-server (623 passed). The web and API test files also ran against Postgres 17 at the top of this stack (320 passed).

The desktop app asks the agent for each suite app's shared recents when
the Files tab opens or is refreshed, merges them newest first, and lists
them with app, computer and time. Open hands a file that exists in the
synced folder on this computer to the OS's default program; the path is
joined and checked against the folder first, and programs and scripts are
never started. The overview links to the page.
…ccount pages in the new layout

The admin landing page gets Accounts, Storage and Audit log tabs. The Storage tab shows the server's room as one number and a
bar, with a row for the server's disk and one for the store, built from the optional capacity fields of the storage overview.
Every number an older server, Postgres or S3 leaves out reads as a dash or a one-line hint. The warning starts at 90 percent,
the line the server uses for an account near its quota. The accounts are a table that scrolls sideways in its own area, and
the account, new account and shown-code pages use the same panels.
…r capacity, and review fixes

A disk store's all storage line and bar now use the store's filesystem numbers, because the accounts' stored total counts per
account and can pass the disk's size. A capacity set on a disk is a second measure with its own warning, and the tags say which
one is near (disk 92 %, capacity 95 %). A store with only a capacity (S3) shows the accounts' total against it, said to be an
upper bound. The bars carry a label with their share, a disk store without numbers stays a disk, the hint about S3 is only shown
for S3, an account shows every state it is in, and the server's own counters stay when the storage call fails.
… names its account and says whether it is destructive
…cells, segmented switcher with the audit log inside, panels for new account, invite, confirmation and account detail, storage table with flexible bars and over-capacity)
…ute table has 13 paths; the PUT's totals are not kept for the overview
…udit and concurrency tests

- accept, decline and leave answer NotFound("space") for an account with no row in the space, the answer for a space that
  does not exist; NotFound("invitation") and NotFound("member") remain for the wrong state of the caller's own row
- the command line's events are chosen by an exhaustive list (created, invited, role changed, removed); any other change
  it is asked to make is refused instead of being named cli.space_member_changed
- space.created and cli.space_created always carry quota_bytes, as null when the space has no limit of its own
- document that a SpaceActor::User without a device is for tests and the command line, and that no space route takes a
  browser session
- the last-owner concurrency test starts both changes at a barrier and runs more rounds of the cases that depend on the
  space's row lock; removing FOR UPDATE on Postgres now fails it (SQLite is ordered by its single writer by design)
- drop the foreign_keys pragma check on a fresh pool, which proved nothing
…ze, and measure goes on past a failing blob

store recompress records its result only if the row still holds the size the
pass read (Meta::set_physical_if), so an upload that replaced the object in
between keeps its own size and level. store measure counts and lists blobs
it can't measure like recompress does, finishes the pass and exits with an
error.
…w their content down and are kept per account
…evel outside 1 to 19 is ignored at start

Concurrent PUTs could leave the store at another level than the database
holds, because the store was set after the commit, outside the transaction.
A mutex now covers both. At start, a stored value outside 1 to 19 is ignored
with a warning, like the overview does, instead of being clamped.
… read_only

A request names its space with the craft-space header (absent: the device's personal space, as before). The new
require_space layer reads the account's role on every request, refuses a role below the route's minimum with 403
read_only, answers a space without access like one that does not exist (404), and echoes the space on every answer. The
roles per route are listed in one place in api.rs. Every space-scoped handler takes its space from SpaceAccess.

Writes re-check the role inside their write transaction (Meta::guarded), a change-feed long-poll re-reads its access when
the space's members change, and the live upgrade refuses team spaces until live protocol 3. PROTOCOL_MAX is 2.
… measure test fits this branch's Backend trait
…p the epoch off the layer's refusals

On Postgres a guarded upload took FOR SHARE on the space's row and then updated the same row to charge the bytes, so two
concurrent uploads to one team space deadlocked (40P01, answered as a 500). The guard now takes FOR NO KEY UPDATE, the
strength the write needs anyway; a test starts two guarded uploads together, many times, and fails with the old lock.

The space layer's own 404 and 403 no longer carry the personal space's craft-epoch, which said nothing true about the
requested space. The malformed-header answer without an echo is documented.
GET /spaces lists an account's spaces with its invitations (an archived space for its owners only). Invitations are
accepted or declined; an owner invites (30 a minute per account), changes roles and removes members, members leave; the
last owner stays (409 last_owner). Members and usage are readable by members. PATCH /spaces/{id} renames (an owner) and
sets the limit (an administrator); owners archive and unarchive with their password.

For administrators: POST and GET /admin/spaces, archive and unarchive, and PUT .../owner for a space whose owners are gone.
An administrator who is not an owner carries the usual confirmation on the member routes and has no access to the files.
Every change calls Notifier::members_changed so waiting long-polls re-read their access, and so do disabling and removing
an account. GET /space gains kind, name and role. GET /account/activity also shows events that name the account.

The command line gains space create, list, members, add and archive (cli.space_archived is a new audit name), and
reset-epoch --space. The meta store gains set_owner_accepted, rename, archive and limit changes, user_spaces and the
administrator's list.
…he review findings of the space routes

A member whose role is too low inside the change's transaction is read_only, not forbidden. An invitation to an archived space can be neither accepted nor declined. One PATCH checks its name and limit before it writes. The invitation limit's Retry-After rounds up, a space name shaped like a space ID is refused, the command line accepts a pending invitation when it sets a role, a confirming administrator who owns a space acts as an administrator, and the administrator's search is folded text instead of a validated name. Disabling or removing an account over the admin pages wakes its spaces' long-polls.
A viewer's engine applies the server's changes and sends nothing. A local edit is kept and listed, never overwritten (a changed server version lands under the original name and the edit stays as a conflict copy); a new local file stays local; a file or folder deleted here that the server still has is downloaded again and reported. The first read_only refusal switches the engine over within the same pass, with no retry; while read-only it asks for the role once per pass and resumes uploads when the role reaches editor. The agent's HTTP transport maps 403 read_only, can be bound to a team space with the craft-space header, and the pass summary says when a space is read-only.
The paused run state in the local API's status carries an optional
`reason_code`: `server_full` where the server refuses uploads because it is
full, `quota_exceeded` where the account is over its quota, and nothing for
any other pause. The loop state in status.json keeps it too.

An app can match the code instead of the words of `reason`, which are free
to change. The field is serde-optional, so an older desktop ignores it and
an older agent sends none.
AdminServerInfo gains optional started_ms, database_ok, store_ok with store_check_ms (a cached,
2 s probe that asks the store for the size of an object that never exists, writing nothing) and
refusing_uploads. A failing store is store_ok false, never an error of the call. ApiError is kept
small by boxing the missing chunk list.
…probe

The size query says no such object for every key when the directory is missing, so the probe also
checks that the directory is there. The overview's refusing_uploads is documented as true when even a
smallest upload would not fit.
…ed pushes as they happened

Review fixes of the read-only mode.

A tracked folder whose place holds a file or a link (or whose parent does) is
the user's: nothing that was in it is put back through it, and nothing is
moved aside to make room. A folder put back is reported once, as restored.

A read-only pass no longer reads every edited file each time: a different size
says "edited" without reading, and the answer for the same size is kept per
file while its size, time and ID and the agreed content stay the same.

A refusal in a later batch keeps what the committed batches did in the
report; content sent for files no commit took is not reported as uploaded.
The two limits that stay (a delete held after the role rises, a swapped
tracked file not listed) are written down in the code.
…pened, show it in its folder

Open used a deny-list of program extensions, which missed whatever the OS
runs that was not on it. It is now an allow-list: the Craft suite's own
formats, office documents without macros, PDF, text, images, audio, video,
fonts, 3D and CAD interchange, and archives an opener only lists. A name
without an extension, one ending in a bundle suffix (.app, .pkg, .mpkg,
.prefPane, .workflow, .saver, .bundle, .framework, .plugin) in any path
part, and on Unix any file with an exec bit are refused. Such a file shows
"Open it from its folder" on the disabled Open, and a new Folder button
shows the file in its folder, for any file that is here.

Open also looks at the disk again when it is clicked, off the UI thread,
requires the file to be here still and opens the path that was checked, so a
file swapped for a link after the list was placed is not followed.
…ges off, and secrets-init makes the two-factor key once

The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists

The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could
write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made
exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code
is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
…nd a lost creation answer keeps the journal

A failed journal write was only logged, so the server space was made and the copy and deletion went on with no record to
recover from. The write before the space is made now has to succeed (otherwise nothing is made and the call fails with a clear
message), and so does every checkpoint of the job: the job fails before the next step, and nothing is deleted without its
record.

When the request that makes the space went out and no answer came, the server may have made it. The journal is kept at
Creating (with the server time it holds), the status says it is not known, and the next try, or the next start, looks for the
space first and adopts it instead of failing on the taken name.
… join

An open that waited in the hub's lookups (for its file's node) or in its waiting list (for its document's join) got no answer
when the hub stopped: only documents already open were told. The personal hub's docs socket goes on after the hub stops, so
nothing else ever answered, and the app library waited for its own 30 second timeout. The stop now answers each of them as
not live (the file opens as a plain file) and forgets them.
Follows the port of the teardown fix: writes to the app now report whether the app is gone or too slow, so the
refusal of an over-size held change goes through the same path as the other refusals.
…es such names

APFS answers EILSEQ when a file is created with a name that is not valid UTF-8, so the test panicked on macOS. The refusal
(InvalidFilename, or EILSEQ) is detected when the file is created and the test then has nothing to check; any other
error still fails it.
…s busy for minutes

The simulation holds simulated time still while SQLite's own thread works, with a busy wait on the real clock. If the
database never goes idle, that wait never ends and a CI job runs into its timeout without a word (a_live_seed_replays_exactly
did, once, on Ubuntu). The wait now aborts the test binary after 180 s of real time with a message saying the simulation is
stuck rather than slow.
…ges off, and secrets-init makes the two-factor key once

The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists

The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could
write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made
exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code
is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
The browser admin gains a Storage page with sizes, capacity, the
filesystem and the largest accounts, and a form to set the compression
level. The form uses the session's CSRF token, asks for the password or
second factor again, and goes through the same operation as the JSON
route, so it writes the same audit entry.

The overview now shows server health and whether the server refuses
uploads because it is full; an account's page shows its usage. The
server and storage operations that both admin surfaces use moved into
shared functions in admin.rs. Fields the server does not measure per
account are labelled as such instead of being left out.
Cover who can open the storage page and its security headers, the
compression form's CSRF check, reconfirmation (password and second
factor), level validation and its single audit entry, and the health
and usage rows on the overview and account pages.
Skyfuller1303 added a commit to Skyfuller1303/craft-cloud that referenced this pull request Nov 11, 2026
…sbad#74, itrycodinganditsbad#75) into integration/all

Conflicts: the navigation (integration/all already had the current-page
marking from itrycodinganditsbad#43; the new branch adds Storage), one navigation test
message, and the backup paragraph in docs/server.md (kept the SFTP
backup sentence from integration/all after the new update section).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant