Skip to content

Files page: who has a recent file open (after P1-8 and P2-4) - #72

Open
Skyfuller1303 wants to merge 500 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:team-p2-5-desktop
Open

Skyfuller1303 wants to merge 500 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:team-p2-5-desktop

Conversation

@Skyfuller1303

Copy link
Copy Markdown

Merge after #71 (P1-8) and #67 (P2-4). The branch is P1-8's head with P2-4 and the Docker fix merged in. The diff of this PR alone is the commits that are in neither parent. GitHub compares one branch at a time: team-8-from-folder...team-p2-5-desktop and team-p2-4-kit...team-p2-5-desktop each also list the other parent's commits, which disappear once that parent is merged.

The Files page now shows the recent files of the team spaces that sync on this computer, and on each recent file who else has it open right now.

What

  • The agent says who has a file open. A recent entry may carry open_now: [{name, colour}]: the username and palette index of the other accounts that have the file open. It is filled in when a list is handed out, from the presence the hub already hears. The field is additive: absent when nobody else is there and from an older agent, ignored by a reader that does not know it. It never lists the account the agent is signed in as, and leaves out an account the member list does not name. An account fades ten seconds after its last presence (the kit's own time to live): the hub keeps when the next counted account may fade and hands every followed list out again then, so a follower (the kit, the egui lists, the desktop) is told that somebody left without a new presence frame. A list is also handed out again when somebody new appears or the member list changes, and to every new subscription; a follower is only woken for a list that differs. Nothing is sent to the server for any of this and nothing is polled. Debug output of an entry names nobody.
  • The shared list never holds it. open_now is not written to a recents document and is dropped from what another machine wrote (a hostile device, a bug), so it is never handed out from the shared list.
  • What it cannot know, and what that means for the wording. Presence only reaches the agent for a document it follows, which is a file an app on this computer has open: the server relays presence only to connections that subscribed to the document, and a subscription downloads the document. So this shows people on a file you also have open here. Showing them on every team recent needs a presence-only watch frame in the live protocol (a server and protocol change); I left that for a follow-up. The role is not in presence or in members either, so viewing cannot be told from editing: the column and the tooltip say "open now", not "editing".
  • One connection for all the lists: Client::recents_of. The desktop used to need one connection per app and space (a handshake each). recents_of opens one recents socket, sends every subscribe {app, space} at once, collects the lists by app and space until all have come or three seconds have passed, and closes. No protocol change; an older agent still works. The first refresh after a team space starts to sync makes the agent follow 12 recents documents (one per app of the suite) for that space; later refreshes read them. docs/agent.md says so. A list whose space is not the one asked for (an agent from before team spaces answers the personal list to every subscribe) is refused as agent_too_old, never taken for the space's list. I used that code and not Error::Version, which carries the local API version and which the desktop treats as an agent that cannot be reached at all.
  • The desktop asks for the personal lists and those of every synced team space with that one call, places each file against its own space's folder, and sorts them together. While the Files page shows the recent files and nothing loads, it asks again every 15 seconds (only the recents, no spinner, the list stays; a failed quiet look keeps it), so people who left drop out and people who came show up. The table gets a Space column (90 px; "Personal" or the space's name) and an Open now column (two people, then "+N"), each only when some file has something to show, so the page looks as before otherwise. The search also matches a space's name. Open and Show on a team file use that space's folder; the events carry the space so a personal file with the same path is not mistaken for it. A space that does not sync here, an invitation and an agent from before team spaces add nothing.
  • The people are drawn by the desktop itself (ui/people.rs): initials in the account's palette colour for the theme (the palette comes from craft-cloud-proto), the names in the tooltip ("Open now: ...") and as accessible labels. It does not use craft-cloud-egui's people badge, because the desktop depends on app and proto only.
  • Merge fix. The desktop line added admin.compression_changed to the audit names and the kit line added cli.live_reset, so the merged list has 46 names; the array lengths and their asserts are updated (own commit), and the desktop words the new name ("Live document emptied on the server").
  • docs/agent.md describes the field and the call; AGENTS.md mentions the new file.

Tests

  • Hub: open_now.rs unit tests for join, leave, the account itself hidden, a name the member list lacks, bounds, the next expiry and the sweep, and no name in a debug print. Recents document: the shared list never carries or hands out open_now (own add and a hostile entry). API: an older agent's entry parses without the field and leaves it out when empty.
  • App: lists are put at their app and space whatever the order, an agent from before team spaces gets agent_too_old for a space and keeps the personal list, and an error waits until it is known which request it was.
  • Desktop model: the people of a file follow each answer (nobody, one joins, five, all leave); the page asks again on its timer, not while loading, not on the conflict copies, not after leaving, and a failed quiet look keeps the list; the search finds a team space's files by its name; a team file is opened and shown against its space's folder. Fake agent: the synced team spaces' lists come with who has them open and with the space's folder, in one call; a space that is off, an invitation and an agent that is too old add nothing.
  • Agent against a real server (team_spaces.rs): two accounts with the file open: ana's list names ben and ben's names ana, never themselves, with the colour the palette derives; a follower of the list is told that ben has faded with no new presence frame; several lists (personal, a space, an app with no files, a space that does not sync here) come over one connection, each at its place.
  • kittest snapshots (new only): files-team-recents, files-open-now, files-open-now-dark, files-open-now-overflow (five people: two, then +3), files-open-now-zoom-160. A test reads the names through the accessibility tree and checks that nothing extra shows without a team file or a person. The desktop tests also ran once on a single thread and twice in parallel: all passed (the STATUS_HEAP_CORRUPTION I saw once while rewriting snapshots did not come back).

Checks

On Windows, through a memory-limited runner (peak 2.5 GiB):

  • cargo fmt --all -- --check, cargo xtask licenses, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo +1.95 check --workspace --all-targets --locked: all passed.
  • cargo test --workspace --locked --no-fail-fast against Postgres 17 and SeaweedFS 4.48 with CRAFT_CLOUD_REQUIRE_BACKENDS=1 at f14e877: 2497 passed, 0 failed. The desktop lib tests (431) also passed three times in a row.

Latest check of this head

fmt, licenses, clippy and cargo +1.95 check pass at f14e877; the full workspace test passed with 2497 tests and no failure (see Checks).

Known issues for reviewers

  • Windows CI timeout on very large trees. On the fork's integration branch the Windows job hit the 45-minute step limit once, while compiling a tree with all open PRs in it, and passed on a rerun. No test failed. PR branches have small diffs and use the repository's normal CI settings.
  • fe80 flake. meta behaviour on_sqlite::no_event_holds_a_name_or_a_secret scans event text for the string fe80, which a random device id can contain. It passes alone and on a rerun; the test exists on main too.
  • Other timing flakes, each green alone: cli::healthcheck_gives_up_on_an_answer_that_never_comes (under load), live_autosave::a_save_made_while_the_agent_was_away_..., and oversized_manifests_are_refused (one Windows connection abort on the Postgres variant). A rare simulation hang, a_live_seed_replays_exactly (about 1 in 110 runs on a loaded 2-core Linux machine), now aborts after 180 s with a message instead of hanging.
  • Desktop tests on Windows: parallel wgpu rendering corrupts the heap now and then; Desktop tests: share one wgpu device (fixes a crash in parallel runs on Windows) #35 serialises it, and this branch contains that change.
  • Migration numbers differ along the stacks (compression is 0007 in its own PR and 0015 in the stacked branches; team spaces take 0015 on the server line and 0016 on the desktop line); whichever PR lands second renumbers.
  • Found by an end-to-end run and not fixed by these PRs: the --name flag of join and login is described inconsistently in docs/agent.md; GET /api/info reports protocol_max: 2 although team live connections use protocol 3; a lost two-factor setup secret leaves a 409 two_factor_pending until the pending setup is cleared.

…, a footer with the step and a sidebar of the steps
The desktop app asks the agent for each suite app's shared recents when
the Files tab opens or is refreshed, merges them newest first, and lists
them with app, computer and time. Open hands a file that exists in the
synced folder on this computer to the OS's default program; the path is
joined and checked against the folder first, and programs and scripts are
never started. The overview links to the page.
…ccount pages in the new layout

The admin landing page gets Accounts, Storage and Audit log tabs. The Storage tab shows the server's room as one number and a
bar, with a row for the server's disk and one for the store, built from the optional capacity fields of the storage overview.
Every number an older server, Postgres or S3 leaves out reads as a dash or a one-line hint. The warning starts at 90 percent,
the line the server uses for an account near its quota. The accounts are a table that scrolls sideways in its own area, and
the account, new account and shown-code pages use the same panels.
…r capacity, and review fixes

A disk store's all storage line and bar now use the store's filesystem numbers, because the accounts' stored total counts per
account and can pass the disk's size. A capacity set on a disk is a second measure with its own warning, and the tags say which
one is near (disk 92 %, capacity 95 %). A store with only a capacity (S3) shows the accounts' total against it, said to be an
upper bound. The bars carry a label with their share, a disk store without numbers stays a disk, the hint about S3 is only shown
for S3, an account shows every state it is in, and the server's own counters stay when the storage call fails.
… names its account and says whether it is destructive
…cells, segmented switcher with the audit log inside, panels for new account, invite, confirmation and account detail, storage table with flexible bars and over-capacity)
…ute table has 13 paths; the PUT's totals are not kept for the overview
…udit and concurrency tests

- accept, decline and leave answer NotFound("space") for an account with no row in the space, the answer for a space that
  does not exist; NotFound("invitation") and NotFound("member") remain for the wrong state of the caller's own row
- the command line's events are chosen by an exhaustive list (created, invited, role changed, removed); any other change
  it is asked to make is refused instead of being named cli.space_member_changed
- space.created and cli.space_created always carry quota_bytes, as null when the space has no limit of its own
- document that a SpaceActor::User without a device is for tests and the command line, and that no space route takes a
  browser session
- the last-owner concurrency test starts both changes at a barrier and runs more rounds of the cases that depend on the
  space's row lock; removing FOR UPDATE on Postgres now fails it (SQLite is ordered by its single writer by design)
- drop the foreign_keys pragma check on a fresh pool, which proved nothing
…ze, and measure goes on past a failing blob

store recompress records its result only if the row still holds the size the
pass read (Meta::set_physical_if), so an upload that replaced the object in
between keeps its own size and level. store measure counts and lists blobs
it can't measure like recompress does, finishes the pass and exits with an
error.
…w their content down and are kept per account
…evel outside 1 to 19 is ignored at start

Concurrent PUTs could leave the store at another level than the database
holds, because the store was set after the commit, outside the transaction.
A mutex now covers both. At start, a stored value outside 1 to 19 is ignored
with a warning, like the overview does, instead of being clamped.
… read_only

A request names its space with the craft-space header (absent: the device's personal space, as before). The new
require_space layer reads the account's role on every request, refuses a role below the route's minimum with 403
read_only, answers a space without access like one that does not exist (404), and echoes the space on every answer. The
roles per route are listed in one place in api.rs. Every space-scoped handler takes its space from SpaceAccess.

Writes re-check the role inside their write transaction (Meta::guarded), a change-feed long-poll re-reads its access when
the space's members change, and the live upgrade refuses team spaces until live protocol 3. PROTOCOL_MAX is 2.
… measure test fits this branch's Backend trait
…p the epoch off the layer's refusals

On Postgres a guarded upload took FOR SHARE on the space's row and then updated the same row to charge the bytes, so two
concurrent uploads to one team space deadlocked (40P01, answered as a 500). The guard now takes FOR NO KEY UPDATE, the
strength the write needs anyway; a test starts two guarded uploads together, many times, and fails with the old lock.

The space layer's own 404 and 403 no longer carry the personal space's craft-epoch, which said nothing true about the
requested space. The malformed-header answer without an echo is documented.
GET /spaces lists an account's spaces with its invitations (an archived space for its owners only). Invitations are
accepted or declined; an owner invites (30 a minute per account), changes roles and removes members, members leave; the
last owner stays (409 last_owner). Members and usage are readable by members. PATCH /spaces/{id} renames (an owner) and
sets the limit (an administrator); owners archive and unarchive with their password.

For administrators: POST and GET /admin/spaces, archive and unarchive, and PUT .../owner for a space whose owners are gone.
An administrator who is not an owner carries the usual confirmation on the member routes and has no access to the files.
Every change calls Notifier::members_changed so waiting long-polls re-read their access, and so do disabling and removing
an account. GET /space gains kind, name and role. GET /account/activity also shows events that name the account.

The command line gains space create, list, members, add and archive (cli.space_archived is a new audit name), and
reset-epoch --space. The meta store gains set_owner_accepted, rename, archive and limit changes, user_spaces and the
administrator's list.
…he review findings of the space routes

A member whose role is too low inside the change's transaction is read_only, not forbidden. An invitation to an archived space can be neither accepted nor declined. One PATCH checks its name and limit before it writes. The invitation limit's Retry-After rounds up, a space name shaped like a space ID is refused, the command line accepts a pending invitation when it sets a role, a confirming administrator who owns a space acts as an administrator, and the administrator's search is folded text instead of a validated name. Disabling or removing an account over the admin pages wakes its spaces' long-polls.
A viewer's engine applies the server's changes and sends nothing. A local edit is kept and listed, never overwritten (a changed server version lands under the original name and the edit stays as a conflict copy); a new local file stays local; a file or folder deleted here that the server still has is downloaded again and reported. The first read_only refusal switches the engine over within the same pass, with no retry; while read-only it asks for the role once per pass and resumes uploads when the role reaches editor. The agent's HTTP transport maps 403 read_only, can be bound to a team space with the craft-space header, and the pass summary says when a space is read-only.
The paused run state in the local API's status carries an optional
`reason_code`: `server_full` where the server refuses uploads because it is
full, `quota_exceeded` where the account is over its quota, and nothing for
any other pause. The loop state in status.json keeps it too.

An app can match the code instead of the words of `reason`, which are free
to change. The field is serde-optional, so an older desktop ignores it and
an older agent sends none.
AdminServerInfo gains optional started_ms, database_ok, store_ok with store_check_ms (a cached,
2 s probe that asks the store for the size of an object that never exists, writing nothing) and
refusing_uploads. A failing store is store_ok false, never an error of the call. ApiError is kept
small by boxing the missing chunk list.
…probe

The size query says no such object for every key when the directory is missing, so the probe also
checks that the directory is there. The overview's refusing_uploads is documented as true when even a
smallest upload would not fit.
…ed pushes as they happened

Review fixes of the read-only mode.

A tracked folder whose place holds a file or a link (or whose parent does) is
the user's: nothing that was in it is put back through it, and nothing is
moved aside to make room. A folder put back is reported once, as restored.

A read-only pass no longer reads every edited file each time: a different size
says "edited" without reading, and the answer for the same size is kept per
file while its size, time and ID and the agreed content stay the same.

A refusal in a later batch keeps what the committed batches did in the
report; content sent for files no commit took is not reported as uploaded.
The two limits that stay (a delete held after the role rises, a swapped
tracked file not listed) are written down in the code.
…ges off, and secrets-init makes the two-factor key once

The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists

The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could
write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made
exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code
is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
…nd a lost creation answer keeps the journal

A failed journal write was only logged, so the server space was made and the copy and deletion went on with no record to
recover from. The write before the space is made now has to succeed (otherwise nothing is made and the call fails with a clear
message), and so does every checkpoint of the job: the job fails before the next step, and nothing is deleted without its
record.

When the request that makes the space went out and no answer came, the server may have made it. The journal is kept at
Creating (with the server time it holds), the status says it is not known, and the next try, or the next start, looks for the
space first and adopts it instead of failing on the taken name.
… join

An open that waited in the hub's lookups (for its file's node) or in its waiting list (for its document's join) got no answer
when the hub stopped: only documents already open were told. The personal hub's docs socket goes on after the hub stops, so
nothing else ever answered, and the app library waited for its own 30 second timeout. The stop now answers each of them as
not live (the file opens as a plain file) and forgets them.
Follows the port of the teardown fix: writes to the app now report whether the app is gone or too slow, so the
refusal of an over-size held change goes through the same path as the other refusals.
…es such names

APFS answers EILSEQ when a file is created with a name that is not valid UTF-8, so the test panicked on macOS. The refusal
(InvalidFilename, or EILSEQ) is detected when the file is created and the test then has nothing to check; any other
error still fails it.
…s busy for minutes

The simulation holds simulated time still while SQLite's own thread works, with a busy wait on the real clock. If the
database never goes idle, that wait never ends and a CI job runs into its timeout without a word (a_live_seed_replays_exactly
did, once, on Ubuntu). The wait now aborts the test binary after 180 s of real time with a message saying the simulation is
stuck rather than slow.
…ges off, and secrets-init makes the two-factor key once

The template had neither [two_factor] nor [web_admin], and only 'craft-cloud-server init' made a key, so a Docker install could not turn two-factor sign-in on without manual edits. secrets-init now makes 32 random bytes in the data volume (mode 0600, owner 65532) when there is no key, and never replaces one. docs/server.md describes it.
…that exists

The key was written through one fixed temporary name that every run removed first. A run that was paused after the check could
write into the file another run had already hard-linked to the key's name. Each run now writes its own temporary file, made
exclusively with mktemp beside the key, and links it to the key's name; a failed link removes only that run's file. The code
is make-key.sh, sourced by secrets-init.sh, with a shell test that runs many overlapping runs and checks the key never changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant