Skip to content

Trash: list, restore and mark deleted files and folders - #24

Open
Skyfuller1303 wants to merge 19 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:trash
Open

Skyfuller1303 wants to merge 19 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:trash

Conversation

@Skyfuller1303

@Skyfuller1303 Skyfuller1303 commented Nov 4, 2026 •

Copy link
Copy Markdown

Stacked on #21 (version history and restore): this branch contains #21's commit, so please review #21 first; the diff of this PR alone is ec83bd7..trash. The history/retention design (CP-06) isn't agreed yet. The trash only lists, restores and marks; it frees nothing. The assumptions are listed below.

What

  • GET /api/v1/trash?cursor&limit: the deleted nodes of your space that were not removed from the trash, newest deletion first by (deleted_ms, id), paged (default 50, max 200). A node whose folder is itself in the trash is not listed: the folder stands for it. Each item has id, parent, name, is_dir, size, deleted_ms and purge_after_ms (deleted_ms plus [retention] trash_days).
  • POST /api/v1/trash/{id}/restore: brings a node back with its deleted folders, root-most first, keeping their IDs, in one transaction under the space's write lock and with one sequence number.
    • Where a deleted folder's name is held by a live folder, that folder is used and the rest moves into it.
    • Where a name is held by a file, or the node's own name is taken, the node comes back as name (conflict, <device>, <UTC date and minute>) like the engine's conflict copies, with , 2 to , 100. No free name gives 409 conflict and changes nothing.
    • A restored file gets a new version: a copy of its last one (same manifest, mtime and exec bit, no save record).
    • A restored folder brings back everything in the trash below it, whenever it was deleted, so a file deleted on its own inside it earlier comes back too. A node removed from the trash, and what hangs only below it, stays deleted. The nodes below keep their names; of two deleted nodes of one name in one folder, the one deleted last keeps it and the other gets a conflict name.
    • One call takes at most 10,000 nodes below the folder, shallowest first, so a node never comes back without its folder. The rest stays in the trash and is then listed on its own, so a further restore brings it back.
  • DELETE /api/v1/trash/{id}: only marks the node (purge_requested_ms, 204, idempotent). Versions, manifests, chunks and the tombstone stay; the change feed and the sequence number don't change. For a folder it marks what is in the trash below it too, with the same 10,000-per-call limit; the rest is then listed on its own.
  • [retention] trash_days = 30 (1 to 3650). It is shown to clients and removes nothing yet.
  • One migration (nominally 0009; numbered when merged): nodes.deleted_ms, nodes.purge_requested_ms, and a partial index for the trash list. Nodes deleted before the migration are dated at the migration.

Sync engine

A restore must not be undone by a delete that was in flight:

  • Files: a machine that deleted a file but never heard the answer sends the delete again with the old version. Because the restore makes a new version, the server refuses that delete as a version mismatch, and the engine downloads the restored file.
  • Folders have no version. The engine now remembers folder deletes it sent without hearing the answer (State::sent_dir_deletes). When the next pull or a relist shows such a folder live, it lets it go instead of deleting it again. Nested folders are settled deepest first, and a parent is judged only after its children.
  • An older agent lacks the folder rule. In the short window after a lost delete answer, it can delete a restored folder a second time (the folder is empty and can be restored again). docs/server.md says to update agents with or soon after the server.

Assumptions (please confirm or redirect)

  1. The trash is the set of tombstones plus a deletion time; there is no separate trash table.
  2. deleted_ms is set when a node goes from live to deleted (not on a second delete) and cleared when it comes back by any means.
  3. Any revival also clears the purge mark.
  4. The trash list is read from a snapshot, not under the write lock.
  5. A deleted node without a time (damaged data) is dated at the next read, logged without names, and never counts as expired.
  6. Restore merges deleted ancestors into live folders of the same name, but never merges the restored node itself into an existing one.
  7. Conflict names reuse the engine's format, with at most 100 tries.
  8. A restored file gets a new version rather than reviving the old head.
  9. Restoring a version of a deleted file (Version history and restoring an old version #21's route) stays refused; bring the file back from the trash first.
  10. DELETE /trash/{id} is a mark only, until GC exists.
  11. On a same-epoch relist, a folder whose delete is in doubt and that the server lists live is kept. This also undoes a delete that never reached the server, because a listing can't tell the two apart.
  12. A folder stands for everything in the trash below it, including files deleted on their own before the folder. The server only lets a folder be deleted once it's empty, so its contents were always deleted before it or with it, and the deletion times can't tell "deleted with the folder" from "deleted earlier" reliably.
  13. Up to 10,000 nodes per call, so one call's work under the write lock stays bounded.

Testing

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test -p craft-cloud-sync --locked; agent tests end_to_end, local_api, binary, network (turmoil), disk, run_loop
  • cargo test -p craft-cloud-meta -p craft-cloud-server -p craft-cloud-api -p craft-cloud-proto --locked on SQLite
  • Postgres 17: meta with CRAFT_CLOUD_REQUIRE_BACKENDS=1
  • MSRV 1.94 cargo check of proto, meta and server
  • Meta tests on both databases: paging and the order of the list, deletion times, purge marks, restores with deleted ancestors, merges into live folders, conflict names up to the limit, a restored file's new version, and refusals.
  • Sync engine regressions for a lost delete answer followed by a restore: a single file; one-level folders; d/sub/f and d/a/b/f with only the top, a middle or the deepest node restored; an empty folder restored before a relist. The fixes were reverted once to check that the regressions fail without them (4 of the new folder tests fail; the other 3 are guards).
  • Further engine tests pin the edge cases: a folder delete that never reached the server followed by a relist (the folder stays, no delete is sent again), a nested relist, a folder still holding a tracked file (never released), and a new local file at the path of a file whose delete answer was lost (the local content survives as a conflict copy).

Folder contents (added later):

  • Meta tests on SQLite and Postgres 17: the list shows a folder without what is below it (paging with limits 1, 2, 3 and 200 returns each listed node once); a folder restore brings back an earlier-deleted file and nested folders while removed nodes stay deleted; conflict names for two deleted nodes of one name; removing a folder marks what is below it; the 10,000 limit (lowered in tests) leaves the rest listed and a further call finishes the job.
  • Server HTTP test a_deleted_folder_comes_back_and_goes_with_what_was_deleted_in_it, and an agent end-to-end test in which two agents get a restored folder's files and subfolder back on disk.
  • Eight sync engine tests assumed a folder comes back without its file. They now take the file out of the trash first, so they still cover "folder back, its file still deleted".
  • fmt and clippy; meta, server and agent end-to-end tests with Postgres and S3; cargo test -p craft-cloud-sync (142 engine tests).

Not tested

Adds the request and answer types of GET /nodes/{id}/history and POST /nodes/{id}/restore, the opaque history cursor with a single accepted spelling, and the paging limits (default 50, at most 200).
history() pages a file's versions newest first by (seq, id) with an exclusive cursor. restore_version() records a new version with the content of an old one as an ordinary update, checking the source, the current version, deletion and live documents under the same space lock as a commit. No migration is needed.
GET /api/v1/nodes/{id}/history and POST /api/v1/nodes/{id}/restore, resolved in the caller's own space. A stale expected_version answers 409 conflict. Covers permissions, paging, conflicts, restoring old and missing versions, and parallel restores in the API tests.
Updates the API table and the milestone note in the sync design, and adds a short section to the server guide.
A history page now reads versions through an index on (space_id, node, seq, id) in the order of the query, so the page size bounds the work. The migration adds the index on both databases. The restore note in the sync design no longer claims the engine prefers a newer live-document state: a restore is an ordinary new version and files with a live document are refused.
Adds the list, restore and cursor types of GET /trash and POST /trash/{id}/restore, with the same page limits as the history.
…und taken names

A migration adds nodes.deleted_ms and nodes.purge_requested_ms and an index for the trash list, and dates old deletions at the migration. The commit path stamps the time when a node goes from live to deleted and clears it when the node comes back. restore_from_trash brings a deleted node and its deleted folders back in one transaction, merging into live folders of the same name and renaming around taken names with the engine's conflict pattern. request_purge only marks a node, so nothing is freed.
GET /api/v1/trash, POST /api/v1/trash/{id}/restore and DELETE /api/v1/trash/{id}, resolved in the caller's own space, and the [retention] trash_days setting (default 30, 1 to 3650) that sets purge_after_ms.
Adds the trash routes and rules to the sync design and a section with the new setting and the rollout note to the server guide.
…ders within the depth limit

A file restored from the trash now gets a new current version with the content of its last one (no save record). A machine that sent a delete against the old version and never heard the answer can no longer delete the restored file again: the server refuses that delete as a version mismatch, and the engine already lets go of a node that came back at another version. The restore answer reports the new version. A folder that comes back below a folder that has since moved deep is refused as too deep, and nothing changes.
A folder has no version to tell a delete that went through and a later restore from one that never arrived. The engine now remembers the folder deletes it sent without hearing the answer, and when the next pull reports such a folder changed and live, lets it go instead of deleting it again. A folder the feed does not mention was never touched and its delete is sent again. Scenario tests cover a file and a folder restored after a lost answer, at its own path, under a conflict name, and merged into a live folder.
A restored file has a new version, and agents need the folder rule for the rare case of a folder restored after a delete whose answer was lost. Reports the new version in the restore answer.
…d are settled

A folder deleted together with its file, whose answer was lost, and then restored alone, still had the file tracked when the pull judged it, so it was kept and deleted again once the file was let go. The pull now only notes which folders the feed reports live, and the decision is made in apply_remote after the files whose deletes went through are let go. Scenarios cover the non-empty folder restored alone, the folder and its file both restored, and only the file restored.
A folder this machine deleted with a lost answer and that came back from the trash was judged before the folders below it were settled. A tombstoned subfolder still tracked in it kept it, and the next push deleted the restored folder again. The folders of this machine's own delete are now settled deepest first: one the server lists as live, or no longer has and that is gone here too, is let go, and only then are the parents judged.

A relist at the same epoch (the change feed answered resync required) did not note which of those folders the server lists as live, so a restored empty folder stayed tracked and was deleted again. The relist now notes them, so a folder the server has is never pushed as deleted from a stale request.

Regressions cover d/sub/f and d/a/b/f with only the top folder, a middle folder, or only the deepest node restored, and a relist after the restore.
…he trash

Restoring a folder from the trash now also brings back every node below it that
is in the trash (deleted and not removed from the trash), however long before it
was deleted, in the same transaction; the trash list shows such a node only
through its folder; and removing a folder from the trash marks them too. A
folder with more than 10,000 such nodes below it is refused whole as invalid
for both, and Meta::with_trash_tree_cap lowers the limit for tests.
… trash first

Restoring a folder from the trash now brings back all that is in the trash below
it, so the lost-delete tests that restored a folder alone and expected its file
to stay deleted take that file (or sub folder) out of the trash first.
… rest is listed

Instead of refusing a folder with more than 10,000 items in the trash below it
(which left it impossible to restore or remove from a list that hides what is
below a trashed folder), a restore or removal now takes the folder and at most
the cap of items below it, shallowest first. The rest stays in the trash and is
listed on its own because its folder is live or removed, so a further call takes
it. The per-row work under the write lock is bounded by the cap.
Another test helper in a later branch is called sorted; a distinct name keeps the
trash tests compiling wherever they are merged.
@Skyfuller1303

Copy link
Copy Markdown
Author

Update: restoring a folder from the trash now brings back everything in the trash below it, including a file that was deleted on its own inside it earlier. Before, the folder came back empty. Removing a folder from the trash marks its contents too, and the list shows only the folder. One call handles at most 10,000 nodes below the folder, shallowest first. Anything beyond that is listed on its own and can be handled with a further call. The details are in the description (new items under "What", assumptions 12 and 13, and the testing notes). The change is merged into #25, #30, #31, #32, #33 and #34.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant