Skip to content

Security audit log - #18

Open
Skyfuller1303 wants to merge 25 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:audit-log
Open

Skyfuller1303 wants to merge 25 commits into
itrycodinganditsbad:mainfrom
Skyfuller1303:audit-log

Conversation

@Skyfuller1303

@Skyfuller1303 Skyfuller1303 commented Nov 4, 2026 •

Copy link
Copy Markdown

Builds on #17 (admin API), which builds on #14 (device sessions); please review those first. This branch contains their commits until they are merged.

What

  • A security audit log: the events that decide who may do what become rows of a new audit_events table (migration 0009, SQLite and Postgres). Each change writes its event in its own transaction before the commit, so a change whose event cannot be written is not made.
  • Event names are a closed list (auth.*, device.*, account.*, admin.*, join.*, cli.*). The two-factor names are already in the list so the vocabulary clients filter by doesn't change later; their writers come with two-factor sign-in.
  • details is typed. Every field is a closed list of words, a number or an ID; a device's self-reported platform outside the list is stored as other. The text is at most 2,048 bytes, and whole fields are dropped rather than bytes cut. No address, file or device name, unknown account name, password, code, token or key is ever stored.
  • A failed sign-in is aggregated to one row per account, action and minute (ON CONFLICT … DO UPDATE). It is written in its own transaction after the answer is decided and never changes the answer or the limits on tries. Tries with an unknown name share one row per minute with no account in it.
  • A repeat or a no-op writes nothing: disabling a disabled account, giving a role the account already has, and so on.

Server

  • GET /api/v1/account/activity: the caller's own events, as actor or as target. The device appears only on the caller's own acts. An event about another account (for example a former admin's past act on someone else) shows no details.
  • GET /api/v1/admin/audit?user&action&before_ms&cursor&limit: newest first by (at_ms, id), with an opaque, exclusive cursor.
  • GET /api/v1/admin/audit/export?user&action&before_ms: NDJSON. It reads 500 rows per query through a channel of two pages, so a slow client slows the reading rather than filling memory. The end of the log is fixed at the start, at most 100,000 events are sent, and the last line says complete or truncated. A database error breaks the connection instead, so a partial export can't pass for a whole one.
  • [audit] retention_days (1–3650, default 365). Old events are pruned by audit prune [--dry-run], and opportunistically after a write's commit (at most hourly, bounded batches, never inside a security transaction).
  • CLI: audit list. CLI changes (user reset-password, device revocation) write cli.* events with no actor.
  • Append-only is enforced in the code: Meta has no statement that changes or deletes an audit row except pruning and the aggregation counter, and a test checks the sources for exactly those two. Using database triggers and roles instead was a choice against: SQLite has no roles, and this way the same code runs on both databases. Someone with direct database access can still change the log.

Docs

docs/design/sync.md (security audit log section, API table) and docs/server.md (the [audit] config, the audit commands, an upgrade note).

Testing

  • cargo fmt --all -- --check, cargo check --workspace --all-targets --locked, cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test -p craft-cloud-meta -p craft-cloud-server -p craft-cloud-api -p craft-cloud-proto --locked on SQLite
  • Postgres 17: meta with CRAFT_CLOUD_REQUIRE_BACKENDS=1, server with CRAFT_CLOUD_TEST_PG
  • MSRV 1.94 cargo check of the changed crates
  • Tests cover:
    • each operation's event, and that no-ops write none;
    • the fail-closed path (no change without its event);
    • aggregation under concurrent failed sign-ins;
    • the privacy rules (unknown names, platform, size cap);
    • own-activity scoping, including a demoted admin;
    • the cursor at equal timestamps;
    • the export's limits, cutoff, before_ms and terminal line;
    • pruning, and the append-only source check.
  • 65 repeated runs showed no flakes.

Not tested

Left open

  • Two-factor events are named but not written until two-factor exists.
  • No audit export CLI command; the HTTP export covers it.
  • No UI. The desktop admin pages come in a separate PR.

Skyfuller1303 and others added 22 commits November 4, 2026 00:04
Migration 0007 gives a device its client kind, platform and app version
and, once revoked, the time and the reason, and gives a user the time of
the last password change (plus a disabled time nothing reads yet). All
of it is additive, BIGINT on Postgres, and the flag stays the check.

Meta gets create_device_with and redeem_with, rename_device,
revoke_device_as, revoke_other_devices, revoke_all_devices, account,
change_password and reset_password. Flag, time and reason are set
together and only by the first revoke. A password change or reset sets
the hash, revokes the devices and deletes the account's invite and
pairings in one transaction.

Tests run on both databases, including the migration from 0006.

Co-Authored-By: Claude <noreply@anthropic.com>
New: GET /account, POST /account/password, PATCH /devices/{id} and
POST /devices/revoke-others. Sign-in and join take an optional client
kind, platform and app version, checked before a try is counted. The
device list shows them with the time and reason of a revoke; all new
fields are optional on the wire.

Changing the password answers a wrong current password with 403
wrong_password, never 401, since the agent reads every 401 on a
signed-in route as a revoked device. It is limited to five tries per
account in 15 minutes, takes a password-check slot, and ends the revoked
devices' live connections at once. Password and name bodies are read up
to the new TINY_JSON_BYTES (4 KiB).

wrong_password and forbidden move into the shared error codes; the local
API refers to them. The agent, app and desktop only change where they
build the extended types.

Co-Authored-By: Claude <noreply@anthropic.com>
user reset-password now revokes every device of the account (reason
reset) and deletes its open invite and pairings, so a stolen token does
not outlive the old password. --keep-devices restores the old behaviour
for the devices. device revoke-all signs out an account's devices
without touching the password. Both print how many devices they ended;
device revoke records the admin as the reason.

Co-Authored-By: Claude <noreply@anthropic.com>
The API table and error codes in the design document, the password and
sign-out commands, the live-connection delay of the command line and the
rollout note (an older server refuses the newer database) in the
server guide.

Co-Authored-By: Claude <noreply@anthropic.com>
…evice

A /changes long-poll that was waiting when its device was signed out
through the API now ends at once, and one that wakes for any other reason
looks the device up again before it answers, so a device signed out by
the command line gets 401 instead of the next change set.

The live endpoint listens for revocations and looks the device up before
the upgrade, so a revoke between the sign-in check and the subscription
is not missed.

Co-Authored-By: Claude <noreply@anthropic.com>
change_password checks, inside its transaction, that the calling device
is still signed in (it may have been revoked while the password was
hashed) and answers 401 if not. A password that changed meanwhile is now
409 conflict instead of 400 invalid. Both are new MetaError variants
mapped in the server.

set_password_hash had no caller outside a test and is removed. A new test
breaks the pairing deletes with a trigger and checks that a password
change, an admin reset and a revoke-others undo everything on SQLite and
Postgres.

Co-Authored-By: Claude <noreply@anthropic.com>
The long-poll now listens for revocations and looks its device up before
it waits, so a revoke between the sign-in check and the wait is not
missed, and looks it up once more after the change set is fetched, as
the last step before the answer, so a revoke during the fetch is not
answered with changes.

Co-Authored-By: Claude <noreply@anthropic.com>
…changes a password

change_password locked the caller's device row and then the user's row,
while revoke_other_devices from another device locked its own device and
then the user's, and reset_password locked the user before the devices.
On Postgres two of them at once could deadlock and fail with a database
error. All of them now take the user's row first (a no-op update), then
the devices, then the pairings.

A new scenario runs password changes, sign-outs and an admin reset at the
same moment, over and over, on both databases and expects only successes,
conflicts and signed-out callers. The test trigger helper also drops its
Postgres function.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…-in adds a device

redeem consumed the invite and then updated the user, while a password
reset locks the user and then deletes the invite, so the two could
deadlock on Postgres. redeem now finds the invite or pairing with a plain
read, locks the account, and consumes the code again under the lock; a
code that is gone by then is invalid_code. create_pairing locks the
account first too.

A password sign-in verified the password outside any transaction, so a
reset or change that committed meanwhile could be followed by a new
device. sign_in_device locks the account and compares the stored hash
again before it creates the device; if it differs, login answers as for
a wrong password.

New scenarios run redeems, a sign-in, a pairing and a reset at the same
moment on both databases and expect no database error and no live device
once the reset is done.

Co-Authored-By: Claude <noreply@anthropic.com>
The invite delete and the pairing update now name the account the plain
read found, so a code hash removed and made again for another account
between that read and the lock is refused as invalid_code instead of
adding a device to the wrong account.

Co-Authored-By: Claude <noreply@anthropic.com>
The admin API's server overview shows whether blobs are on disk, in S3 or in memory. A backend that does not say is reported as other, so nothing outside the store crate has to change.

Co-Authored-By: Claude <noreply@anthropic.com>
New request and answer types for the routes under /api/v1/admin, with a redacting Debug for everything that holds a password or a code, and the error codes account_disabled and last_admin. The confirmation is a body with an optional password today, so an optional totp_code can join it later without breaking a client.

Co-Authored-By: Claude <noreply@anthropic.com>
Migration 0008 adds the purpose of an invite (invite or reset), a quota per space and a removal mark per user. The new Meta methods for the admin API each re-check the acting admin inside their own transaction and take the locks in one order, with a transaction-wide lock before the last-admin count. Disabled accounts are refused by the token check, sign-in, redeem, pairing and device creation, decided under the account's row lock. A reset code always sets a new password, signs every device out and then signs the redeeming computer in. A space's quota replaces the server-wide cap in the preflight check and in the count that follows.

Co-Authored-By: Claude <noreply@anthropic.com>
The require_admin layer and the routes under /api/v1/admin: counters, a paged and searchable user list, creating accounts, changing role and quota, disabling, enabling and removing, invites, password reset codes and a user's devices. Every change asks for the acting admin's password in its body. Sign-in and join answer account_disabled for a disabled account, and a redeemed reset code ends the live connections of the devices it signed out. Documents the API, the confirmation and the rollout in the design document and the server guide.

Co-Authored-By: Claude <noreply@anthropic.com>
Limiter::count added the attempt before comparing it with the maximum, so tries refused at the limit still raised the count. A refund of an attempt that had been accepted at the same moment then left the key above its limit with fewer wrong passwords than allowed. A refused try now leaves the count alone.

Co-Authored-By: Claude <noreply@anthropic.com>
Disabling and enabling an account leaves the password hash as it was, so a sign-in that verified the password before the disable could still create a usable device after the enable. Users get an auth_gen counter (migration 0008) that a disable, an enable, a removal and every password change or reset bump. The sign-in reads it with the hash and sign_in_device compares both under the account's lock. The server's refunds test covers the limiter change.

Co-Authored-By: Claude <noreply@anthropic.com>
The typed details of a security event, an event, a page of them and the last line of the NDJSON export. Every
field of the details is a closed list of words or a number, and fields a later release adds are ignored.

Co-Authored-By: Claude <noreply@anthropic.com>
Migration 0009 adds audit_events, with no foreign keys so that an event outlives its account. Every change that the
log names writes its event in its own transaction before the commit, so a change whose event cannot be written is
not made. A change that changes nothing writes none. A failed sign-in is one row per account and minute whose
counter goes up, written in a transaction of its own. The values of an event are typed and come from closed
lists, never from what a client typed. Old events are pruned in batches outside those transactions.

Meta methods that had no way to say who asked get one: revoke_device_by takes the origin, and admin_enable_user and
admin_revoke_invite take the time.

Co-Authored-By: Claude <noreply@anthropic.com>
GET /account/activity, GET /admin/audit and the streamed GET /admin/audit/export; a failed sign-in is written
after the answer is decided and never changes it. [audit] retention_days (1 to 3650, default 365) and
audit list and audit prune on the command line. The export reads fixed pages through a bounded channel up to a
fixed end of the log and ends with a line that says complete or truncated, or breaks the connection.

Co-Authored-By: Claude <noreply@anthropic.com>
The routes and events in the design, and the retention, the personal data and the upgrade note in the server guide.

Co-Authored-By: Claude <noreply@anthropic.com>
…in the export, reword the append-only note

The account's own activity no longer shows the details (the device an admin signed out, roles, counts) of an event about another
account, which a former administrator would otherwise keep seeing. The export validates before_ms like the list and applies it
to every page and the probe besides its fixed end. The docs now say the append-only choice is a design decision, not that SQLite
has no triggers.

Co-Authored-By: Claude <noreply@anthropic.com>
The server (device sessions) shows a device's client_kind, platform and
app_version in the device list, but the agent never sent them. Every
sign-in and join of the agent now says it is an agent, which operating
system it runs on and its own version, cut to the server's 32-byte limit.
A server-backed test checks that the device list shows them after both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant