Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/layerfs-js-sdk/references/methods/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Store on another machine is read through its worker; a stopped worker is
| Call | What it does | Notes |
| --- | --- | --- |
| `create(name)` | An empty sqlite Store on `native` | Live result: the record above, with `projects: []`. `Exists` for a taken name |
| `connect(name, url, …)` | Attaches a PostgreSQL database | `url` is `postgresql://<user>@<host>:<port>/<database>[?sslmode=…]` without a password (one in the user part or in any query parameter whose name contains `password` is `InvalidInput`); `password` is a reference, `env:LAYERFS_<NAME>` or `file:/path`. `synchronous_commit` defaults to `off`. The same database under a second name is `Exists` |
| `connect(name, url, …)` | Attaches a PostgreSQL database | `url` is `postgresql://<user>@<host>:<port>/<database>[?sslmode=…]` without a password (one in the user part or in any query parameter whose name contains `password` is `InvalidInput`); `password` is a reference, `env:LAYERFS_<NAME>` or `file:/path`. `synchronous_commit` defaults to `local`. The same database under a second name is `Exists` |
| `attach(name, machine, …)` | Lets a machine run Sandboxes of a PostgreSQL Store | Checks from that machine that it reaches the database and its role may write; `url` and `password` replace the Store's for that machine, either alone. `InvalidInput` for sqlite; `Unsupported` when the platform runs no PostgreSQL Sandboxes; `Unavailable` (not reached, machine stopped); `Denied` (login or privilege); `Conflict` while the Store is disconnected or reconnected |
| `detach(name, machine)` | Stops a machine running Sandboxes of a PostgreSQL Store | Dials nothing; detaching `native` is allowed. `InvalidInput` (sqlite, or not attached there); `InUse` while a Sandbox of the Store is recorded on the machine |
| `push(name, machine, …)` | Sends a sqlite Store's file from `native` to `machine` | `InvalidInput` unless the Store is on `native` (pull it first). Progress `{ phase, done, total }` on `on_progress` / `--progress`: `done` and `total` are bytes during `send`, null in the other phases (`check`, `snapshot`, `verify`, `install`, `record`, `retire`). `InUse` while the Store is held (Sandbox records, processes with it open; `detail.related` lists them as `{ kind, name }`); `Capacity` when the target has less free disk than the Store plus a headroom of 1 GiB (or a tenth of the Store), checked before anything changes; `Unavailable` naming `layerfs machine start <m>` for a stopped machine; while it runs, Store calls and operations on it are `Busy`, from the caller's process too. An uncertain push whose process is gone (it ended during install or record, or a step's answer was lost) leaves every later call `Unresolved`, naming `store status` and the push that clears it: a new `push` (or `pull`) first deletes the copy on the machine the record does not name, then runs. A push whose `retire` did not finish leaves the Store in use on `machine` and `last_transfer.retired: false`; the next `pull` deletes the old copy first |
Expand Down
39 changes: 24 additions & 15 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ assignment. Earlier wording of these guides is in the

| Rule | Where |
| --- | --- |
| Disposable / WAL / synchronous=OFF is the only SQLite Store profile; Durable execution is `NOT_RUN` | [Persistence profile](#active-persistence-profile--owner-direction-2026-10-07) |
| Normal / WAL / synchronous=NORMAL is the product SQLite Store profile; Durable execution is `NOT_RUN` | [Persistence profile](#active-persistence-profile--owner-direction-2026-11-17) |
| One attempted operation: no automatic retry, busy handler, refresh or replay | [§4](#4-code-build-and-docs) |
| Build tests with `--no-run` first; every test command stops at 120 s, and reaching it is a hang | [§4](#4-code-build-and-docs) |
| CI reports and never gates; verify your own change and report checks and gaps | [§4](#4-code-build-and-docs) |
Expand Down Expand Up @@ -98,24 +98,33 @@ assignment. Earlier wording of these guides is in the
not resume old loops, reopen closed rows or run old campaigns unless the
task selects that work, and do not choose authority by issue number alone.

## Active persistence profile — owner direction 2026-10-07

- **Disposable / WAL / synchronous=OFF is the only permitted profile for a
SQLite Store.** Durable execution is disabled indefinitely, until the owner
explicitly allows it again; this supersedes earlier instructions to exercise
both profiles or resume Durable. A milestone, old benchmark selection,
default argument or passing build does not reauthorize Durable.
- Select Disposable explicitly for every new Store, application run, test,
diagnostic and measurement; never execute Durable or fall back to it.
Retained Durable code may compile; report its execution as
## Active persistence profile — owner direction 2026-11-17

- **Normal / WAL / synchronous=NORMAL is the profile every product path opens
a SQLite Store with** (owner, 2026-11-17; it was Disposable / synchronous=OFF
from 2026-10-07). The owner enabled it ahead of its safety and performance
gate ([layerfs-project#155](https://github.com/Ephemeral-AI-Lab/layerfs-project/issues/155)
§7.5 and §7.6), which has **not** been run: claim no power-loss guarantee
from it yet. Store files written outside SQLite (install, transfer, restore,
snapshot) are still not synchronized (#155 §7.3).
- Disposable / synchronous=OFF remains selectable, and existing tests,
diagnostics and measurements that select it explicitly keep doing so. A
Store's manifest and record still carry the `disposable` label; it names the
non-durable WAL Store, not the connection's synchronization.
- Durable execution is disabled indefinitely, until the owner explicitly
allows it again. A milestone, old benchmark selection, default argument or
passing build does not reauthorize Durable; never execute it or fall back to
it. Retained Durable code may compile; report its execution as
`NOT_RUN — disabled by owner until explicit reauthorization`. Historical
Durable receipts keep their verdicts. This policy does not claim the Durable
API was removed.
- Disposable's COMMIT does not checkpoint (`wal_autocheckpoint=0`); its writing
session's own thread checkpoints the WAL off the commit path (owner approval,
2026-11-07; [SQLite checkpoints](docs/architecture/engine/store/sqlite.md#checkpoints)).
- A Normal or Disposable COMMIT does not checkpoint (`wal_autocheckpoint=0`);
the writing session's own thread checkpoints the WAL off the commit path
with the session's synchronization (owner approval, 2026-11-07;
[SQLite checkpoints](docs/architecture/engine/store/sqlite.md#checkpoints)).
- A PostgreSQL Store has no profile; its write setting is `synchronous_commit`
(default `off`), recorded by `store.connect`.
(default `local` since 2026-11-17, `off` before), recorded by
`store.connect`. A Store connected earlier keeps its recorded value.
- Each Workspace's daemon overlay is a separate MEMORY/OFF/EXCLUSIVE database
file.

Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-bucket/src/backup/restore.rs
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ fn fetch(
progress::begin("verify", None);
let opened = open_store_observed(
PersistenceConfig::sqlite(&temporary)
.with_sqlite_profile(SqlitePersistenceProfile::Disposable),
.with_sqlite_profile(SqlitePersistenceProfile::Normal),
&binding,
cursor_key,
1,
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/compact.rs
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ pub fn compact(home: &Path, _events: &mut Events<'_>, store: &str) -> Outcome {
};
let identity = identity::identity(&args)?;
let handles = SqliteHandles::open_writable(
PersistenceConfig::sqlite(path).with_sqlite_profile(SqlitePersistenceProfile::Disposable),
PersistenceConfig::sqlite(path).with_sqlite_profile(SqlitePersistenceProfile::Normal),
&identity.binding,
identity.cursor_key,
)
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/identity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ fn create_empty(
let sealed = ProjectApi::new()
.create_store(CreateRequest {
store: PersistenceConfig::sqlite(&temporary)
.with_sqlite_profile(SqlitePersistenceProfile::Disposable)
.with_sqlite_profile(SqlitePersistenceProfile::Normal)
.into(),
policy,
catalog: HistoryCatalogConfig {
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/open_args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ impl PgSelection {
schema_of(self.schema.as_deref())
}

/// Its `synchronous_commit`, `off` by default.
/// Its `synchronous_commit`, `local` by default.
pub fn setting(&self) -> Result<String> {
setting_of(self.synchronous_commit.as_deref())
}
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/postgresql/connect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ pub fn connect(home: &Path, request: &StoreConnect) -> Outcome {
let name = request.name.as_str();
let given = request.url.as_str();
refuse_password(given)?;
let sync = request.synchronous_commit.as_deref().unwrap_or("off");
let sync = request.synchronous_commit.as_deref().unwrap_or("local");
if !matches!(
sync,
"off" | "local" | "remote_write" | "on" | "remote_apply"
Expand Down
11 changes: 6 additions & 5 deletions control/crates/layerfs-store/src/postgresql/open.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ use layerfs_home::outcome::{refused, Outcome};
use layerfs_store_service::StoreService;
use std::path::Path;

/// `synchronous_commit` as the call or record names it: off (default),
/// local, remote_write, on or remote_apply.
/// `synchronous_commit` as the call or record names it: off, local (the
/// default since 2026-11-17), remote_write, on or remote_apply. A record
/// keeps the value it was connected with.
pub fn setting_of(given: Option<&str>) -> Result<String, SdkRefusal> {
let given = given.unwrap_or("off");
let given = given.unwrap_or("local");
match given {
"off" | "local" | "remote_write" | "on" | "remote_apply" => Ok(given.to_owned()),
other => Err(refused(
Expand Down Expand Up @@ -63,11 +64,11 @@ pub fn url_of(url: Option<&str>) -> Result<&str, SdkRefusal> {
pub fn selection(url: &str, schema: &str, sync: &str) -> layerfs_persistence::PostgresSelection {
use layerfs_persistence::SynchronousCommit as S;
let setting = match sync {
"local" => S::Local,
"off" => S::Off,
"remote_write" => S::RemoteWrite,
"on" => S::On,
"remote_apply" => S::RemoteApply,
_ => S::Off,
_ => S::Local,
};
layerfs_persistence::PostgresSelection::new(url, schema).with_synchronous_commit(setting)
}
Expand Down
4 changes: 2 additions & 2 deletions control/crates/layerfs-store/src/project/add.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ pub fn save_entry(
let project = Project {
store: SealedStore {
path: entry.join("sealed.sqlite"),
profile: SqlitePersistenceProfile::Disposable,
profile: SqlitePersistenceProfile::Normal,
sqlite_version: manifest.host_sqlite.clone(),
bytes: manifest.bytes,
},
Expand Down Expand Up @@ -170,7 +170,7 @@ pub fn add_project(request: &ProjectAdd) -> Outcome {
let request = AddRequest {
source,
store: PersistenceConfig::sqlite(&store_file)
.with_sqlite_profile(SqlitePersistenceProfile::Disposable)
.with_sqlite_profile(SqlitePersistenceProfile::Normal)
.into(),
binding: identity.binding.clone(),
cursor_key: identity.cursor_key,
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/project/discovery.rs
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ pub fn project(home: Option<&Path>, store: Option<&str>, name: Option<&str>) ->
Ok(Project {
store: SealedStore {
path: path.join("sealed.sqlite"),
profile: SqlitePersistenceProfile::Disposable,
profile: SqlitePersistenceProfile::Normal,
sqlite_version: manifest.host_sqlite.clone(),
bytes: manifest.bytes,
},
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/src/sqlite/transfer/steps.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ fn checkpoint_leftover_wal(file: &Path, frame: &StepFrame) -> Result<(), SdkRefu
let identity = crate::identity::identity_of(record)?;
if let Ok(handles) = layerfs_persistence::SqliteHandles::open_writable(
layerfs_persistence::PersistenceConfig::sqlite(file)
.with_sqlite_profile(layerfs_persistence::SqlitePersistenceProfile::Disposable),
.with_sqlite_profile(layerfs_persistence::SqlitePersistenceProfile::Normal),
&identity.binding,
identity.cursor_key,
) {
Expand Down
2 changes: 1 addition & 1 deletion control/crates/layerfs-store/tests/open_args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ fn reads_the_routing_fields_and_refuses_a_missing_store_path() {
);
let postgresql = open.postgresql.as_ref().unwrap();
assert_eq!(postgresql.url().unwrap(), "postgresql://u:pw@x/db");
assert_eq!(postgresql.setting().unwrap(), "off");
assert_eq!(postgresql.setting().unwrap(), "local");
assert_eq!(
postgresql.schema().unwrap_err().message,
"postgresql.schema is required"
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/api/sdk/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ CLI layerfs store connect <name> <url> [--password <password>] [--synchronou
| --- | --- | --- |
| `url` | Text | `postgresql://` or `postgres://`, holding no password (no `user:pass@`, no query parameter whose name contains `password`); names its user when `password` is given |
| `password` option | Text | a reference, `env:NAME` or `file:/absolute/path` (trailing newline dropped), never the password; it must resolve now and is resolved again in the caller on every later call ([`secrets.rs`](../../../../control/crates/layerfs-home/src/secrets.rs)) |
| `synchronous_commit` option | Text | `off` (default), `local`, `remote_write`, `on` or `remote_apply`; applied to the Store's write transactions |
| `synchronous_commit` option | Text | `off`, `local` (default), `remote_write`, `on` or `remote_apply`; applied to the Store's write transactions |

```text
{ name, kind: "postgresql", schema: "layerfs", synchronous_commit,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ another operation holds the Store
```text
store.connect(name, url, {password: "env:PG", synchronous_commit})
refuse_password(url) user:pass@ or any *password* param -> InvalidInput
synchronous_commit off (default) | local | remote_write | on
synchronous_commit off | local (default) | remote_write | on
| remote_apply, else InvalidInput
resolve(password) checked once now
Exists? same name, or same server+database, or
Expand Down
10 changes: 5 additions & 5 deletions docs/architecture/engine/store/sqlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The SQLite backend of the [Store](README.md): one WAL file every opener shares t
| --- | --- | --- |
| `sqlite_pack_layout` | `Monolithic` (`user_version` 36): whole `pack.body`; `GroupRows` (37): `control` row plus one `pack_unit` row per group; `GroupRowsIndexed` (38): adds index `pack_unit_mapping` | [`config.rs`](../../../../engine/crates/layerfs-persistence/src/store/config.rs), [`objects_units.sql`](../../../../engine/crates/layerfs-persistence/sql/sqlite/objects_units.sql) |
| `sqlite_acquisition` | `Absent` (+0) or `Tables` (+6, versions 42–44, created with `auto_vacuum=INCREMENTAL`) | [`config.rs`](../../../../engine/crates/layerfs-persistence/src/store/config.rs), [`acquisition/schema.sql`](../../../../engine/crates/layerfs-persistence/sql/sqlite/acquisition/schema.sql) |
| `sqlite_profile` | `Disposable` in every product opener (see [Store](README.md)) | [`profile.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/profile.rs) |
| `sqlite_profile` | `Normal` in every product opener since 2026-11-17 (see [Store](README.md)); a test may select `Disposable` | [`profile.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/profile.rs) |

Every table is `STRICT`; history keys are `WITHOUT ROWID` blobs whose first
byte tags the kind (`0x31` stack, `0x32` Layer, `0x12` Commit, `0x11`
Expand All @@ -55,13 +55,13 @@ UPDATE and DELETE on `pack` and `pack_unit`. Open checks `application_id`,
| Open | `READ_WRITE` or `READ_ONLY`, `NO_MUTEX`; statement cache 64 | [`connection.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/connection.rs) |
| Guard | `busy_timeout=0`; `SQLITE_DBCONFIG_DEFENSIVE` on | same |
| Settings | `profile::apply`, then read back into `ConnectionProfile`; a mismatch is `Integrity`; an existing Store not in `wal` mode is refused | [`profile.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/profile.rs) |
| Checkpoint thread | writable Disposable session only: a second connection on its own thread; failure to start it refuses the open | [`background.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/background.rs) |
| Checkpoint thread | writable Normal or Disposable session only: a second connection on its own thread, opened with the session's `synchronous` value and read back; failure to start it refuses the open | [`background.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/background.rs) |
| Last close | A writable session's close waits for `Writer::acquire(path)` like any write (or reuses the lane an uncertain transaction already holds), so its close, which may be SQLite's last, checkpoints and removes `-wal`/`-shm` under the lane; a pull's verify close and a restore's in-use check rely on that. If the wait itself fails (interrupted, or the directory cannot be opened), the close sets `SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE` first, so SQLite takes no exclusive lock, runs no checkpoint and removes nothing; there is no second attempt. If even that setting fails, the connection is kept until process exit. A read-only or diagnostic session cannot checkpoint or remove those files and takes no lane, so it never waits behind a writer kept after an uncertain write. The checkpoint thread's connection closes first and never takes it. One exception: an open whose checkpoint thread fails to start drops its new connection after the open's lane is released | [`lifecycle.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/lifecycle.rs) |

| Setting | Disposable | Source |
| Setting | Normal (Disposable where it differs) | Source |
| --- | --- | --- |
| identity | `sqlite-wal-off-v2` | [`profile.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/profile.rs) |
| `journal_mode`, `synchronous`, `fullfsync` | `wal`, 0 (OFF), 0 | same |
| identity | `sqlite-wal-normal-v2` (`sqlite-wal-off-v2`) | [`profile.rs`](../../../../engine/crates/layerfs-persistence/src/backend/sqlite/session/profile.rs) |
| `journal_mode`, `synchronous`, `fullfsync` | `wal`, 1 (NORMAL; Disposable 0, OFF), 0 | same |
| `wal_autocheckpoint` | 0: the session's checkpoint thread copies the log | same |
| shared | `foreign_keys=1`, `checkpoint_fullfsync=1`, `journal_size_limit=4194304`, `cache_size=-2048`, `mmap_size=0`, `temp_store=2`, `page_size=4096`, `busy_timeout=0` | same |

Expand Down
2 changes: 1 addition & 1 deletion docs/versioned/2.0.0/machines-and-stores.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ layerfs project init pg-main/load ./load
(`password`, `sslpassword`, any case). Give the password as a reference
with `--password` (`env:NAME` or `file:/absolute/path`). It is resolved on
each call and never written to the home.
- `synchronous_commit` is `off` (the default), `local`, `remote_write`,
- `synchronous_commit` is `off`, `local` (the default), `remote_write`,
`on` or `remote_apply`.
- The Store lives in the fixed schema `layerfs`. An empty database gets the
schema with a new identity. A database that already holds the `layerfs`
Expand Down
9 changes: 5 additions & 4 deletions engine/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,10 +228,11 @@ is the working procedure. A docs-only change needs no campaign.
uncertain custody: no resend, rollback, deletion or success claim from an
unfenced read. An unsupported required platform or capability fails
explicitly, never as a silent no-op or fallback.
- **SQLite Stores are Disposable/WAL/OFF only**, on macOS and Linux
([profile](../AGENTS.md#active-persistence-profile--owner-direction-2026-10-07));
select it explicitly, never by default argument. Disposable claims
process-crash survival only. Open verifies WAL and never converts an existing
- **Product paths open SQLite Stores Normal/WAL/NORMAL**, on macOS and Linux
([profile](../AGENTS.md#active-persistence-profile--owner-direction-2026-11-17));
tests may still select Disposable/WAL/OFF. Select a profile explicitly,
never by default argument. Both claim process-crash survival only until
Normal's safety gate is run. Open verifies WAL and never converts an existing
Store; select profile and layout before open. WAL stays on throughout Init
and Commit (owner supersession 2026-10-07): no private MEMORY import or
journal promotion. Checkpointing is described on the
Expand Down
Loading