refactor(server): let the S3 backend own its proxy read slot and route storage I/O through the backend

[luminoso]

Summary

The S3 proxy read slot has leaked three times, and each fix landed in utils/file.ts rather than the backend: #497 (reverted in #499), #1104 (the idle timer) and #1106 (the client gone before the stream exists). The slot was freed only if the HTTP layer destroyed the stream on every exit path, which the StorageBackend interface never said. One leak was still open: a throw in sendFile after the stream existed but before pipe left the stream undestroyed and its slot held. The S3 adapter now owns the slot from acquire to release, so sendFile only has to destroy a source it stops piping. Services also stop branching on path shape where the backend can do the work.

How the slot lifetime works

sendFile creates an AbortController that aborts when the response closes, before the handler runs, and passes its signal to the handler through an AsyncLocalStorage (getResponseSignal()). serveFromBackend forwards it as ServeOptions.signal. In proxy mode the S3 adapter rejects a read whose client already left, both before queueing for a slot and after getting one. It passes the signal to GetObject, and wraps the body in a Transform that runs the idle watchdog, joined to the body with pipeline and to the signal with addAbortSignal. The slot is released on the Transform's close, which follows end, error, consumer destroy, abort and idle alike. sendFile registers "destroy the source when the response closes" before setting any header, so a throw before pipe is covered as well: its error response closes the response.

Changes

• S3 backend (backends/s3-storage.backend.ts): owns the slot as above. S3_STREAM_IDLE_TIMEOUT_MS and the zombied-socket explanation moved here from utils/file.ts. Adds readAll.
• HTTP layer (utils/file.ts): the response signal and its AsyncLocalStorage. The S3 idle timer is gone. An AbortError that follows the client leaving is no longer logged or answered.
• Interface (interfaces/storage-backend.interface.ts): ServeOptions.signal, readAll, and an optional { readable } on exists.
• Disk backend (backends/disk-storage.backend.ts): exists, readAll and delete go through StorageRepository, the calls they replace in upstream code paths, so upstream specs that mock the repository still apply. A relative key with no media location now throws instead of resolving against the working directory.
• Services: BaseService.backendFor(key) picks the backend for a path. Absolute paths get a disk backend over the service's own StorageRepository; relative keys go through StorageService.resolveBackendForKey. ensureLocalFile, getProbeInput, serveFromBackend, the metadata sidecar check and sidecar download, and StorageService.handleDeleteFiles use it.
• Notification and the machine-learning repository share readAll instead of two hand-written read-into-a-buffer loops.
• Specs: the idle timeout tests moved from file.spec.ts to s3-storage.backend.spec.ts, which asserts that active slots return to 0. The ML and metadata specs dropped their disk-versus-S3 routing pairs. The MinIO integration spec gained two tests and now uses the same pinned Chainguard image as the storage-migration e2e, since MinIO withdrew its own images.

isAbsolute sites

Site	Outcome	Reason
base.service ensureLocalFile	Deleted	Disk downloadToTemp already returns the absolute path with a no-op cleanup
base.service getProbeInput	Deleted	Disk getReadableUrl already returns the absolute path
metadata handleMetadataExtraction sidecar download	Collapsed	ensureLocalFile handles both path shapes
metadata handleSidecarCheck	Collapsed	backend.exists(candidate, { readable: true }) keeps upstream's R_OK check on disk
machine-learning.repository getFormData	Collapsed	readAll on either backend; disk reads through readFile as upstream does
storage.service handleDeleteFiles	Collapsed	Disk delete is upstream's storageRepository.unlink
base.service backendFor	New, single dispatch	The one place a service picks the backend from the path shape
metadata motion photo path choice	Kept, policy	Chooses where a new motion video is placed
media handleAssetMigration	Kept, policy	fs.rename migration does not apply to S3 keys
person handlePersonMigration	Kept, policy	Same as above for face thumbnails
storage-template job and moveAsset (2)	Kept, policy	Storage templates do not apply to S3 keys
asset trim external-library check	Kept, policy	Blocks trimming external-library files
metadata handleSidecarWrite	Kept, local path	exiftool writes to a local file
download downloadArchive	Kept, local path	archiver adds disk files by real path
notification album thumbnail	Kept, local path	The mailer attaches a disk file by path; only the S3 side changed, to readAll
metadata motion video write	Kept, disk write	Disk creates the file only if it is new, after ensureFolders, unlike an S3 put
asset copySidecar	Kept, disk write	Disk uses copyFile; collapsing would change it to a stream copy

Behaviour changes

• The leak is fixed: a throw in sendFile between getting the stream and piping it no longer leaves the slot held.
• A client that leaves while its proxied read is queued or in flight now aborts the GetObject and frees the slot immediately, instead of after the fetch completes.
• When the client has already gone, an AbortError from that read is dropped silently. Previously it was logged as Unable to send file and answered with a 404 on a dead response. An AbortError while the response is still open is still logged and answered.
• Disk delete now goes through StorageRepository.unlink, which logs and ignores a file that is already gone. Its callers, handleDeleteFiles and the storage migration's source delete, already caught that error and logged a warning, so the outcome is the same.
• Disk streams served as stream responses (face thumbnail crops) no longer get the S3 idle timer, which never applied to them in practice.
• Each proxied read now passes through a Transform. A stalled client can buffer up to about 128 KiB more per slot than a direct pipe did. That is a bounded cost and is what lets the idle watchdog see chunks without consuming the body.

Testing

• Unit, pnpm test -- --run: 201 files passed, 6445 tests passed.
• pnpm lint (zero warnings) and pnpm check: clean.
• MinIO integration spec on podman (DOCKER_HOST=unix:///run/user/1002/podman/podman.sock TESTCONTAINERS_RYUK_DISABLED=true IMMICH_TEST_DOCKER=true npx vitest --config test/vitest.config.mjs --run src/backends/s3-storage.backend.integration.spec.ts): 14/14 passed locally. CI never runs this spec, because IMMICH_TEST_DOCKER is not set in any workflow.
• Medium metadata, storage, asset-media, asset, download, user and person specs: all passed. The full medium suite had 36 failures in exif, library and workflow-core-plugin, which also fail on main. It also had timeouts in people-identity-rbac, memory and database-migration under load; those three pass 150/150 run on their own.
• Mutation checks, each killed by at least one test: no release on close, no wait-time abort check, no pre-queue abort check, no addAbortSignal, no SDK abortSignal, sendFile registering the destroy-on-close after the headers, a re-arm that always fires, pipe instead of pipeline (the no-unhandled-error test), the AbortError suppression removed or ignoring whether the response closed, disk exists always requiring readability (the storage-migration test), and the relative-key guard removed.

Follow-ups

• Replace the static backend state in StorageService and the remaining lazy imports with an injected provider. Deferred because specs touch the static state 88 times and spy on resolveBackendForKey about 50 times, so the change would mostly be spec rewrites across upstream-derived spec files. backendFor already cuts the lazy imports from 5 to 3.
• Run s3-storage.backend.integration.spec.ts in CI with IMMICH_TEST_DOCKER=true. It is the only test that releases slots against a real HTTP body, and the storage-migration e2e already pulls the same pinned MinIO image.