security(macos): give archive deduplication a private, unpredictable scratch directory - #999
Conversation
…scratch directory Closes #889. macOS whole-archive deduplication wrote its copies into `<tmp>/elephc-link-dedup-<pid>`, created with `create_dir_all` — which SUCCEEDS on a directory that already exists — and copied archives in with `fs::copy`, which follows a symlink at the destination. On a multi-user machine another local user could predict or race the compiler's pid, pre-create that directory with permissive access, and plant a symlink named after a bridge archive. The copy would then truncate and overwrite the symlink's target with the compiler user's permissions. The path is reached whenever at least two whole-archived bridges need deduplication. The scratch directory now comes from `mkdtemp(3)`: six characters of kernel-chosen randomness, EXCLUSIVE creation so a planted directory cannot be adopted, and mode `0700` from the moment it exists — no window between creation and a permission fixup. Failure to obtain one FAILS CLOSED: the plan is returned unchanged and the link proceeds with the original archives, because deduplication is an optimization and a predictable scratch path is not an acceptable price for it. Archive copies are written through `OpenOptions::create_new` (`O_CREAT | O_EXCL`), which refuses to follow a symlink and fails outright if anything already sits at the destination. The private directory already makes a planted symlink unreachable; this is the second lock on the same door. Cleanup is unchanged — `PreparedArchives::cleanup` still removes the directory after the linker has consumed the plan. Tests cover both halves: two scratch paths differ, carry no pid, are `0700` and empty; and a pre-existing destination (a symlink among them) is refused with its target left untouched. Claude-Session: https://claude.ai/code/session_01KSAAWPyNBq6dP2b5puN3wr
Greptile SummaryThis PR secures macOS whole-archive deduplication and updates the copy path following the previous review.
Confidence Score: 5/5The PR appears safe to merge; the previously reported whole-archive memory spike has been fully addressed by streaming the source into the exclusive destination. No blocking or non-blocking actionable issues remain. The prior Greptile thread is manually resolved, and the current implementation replaces the full-file read with
|
| Filename | Overview |
|---|---|
| src/linker/archive_dedup.rs | Replaces predictable scratch storage and symlink-following copies with private exclusive creation, bounded-memory streaming, fail-closed behavior, and focused regression tests. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Link plan] --> B{Deduplication needed?}
B -- No --> C[Return unchanged plan]
B -- Yes --> D{Create private scratch directory}
D -- Failure --> C
D -- Success --> E[Open source archive]
E --> F{Exclusively create destination}
F -- Failure --> G[Keep original archive]
F -- Success --> H[Stream archive contents]
H --> I[Strip duplicate members]
I --> J[Replace archive in prepared plan]
J --> K[Linker consumes plan]
K --> L[Cleanup scratch directory]
Reviews (2): Last reviewed commit: "perf(linker): stream the deduplication c..." | Re-trigger Greptile
…archive Review follow-up on #889: the exclusive `create_new` destination was being fed from `fs::read`, which materializes the ENTIRE source archive in memory before a byte is written. A whole-archived bridge runs to tens of megabytes and a macOS link deduplicates several of them in sequence, so the peak was proportional to the archive set for no reason at all. `io::copy` streams it instead — the same thing `fs::copy` does internally, minus the symlink-following open that made `fs::copy` unusable here in the first place. The `O_CREAT | O_EXCL` destination, which is the security property #889 added, is untouched: only the source side changes. Claude-Session: https://claude.ai/code/session_01KSAAWPyNBq6dP2b5puN3wr
|
@Guikingone is this ready? for me can be merged, but you didn't ask for my review :) let me know if I can merge it |
|
Ah yes, forgot about it 😄 Ready for review / merge 🙂 |
Closes #889.
What was wrong
macOS whole-archive deduplication wrote its copies into
<tmp>/elephc-link-dedup-<pid>, created withcreate_dir_all— which succeeds on a directory that already exists — and copied archives in withfs::copy, which follows a symlink at the destination.On a multi-user machine another local user could predict or race the compiler's pid, pre-create that directory with permissive access, and plant a symlink named after a bridge archive. The copy would then truncate and overwrite the symlink's target with the compiler user's permissions. The path is reached whenever at least two whole-archived bridges need deduplication.
Fix
mkdtemp(3)— six characters of kernel-chosen randomness, no pid in the namemkdtempcreates or fails; a planted directory cannot be adopted0700from the moment it exists, with no window between creation and a permission fixupOpenOptions::create_new(O_CREAT | O_EXCL), which refuses to follow a symlink and fails if anything already sits at the destinationFailing closed is the right trade here: deduplication is an optimization, and a predictable scratch path is not an acceptable price for it.
create_newis the second lock on the same door — the private directory already makes a planted symlink unreachable — but the issue asks for it and it costs nothing.Cleanup is unchanged:
PreparedArchives::cleanupstill removes the directory after the linker has consumed the plan.Tests
scratch_is_unpredictable_private_and_fresh— two scratch paths differ, neither contains the pid, both are0700directories and emptyan_existing_destination_is_refused_not_followed— a symlink planted at a destination name is refused bycreate_new, and its target still holds its original bytescargo test --bin elephc(1903) passes;cargo buildis warning-free.🤖 Generated with Claude Code
https://claude.ai/code/session_01KSAAWPyNBq6dP2b5puN3wr