Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
5efb047
refactor: move backend, io and profiling experiments to benchmarks/ex…
unbalancedparentheses Nov 16, 2026
c402e09
refactor: drop experiment features from published crates; update driv…
unbalancedparentheses Nov 16, 2026
6054b2c
style: rustfmt after removing experiment arguments; regenerate patch
unbalancedparentheses Nov 16, 2026
b064eff
refactor: split tasks and threads actor.rs into envelope, context, ac…
unbalancedparentheses Nov 16, 2026
dedcf95
refactor: split supervision.rs into child, state, builder, run and po…
unbalancedparentheses Nov 16, 2026
14370e6
refactor: split macros/src/lib.rs into protocol, actor, lints, send_h…
unbalancedparentheses Nov 16, 2026
4bc52c7
chore: map split files in model-map, retarget planted supervision pat…
unbalancedparentheses Nov 16, 2026
26b475e
chore: rebase backend-experiments.patch onto the split actor modules
unbalancedparentheses Nov 16, 2026
553ef69
chore: restore blank line after patch header
unbalancedparentheses Nov 16, 2026
0286f1a
chore: restore blank line after patch header
unbalancedparentheses Nov 16, 2026
61caab1
Merge branch 'refactor/experiments-to-benchmarks' into refactor/split…
unbalancedparentheses Nov 16, 2026
bd96ae0
fix: keep the experiment feature names as no-ops for 0.8 patch compat…
unbalancedparentheses Nov 16, 2026
0fdeae0
Merge branch 'refactor/experiments-to-benchmarks' into refactor/split…
unbalancedparentheses Nov 16, 2026
07d8163
chore: rebase backend-experiments.patch after the no-op feature stubs
unbalancedparentheses Nov 16, 2026
2ff677f
fix: experiment feature stubs keep enabling async (semver feature_no_…
unbalancedparentheses Nov 16, 2026
af18085
Merge branch 'refactor/experiments-to-benchmarks' into refactor/split…
unbalancedparentheses Nov 16, 2026
b9a36c0
chore: rebase backend-experiments.patch onto the split modules
unbalancedparentheses Nov 16, 2026
deea239
Merge origin/main (#276 Loom lint) into refactor/experiments-to-bench…
unbalancedparentheses Nov 16, 2026
3c2f787
Merge branch 'refactor/experiments-to-benchmarks' into refactor/split…
unbalancedparentheses Nov 16, 2026
7a1add3
fix(verification): regenerate stale planted-bug patches and check the…
unbalancedparentheses Nov 16, 2026
b8f1526
Merge remote-tracking branch 'origin/main' into refactor/split-large-…
unbalancedparentheses Nov 16, 2026
7c3b264
Merge branch 'refactor/split-large-files' into fix/planted-patches
unbalancedparentheses Nov 16, 2026
c85fb46
Merge remote-tracking branch 'origin/main' into fix/planted-patches
unbalancedparentheses Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/model-sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ jobs:
run: python3 scripts/check-model-sync.py events
- name: Every model is trace-validated (or allowlisted), coverage floors hold
run: python3 scripts/check-model-sync.py models
# The planted-bug patches (docs/TESTING.md, "Planted-bug sensitivity")
# must keep applying to the code they mutate, or the matrix rots silently.
# A PR that moves planted code regenerates the affected patches.
- name: Planted-bug patches apply
run: |
for p in verification/planted/patches/*.patch; do
git apply --check "$p" || { echo "::error file=$p::no longer applies; regenerate it against this tree"; exit 1; }
done
# On merge_group and push the script passes: the pull_request run
# already enforced it (labels are not visible in the merge queue).
- name: Modelled code changes update their model
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

## Unreleased

- Regenerate the 11 planted-bug patches (04, 10-17, 19, 20) that no longer
applied, each planting the same fault where the code now lives, and check in
the required `model-sync` job that every planted patch still applies.
- Move the private `backend-experiments`, `io-experiments` and
`profiling-experiments` features out of `spawned-rt` and
`spawned-concurrency` into a standalone, unpublished crate in
Expand Down
26 changes: 22 additions & 4 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -1538,7 +1538,25 @@ The runner applies a patch, runs each layer, records `caught`/`missed` and the
wall time in `OUT/results.tsv`, then reverts the patch and deletes any proptest
seed files the run created (a seed found under one bug must not help the next).
Simulator and fuzzer tapes go to `OUT/tmp`. Patches touch
`concurrency/src` only; they are never applied in CI.
`concurrency/src` only. CI never runs them, but the required `model-sync`
check runs `git apply --check` on every patch, so a PR that moves planted
code must regenerate the affected patches (same fault, new location).

Regenerated after 0.8.0 (the 0.8 lifecycle rewrite and the file splits had
left 11 patches unapplicable), each planting the same fault where the code now
lives: 04 (`supervision.rs`), 10, 12, 13 (`link.rs`), 11 (`link/table.rs`),
14 (`child_handle.rs`), 15, 16 (`monitor/fence.rs`, shared by both modes),
17 (`monitor.rs`), 19 (`tasks/actor/lifecycle.rs`), 20
(`tasks/actor/actor_ref.rs`). Three are adapted to the 0.8 design: 11 keeps
the *caller's* record (the receiver-side fence makes dropping only the peer's
record harmless); 12 delivers `Normal` instead of `NoProc` (0.8 linking to a
dead peer reports `NoProc`, OTP's `noproc`); 14 plants the fault in
`ChildHandle::stop_with_reason(Shutdown)` (`shutdown()` was removed in 0.8).
A spot check with each applied (unit and integration tests only) caught 04,
10, 11, 12, 13, 15, 16 and 19; 15, a miss in every layer below, is now caught
by `demonitor_before_target_dies_suppresses_down` and
`demonitor_releases_waiter` in both modes. The table predates the
regeneration; rerun the matrix before citing rows 04, 10-17, 19 and 20.

**Results** (2026-11-12, after the follow-ups below, `53c0995d`, the 30-core
shared server at `nice 19`, one patch at a time). Times are seconds for the
Expand All @@ -1561,10 +1579,10 @@ cells are from that rerun; the first matrix (without that oracle) missed both.
| 08 | Exhausted budget retires the child instead of failing the supervisor | 212 (19 tests) | 14 | 8 | 2 (V5) | 14 | - |
| 09 | Nested supervisor's restart mode ignored (always Permanent) | 44 (1 test) | 14 | 8 | <1 (V1) | - | - |
| 10 | `Normal` exit kills a non-trapping linked peer | 46 | 15 | 8 | 1 (X3) | - | - |
| 11 | `unlink` removes only the caller's edge | 43 (2 unit tests) | - | - | 17 (X5) | - | - |
| 12 | Linking to a dead peer reports `Normal`, not its reason | 54 | - | 8 | <1 (X3) | - | - |
| 11 | `unlink` is one-sided (the peer's death still signals the caller) | 43 (2 unit tests) | - | - | 17 (X5) | - | - |
| 12 | Linking to a dead peer reports `Normal`, not `NoProc` | 54 | - | 8 | <1 (X3) | - | - |
| 13 | Actor killed through a link exits `Normal` (no transitive propagation) | 43 | 17 | 9 | <1 (X3) | - | - |
| 14 | `ChildHandle::shutdown` commits `Normal` instead of `Shutdown` | 44 | 17 | 9 | <1 (X8) | - | - |
| 14 | An administrative `Shutdown` stop commits `Normal` | 44 | 17 | 9 | <1 (X8) | - | - |
| 15 | Monitor waiter delivers `Down` after `demonitor` removed it | - | - | - | - | - | - |
| 16 | Monitoring an already-dead actor gives no `Down` | 137 | - | 9 | 5 (X1) | - | - |
| 17 | Exiting observer does not cancel its monitor waiters | 59 | 14 | - | - | - | - |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
Planted bug 04 (sup-transient-restarts-on-shutdown): Transient restarts on Shutdown (only Normal treated as clean)

diff --git a/concurrency/src/supervision.rs b/concurrency/src/supervision.rs
index 328ed91b..ff3b2a99 100644
index 3ed4dfe6..6e80f9ce 100644
--- a/concurrency/src/supervision.rs
+++ b/concurrency/src/supervision.rs
@@ -41,7 +41,7 @@ impl RestartMode {
fn restarts(self, reason: Option<&ExitReason>) -> bool {
@@ -54,7 +54,7 @@ impl RestartMode {
match self {
Self::Permanent => true,
- Self::Transient => reason.map(ExitReason::is_abnormal).unwrap_or(true),
+ Self::Transient => reason.map(|r| *r != ExitReason::Normal).unwrap_or(true),
Self::Transient => {
- cleanup_failed || reason.map(ExitReason::is_abnormal).unwrap_or(true)
+ cleanup_failed || reason.map(|r| *r != ExitReason::Normal).unwrap_or(true)
}
Self::Temporary => false,
}
}
22 changes: 11 additions & 11 deletions verification/planted/patches/10-link-normal-exit-kills.patch
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
Planted bug 10 (link-normal-exit-kills): A Normal exit kills a non-trapping linked peer

diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs
index 48cf54a8..a05dd0bd 100644
index ff5b35ef..e350368a 100644
--- a/concurrency/src/link.rs
+++ b/concurrency/src/link.rs
@@ -95,9 +95,6 @@ pub(crate) fn make_signal(
// stop_with_reason(Kill) bypasses the target's trap_exit flag.
// Normal exit signals are silently dropped unless the peer is trapping
let trapping = peer_trap_exit.load(Ordering::Acquire);
- if matches!(reason, ExitReason::Normal) && !trapping {
- return;
- }
if trapping {
// Send Exit message to the peer's mailbox
let exit = Exit {
@@ -307,7 +307,8 @@ fn apply_signal(
let _ = send(exit); // admission may be closed if the peer just died
return;
}
- if exit.reason == ExitReason::Normal {
+ // Planted: only a direct Normal signal is ignored; a Normal link notice kills.
+ if exit.reason == ExitReason::Normal && origin == Origin::Direct {
return;
}
if request.untrapped(exit.reason) {
26 changes: 14 additions & 12 deletions verification/planted/patches/11-link-unlink-one-sided.patch
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
Planted bug 11 (link-unlink-one-sided): unlink removes only the caller's edge; the peer still signals it
Planted bug 11 (link-unlink-one-sided): unlink is one-sided: the caller keeps its record, so the peer's death still signals it

diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs
index 48cf54a8..c68b389b 100644
--- a/concurrency/src/link.rs
+++ b/concurrency/src/link.rs
@@ -192,7 +192,7 @@ pub(crate) fn unregister_link(
if own_id == peer_id || Arc::ptr_eq(own, peer) {
return;
}
- let removed = with_pair(own, peer, |a, b| (remove(a, peer_id), remove(b, own_id)));
+ let removed = with_pair(own, peer, |a, _b| remove(a, peer_id));
drop(removed);
diff --git a/concurrency/src/link/table.rs b/concurrency/src/link/table.rs
index c7ce16e5..f57cc071 100644
--- a/concurrency/src/link/table.rs
+++ b/concurrency/src/link/table.rs
@@ -157,9 +157,7 @@ pub(crate) fn remove_pair(
peer_id: ActorId,
peer: &LinkTable,
) -> (Option<LinkEntry>, Option<LinkEntry>) {
- with_pair(own_id, own, peer_id, peer, |a, b| {
- (remove(a, peer_id), remove(b, own_id))
- })
+ with_pair(own_id, own, peer_id, peer, |_a, b| (None, remove(b, own_id)))
}

/// Receiver-side delivery: under the receiver's table lock, apply the notice
23 changes: 11 additions & 12 deletions verification/planted/patches/12-link-dead-peer-normal.patch
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
Planted bug 12 (link-dead-peer-normal): Linking to an already-exited peer reports Normal instead of its reason
Planted bug 12 (link-dead-peer-normal): Linking to an already-exited peer reports Normal instead of NoProc

diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs
index 48cf54a8..a85bdb9a 100644
index ff5b35ef..45e66107 100644
--- a/concurrency/src/link.rs
+++ b/concurrency/src/link.rs
@@ -154,7 +154,8 @@ pub(crate) fn register_link(
return None;
}
if let Some(reason) = &b.exited {
- return Some(reason.clone());
+ let _ = reason;
+ return Some(ExitReason::Normal);
}
a.entries.push(LinkEntry {
peer_id,
@@ -383,7 +383,7 @@ fn link_in_section(
});
}
let delivery = if outcome == LinkOutcome::NoProc && !try_only {
- deliver_notice_in_section(own_links, own, peer.id, gen, ExitReason::NoProc)
+ deliver_notice_in_section(own_links, own, peer.id, gen, ExitReason::Normal)
} else {
DeliveryCleanup::default()
};
32 changes: 19 additions & 13 deletions verification/planted/patches/13-exit-linked-cause-dropped.patch
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
Planted bug 13 (exit-linked-cause-dropped): An actor killed through a link exits Normal, so the crash stops propagating

diff --git a/concurrency/src/actor_cleanup.rs b/concurrency/src/actor_cleanup.rs
index 69e376ff..0fa2c19d 100644
--- a/concurrency/src/actor_cleanup.rs
+++ b/concurrency/src/actor_cleanup.rs
@@ -72,7 +72,7 @@ impl<C: Fn()> ActorCleanup<C> {
}
}
if matches!(reason, ExitReason::Normal) {
- if let Some(linked) = cause.as_ref() {
+ if let Some(linked) = cause.as_ref().filter(|c| !c.is_abnormal()) {
reason = linked.clone();
}
if reason == ExitReason::Normal {
diff --git a/concurrency/src/link.rs b/concurrency/src/link.rs
index ff5b35ef..cf77cea9 100644
--- a/concurrency/src/link.rs
+++ b/concurrency/src/link.rs
@@ -310,7 +310,13 @@ fn apply_signal(
if exit.reason == ExitReason::Normal {
return;
}
- if request.untrapped(exit.reason) {
+ // Planted: a link notice ends the receiver Normal, dropping the cause.
+ let reason = if origin == Origin::Link {
+ ExitReason::Normal
+ } else {
+ exit.reason
+ };
+ if request.untrapped(reason) {
drop(request);
(target.wake)();
}
23 changes: 13 additions & 10 deletions verification/planted/patches/14-exit-shutdown-reports-normal.patch
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
Planted bug 14 (exit-shutdown-reports-normal): ChildHandle::shutdown commits Normal instead of Shutdown
Planted bug 14 (exit-shutdown-reports-normal): An administrative Shutdown stop (ChildHandle::stop_with_reason(Shutdown); `shutdown()` before 0.8) commits Normal

diff --git a/concurrency/src/child_handle.rs b/concurrency/src/child_handle.rs
index dbcd8611..76b5df14 100644
index 3b94d213..541fee9c 100644
--- a/concurrency/src/child_handle.rs
+++ b/concurrency/src/child_handle.rs
@@ -190,7 +190,7 @@ impl ChildHandle {
/// Request an ordered cooperative shutdown without replacing an earlier
/// committed cause. A cleanup panic remains an abnormal exit.
pub fn shutdown(&self) {
- crate::link::stop_with_reason(&self.linked_reason, ExitReason::Shutdown, || {
+ crate::link::stop_with_reason(&self.linked_reason, ExitReason::Normal, || {
(self.cancel)()
@@ -170,6 +170,11 @@ impl ChildHandle {
/// runs. `stop_with_reason(Kill)` ends `Kill`; use [`kill`](Self::kill)
/// for the untrappable kill signal that ends `Killed`.
pub fn stop_with_reason(&self, reason: ExitReason) {
+ let reason = if reason == ExitReason::Shutdown {
+ ExitReason::Normal
+ } else {
+ reason
+ };
crate::link::stop_with_reason(self.target.id, &self.target.request, reason, || {
(self.target.wake)()
});
}
52 changes: 39 additions & 13 deletions verification/planted/patches/15-mon-down-after-demonitor.patch
Original file line number Diff line number Diff line change
@@ -1,15 +1,41 @@
Planted bug 15 (mon-down-after-demonitor): The monitor waiter delivers Down even when demonitor already removed it

diff --git a/concurrency/src/tasks/actor.rs b/concurrency/src/tasks/actor.rs
index a7fb0ec2..09ec5c06 100644
--- a/concurrency/src/tasks/actor.rs
+++ b/concurrency/src/tasks/actor.rs
@@ -458,7 +458,7 @@ impl<A: Actor> Context<A> {
.as_mut()
.and_then(|entries| entries.remove(&monitor_ref))
.is_some();
- if deliver {
+ if deliver || reason.is_abnormal() {
let _ = actor_ref.send(Down {
monitor_ref,
reason,
diff --git a/concurrency/src/monitor/fence.rs b/concurrency/src/monitor/fence.rs
index f7bbaa27..d7778cf3 100644
--- a/concurrency/src/monitor/fence.rs
+++ b/concurrency/src/monitor/fence.rs
@@ -83,7 +83,7 @@ pub(crate) fn register<K: Hash + Eq, V>(
/// and enqueue the `Down` (`deliver`) in one critical section, so once
/// [`demonitor`] returns no `Down` of this monitor is enqueued, and one
/// already enqueued is flushed. Returns whether it was delivered.
-pub(crate) fn deliver<K: Hash + Eq, V>(
+pub(crate) fn deliver<K: Hash + Eq + Clone, V>(
table: &Entries<K, V>,
key: &K,
deliver: impl FnOnce(),
@@ -95,7 +95,14 @@ pub(crate) fn deliver<K: Hash + Eq, V>(
*entry = Entry::Queued;
true
}
- _ => false,
+ Some(Entry::Queued) => false,
+ // Planted: a removed (demonitored) entry is delivered anyway.
+ None => {
+ if let Some(entries) = guard.as_mut() {
+ entries.insert(key.clone(), Entry::Queued);
+ }
+ true
+ }
};
if active {
deliver();
@@ -155,7 +162,8 @@ pub(crate) fn demonitor<K: Hash + Eq, V>(
let entry = guard.as_mut().and_then(|entries| entries.remove(key))?;
removed();
match entry {
- Entry::Active(value) => Some(value),
+ // Planted: the waiter is not woken, so it still delivers.
+ Entry::Active(_value) => None,
Entry::Queued => None,
}
}
29 changes: 16 additions & 13 deletions verification/planted/patches/16-mon-dead-target-no-down.patch
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
Planted bug 16 (mon-dead-target-no-down): Monitoring an already-dead actor returns an inert ref and no Down

diff --git a/concurrency/src/tasks/actor.rs b/concurrency/src/tasks/actor.rs
index a7fb0ec2..92ed728e 100644
--- a/concurrency/src/tasks/actor.rs
+++ b/concurrency/src/tasks/actor.rs
@@ -424,7 +424,7 @@ impl<A: Actor> Context<A> {
// The token check makes registration inert once shutdown starts.
// The closed table below serializes registration with exit cleanup,
// even on paths that unwind before cancelling the owner token.
- if self.shared.cancellation_token.is_cancelled() {
+ if self.shared.cancellation_token.is_cancelled() || !target.is_alive() {
return monitor_ref;
}
let Some(monitors) = monitors.as_mut() else {
diff --git a/concurrency/src/monitor/fence.rs b/concurrency/src/monitor/fence.rs
index f7bbaa27..d40d7c7e 100644
--- a/concurrency/src/monitor/fence.rs
+++ b/concurrency/src/monitor/fence.rs
@@ -70,9 +70,8 @@ pub(crate) fn register<K: Hash + Eq, V>(
return Registration::Inert;
};
if target_dead() {
- late();
- entries.insert(key, Entry::Queued);
- return Registration::Late;
+ let _ = late;
+ return Registration::Inert;
}
entries.insert(key, Entry::Active(value));
active();
12 changes: 6 additions & 6 deletions verification/planted/patches/17-mon-cleanup-keeps-waiters.patch
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
Planted bug 17 (mon-cleanup-keeps-waiters): An exiting observer does not cancel its monitor waiters (they hold its handle until the target dies)

diff --git a/concurrency/src/monitor.rs b/concurrency/src/monitor.rs
index 578f835c..40a53974 100644
index 2a7c5d66..e4459eda 100644
--- a/concurrency/src/monitor.rs
+++ b/concurrency/src/monitor.rs
@@ -69,9 +69,7 @@ impl Drop for MonitorCleanup {
#[cfg(spawned_sim)]
let monitors: std::collections::BTreeMap<u64, _> =
monitors.into_iter().map(|(r, t)| (r.0, t)).collect();
- for token in monitors.into_values() {
@@ -94,9 +94,7 @@ impl Drop for MonitorCleanup {
monitors.sort_unstable_by_key(|(r, _)| r.0);
monitors
};
- for (_, token) in monitors {
- token.cancel();
- }
+ std::mem::forget(monitors);
Expand Down
29 changes: 15 additions & 14 deletions verification/planted/patches/19-mb-handle-after-stop.patch
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
Planted bug 19 (mb-handle-after-stop): The run loop keeps handling queued messages after stop (D4)

diff --git a/concurrency/src/tasks/actor.rs b/concurrency/src/tasks/actor.rs
index a7fb0ec2..200d1e8c 100644
--- a/concurrency/src/tasks/actor.rs
+++ b/concurrency/src/tasks/actor.rs
@@ -1273,9 +1273,6 @@ async fn run_actor_lifecycle<A: Actor>(
// Stop and closed-mailbox both take the normal lifecycle exit,
// just like the cancellation arm below. Check before consuming
// work, including when receive resumes from a pending poll.
- if cancellation_token.is_cancelled() {
- return std::task::Poll::Ready(None);
- }
let result = rx.poll_recv(cx);
#[cfg(feature = "backend-experiments")]
if let Some(turn) = &mut turn {
diff --git a/concurrency/src/tasks/actor/lifecycle.rs b/concurrency/src/tasks/actor/lifecycle.rs
index a88489ac..443f67ff 100644
--- a/concurrency/src/tasks/actor/lifecycle.rs
+++ b/concurrency/src/tasks/actor/lifecycle.rs
@@ -443,9 +443,7 @@ pub(super) async fn run_actor_lifecycle<A: Actor, C: Fn()>(
#[cfg(all(test, not(feature = "lifecycle-trace"), not(spawned_sim)))]
stop_receive_probe::before_receive();
let received = rx.poll_recv(cx);
- let result = cancellation_token.after_receive(received, |item| {
- rx.hold_for_shutdown(item);
- });
+ let result = received;
#[cfg(feature = "lifecycle-trace")]
if let std::task::Poll::Ready(Some(envelope)) = &result {
crate::lifecycle_trace::life_emit(trace_id, || {
Loading
Loading