[viranovskaya]

Closes #88.

I added an opt-in interval-based comparison, following the scope agreed in the issue. This makes it possible to compare complete event intervals rather than only their start times, including detections with different sampling frequencies.

method="distance" remains the default, with its existing behavior and positional calling conventions unchanged.

With method="intersection":

• Inputs are start/end intervals, compared directly in their common units without rounding or resampling.
• A pair must overlap for a positive duration and have IoU >= iou_threshold (default 0.2), allowing a documented relative roundoff tolerance of 1e-9 and no absolute tolerance.
• Matching is one-to-one and maximizes the number of valid pairs, not their summed IoU. Split, merged or duplicate detections therefore cannot all count as true positives against the same reference event.
• The SpindlesResults, SWResults and REMResults detection-comparison wrappers use start/end times in seconds and retain per-channel comparison. Spindle and slow-wave compare_channels also support the new method; REMs still represent one combined channel and do not support inter-channel comparison.
• max_distance / max_distance_sec must be zero in intersection mode, including numeric 0.0.
• TP/FP/FN interval arrays preserve the endpoint dtype of their corresponding input. Invalid intersection inputs raise ValueError, including under Python -O; wrapper errors identify self or other independently of the scoring direction.

The implementation builds a sparse overlap graph rather than sample-wise masks or a dense pairwise matrix. The new parameters are keyword-only. Both wrappers share _intervals_by_channel, and interval validation is shared with the standalone function. Docstrings and executable examples are updated. The changelog change has been removed, as requested.

Tests cover threshold boundaries, decimal endpoints at the beginning and end of a 24-hour recording, touching and nested intervals, split/merged/duplicate events, a case where greedy matching loses a valid pair, empty and invalid inputs, endpoint dtypes, reversed scoring, channel separation, inter-channel precision/recall orientation and sub-100 ms endpoints.

Fresh local validation: 739 tests passed (two existing pytest fixture warnings), 85 intersection checks passed under Python -O (one expected pytest warning), 8 doctests passed and 4 skipped; Ruff and the full Sphinx HTML build with -E -a -W --keep-going passed. I also checked 800 small exact-rational cases and their reversed comparisons against exhaustive one-to-one assignments. These are local checks; cross-platform and minimum-dependency validation of the revised patch remains for CI.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.60%. Comparing base (e684111) to head (0dbdd9a).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #278      +/-   ##
==========================================
+ Coverage   98.57%   98.60%   +0.02%     
==========================================
  Files          13       13              
  Lines        3093     3159      +66     
  Branches      416      429      +13     
==========================================
+ Hits         3049     3115      +66     
  Misses         23       23              
  Partials       21       21              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[raphaelvallat]

Hi @viranovskaya, thank you for your contribution. Can you please address /reply to these comments from an AI review? Can you also please remove the changelog entry? I typically update the changelog based on the merged PRs just before a version release.

1. detection.py:2811: The IoU check uses a strict >= on floats, so a pair whose true IoU equals the threshold can flip between TP and FP/FN depending on rounding. For example, [0.1, 0.4] vs [0.3, 0.6] gives an IoU of 0.2000000000000001. The docstring says "IoU == threshold counts", and the existing test only uses integer endpoints, so it doesn't catch this. A small tolerance or np.isclose would fix it.
2. detection.py:591: The End-column check only runs on other and raises a bare assert. When other_is_groundtruth=False, the finite and start<end error messages name the wrong side ("detection" when the bad data is in other). These checks also vanish under python -O.
3. detection.py:2968: iou_threshold is validated with assert, while method and max_distance raise ValueError. A string value like '0.5' raises a confusing TypeError instead of the intended message.
4. detection.py:2973: In intersection mode, max_distance=0.0 (a float) hits the earlier int-only check and fails with "must be 0 or a positive integer". The intersection-specific check never runs.
5. detection.py:2955: Intersection mode always casts inputs to float, so sample-index intervals come back as float tp/fp/fn arrays. Distance mode returns ints, and float values break downstream indexing.
6. detection.py:481: compare_channels didn't get method/iou_threshold, so you can't compute IoU-based agreement between channels.
7. detection.py:594: The per-channel interval grouping is an inline nested generator. A _intervals_by_channel staticmethod next to _starts_by_channel would read better and could be reused by Add bandpass=True to yasa.bandpower #7.

[raphaelvallat]

Thanks for this PR @viranovskaya! I pushed a few follow-up commits (6c4bf73, 6dc7a5e, 0dbdd9a) on top of your branch. Here is a summary of the changes.

Performance

• _match_intervals no longer uses a Python endpoint sweep. Candidate pairs are now found with sorted searches (np.searchsorted), using the fact that in any overlapping pair, one interval starts within the other. The edge set passed to maximum_bipartite_matching is unchanged. On 20,000 events per side, a single comparison goes from ~0.32 s to ~0.006 s, which matters for compare_channels on many channels.
• compare_channels no longer re-sorts the events of each channel for every pair of channels. Events are sorted once per channel, and _match_intervals searches in sorted order, which also makes np.searchsorted much faster. The diagonal (a channel compared with itself, always 1) is no longer computed. On 64 channels with 5,000 events each, compare_channels(method="intersection") goes from 11.8 s to 4.8 s, with identical scores.

Validation

• _starts_by_channel and _intervals_by_channel are merged into a single _events_by_channel(df, method, name) helper. It now:
  • raises a ValueError for missing columns in both methods (the distance method previously relied on bare asserts),
  • raises a ValueError when the Channel column has missing values, instead of silently dropping these events from the comparison,
  • supports nullable and pyarrow-backed Start/End columns, e.g. from pd.read_csv(..., dtype_backend="numpy_nullable").
• The method / max_distance checks, previously repeated in three places, are now in a single _check_method helper.
• The fallback error message for an invalid other now mentions the End column.

Behaviour change (distance method)

• max_distance_sec is now rounded to the nearest decisecond instead of truncated. For example, max_distance_sec=0.29 is now 300 ms instead of 200 ms.

Documentation

• The docstrings of the new method and iou_threshold parameters are shorter and focus on what each method does. The full explanation is in yasa.compare_detection, and the methods of the detection results classes refer to it.

Tests

• The new tests are consolidated into two:
  • test_compare_detection_intersection checks the matching against a brute-force dense IoU matrix on 200 random cases, plus key cases: maximum matching vs greedy, unsorted inputs, one-to-one matching, inclusive threshold at large absolute times, touching intervals, empty inputs and invalid inputs.
  • test_results_compare_intersection covers the compare_detection and compare_channels methods: unrounded endpoints, per-channel scores, other_is_groundtruth, nullable columns and DataFrame validation errors.
• All new code is covered.

🤖 Generated with Claude Code

[raphaelvallat]

Distance vs intersection: how they differ and when to use each

How each method decides two events are the same

Distance looks only at when each event starts. It rounds both lists of start times to the nearest 100 ms and calls a detected event a hit if any reference event starts within max_distance_sec of it. Durations and end times are ignored.

Intersection looks at the whole event, from start to end. For each pair of events it computes the IoU (intersection over union): how long they overlap, divided by the total time covered by either one. A pair counts as a match if that ratio reaches iou_threshold (0.2 by default). Each event can match at most one event on the other side, and YASA picks the pairing that gives the most matches overall.

Practical differences

	Distance	Intersection
Needs from the reference	Start only	Start and End
Time precision	Rounded to 100 ms	Exact, no rounding
Event duration	Ignored	Central to the score
One event matching several	Allowed	Not allowed (strictly one-to-one)
Events whose starts round to the same 100 ms (same channel)	Counted as one event	Kept as separate events
Nearby events that don't overlap	Can match if within max_distance_sec	Never match
Tuning knob	max_distance_sec	iou_threshold

Three examples:

• A long detection covering two short reference spindles. Distance can count both reference spindles as found if their starts are close enough. Intersection matches at most one of them, so you get one hit and one miss. That is usually the fairer answer.
• The same onset but very different durations (0.5 s vs 3 s). Distance calls it a match. Intersection gives an IoU of about 0.17, which is a miss at the default threshold of 0.2.
• Boundaries offset by a small amount (a detection at 10.0–10.8 s and a reference at 10.9–11.5 s). Distance with max_distance_sec=1 calls it a match. Intersection never does, because the two events don't overlap.

Pros and cons

Distance

• Pros:
  • It works when the reference only marks a single point in time, such as an onset or a slow-wave trough.
  • It forgives annotators who disagree about where an event ends.
  • It is the existing behaviour, so results stay comparable with older analyses.
• Cons:
  • It ignores duration, so a detection that is far too long or too short still counts as correct.
  • Because one event can match several, precision and recall can be inflated.
  • The 100 ms rounding hides differences smaller than that.
  • Two events on the same channel that start within the same 100 ms window are merged into one.

Intersection

• Pros:
  • It checks that the event's position and its extent both agree.
  • One-to-one matching gives true positive, false positive and false negative counts that add up properly, so precision, recall and F1 are better behaved.
  • There is no rounding, and the two recordings don't need the same sampling rate.
  • It follows the usual way spindle detectors are evaluated, where an overlap threshold of 0.2 is a common choice.
• Cons:
  • It needs end times you trust. Short events like spindles or REMs are sensitive to annotators placing the boundaries differently, and a few hundred milliseconds can push the IoU under the threshold.
  • Results depend on the threshold, so report the value you used. It is worth checking results at a couple of thresholds, for example 0.2 and 0.5.
  • It can't handle point annotations.
  • The matching maximizes the number of pairs, not their total overlap. When several equally good pairings exist, which specific events get matched can vary, but the counts and scores don't.

When to use which

• Use intersection when both sides have reliable start and end times, for example when comparing a detector with expert-scored spindles or slow waves, or when tuning detection thresholds against a ground truth. It is the more rigorous choice, and probably the better default for new analyses that have full annotations.
• Use distance when the reference only has onsets or peaks, when boundaries are known to be unreliable, or when you need results that stay comparable with earlier YASA-based work.

Scaling to long recordings and many channels

Per comparison, intersection does roughly what distance does: it sorts the events and does binary searches, then pairs up the events that overlap. In sleep data each event overlaps only a few others, so the cost grows close to linearly with the number of events. There is no full events × events table, so memory stays small. After the follow-up commit, 20,000 events against 20,000 events took about 6 ms, down from about 320 ms with the original sweep. That is roughly the number of slow waves in a full night of high-density EEG.

The worst case is very long reference events, such as an artifact annotation lasting several minutes that covers hundreds of detections. Each long event creates many candidate pairs, so the cost grows with the number of pairs. It stays correct and doesn't use a lot of memory, but it is slower. That isn't the normal situation for spindles, slow waves or REMs.

Across channels, compare_channels compares every channel with every other channel in both directions, so the number of comparisons grows with the square of the channel count. The same is true for both methods.

Channels	Comparisons
8	64
64	4,096
256	65,536

Two follow-up changes (6dc7a5e) reduce the cost of each comparison: the events of each channel are now sorted once instead of once per comparison, and the diagonal (a channel compared with itself, always 1) is skipped. On 64 channels with 5,000 events each, compare_channels now takes about 4.8 s with method="intersection" (down from 11.8 s) and about 6.3 s with method="distance". That is fine for a one-off analysis. Fewer events per channel, which is typical for spindles, make it much faster. These changes reduce the constant factor, not the squared growth with channel count.

compare_detection against a ground truth runs one comparison per channel, not one per pair, so it is cheap even on high-density EEG.

The distance method doesn't benefit much from sorting once, because the public compare_detection still checks and sorts its inputs on every call. I'll address this in a separate PR, since it touches the existing distance code path.

🤖 Generated with Claude Code