Skip to content

fix(fonts): paragraph direction skips bidi isolates; harakat no longer block the Lam-Alef ligature - #1034

Merged
skylartaylor merged 1 commit into
storytold:mainfrom
vangel76:fix/fonts-bidi-hardening
Nov 9, 2026
Merged

skylartaylor merged 1 commit into
storytold:mainfrom
vangel76:fix/fonts-bidi-hardening

Conversation

@vangel76

Copy link
Copy Markdown
Contributor

Fixes #1033

Two fixes in crates/fonts/src/bidi.rs only.

Causes and fixes

  • Paragraph direction looked inside isolates. base_rtl took the first L/R/AL character anywhere, so "\u{2067}שלום\u{2069} abc" gave Some(true). UBA P2 skips everything between an isolate initiator (LRI/RLI/FSI) and its matching PDI, or to the end of the paragraph when unmatched. It now tracks isolate depth (saturating, so a stray PDI is harmless) and only considers strong characters at depth 0. The example gives Some(false).
  • Harakat blocked the Lam-Alef ligature. presentation_forms required the Alef to be the next code point. Marks are transparent for joining (joining_forms already skips them), so Lam + fatha + Alef kept two separate glyphs. After a Lam it now looks past combining marks for the Alef, emits the ligature, then the skipped marks (logical order, each with its own source index), then continues after the Alef. A non-mark between Lam and Alef still blocks the ligature. The scan only runs for a Lam.

Tests (in bidi.rs)

  • paragraph_direction_skips_isolate_contents: the example, LRI/FSI variants, nested isolates, unmatched initiator, stray PDIs, no strong character outside the isolates, unchanged plain case.
  • harakat_between_lam_and_alef_keep_the_ligature: Lam+fatha+Alef, medial with fatha+shadda, plain Lam+Alef unchanged, mark after the Alef, Lam+mark+non-Alef, Lam+mark at the end, non-mark between Lam and Alef.
  • Both tests fail without their fix: I temporarily disabled each fix, ran cargo test -p cadcraft-fonts --lib bidi, saw both tests fail, then restored the fixes.

Commands that ran

All with a separate CARGO_TARGET_DIR:

  • cargo fmt -p cadcraft-fonts (a --check first reported one wrapping diff in my code, now formatted)
  • cargo clippy -p cadcraft-fonts --all-targets -- -D warnings: clean
  • cargo test -p cadcraft-fonts: 55 unit + 9 integration tests pass
  • cargo test -p cadcraft-render: all pass
  • cargo xtask layers: OK
  • cargo check --target wasm32-unknown-unknown -p cadcraft-fonts: OK

Not run: the full cargo xtask ci and workspace-wide builds/tests (shared machine; only the touched crate and its direct consumer were checked).

Left out

🤖 Generated with Claude Code

…r block the Lam-Alef ligature (storytold#1033)

base_rtl now ignores strong characters between an isolate initiator and its PDI (or the end, when unmatched), per UBA P2. presentation_forms looks past combining marks for the Alef after a Lam and emits the marks after the ligature.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@FredApps FredApps left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReviewerA — source-read COMMENT (not an approval)

I read the diff: the paragraph direction skips bidi isolates, and harakat no longer block the Lam-Alef ligature. The function that finds the first strong character used find_map over the characters and now, "as rule P2 requires", skips the characters between an isolate initiator and its matching PDI (or the end of the paragraph when there is no match). UAX #9 rule P2 does say that characters between an isolate initiator and its matching PDI are ignored when finding the first strong character, so this is the specified behaviour, and it matters for text like an isolated English name inside Arabic. Isolate nesting needs a depth counter and the unmatched-PDI case needs care (a PDI with no initiator is not an end of anything); I saw the doc comment and the start of the loop, not the counter. The second half of the title, harakat (Arabic diacritics) between Lam and Alef blocking the ligature, is in a part I did not read; a test with the Arabic string for "no" with a fatha between them would pin it. I have not run it, and no exact-head CI result exists for this head. Nothing was run.

I ran nothing and have no exact-head CI result for this PR, so this is not an approval. Where a diff was long I read its code hunks rather than every generated or documentation line.

Status: waiting_author.

@austin62jake austin62jake left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I merged it onto current main with seven other PRs (#1460, #1452, #1038, #1020, #1034, #1588, #1673, #1292). fmt, clippy and the web build pass, and 3188 of 3189 tests pass. The one failure is #1452's missing Chinese label.

Skipping characters inside an isolate (LRI/RLI/FSI…PDI, or to the end when unmatched) when finding the paragraph direction is rule P2. Letting harakat between lam and alef keep the ligature, staying after it in logical order, is how Arabic joins. It doesn't conflict with my #2618, which sets single-line TEXT's base direction explicitly.

@FredApps FredApps left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed c32c0f5e30eecdfe6dce20c5a0948eed4e4293fb. No findings in the complete one-file source review. The isolate-depth counter saturates at both ends, so nested/unmatched isolates and stray PDI controls are handled without underflow. The Lam–Alef lookahead skips combining marks and preserves their source indices and logical order. The new tests already cover nested/unmatched isolates, stray PDI, intervening harakat, word-medial joining and non-Alef boundaries. This supersedes my earlier partial review, which had not inspected those paths.

This is a review comment, not an approval: GitHub reports no checks for this head, and no local Rust tests were run because the available workspace is out of disk space. Please refresh the branch onto the current CI-enabled base and obtain a passing gate, or provide reproducible validation for this exact head. The missing validation is the remaining gap; I am not asking for duplicate tests that are already in the diff.

@skylartaylor

Copy link
Copy Markdown
Member

Same as #1273, CI never ran here, so I merged it onto current main (909af5e3) with this batch and ran everything on Linux. The workspace tests (3,732) and clippy on cadcraft-fonts pass. Merging.

@skylartaylor
skylartaylor merged commit 4744b4c into storytold:main Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(fonts): paragraph direction looks inside bidi isolates; harakat block the Lam-Alef ligature

4 participants