Repository navigation
Integrate rustedbytes-tl 0.3 through an explicit optional parser - #18
Merged
Merged
Conversation
Merge current master and add an optional TL adapter with compatibility tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #17 with a tested optional integration of the now-published rustedbytes-tl 0.3.0.
Add the non-default tl-parser feature, owned TlDocument and borrowing TlElement APIs, plus HtmlResponse::tl_document(). The response adapter snapshots the existing decoded, byte-limited source; callers keep the document for repeated and nested queries. It uses TL's safe owned constructor and preserves selector errors separately from empty results. Text decoding is enabled; raw_attr deliberately exposes raw attribute spelling. No unsafe code is introduced.
Existing scraper-based selection, compiled scraper::Selector APIs, utility return types, XPath and default dependency behavior remain unchanged. This is an explicit alternative, not transparent HTML5 equivalence: omitted tbody, malformed-tree recovery, raw attributes, serialization and unsupported CSS still differ. There is no automatic fallback. Test site fixtures before opting in.
Update the independent evaluation to exact published TL 0.3.0 + entities. The catalog's 14 checked selectors and decoded entity text now agree with scraper 0.27.0; retain tests demonstrating remaining differences. Document ownership, lifetimes, memory/parse boundaries and limitations. Add TL feature checks and a path-filtered independent evaluation workflow. Historical timings are clearly labeled as TL 0.2.0, not performance evidence for this adapter.
Validation:
No new timing, allocation or whole-crawler speedup claim is made. A broad real-site differential corpus and complete-adapter benchmarks remain prerequisites for replacing scraper as the default. GitHub Actions results are pending.