Skip to content

Reject implausible temporal year matches - #3259

Open
koriyoshi2041 wants to merge 2 commits into
vectorize-io:mainfrom
koriyoshi2041:rios/fix-implausible-temporal-years
Open

Reject implausible temporal year matches#3259
koriyoshi2041 wants to merge 2 commits into
vectorize-io:mainfrom
koriyoshi2041:rios/fix-implausible-temporal-years

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Problem

Dateparser treats bare integers such as port and ticket numbers as years. Because digit-bearing matches receive the highest temporal score, a query like port 9077 produces a non-null year-9077 constraint and silently empties the temporal retrieval arm.

Closes #3250.

Fix

Filter parsed candidates to a reference-relative plausibility window before selecting the strongest match. Filtering before max() preserves a genuine relative or explicit date elsewhere in the same query.

Test

  • uv run pytest tests/test_query_analyzer.py -q (429 passed)
  • ./scripts/hooks/lint.sh
  • repository pre-commit hooks
  • git diff --check

Risk

Low to moderate. Dates more than 120 years in the past or 20 years in the future are no longer inferred by the fallback dateparser path. Explicit period parsing remains unchanged, and the bounds are clamped for low/high reference years.

@ebarkhordar ebarkhordar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both cases from #3250 do go away at 29ac311: port 9077 and job 4417 return None.

The window is applied to every surviving match rather than to the bare-integer class specifically, so a fully written-out date outside it loses its constraint too. Reference date 2026-08-07, main vs this branch:

notes from 1890-03-05                 1890-03-05  ->  None
the roadmap milestone on 2050-01-15   2050-01-15  ->  None
the mortgage matures on 2047-01-15    2047-01-15  ->  None
the mortgage matures on 2046-01-15    2046-01-15  ->  2046-01-15

So the future edge sits exactly at reference_date.year + 20. Twenty years is inside the range of dates a user might genuinely have written down (a mortgage maturity, a long-dated contract), and unlike a port number those spans carry a day and a month.

The obvious narrowing does not work, which took a run to establish, so it seems worth recording here. Carrying the matched span into the tuple and applying the window only when span.strip().isdigit() does keep 1890-03-05 and 2050-01-15, and still drops the plain 9077 from the issue's query. It fails your own test_query_analyzer_keeps_plausible_date_with_implausible_number: for port 9077 and also last Tuesday, search_dates returns the span as '9077 and' rather than '9077', so the digit test misses it and 9077 wins the ranking again. The span boundary moves with the neighbouring word, so anything keyed on the span text alone is fragile in the same way.

Is dropping explicit out-of-window dates intended? If it is, a line saying so in the comment would be worth having, since the comment currently describes the bare-integer case only.

Checked at 29ac311 in a clean python:3.12-slim container with an editable install of hindsight-api-slim. tests/test_query_analyzer.py is 429 passed on the branch, and 428 passed / 1 failed with the isdigit variant above.

@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Good catch. I narrowed the window to isolated year matches, including spans where dateparser absorbs padding such as 9077 and. Full dates and month/year expressions outside the window now remain valid.

Added regressions for 1890-03-05, 2050-01-15, and March 1890; the complete query-analyzer suite passes (432 tests). Updated in 6d426a0.

@koriyoshi2041
koriyoshi2041 force-pushed the rios/fix-implausible-temporal-years branch from 6d426a0 to 2a1d4ed Compare August 14, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Bare integers score as the strongest date signal, producing implausible-year temporal constraints (same class as #2768)

2 participants