Fix document replacement, watcher synchronization, and search weighting

.github/workflows/regressions.yml

@@ -0,0 +1,22 @@
+name: Document lifecycle regressions
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+jobs:
+  regression-tests:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python: ['3.11', '3.13']
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - name: Install storage and extraction test dependencies
+        run: python -m pip install pytest numpy 'lancedb>=0.25' pyarrow typer pyyaml watchdog pillow pymupdf python-docx openpyxl python-pptx
+      - name: Test without model downloads or autostart changes
+        run: python -m pytest tests/test_document_lifecycle.py tests/test_watchlock.py tests/test_new_features.py -q

README.md

@@ -135,9 +135,13 @@ The installer copies the skill to `~/.hermes/skills/research/rag-kit/SKILL.md`.
 ```bash
 cd rag-kit
 
-# Run the full suite (66 tests, no model downloads needed)
+# Full suite: CLI integration and VLM tests require locally cached models.
+# Autostart tests change OS startup settings; run those only in a disposable environment.
 python -m pytest tests/ -v
 
+# Storage/extraction regressions: no model downloads or autostart changes.
+python -m pytest tests/test_document_lifecycle.py tests/test_watchlock.py tests/test_new_features.py -q
+
 # Run a specific module
 python -m pytest tests/test_rag_pipeline.py -v   # core pipeline + version check
 python -m pytest tests/test_ingest_pipeline.py -v
@@ -210,6 +214,20 @@ rag ingest ~/Documents/rag-ingest
 
 ## Changelog
 
+### Unreleased — document lifecycle fixes
+
+- Re-importing a document atomically replaces its previous chunks. Empty documents remove previous content; extraction and embedding failures retain it.
+- All text and visual chunks share a resolved source path and consistent page numbering. Windows CLI output uses UTF-8 when captured by agents.
+- An explicit `RAG_KIT_CONFIG` path is respected even before the file exists, so creating a separate config does not edit the user's default config.
+- Watch startup reconciles changes made while stopped. File and directory moves/deletions remove old sources; moves outside the watched folder are not ingested.
+- Watcher ownership uses an OS file lock, released when the process exits. Lock files remain on disk and are safe to reuse.
+- Existing indexes gain missing metadata columns without dropping documents. Changing embedding dimensions requires a separate database.
+- Long-lived database readers see commits from other processes, including watcher updates; queries tolerate a first ingestion that is still creating its table.
+- `search_alpha` weights reciprocal-rank fusion and retains zero for keyword-only search, including source-filtered queries. LanceDB 0.25+ is required for native full-text indexing.
+- Cached sentence-transformers snapshots load directly, including short model names in the standard namespaced Hub cache.
+
+New imports replace sources stored under the same absolute path. Older releases may have left relative paths or basename-only visual chunks; review `rag list-files --json` and remove those obsolete source entries explicitly after re-importing. Basename-only entries cannot be safely assigned to a document when several files share a name.
+
 ### v0.1.1 — RAG overhaul
 
 - **Hybrid search**: FTS + RRF reranker now blended into `rag query`, tuned via `search_alpha`; store dimension derived from the embedding model instead of hardcoded

pyproject.toml

@@ -31,7 +31,7 @@ dependencies = [
     "pymupdf>=1.24",
     "python-docx>=1.1",
     "sentence-transformers>=3.0",
-    "lancedb>=0.6",
+    "lancedb>=0.25",
     "transformers>=4.54,<6",
     "tokenizers>=0.22.0,<0.23.1",
     "pillow>=10.0",
@@ -46,8 +46,8 @@ vlm = ["optimum[onnxruntime]>=1.20"]  # optional ONNX fallback for VLM (PyTorch
 dev = ["pytest>=7.0", "pytest-cov"]
 
 [project.scripts]
-rag = "rag_kit.cli.main:app"
-rag-kit = "rag_kit.cli.main:app"
+rag = "rag_kit.cli.main:run_cli"
+rag-kit = "rag_kit.cli.main:run_cli"
 
 [project.urls]
 Homepage = "https://github.com/lychee888/Rag-Kit"