Visitar URL original
docs(examples): add runnable index sync example (updates, deletes, folder sync) (#199) by webdevsamran · Pull Request #201 · priya-sundaram-dev/whoosh · GitHub
Skip to content

docs(examples): add runnable index sync example (updates, deletes, folder sync) (#199) - #201

Merged
priya-sundaram-dev merged 1 commit into
priya-sundaram-dev:mainfrom
webdevsamran:docs/issue-199-index-sync-example
Sep 19, 2026
Merged

priya-sundaram-dev merged 1 commit into
priya-sundaram-dev:mainfrom
webdevsamran:docs/issue-199-index-sync-example

Conversation

@webdevsamran

Copy link
Copy Markdown
Contributor

Summary

Adds examples/index_sync.py, a runnable, standalone script demonstrating the three core patterns for keeping a Whoosh index synchronized with changing source data:

  1. Upsert (update_document): Shows that indexing on a unique=True ID field replaces an existing document rather than duplicating it, while adding new documents seamlessly.
  2. Delete (delete_by_term): Removes documents by unique key and confirms they are completely excluded from search results.
  3. Folder Sync (sync_folder): A 3-pass reconciliation demo comparing stored file modification times against files on disk:
    • Pass 1 (Initial sync): indexes new files.
    • Pass 2 (Incremental sync): detects modified files, adds new files, and deletes removed files in a single atomic commit.
    • Pass 3 (Idempotent check): confirms zero work is done when files haven't changed.

Also:

  • Added an ## Index management section to examples/README.md housing index_sync.py and resource_management.py.
  • Added "index_sync.py" to the _RUNNABLE allowlist in tests/test_examples_runnable.py so CI automatically smoke-tests it end-to-end.
  • Updated CHANGELOG.md under ## [Unreleased].

Related issues

Fixes #199

Checklist

  • Tests pass locally (pytest tests/)
  • Added/updated tests for the change where it makes sense
  • Updated docs / CHANGELOG if user-facing
  • I have read the CONTRIBUTING guide

Authored with AI assistance via Antigravity (DeepMind agentic coding system) with complete human validation, testing, and DCO sign-off.

…lder sync) (priya-sundaram-dev#199)

Signed-off-by: Samran Asif <samranwebdev2000@gmail.com>

@priya-sundaram-dev priya-sundaram-dev left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is excellent work, @webdevsamran — thank you! I ran it locally and read through carefully:

  • The three patterns are each demonstrated crisply, and the output is genuinely instructive (the Query 'jumps' matches: ['doc1.txt'] line proving no duplicates is a nice touch).
  • sync_folder batching adds/updates/deletes into a single atomic commit is exactly the right pattern, and the Pass 3 idempotent no-op check is a thoughtful way to show reconciliation converges.
  • Everything runs in RamStorage + a throwaway tempdir, so it leaves nothing behind — and you wired it into test_examples_runnable.py so it can't rot.
  • Creating a dedicated Index management section in the README (and folding resource_management.py into it) is a genuine improvement to the docs structure.

All 17 CI checks pass. Merging. Really appreciate the care you put into this.

@priya-sundaram-dev
priya-sundaram-dev merged commit d43f8db into priya-sundaram-dev:main Sep 19, 2026
17 checks passed
@priya-sundaram-dev

Copy link
Copy Markdown
Owner

Merged — thank you again, @webdevsamran! That's two solid contributions in a row (the whoosh.index typing in #200 and now this runnable index-sync example). The examples/ directory is meaningfully friendlier for it.

If you'd like a third, #195 (typing the remaining IndexWriter base-class helpers in whoosh.writing) is a natural follow-on to your #200 work, and #121 tracks the broader ty cleanup. No pressure either way — really appreciate you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a runnable example: keeping a Whoosh index in sync (updates & deletes)

2 participants