Whoosh

Warnings

• gotcha When adding documents, ensure text fields are passed as Unicode strings (e.g., `u"my text"` in Python 2 or regular strings in Python 3). Non-text fields that are stored but not indexed (STORED type) can be any pickle-able object.
  Fix: Prefix string literals with 'u' in Python 2 (e.g., `u'your string'`). In Python 3, all strings are Unicode by default, so `"your string"` is sufficient.
• gotcha Whoosh does not inherently enforce uniqueness for documents. Calling `add_document` multiple times with identical data will result in multiple duplicate documents in the index. Use `update_document` with a `unique=True` field in your schema to overwrite existing documents.
  Fix: Define a unique field in your Schema (e.g., `path=ID(unique=True)`), then use `writer.update_document()` instead of `writer.add_document()` when you intend to replace or update an existing document. If no match is found for the unique field, `update_document` acts like `add_document`.
• gotcha The `whoosh.index.create_in()` function requires the directory to exist before it's called. If the directory does not exist, a `FileNotFoundError` will occur.
  Fix: Always ensure the directory for your index exists by creating it with `os.makedirs(indexdir, exist_ok=True)` or `os.mkdir(indexdir)` before calling `create_in()`.
• deprecated Direct manipulation of index files or relying on undocumented internal structures can lead to issues with future updates. Always use the public API for index management. Some older examples might show direct `FileStorage` usage without `index.create_in` or `index.open_dir` convenience functions.
  Fix: Stick to high-level functions like `whoosh.index.create_in()` and `whoosh.index.open_dir()` for managing your index to ensure compatibility and stability.

Install

• pip install whoosh Install stable version

Imports

• create_in

  from whoosh.index import create_in

• Schema

  from whoosh.fields import Schema, TEXT, ID, STORED

• QueryParser

  from whoosh.qparser import QueryParser

• index

  from whoosh import index

  While 'import whoosh.index' works, direct import from whoosh.index is more common for specific functions like create_in or open_dir, and 'from whoosh import index' is used to access general index-related functions and objects.

Quickstart

This quickstart demonstrates how to define a schema, create or open an index, add documents to the index, and then perform a basic text search. It includes handling the creation of the index directory if it doesn't exist. Documents are added with `title`, `path`, and `content` fields, and a `QueryParser` is used to search the 'content' field.

import os
from whoosh.index import create_in, open_dir
from whoosh.fields import Schema, TEXT, ID
from whoosh.qparser import QueryParser

# 1. Define schema
schema = Schema(title=TEXT(stored=True), path=ID(stored=True), content=TEXT)

# 2. Create or open index directory
indexdir = "indexdir"
if not os.path.exists(indexdir):
    os.mkdir(indexdir)
    ix = create_in(indexdir, schema)
else:
    ix = open_dir(indexdir)

# 3. Add documents
writer = ix.writer()
writer.add_document(title=u"First document", path=u"/a",
                    content=u"This is the first document we've added!")
writer.add_document(title=u"Second document", path=u"/b",
                    content=u"The second one is even more interesting!")
writer.commit()

# 4. Search documents
with ix.searcher() as searcher:
    query_parser = QueryParser("content", ix.schema)
    query = query_parser.parse("first")
    results = searcher.search(query)
    for hit in results:
        print(f"Found: {hit['title']} at {hit['path']}")

# Clean up (optional: remove the index directory)
# import shutil
# shutil.rmtree(indexdir)