LlamaIndex File Readers

0.6.0 · active · verified Fri Apr 10

The `llama-index-readers-file` library provides specialized data loaders for various local file formats (e.g., PDF, DOCX, CSV, TXT, Image) within the LlamaIndex ecosystem. It allows users to ingest different file types into LlamaIndex Document objects for indexing and retrieval. Current version is 0.6.0, with releases typically aligning with LlamaIndex core library updates.

Warnings

breaking Prior to LlamaIndex v0.10.x, some file readers might have been directly available within the main `llama_index` package. With the modularization, specific readers now reside in sub-packages like `llama-index-readers-file` and require a separate installation.
Fix: Ensure `pip install llama-index-readers-file` is executed. Update import paths from `llama_index.readers.*` to `llama_index.readers.file.*`.
gotcha Many specific file type readers (e.g., PDFReader, DocxReader, CSVFileReader) rely on additional third-party libraries that are not installed by default with `llama-index-readers-file`. You must install these optional dependencies explicitly.
Fix: Install the required extras, e.g., `pip install llama-index-readers-file[pdf]` for PDF support, or `pip install pypdf` manually. Check the `pyproject.toml` or documentation for a full list of extras.
gotcha When dealing with complex documents (e.g., PDFs with tables, scanned images, or nested structures), `FlatReader` or basic type-specific readers may not extract content optimally. For advanced parsing, consider `UnstructuredReader` (available in `llama-index-readers-unstructured`).
Fix: For basic text extraction, `FlatReader` from `llama-index-readers-file` is sufficient. For rich content, install `pip install llama-index-readers-unstructured` and use `UnstructuredReader`.

Install

pip install llama-index-readers-file Basic Install
pip install llama-index-readers-file[pdf,docx,xlsx] Install with common extras

Imports

FlatReader

from llama_index.readers.file import FlatReader

For loading plain text files.

PDFReader
```
from llama_index.readers.file import PDFReader
```
For loading PDF documents. Requires 'pypdf' extra.
DocxReader
```
from llama_index.readers.file import DocxReader
```
For loading DOCX documents. Requires 'docx2txt' extra.
CSVFileReader
```
from llama_index.readers.file import CSVFileReader
```
For loading CSV files. Requires 'pandas' extra.

Quickstart

This quickstart demonstrates how to use the `FlatReader` from `llama-index-readers-file` to load a plain text document. It creates a temporary file, loads its content into LlamaIndex Document objects, and prints a snippet of the loaded text. Remember to install the `llama-index-readers-file` package.

import tempfile
from pathlib import Path
from llama_index.readers.file import FlatReader

# Create a dummy text file
file_content = "This is a sample document for LlamaIndex. It contains some text."
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as tmp_file:
    tmp_file.write(file_content)
    tmp_file_path = Path(tmp_file.name)

# Initialize the FlatReader
reader = FlatReader()

# Load data from the temporary file
documents = reader.load_data(file=tmp_file_path)

# Print the content of the first document
if documents:
    print(f"Loaded document content: {documents[0].text[:100]}...")
    print(f"Metadata: {documents[0].metadata}")

# Clean up the temporary file
tmp_file_path.unlink()

view raw JSON →