html-text

0.7.1 · active · verified Sat Apr 11

html-text is a Python library designed to extract clean, readable plain text from HTML content. It goes beyond simple text extraction by removing invisible non-text content like inline styles, JavaScript, and comments. The library intelligently normalizes whitespace and can optionally add newlines after block-level elements (e.g., headers, paragraphs) to produce text that more closely resembles browser rendering, making it suitable for text classification or further natural language processing. The current version is 0.7.1, and it maintains an active development status.

Warnings

gotcha By default, `html_text.extract_text()` attempts to 'guess layout' and inserts newlines after block-level HTML elements (e.g., <h1>, <p>) to improve readability. If a flat, single-line text output is desired, explicitly set `guess_layout=False`.
Fix: For flatter output, call `html_text.extract_text(html_string, guess_layout=False)`.
gotcha When working with pre-parsed `lxml.html.HtmlElement` trees or `parsel.Selector` objects, lower-level functions like `html_text.etree_to_text()` or `html_text.selector_to_text()` do NOT automatically clean the HTML. You must manually apply cleaning using `html_text.cleaner.clean_html()` or `html_text.cleaned_selector()` first.
Fix: Always clean manually before using lower-level extraction functions, e.g., `cleaned_tree = html_text.cleaner.clean_html(tree)` or `cleaned_sel = html_text.cleaned_selector(html_content)`.
gotcha The `html-text` library depends on `lxml`. Installing `lxml` can sometimes be complex on various operating systems, requiring system-level development packages (e.g., `libxml2-dev` and `libxslt-dev` on Debian/Ubuntu, or Xcode Command Line Tools on macOS) for its C extensions to compile correctly.
Fix: Refer to the `lxml` installation guide (often linked from its PyPI page or documentation) for specific system dependencies required before `pip install lxml` or `pip install html-text`.

Install

pip install html-text Install with pip

Imports

extract_text
```
from html_text import extract_text
```
parse_html
```
from html_text import parse_html
```
cleaner
```
from html_text import cleaner
```
etree_to_text
```
from html_text import etree_to_text
```
cleaned_selector
```
from html_text import cleaned_selector
```

Quickstart

Demonstrates the basic usage of `html_text.extract_text` to convert an HTML string into plain text, including an example of disabling layout guessing for a flatter output.

import html_text

html_content = '<h1>Hello</h1><p>This is a <b>paragraph</b> with <span>inline</span> text.</p>'
plain_text = html_text.extract_text(html_content)
print(plain_text)

# To get text without layout-driven newlines (e.g., after h1)
plain_text_flat = html_text.extract_text(html_content, guess_layout=False)
print(plain_text_flat)

view raw JSON →