Beautiful Soup

3.2.2 · maintenance · verified Wed Apr 15

Beautiful Soup (version 3.x) is a Python 2 library for parsing HTML and XML documents, including those with malformed markup. It creates a parse tree that can be used to extract data from web pages, making it useful for screen-scraping tasks. Version 3.2.2 is the final release in this series. Development on the 3.x series ended in 2011, and it has been largely superseded by `beautifulsoup4` for Python 3.

Warnings

deprecated Beautiful Soup 3.x is no longer actively developed or maintained. The current, actively developed version is Beautiful Soup 4.x (package name `beautifulsoup4`). New projects should use `beautifulsoup4` which offers improved parsing, better Python 3 compatibility, and is actively supported.
Fix: Migrate to `beautifulsoup4`. Install with `pip install beautifulsoup4` and update imports from `BeautifulSoup` to `bs4`. Be aware of API changes.
breaking Beautiful Soup 3.x is primarily a Python 2 library and is largely incompatible with Python 3. It relies on `SGMLParser`, which was deprecated and removed in Python 3.0. Running BS3 code directly in Python 3 will result in `ImportError: No module named HTMLParser` or `SyntaxError: Invalid syntax`.
Fix: Use Python 2.x for Beautiful Soup 3.x, or migrate your codebase to Beautiful Soup 4.x if using Python 3.
breaking Between Beautiful Soup 3.x and 4.x, some attribute names were renamed for PEP 8 compliance (e.g., `contents` and `findAll` were common in BS3, now `children` and `find_all` in BS4). This causes `AttributeError` if old names are used with BS4.
Fix: When migrating from BS3 to BS4, consult the porting guide and update method/attribute names to their BS4 equivalents (e.g., `findAll` to `find_all`, `contents` to `children`).
gotcha Different parsers (like Python's built-in `SGMLParser` in BS3, or `html.parser`, `lxml`, `html5lib` in BS4) can produce different parse trees for malformed HTML. This might lead to unexpected results or missing elements if the parser interprets the markup differently than expected.
Fix: Always inspect the HTML you are scraping. If encountering issues, try to use a more robust parser (e.g., `lxml` or `html5lib` with BS4) and compare the parse tree using `prettify()` or `diagnose()` (BS4 only).
gotcha Accessing an attribute on a `NoneType` object (e.g., `soup.find('nonexistent_tag').text`) will raise an `AttributeError`. This typically happens when a `find()` call doesn't locate any matching tag and returns `None`.
Fix: Always check if the result of `find()` is not `None` before attempting to access its attributes or children, e.g., `tag = soup.find('mytag'); if tag: print(tag.text)`.
gotcha Using dictionary-style attribute access (e.g., `tag['href']`) will raise a `KeyError` if the specified attribute does not exist on the tag.
Fix: Use the `tag.get('attribute_name')` method instead, which returns `None` if the attribute does not exist, preventing a `KeyError`. Example: `href = tag.get('href')`.
gotcha The `prettify()` method in Beautiful Soup 3.x returns a bytestring, while in Beautiful Soup 4.x it returns a Unicode string. This can cause encoding issues if not handled carefully during migration or when mixing code.
Fix: Be mindful of the return type of `prettify()`. If expecting Unicode in Python 2 or dealing with bytestrings in Python 3, explicit encoding/decoding might be necessary during migration.

Install

pip install beautifulsoup Install Beautiful Soup 3.x

Imports

BeautifulSoup
wrong:
```
from bs4 import BeautifulSoup
```
correct:
```
from BeautifulSoup import BeautifulSoup
```
This import path is specifically for Beautiful Soup 3.x. For Beautiful Soup 4.x (the current, actively maintained version), the correct import is `from bs4 import BeautifulSoup`.

Quickstart

This quickstart demonstrates basic HTML parsing and element extraction using Beautiful Soup 3.x. It creates a `BeautifulSoup` object from an HTML string and then accesses elements by tag name and attributes. Note that Beautiful Soup 3.x does not take an explicit parser argument like `html.parser` which is common in Beautiful Soup 4.x.

from BeautifulSoup import BeautifulSoup

html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>

<p class="story">...</p>
</body></html>
"""

# For Beautiful Soup 3.x, you pass the HTML string directly.
# It defaults to Python's SGMLParser.
soup = BeautifulSoup(html_doc)

print("Document Title:", soup.title.string)
print("First paragraph's class attribute:", soup.p['class'])
print("First anchor tag (link):", soup.a)
print("Text of the first link:", soup.a.string)

view raw JSON →