MechanicalSoup

1.4.0 verified Tue May 12 auth: no python install: verified quickstart: stale

Python library for automating website interaction — form submission, cookie handling, link following. Built on requests + BeautifulSoup. Current version is 1.4.0 (2023). Install name is MechanicalSoup (capital M and S), import is mechanicalsoup (all lowercase). Does NOT execute JavaScript — use Playwright or Selenium for JS-rendered pages. Maintenance status: low activity since 2023.

pip install MechanicalSoup

Common errors

error mechanicalsoup.utils.LinkNotFoundError ↓

cause This error occurs when the specified link cannot be found on the current page.

fix

Ensure the link exists on the page and is correctly specified. Use the find_link method to verify its presence before attempting to follow it.

error UserWarning: No parser was explicitly specified, so I’m using the best available HTML parser for this system (“lxml”). ↓

cause This warning appears when BeautifulSoup defaults to an HTML parser because none was specified.

fix

Specify the parser explicitly by setting the soup_config parameter, e.g., soup_config={'features': 'lxml'}.

error ReferenceError: weakly-referenced object no longer exists ↓

cause This error can occur when the browser session is not properly closed, leading to issues during garbage collection.

fix

Ensure to call browser.close() before the end of the script or use a context manager with the with statement to handle the browser session.

error mechanicalsoup.utils.LinkNotFoundError: No link found matching the criteria ↓

cause This error occurs when attempting to follow a link that does not exist on the page.

fix

Verify the link's presence and correctness on the page before attempting to follow it.

error mechanicalsoup.utils.LinkNotFoundError: No form matching the criteria ↓

cause This error occurs when attempting to select a form that does not exist on the page.

fix

Ensure the form exists and is correctly specified before attempting to select it.

Warnings

breaking MechanicalSoup does NOT execute JavaScript. Pages that render content via JS (React, Vue, Angular SPAs) will appear empty or broken. This is the most common misuse. ↓

fix For JavaScript-rendered pages use Playwright (pip install playwright + playwright install) or Selenium instead.

gotcha Install name and import name differ in capitalization. pip install MechanicalSoup (capital M and S), but import mechanicalsoup (all lowercase). import MechanicalSoup raises ModuleNotFoundError. ↓

fix pip install MechanicalSoup then import mechanicalsoup

gotcha raise_on_404=False by default — 404 responses are silently treated as successful. Code that navigates to a missing page gets no error and browser.page will contain the 404 HTML. ↓

fix Always pass raise_on_404=True: StatefulBrowser(raise_on_404=True). This raises LinkNotFoundError on 404 responses.

gotcha Browser (stateless) and StatefulBrowser (stateful) have different APIs. Browser.get() returns a response. StatefulBrowser.open() returns a response AND updates internal state. Mixing them causes confusion. ↓

fix Use StatefulBrowser for multi-page workflows (login flows, form sequences). Use Browser only for simple one-shot requests.

gotcha select_form() with no argument selects the first form on the page. On pages with multiple forms (login + newsletter), this may select the wrong form. ↓

fix Always pass a CSS selector: browser.select_form('#login-form'). Or use the nr= argument to select by index: browser.select_form('form', nr=1).

Install compatibility verified last tested: 2026-05-12

python os / libc status wheel install import disk

3.10 alpine (musl) - - 0.97s 34.6M

3.10 slim (glibc) - - 0.71s 35M

3.11 alpine (musl) - - 1.61s 36.9M

3.11 slim (glibc) - - 1.33s 37M

3.12 alpine (musl) - - 1.21s 28.7M

3.12 slim (glibc) - - 1.29s 29M

3.13 alpine (musl) - - 1.16s 28.4M

3.13 slim (glibc) - - 1.19s 29M

3.9 alpine (musl) - - 0.93s 33.9M

3.9 slim (glibc) - - 0.79s 34M

Imports

StatefulBrowser

wrong

import MechanicalSoup  # ImportError — import name is lowercase

# Browser (stateless) vs StatefulBrowser:
browser = mechanicalsoup.Browser()  # stateless — does NOT track current page
browser.open('https://example.com')  # AttributeError — Browser has no open()

correct

import mechanicalsoup

# StatefulBrowser remembers state (cookies, current page)
browser = mechanicalsoup.StatefulBrowser(
    soup_config={'features': 'lxml'},
    raise_on_404=True  # recommended: raise on 404 instead of silently failing
)
browser.open('https://example.com/login')
browser.select_form('#login-form')
browser['username'] = 'alice'
browser['password'] = 'secret'
browser.submit_selected()

Install with pip install MechanicalSoup (capital). Import as import mechanicalsoup (lowercase). Use StatefulBrowser (not Browser) for multi-step workflows — it tracks the current page and form state.

Quickstart stale last tested: 2026-04-23

StatefulBrowser for form-based workflows. Requires lxml or html.parser.

import mechanicalsoup

# StatefulBrowser is the high-level interface
browser = mechanicalsoup.StatefulBrowser(
    soup_config={'features': 'lxml'},
    raise_on_404=True
)

# Navigate to a page
browser.open('https://httpbin.org/forms/post')

# Select and fill a form
browser.select_form('form')  # or CSS selector like '#login-form'
browser['custname'] = 'Alice'
browser['custtel'] = '555-1234'
browser['comments'] = 'Hello!'

# Submit the form
response = browser.submit_selected()
print(response.status_code)

# Access the resulting page
print(browser.page.title.string)