Python Google Search Scraper

3.0.0 · active · verified Thu Apr 09

This library provides Python bindings for performing web searches on Google.com by directly scraping the search results pages. It is important to note that this is NOT an official Google product or API. Its functionality relies on parsing the HTML structure of Google's search results, making it highly susceptible to breaking changes if Google alters its website layout. Users should be aware of potential rate limiting and IP blocking from Google.

Warnings

breaking This library scrapes Google's website. Any changes to Google's search result page HTML structure (which happens frequently) will break the library's functionality. There is no guarantee of continuous operation.
Fix: Monitor library updates; be prepared for breakage. Consider using official Google Search APIs for reliable, long-term solutions.
gotcha This is NOT an official Google product or API. It's a third-party scraping tool. Do not confuse it with official Google Cloud APIs (e.g., Custom Search API) which offer more robust and legal ways to access search data.
Fix: If you require official Google services, use Google's own client libraries (e.g., `google-api-python-client`) and their respective APIs after setting up a Google Cloud Project.
gotcha Frequent or rapid requests will lead to your IP being temporarily or permanently blocked by Google. The `pause` parameter helps, but aggressive usage is still risky.
Fix: Use the `pause` parameter between requests (e.g., `pause=2.0`). Implement exponential backoff for retries. Avoid making large volumes of requests from a single IP address.

Install

pip install google Install stable version

Imports

search
```
from google import search
```

Quickstart

This quickstart demonstrates how to perform a basic Google search using the `search` function. It includes a `pause` parameter to mitigate the risk of IP blocking from Google, which is a common issue with web scraping libraries.

import time
from google import search

search_query = "checklist.day Python libraries"

print(f"Searching for: '{search_query}'")

# Perform a search for 5 URLs, with a 2-second pause between requests
# to avoid IP blocking. Adjust parameters as needed.
try:
    for i, url in enumerate(search(search_query, tld="com", lang="en", num=5, start=0, stop=5, pause=2.0)):
        print(f"Result {i+1}: {url}")
except Exception as e:
    print(f"An error occurred: {e}")
    print("Consider increasing the 'pause' time or checking your network connection.")

view raw JSON →