{"id":5594,"library":"cloudscraper","title":"Cloudscraper","description":"Cloudscraper is a Python library built on top of the `requests` library, designed to bypass Cloudflare's anti-bot page (also known as \"I'm Under Attack Mode\" or IUAM). It achieves this by mimicking a real web browser, handling JavaScript challenges, and managing cookies automatically, allowing users to scrape websites protected by Cloudflare. The library is actively maintained, with frequent updates to adapt to Cloudflare's evolving security measures. The current PyPI version is 1.2.71, though a major version 3.0.0 has been released on GitHub with significant changes.","status":"active","version":"1.2.71","language":"en","source_language":"en","source_url":"https://github.com/venomous/cloudscraper","tags":["web scraping","cloudflare bypass","anti-bot","automation","requests"],"install":[{"cmd":"pip install cloudscraper","lang":"bash","label":"Install stable release"}],"dependencies":[{"reason":"Core HTTP client functionality, Cloudscraper extends its Session object.","package":"requests","optional":false},{"reason":"Required for certain HTTP utilities.","package":"requests_toolbelt","optional":false},{"reason":"Used for parsing JavaScript challenges.","package":"pyparsing","optional":false},{"reason":"For advanced TLS/SSL capabilities.","package":"pyOpenSSL","optional":false},{"reason":"For cryptographic operations if required by challenges.","package":"pycryptodome","optional":false},{"reason":"For WebSocket handling if Cloudflare uses it.","package":"websocket-client","optional":false},{"reason":"Default pure Python JavaScript interpreter for solving challenges.","package":"js2py","optional":false},{"reason":"For Brotli compression support in HTTP requests.","package":"brotli","optional":false},{"reason":"Provides a curated list of trusted root certificates.","package":"certifi","optional":false},{"reason":"Recommended for faster and more robust JavaScript challenge solving, requires separate installation.","package":"nodejs","optional":true}],"imports":[{"note":"While `CloudScraper()` directly works, `create_scraper()` is the recommended factory function as it handles initial setup and configuration best, especially in newer versions.","wrong":"from cloudscraper import CloudScraper; scraper = CloudScraper()","symbol":"cloudscraper","correct":"import cloudscraper"}],"quickstart":{"code":"import cloudscraper\nimport os\n\n# Instantiate a CloudScraper session. This object works like a requests.Session\nscraper = cloudscraper.create_scraper(\n    # Optionally, provide a `requests` Session object to base it on\n    # sess=requests.Session(),\n    # Or configure an interpreter, e.g., 'nodejs' if installed for better performance\n    # interpreter='nodejs'\n)\n\n# Make a GET request to a Cloudflare-protected site\n# Replace 'http://somesite.com' with your target URL\n# For demonstration, we'll use a placeholder URL or a test site\n# You might need to set headers, e.g., a User-Agent, for more realistic requests\nheaders = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36'}\nurl = os.environ.get('TARGET_URL', 'https://nowsecure.nl') # A common test site for bot detection\n\ntry:\n    response = scraper.get(url, headers=headers)\n    response.raise_for_status() # Raise an exception for bad status codes\n    print(f\"Successfully accessed {url} (Status: {response.status_code})\")\n    # print(response.text[:500]) # Print first 500 characters of content\nexcept Exception as e:\n    print(f\"Failed to access {url}: {e}\")\n    if response.status_code == 403:\n        print(\"Access denied (403 Forbidden). Cloudflare protection might be too strong or settings need adjustment.\")","lang":"python","description":"The simplest way to use `cloudscraper` is by calling `cloudscraper.create_scraper()`, which returns a `CloudScraper` instance. This instance behaves similarly to a `requests.Session` object. Any requests made through this session to Cloudflare-protected websites will automatically attempt to bypass the anti-bot measures. The script includes a placeholder URL and demonstrates basic error handling for a 403 status code, which is common if Cloudflare still blocks the request. It's recommended to set a realistic User-Agent for better stealth."},"warnings":[{"fix":"Upgrade Python to 3.8 or newer. Update all project dependencies to their latest stable versions. Review `cloudscraper`'s GitHub `UPGRADE_SUMMARY.md` for specific changes if migrating an existing codebase. Use `pip install --upgrade cloudscraper` or `pip install cloudscraper>=3.0.0`.","message":"Cloudscraper v3.0.0 (released on GitHub, potentially available via `pip install cloudscraper>=3.0.0` or direct GitHub install) introduces breaking changes, including a minimum Python version requirement of 3.8+ (dropping Python 3.6 and 3.7 support) and significant dependency upgrades. Python 2 compatibility code has also been removed. If upgrading from an older 1.x or 2.x version, ensure your environment meets these new requirements.","severity":"breaking","affected_versions":"< 3.0.0"},{"fix":"Keep `cloudscraper` updated to the latest version. If issues persist, enable debug mode (`cloudscraper.create_scraper(debug=True)`) to diagnose. Consider adjusting scraper settings like `delay`, `browser` profile, or trying different JavaScript interpreters (e.g., Node.js if installed).","message":"Cloudflare continuously updates its anti-bot techniques, leading to an 'arms race' where `cloudscraper` versions can become outdated and cease to work against the latest Cloudflare protections. This often results in `403 Forbidden` errors or being stuck on a Cloudflare challenge page.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Integrate with a third-party CAPTCHA solving service (e.g., 2Captcha, Anti-Captcha) via `cloudscraper`'s built-in support. This typically requires an API key and an active subscription to the service.","message":"While `cloudscraper` can handle JavaScript challenges, it does not natively solve CAPTCHA challenges (like reCAPTCHA or hCaptcha) that Cloudflare might present. Users will encounter blocks if CAPTCHAs are triggered.","severity":"gotcha","affected_versions":"All versions"},{"fix":"For highly dynamic or heavily protected websites, consider using headless browser automation tools like Selenium or Playwright, or utilizing specialized managed scraping APIs that handle these complexities.","message":"Cloudscraper is not a full-fledged browser and may not suffice for complex web scraping scenarios involving heavy client-side JavaScript rendering, dynamic content loaded via XHR, or highly sophisticated Cloudflare protection layers. It primarily tackles JavaScript challenges and cookie management.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}