{"library":"scrapingbee","title":"ScrapingBee Python SDK","description":"ScrapingBee is a web scraping API that handles headless browsers and rotates proxies for you. The Python SDK simplifies interaction with this API, offering features like JavaScript rendering, proxy rotation, AI-powered data extraction, and screenshot capabilities. It is currently at version 2.0.2 and receives regular updates, focusing on reliability and new API features.","language":"python","status":"active","last_verified":"Sun May 17","install":{"commands":["pip install scrapingbee"],"cli":null},"imports":["from scrapingbee import ScrapingBeeClient"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"import os\nfrom scrapingbee import ScrapingBeeClient\n\n# It's highly recommended to store your API key in an environment variable\napi_key = os.environ.get('SCRAPINGBEE_API_KEY', 'YOUR_API_KEY')\n\nif api_key == 'YOUR_API_KEY':\n    print(\"Warning: Replace 'YOUR_API_KEY' or set the SCRAPINGBEE_API_KEY environment variable.\")\n\nclient = ScrapingBeeClient(api_key=api_key)\n\nurl_to_scrape = 'https://www.scrapingbee.com/blog/'\n\ntry:\n    response = client.get(\n        url_to_scrape,\n        params={\n            'render_js': True, # Set to False to save credits if JavaScript rendering is not needed\n            'extract_rules': {\n                'title': 'h1',\n                'subtitle': '#subtitle',\n                'articles': {'selector': 'article h2 a', 'type': 'list', 'output': 'text'}\n            }\n        }\n    )\n\n    if response.ok:\n        # If extract_rules are used, the content is usually JSON\n        if response.headers.get('content-type') == 'application/json':\n            import json\n            data = json.loads(response.content)\n            print(json.dumps(data, indent=2))\n        else:\n            # Otherwise, it's the raw HTML\n            print(response.text[:500]) # Print first 500 characters of HTML\n    else:\n        print(f\"Failed to scrape {url_to_scrape}: Status {response.status_code}, Content: {response.text[:200]}\")\nexcept Exception as e:\n    print(f\"An error occurred: {e}\")","lang":"python","description":"This quickstart initializes the ScrapingBee client with an API key (preferably from an environment variable) and sends a GET request to a URL. It demonstrates using `extract_rules` to automatically parse specific data (title, subtitle, article headings) from the page into a JSON format.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-17","installed_version":"2.0.2","pypi_latest":"2.0.2","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":100,"avg_install_s":2.1,"avg_import_s":0.61,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.6,"mem_mb":9.8,"disk_size":"21.2M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.1,"import_time_s":0.43,"mem_mb":9.8,"disk_size":"22M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.75,"mem_mb":10.8,"disk_size":"23.2M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.3,"import_time_s":0.63,"mem_mb":10.8,"disk_size":"24M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.68,"mem_mb":11.3,"disk_size":"15.0M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":1.9,"import_time_s":0.67,"mem_mb":11.3,"disk_size":"16M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.66,"mem_mb":11.6,"disk_size":"14.8M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":1.9,"import_time_s":0.67,"mem_mb":11.6,"disk_size":"15M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":0.55,"mem_mb":9.4,"disk_size":"20.4M"},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"scrapingbee","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":2.4,"import_time_s":0.46,"mem_mb":9.4,"disk_size":"21M"}]}}