Scrapy Zyte API

0.33.1 verified Mon Apr 27 auth: no python

Client library to process URLs through Zyte API (formerly Crawlera). Enables Scrapy spiders to use Zyte's proxy rotation and data extraction services. Current version: 0.33.1. Release cadence: monthly.

pip install scrapy-zyte-api

Common errors

error ModuleNotFoundError: No module named 'scrapy_zyte_api' ↓

cause Library not installed or installed in wrong environment.

fix

Run: pip install scrapy-zyte-api

error TypeError: __init__() got an unexpected keyword argument 'api_key' ↓

cause Using old class ScrapyZyteAPIDownloaderMiddleware directly with api_key argument, which is no longer supported in latest version (0.33+).

fix

Use ZyteAPI class instead: from scrapy_zyte_api import ZyteAPI

error scrapy.exceptions.NotConfigured: Zyte API key not found. Please set ZYTE_API_KEY environment variable. ↓

cause API key not provided anywhere.

fix

Set the environment variable ZYTE_API_KEY before running Scrapy, or ensure it is passed to middleware.

Warnings

breaking Version 0.33.0 dropped Python 3.9 support (min is 3.10). ↓

fix Upgrade to Python 3.10+ or pin to <0.33.0.

breaking In 0.33.0, the middleware class was renamed from ScrapyZyteAPIDownloaderMiddleware to ZyteAPI. The old import still works as an alias, but the class no longer accepts the same constructor parameters. ↓

fix Use new import syntax and pass api_key via settings or env var.

deprecated The old setting ZYTE_API_DOWNLOADER_MIDDLEWARE_ENABLED is deprecated; use ZYTE_API_ENABLED instead. ↓

fix Replace ZYTE_API_DOWNLOADER_MIDDLEWARE_ENABLED with ZYTE_API_ENABLED in settings.

gotcha The API key must be provided via ZYTE_API_KEY environment variable or explicitly passed, not just set in settings.py when using the middleware. ↓

fix Set env var before running spider: export ZYTE_API_KEY='your_key'.

Imports

ZyteAPI
```
from scrapy_zyte_api import ZyteAPI
```
Correct import for the middleware class.
ScrapyZyteAPIDownloaderMiddleware
```
from scrapy_zyte_api import ScrapyZyteAPIDownloaderMiddleware
```
Alias for backward compatibility; still works.

Quickstart

Quickstart: Instantiate ZyteAPI and configure settings for Scrapy.

import os
from scrapy_zyte_api import ZyteAPI

api_key = os.environ.get('ZYTE_API_KEY', '')
zym = ZyteAPI(api_key=api_key, downloader_middleware_settings={
    'ZYTE_API_DOWNLOADER_MIDDLEWARE_ENABLED': True,
    'ZYTE_API_ENABLED': True,
})

# Then add to your Scrapy settings:
# DOWNLOADER_MIDDLEWARES = {
#     'scrapy_zyte_api.ZyteAPI': 1000,
# }
# ZYTE_API_KEY = api_key