curl-cffi
curl-cffi is a Python binding for the curl-impersonate fork via CFFI. It enables Python applications to impersonate browsers' TLS/JA3 and HTTP/2 fingerprints, effectively bypassing many anti-bot systems. The library provides a high-level API that mimics the popular `requests` library, making it intuitive to use. It supports asynchronous operations, HTTP/2, HTTP/3, and WebSockets. The current stable version is 0.14.0, with active development and frequent updates.
Warnings
- breaking As of version 0.14.0, curl-cffi officially supports Python 3.10 and above. Older Python versions (e.g., 3.9) are no longer officially supported.
- breaking In version 0.3.0, the `Response.cookies` attribute's type changed from `http.cookies.SimpleCookie` to `curl_cffi.requests.Cookies`. This might affect code expecting the standard library's cookie object for manipulation.
- gotcha When configuring HTTPS proxies, specifying the proxy URL with an `https://` scheme might lead to `OPENSSL_internal:WRONG_VERSION_NUMBER` errors. The `http://` scheme is often required for the proxy's URL.
- gotcha The error `ErrCode: 92, Reason: 'HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)'` often occurs with HTTP/2 requests, sometimes due to proxies or server-side issues. Removing `Content-Length` headers or forcing HTTP/1.1 can be workarounds.
- gotcha When handling streamed responses (e.g., `stream=True`), content is buffered in memory by default. If not consumed immediately, this can lead to Out Of Memory (OOM) errors for large responses. Using `iter_content()` or `content_callback` is recommended.
Install
-
pip install curl-cffi
Imports
- requests
from curl_cffi import requests
- Session
from curl_cffi.requests import Session
- AsyncSession
from curl_cffi import AsyncSession
- WebSocket
from curl_cffi import WebSocket
Quickstart
from curl_cffi import requests
def main():
try:
# Make a GET request impersonating Chrome
response = requests.get('https://www.example.com', impersonate='chrome110')
response.raise_for_status() # Raise an exception for HTTP errors (4xx or 5xx)
print(f"Status Code: {response.status_code}")
print("Response Header:\n", response.headers)
print("First 500 chars of Response Body:\n", response.text[:500])
# Using a session for persistent connections and cookies
with requests.Session() as s:
s.impersonate = 'safari15_5'
res_session = s.get('https://httpbin.org/cookies/set/sessioncookie/123')
print(f"\nSession Status Code: {res_session.status_code}")
print(f"Session Cookies: {s.cookies.get('sessioncookie')}")
except requests.exceptions.RequestError as e:
print(f"An error occurred: {e}")
if __name__ == '__main__':
main()