Protego
JSON →Protego is a pure-Python robots.txt parser with support for modern conventions like those defined by Google. As of version 0.6.0, it actively supports Python 3.10 and newer, with regular updates aligning with new Python releases. It is widely used for web scraping and compliance checking.
Traffic · last 30 days ↑100% vs prev 7d
total hits 24
actors 5 distinct systems
last hit 4d ago GPTBot
top countries 🇺🇸 United States · 🇸🇬 Singapore · 🇨🇦 Canada · VN · 🇩🇪 Germany
Resources
packagepypi.org/project/protego/ ↗
API endpoints
full doc /v1/registry/protego
install /v1/registry/protego/install
compatibility /v1/registry/protego/compatibility