Robotex

library 1.0.0 ·ruby

✓ verified Jun 19, 2026

networking utility

A Ruby library for parsing and obeying robots.txt files to control web crawler access.

Resources

docswww.rubydoc.info/gems/robotex/ ↗

githubgithub.com/chriskite/robotex ↗

API endpoints

full doc /v1/registry/robotex

install /v1/registry/robotex/install