Open Crawler MCP Server
JSON →A web crawler and text extractor with robots.txt compliance, rate limiting, and page size protection.
Tools · 3
- crawl Crawl a website starting from a given URL, following links up to a specified depth and page limit.
- search Search the web using a query string and return relevant results.
- extract Extract structured data from a web page using CSS selectors or XPath expressions.
Links
★ 1 GitHub stars