Agent Crawl Readiness MCP server
Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.
Category: Dev tools
● Online, checked 307 min ago · 74 ms
Source: official MCP Registry entry io.github.sadri-dridi/agent-crawl-readiness · repository
Direct MCP URL and setup
CLAUDE CODE
claude mcp add --transport http io-github-sadri-dridi-agent-crawl-readin https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/agent-crawl-readiness/mcpCURSOR / VS CODE (mcp.json)
{ "mcpServers": { "io-github-sadri-dridi-agent-crawl-readin": { "url": "https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/agent-crawl-readiness/mcp" } } }