Robots.txt & AI Crawler Checker MCP server

robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

Category: Search

● Online, checked 348 min ago · 318 ms

Source: official MCP Registry entry dev.workers.quietworklabs.robots-check/robots-check

Direct MCP URL and setup

CLAUDE CODE

claude mcp add --transport http dev-workers-quietworklabs-robots-check-r https://robots-check.quietworklabs.workers.dev/mcp

CURSOR / VS CODE (mcp.json)

{ "mcpServers": { "dev-workers-quietworklabs-robots-check-r": { "url": "https://robots-check.quietworklabs.workers.dev/mcp" } } }