robots-txt-checker MCP server

Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).

Category: Finance

● Online, checked 329 min ago · 367 ms

Source: official MCP Registry entry io.github.ryuamarines/robots-txt-checker

Direct MCP URL and setup

CLAUDE CODE

claude mcp add --transport http io-github-ryuamarines-robots-txt-checker https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp

CURSOR / VS CODE (mcp.json)

{ "mcpServers": { "io-github-ryuamarines-robots-txt-checker": { "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp" } } }