robots-txt-ai-crawler-audit
本地Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
安装配置
{
"mcpServers": {
"robots-txt-ai-crawler-audit": {
"command": "npx",
"args": [
"@readystack/robots-txt-ai-crawler-audit"
]
}
}
}