Scrape-LE
本地Analyse robots.txt content and report whether a path may be crawled.
安装配置
{
"mcpServers": {
"scrape-le": {
"command": "npx",
"args": [
"scrape-le-mcp"
]
}
}
}
Analyse robots.txt content and report whether a path may be crawled.
{
"mcpServers": {
"scrape-le": {
"command": "npx",
"args": [
"scrape-le-mcp"
]
}
}
}