Robots Txt Checker

by ryuamarinesio.github.ryuamarines/robots-txt-checkerv1.0.93058

How the score works

Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).

  • Remote endpoint

Uptime

Checked  · every day

100%from 7 days of checks

Up without downtime on 7 of 7 days with data.

  • No downtime
  • Degraded
  • Down over 5 minutes
  • No data
  • Not checked yet

Response time

Median 2050 ms, slowest 5% 2459 ms for the handshake and the tool list, over 30 days.

2500 ms1250 ms0

How far each check got

  • Reached the address 100%
  • Completed the handshake 71.4%
  • Listed its tools 71.4%

1 tool at the last answer.

Every day
DayResultChecksMedian
6 Nov (today)No downtime1435 ms
5 NovNo downtime1of 1275 ms
4 NovNo downtime1of 1–
3 NovNo downtime1of 12050 ms
2 NovNo downtime1of 12455 ms
1 NovNo downtime1of 12173 ms
31 OctNo downtime1–

A failure counts as down time only when a second check five minutes later fails too. How we check

Connect to the publisher's own endpoint

Add to ~/.bob/settings/mcp.json (global) or .bob/mcp.json (project):

{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
For agents: the install prompt

Every step above as one Markdown text, for an assistant installing it for you.

# Install the MCP server `io.github.ryuamarines/robots-txt-checker`

Its page: https://plug.directory/servers/io.github.ryuamarines/robots-txt-checker

It runs on the publisher's servers; the client connects to its URL.

Use the section for the person's client.

## Claude Desktop

Paste into Settings › Connectors › Add custom connector:

```text
https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp
```

## Claude Code

Run in a terminal:

```sh
claude mcp add --transport http robots-txt-checker https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp
```

## Cursor, Devin Desktop, LM Studio, oterm, Bob Shell, ECA, Kiro, Kiro CLI

Merge into the place for the person's client:

- Cursor: ~/.cursor/mcp.json
- Devin Desktop: ~/.config/devin/mcp_config.json
- LM Studio: mcp.json (Program › Install › Edit mcp.json)
- oterm: config.json in oterm's data folder (~/Library/Application Support/oterm on macOS, ~/.local/share/oterm on Linux, %APPDATA%\oterm on Windows)
- Bob Shell: ~/.bob/settings/mcp.json (global) or .bob/mcp.json (project)
- ECA: ~/.config/eca/config.json (global) or .eca/config.json in the project
- Kiro: ~/.kiro/settings/mcp.json (user) or .kiro/settings/mcp.json (workspace)
- Kiro CLI: ~/.kiro/settings/mcp.json (user) or .kiro/settings/mcp.json (workspace)

```json
{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## VS Code

Merge into .vscode/mcp.json:

```json
{
  "servers": {
    "robots-txt-checker": {
      "type": "http",
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Zed

Merge into ~/.config/zed/settings.json:

```json
{
  "context_servers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Cline

Merge into cline_mcp_settings.json (MCP Servers › Configure › Configure MCP Servers):

```json
{
  "mcpServers": {
    "robots-txt-checker": {
      "type": "streamableHttp",
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Continue

Merge into ~/.continue/config.yaml:

```yaml
mcpServers:
  - name: "robots-txt-checker"
    type: streamable-http
    url: "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
```

## Goose

Merge into ~/.config/goose/config.yaml:

```yaml
extensions:
  "robots-txt-checker":
    type: streamable_http
    name: "robots-txt-checker"
    enabled: true
    uri: "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    headers: {}
    env_keys: []
    envs: {}
    timeout: 300
```

## Gemini CLI

Merge into ~/.gemini/settings.json:

```json
{
  "mcpServers": {
    "robots-txt-checker": {
      "httpUrl": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Codex

Run in a terminal:

```sh
codex mcp add robots-txt-checker --url https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp
```

## mcp-client-cli, TUUI, BoltAI

This server's endpoint speaks streamable HTTP, which this client's documentation does not list among its transports; use another client for it.

## Agenvoy

Merge into ~/.config/agenvoy/mcp.json:

```json
{
  "servers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Amp

Merge into ~/.config/amp/settings.json (user) or .amp/settings.json (workspace):

```json
{
  "amp.mcpServers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## AnythingLLM

Merge into anythingllm_mcp_servers.json in the AnythingLLM storage plugins directory:

```json
{
  "mcpServers": {
    "robots-txt-checker": {
      "type": "streamable",
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

## Augment Code

Merge into Augment Settings Panel – MCP section – Import from JSON (VS Code extension):

```json
{
  "mcpServers": {
    "robots-txt-checker": {
      "type": "http",
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}
```

Then restart the client, or reload its MCP servers, and ask it to list the server's tools to confirm it started.

From the publisher's entry in the Official MCP Registry (the registry entry). Report a problem