CrawlWitness is an MCP server published by com.crawlwitness, hosted at api.crawlwitness.com. When last checked on 2026-11-22 it was working: it answers and lists its tools. It declares 6 tools, including check_ai_consent, can_ai_use, ai_consent_score, signal_history, verify_receipt.
AI consent & licensing check for any domain, with signed on-chain receipts. Paid via x402.
1 connections through 4 shared values. Tap any node.
Squares are this server's values; dots are other servers sharing them. A shared address or publisher is a lead, not proof of a common operator.
Inferred from the host name and headers: what a server runs on, not who runs it.
Full AI-consent report for a domain: robots.txt rules for each AI bot (training, search and user-agent crawlers), RSL license, IETF AIPREF Content-Usage, W3C TDMRep, noai meta tags, llms.txt and C2PA, combined into a 0-100 score and a training stance, plus a verifiable receipt (EAS attestation anchored on Base). Use it before crawling, training on or ingesting a site. Costs 0.005 USD in USDC via x402 (Base or Solana); without a payment it returns the x402 payment options. Unreachable sites are not charged.
| Parameter | Type | Description |
|---|---|---|
| pay | boolean | Pay directly instead of using the small daily free quota |
| fresh | boolean | Force a fresh analysis instead of the cached one (<24 h) |
| domain * | string | Domain to check, without scheme or path (e.g. nytimes.com) |
{
"name": "check_ai_consent",
"title": "Check AI consent & licensing (paid, with signed receipt)",
"description": "Full AI-consent report for a domain: robots.txt rules for each AI bot (training, search and user-agent crawlers), RSL license, IETF AIPREF Content-Usage, W3C TDMRep, noai meta tags, llms.txt and C2PA, combined into a 0-100 score and a training stance, plus a verifiable receipt (EAS attestation anchored on Base). Use it before crawling, training on or ingesting a site. Costs 0.005 USD in USDC via x402 (Base or Solana); without a payment it returns the x402 payment options. Unreachable sites are not charged.",
"inputSchema": {
"type": "object",
"required": [
"domain"
],
"properties": {
"pay": {
"type": "boolean",
"default": false,
"description": "Pay directly instead of using the small daily free quota"
},
"fresh": {
"type": "boolean",
"default": false,
"description": "Force a fresh analysis instead of the cached one (<24 h)"
},
"domain": {
"type": "string",
"maxLength": 253,
"minLength": 3,
"description": "Domain to check, without scheme or path (e.g. nytimes.com)"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"properties": {
"pago": {
"type": "object",
"description": "When paid: network, transaction, signed receipt and EAS attestation"
},
"origen": {
"type": "string",
"description": "cache or fresco (fresh)"
},
"dominio": {
"type": "string"
},
"informe": {
"type": "object",
"description": "Full report: resumen (score, training stance, reasons), robots (per-bot access), rsl, aipref, tdm, html, llms_txt"
},
"analizado_en": {
"type": "string",
"format": "date-time"
}
}
},
"annotations": {
"title": "Check AI consent",
"readOnlyHint": true,
"openWorldHint": true,
"idempotentHint": true,
"destructiveHint": false
}
}Fast yes/no verdict for one domain, one AI bot and one use (train, rag or search), from robots.txt, IETF AIPREF, W3C TDMRep, noai directives and RSL licenses, with the reasons and a signed receipt. Use it in a crawler or RAG pipeline before fetching each new domain. Costs 0.001 USD on Base (0.002 on Solana) via x402. Invalid domain or unknown bot is never charged.
| Parameter | Type | Description |
|---|---|---|
| bot * | string | AI user-agent token (e.g. GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot) |
| use | string | one of ["train","rag","search"] |
| domain * | string | Domain to check, without scheme or path (e.g. nytimes.com) |
{
"name": "can_ai_use",
"title": "Can this AI bot use this site? (paid yes/no, signed)",
"description": "Fast yes/no verdict for one domain, one AI bot and one use (train, rag or search), from robots.txt, IETF AIPREF, W3C TDMRep, noai directives and RSL licenses, with the reasons and a signed receipt. Use it in a crawler or RAG pipeline before fetching each new domain. Costs 0.001 USD on Base (0.002 on Solana) via x402. Invalid domain or unknown bot is never charged.",
"inputSchema": {
"type": "object",
"required": [
"domain",
"bot"
],
"properties": {
"bot": {
"enum": [
"GPTBot",
"ClaudeBot",
"anthropic-ai",
"Google-Extended",
"GoogleOther",
"CloudVertexBot",
"Applebot-Extended",
"meta-externalagent",
"Amazonbot",
"amazon-kendra-",
"Bytespider",
"CCBot",
"cohere-training-data-crawler",
"AI2Bot",
"Diffbot",
"Timpibot",
"ImagesiftBot",
"omgili",
"PetalBot",
"KimiBot",
"ICC-Crawler",
"Cotoyogi",
"Reflectionbot",
"CloudflareBrowserRenderingCrawler",
"LinerBot",
"FishBot",
"Novellum",
"bigsur.ai",
"NavuBot",
"QualifiedBot",
"atlassian-bot",
"make.com",
"magpie-crawler",
"Awario",
"w4mwnpbXf3MFAbxOkJRw",
"SemrushBot-OCOB",
"SemrushBot-SWA",
"BorderxBot",
"SelectikaScraper",
"CitibotSiteCrawler",
"AdpResearchBot",
"WARDBot",
"ygs-scraper-bot",
"OAI-SearchBot",
"Claude-SearchBot",
"PerplexityBot",
"Applebot",
"Amzn-SearchBot",
"Bravebot",
"Cloudflare-AI-Search",
"Cloudflare-AI-Search-External",
"DuckAssistBot",
"YouBot",
"KernelSearchBot",
"ShapBot",
"alphalens-bot",
"Anomura",
"Element451Bot",
"ChatGPT-User",
"Claude-User",
"Perplexity-User",
"meta-externalfetcher",
"meta-externaltest",
"muse-",
"MistralAI-User",
"cohere-ai",
"Google-Agent",
"GrokAgent",
"Devin",
"ApifyWebsiteContentCrawler",
"Instapaper",
"Retool",
"QATechBot",
"TwinAgent",
"CledaraBot",
"HarkBot",
"HIFIBot",
"ChathiveCrawler",
"EasyScan",
"Nava",
"ZipchatBot"
],
"type": "string",
"description": "AI user-agent token (e.g. GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot)"
},
"use": {
"enum": [
"train",
"rag",
"search"
],
"type": "string",
"default": "train"
},
"domain": {
"type": "string",
"maxLength": 253,
"minLength": 3,
"description": "Domain to check, without scheme or path (e.g. nytimes.com)"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"properties": {
"bot": {
"type": "string"
},
"use": {
"type": "string"
},
"pago": {
"type": "object"
},
"domain": {
"type": "string"
},
"allowed": {
"type": [
"boolean",
"null"
]
},
"reasons": {
"type": "array",
"items": {
"type": "string"
}
},
"decision": {
"type": "string"
}
}
},
"annotations": {
"title": "Can AI use",
"readOnlyHint": true,
"openWorldHint": true,
"idempotentHint": true,
"destructiveHint": false
}
}Free summary for a domain: 0-100 signal score and AI-training stance (reservado = reserved, permitido = allowed, sin_senal = no signal), from cache when available.
| Parameter | Type | Description |
|---|---|---|
| domain * | string | Domain to check, without scheme or path (e.g. nytimes.com) |
{
"name": "ai_consent_score",
"title": "AI consent score (free)",
"description": "Free summary for a domain: 0-100 signal score and AI-training stance (reservado = reserved, permitido = allowed, sin_senal = no signal), from cache when available.",
"inputSchema": {
"type": "object",
"required": [
"domain"
],
"properties": {
"domain": {
"type": "string",
"maxLength": 253,
"minLength": 3,
"description": "Domain to check, without scheme or path (e.g. nytimes.com)"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"properties": {
"page": {
"type": "string"
},
"dominio": {
"type": "string"
},
"puntuacion": {
"type": [
"integer",
"null"
]
},
"analizado_en": {
"type": "string"
},
"entrenamiento": {
"type": [
"string",
"null"
]
}
}
},
"annotations": {
"title": "AI consent score",
"readOnlyHint": true,
"openWorldHint": true,
"idempotentHint": true,
"destructiveHint": false
}
}What a domain's robots.txt (or llms.txt, tdmrep.json, RSL license, homepage headers) said at a given instant, with the SHA-256 of each version and its EAS attestation anchored on Base. Without `source` it returns the index of versions; with `source` (and optional `at`, RFC 3339) the document in force at that instant. Covers only domains already analysed.
| Parameter | Type | Description |
|---|---|---|
| at | string | Instant (RFC 3339); default now |
| domain * | string | Domain to check, without scheme or path (e.g. nytimes.com) |
| source | string | portada = homepage headers |
{
"name": "signal_history",
"title": "Signal history on any date (free)",
"description": "What a domain's robots.txt (or llms.txt, tdmrep.json, RSL license, homepage headers) said at a given instant, with the SHA-256 of each version and its EAS attestation anchored on Base. Without `source` it returns the index of versions; with `source` (and optional `at`, RFC 3339) the document in force at that instant. Covers only domains already analysed.",
"inputSchema": {
"type": "object",
"required": [
"domain"
],
"properties": {
"at": {
"type": "string",
"format": "date-time",
"description": "Instant (RFC 3339); default now"
},
"domain": {
"type": "string",
"maxLength": 253,
"minLength": 3,
"description": "Domain to check, without scheme or path (e.g. nytimes.com)"
},
"source": {
"enum": [
"robots",
"llms",
"tdmrep",
"rsl",
"portada"
],
"type": "string",
"description": "portada = homepage headers"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object"
},
"annotations": {
"title": "Signal history",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Returns the signed EAS attestation of a paid query and, once anchored, its Merkle proof (OpenZeppelin format) and the on-chain attestation on Base that contains it.
| Parameter | Type | Description |
|---|---|---|
| uid * | string | Attestation UID (0x + 64 hex) |
{
"name": "verify_receipt",
"title": "Verify a receipt (free)",
"description": "Returns the signed EAS attestation of a paid query and, once anchored, its Merkle proof (OpenZeppelin format) and the on-chain attestation on Base that contains it.",
"inputSchema": {
"type": "object",
"required": [
"uid"
],
"properties": {
"uid": {
"type": "string",
"pattern": "^0x[0-9a-fA-F]{64}$",
"description": "Attestation UID (0x + 64 hex)"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"properties": {
"uid": {
"type": "string"
},
"anclaje": {
"type": "object"
},
"paquete": {
"type": "object"
}
}
},
"annotations": {
"title": "Verify receipt",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Sanctions screening of a crypto address against the US OFAC SDN list (all its digital-currency addresses: EVM, BTC, TRX, SOL, USDT, USDC…). Use it before an agent pays, receives from or interacts with a wallet. Returns sanctioned true/false, the listed entity and programs, the exact list version used and a signed receipt. Costs 0.005 USD via x402. Screening aid, not legal advice.
| Parameter | Type | Description |
|---|---|---|
| address * | string | Crypto address (EVM 0x…, BTC, TRX, SOL…) |
{
"name": "check_wallet",
"title": "Is this crypto wallet sanctioned? (paid, signed)",
"description": "Sanctions screening of a crypto address against the US OFAC SDN list (all its digital-currency addresses: EVM, BTC, TRX, SOL, USDT, USDC…). Use it before an agent pays, receives from or interacts with a wallet. Returns sanctioned true/false, the listed entity and programs, the exact list version used and a signed receipt. Costs 0.005 USD via x402. Screening aid, not legal advice.",
"inputSchema": {
"type": "object",
"required": [
"address"
],
"properties": {
"address": {
"type": "string",
"maxLength": 110,
"minLength": 20,
"description": "Crypto address (EVM 0x…, BTC, TRX, SOL…)"
}
},
"additionalProperties": false
},
"outputSchema": {
"type": "object",
"properties": {
"pago": {
"type": "object"
},
"lists": {
"type": "array",
"items": {
"type": "object"
}
},
"address": {
"type": "string"
},
"matches": {
"type": "array",
"items": {
"type": "object"
}
},
"decision": {
"enum": [
"blocked",
"clear"
],
"type": "string"
},
"sanctioned": {
"type": "boolean"
},
"address_type": {
"type": "string"
}
}
},
"annotations": {
"title": "Check wallet",
"readOnlyHint": true,
"openWorldHint": true,
"idempotentHint": true,
"destructiveHint": false
}
}CrawlWitness: AI consent and licensing oracle. Before crawling a site, training on its content or ingesting it into RAG, call `check_ai_consent` with its domain: it returns what the site allows per AI bot and use (robots.txt, RSL license, IETF AIPREF, W3C TDMRep, noai meta tags, llms.txt, C2PA), a 0-100 score and a signed receipt (EAS attestation anchored on Base) proving what was checked and when. It costs a micro-fee in USDC via x402 (Base or Solana, no account). Free tools: `ai_consent_score` (quick score), `signal_history` (what robots.txt and other signals said on any past date, with on-chain proof) and `verify_receipt`. For crawlers and RAG pipelines, `can_ai_use` gives a cheap signed yes/no per domain, AI bot and use (train, rag, search). Resources: crawlwitness://guide and crawlwitness://pricing. Prompts: audit_domain, before_crawling, robots_on_date.
Names, descriptions and schemas were written by the server and are shown as data.
Pattern matches on what the server declares. They show what it says it can do, not what it does.
None yet. Changes appear after a server's second daily check.