Web Search API Eval is an MCP server published by io.github.psinglap, hosted at websearch-api-eval.psberkeleymba.workers.dev. When last checked on 2026-11-22 it was working: it answers and lists its tools. It declares 11 tools, including get_overall_results, get_scorecard, compare_providers, recommend_provider, list_questions.
Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
Nothing else in the archive shares these values.
Inferred from the host name and headers: what a server runs on, not who runs it.
Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.
No parameters.
{
"name": "get_overall_results",
"title": "Overall results",
"description": "Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.",
"inputSchema": {
"type": "object",
"properties": {}
},
"annotations": {
"title": "Overall results",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}None yet. Changes appear after a server's second daily check.
Squares are this server's values; dots are other servers sharing them. A shared address or publisher is a lead, not proof of a common operator.
Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.
| Parameter | Type | Description |
|---|---|---|
| use_case * | string | overall, gtm, insurance or finance |
{
"name": "get_scorecard",
"title": "Scorecard",
"description": "Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"use_case"
],
"properties": {
"use_case": {
"enum": [
"overall",
"gtm",
"insurance",
"finance"
],
"type": "string",
"description": "overall, gtm, insurance or finance"
}
}
},
"annotations": {
"title": "Scorecard",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').
| Parameter | Type | Description |
|---|---|---|
| use_case | string | overall, gtm, insurance or finance |
| providers * | array | Two or more of: You.com, Tavily, Exa, Parallel |
| metric_filter | string | Optional keyword to keep only matching metrics |
{
"name": "compare_providers",
"title": "Compare providers",
"description": "Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"providers"
],
"properties": {
"use_case": {
"enum": [
"overall",
"gtm",
"insurance",
"finance"
],
"type": "string",
"default": "overall",
"description": "overall, gtm, insurance or finance"
},
"providers": {
"type": "array",
"items": {
"enum": [
"You.com",
"Tavily",
"Exa",
"Parallel"
],
"type": "string"
},
"minItems": 2,
"description": "Two or more of: You.com, Tavily, Exa, Parallel"
},
"metric_filter": {
"type": "string",
"description": "Optional keyword to keep only matching metrics"
}
}
},
"annotations": {
"title": "Compare providers",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.
| Parameter | Type | Description |
|---|---|---|
| agent_need | string | What the agent does, e.g. 'voice agent answering in under a second' or 'finance research' |
{
"name": "recommend_provider",
"title": "Recommend a provider",
"description": "Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"agent_need": {
"type": "string",
"description": "What the agent does, e.g. 'voice agent answering in under a second' or 'finance research'"
}
}
},
"annotations": {
"title": "Recommend a provider",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.
| Parameter | Type | Description |
|---|---|---|
| use_case | string | all, gtm, insurance or finance |
{
"name": "list_questions",
"title": "List questions",
"description": "The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"use_case": {
"enum": [
"all",
"gtm",
"insurance",
"finance"
],
"type": "string",
"default": "all",
"description": "all, gtm, insurance or finance"
}
}
},
"annotations": {
"title": "List questions",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).
| Parameter | Type | Description |
|---|---|---|
| id * | string | Question ID, e.g. G3, I4 or F5 |
{
"name": "get_question",
"title": "Get a question",
"description": "One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"id"
],
"properties": {
"id": {
"type": "string",
"description": "Question ID, e.g. G3, I4 or F5"
}
}
},
"annotations": {
"title": "Get a question",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.
| Parameter | Type | Description |
|---|---|---|
| chart_id | string | e.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation |
| use_case | string | Filter the list |
{
"name": "get_charts",
"title": "Charts",
"description": "The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"chart_id": {
"type": "string",
"description": "e.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation"
},
"use_case": {
"enum": [
"all",
"gtm",
"insurance",
"finance",
"setup"
],
"type": "string",
"default": "all",
"description": "Filter the list"
}
}
},
"annotations": {
"title": "Charts",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.
No parameters.
{
"name": "get_methodology",
"title": "Methodology",
"description": "How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.",
"inputSchema": {
"type": "object",
"properties": {}
},
"annotations": {
"title": "Methodology",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.
No parameters.
{
"name": "how_to_run_locally",
"title": "Run it yourself",
"description": "How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.",
"inputSchema": {
"type": "object",
"properties": {}
},
"annotations": {
"title": "Run it yourself",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.
| Parameter | Type | Description |
|---|---|---|
| query * | string | What to look for |
{
"name": "search",
"title": "Search the benchmark",
"description": "Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"query"
],
"properties": {
"query": {
"type": "string",
"description": "What to look for"
}
}
},
"annotations": {
"title": "Search the benchmark",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).
| Parameter | Type | Description |
|---|---|---|
| id * | string | Document ID from search |
{
"name": "fetch",
"title": "Fetch a benchmark document",
"description": "Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).",
"inputSchema": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"required": [
"id"
],
"properties": {
"id": {
"type": "string",
"description": "Document ID from search"
}
}
},
"annotations": {
"title": "Fetch a benchmark document",
"readOnlyHint": true,
"openWorldHint": false,
"idempotentHint": true,
"destructiveHint": false
}
}Published benchmark of four web search APIs for AI agents (You.com, Tavily, Exa, Parallel): 30 agent-style queries across GTM, insurance and finance, 3 runs each, run on 2026-10-08. Use get_overall_results for the headline and overall scorecard, get_scorecard or compare_providers for numbers, recommend_provider for 'which API should I use', get_question for a specific query, get_charts for the published graphs, get_methodology for how it was measured, and how_to_run_locally to rerun it. Results are a fixed snapshot; always mention the run date and link the site.
Names, descriptions and schemas were written by the server and are shown as data.