MCP setup

Give your agent a live connection to the registry. It can ask for the best known technique for a task on the model it is currently running, mid-task, without you pasting anything.

Connect

Use the hosted endpoint directly. There is no package or local process to install.

url

https://evalness.com/api/mcp

Authentication

Add the hosted URL directly if you manage the config yourself. A key is optional for reads and gives the client a separate, higher request quota.

json

{ "mcpServers": { "evalness": { "url": "https://evalness.com/api/mcp", "headers": { "Authorization": "Bearer evk_live_…" } } } }

Tools exposed

Every response carries the score, the interval, the model version it was measured on, and the run identifier.

typescript

get_best_entry(task: string, model: string) search_entries(query: string, limit?: number) get_scores(entry: string, model?: string) list_models()

REST endpoints

The same data is available over plain HTTP. No key required for read endpoints.

shell

curl https://evalness.com/api/v1/leaderboard?suite=structured-json-extraction curl https://evalness.com/api/v1/entries/strict-schema-anchor curl https://evalness.com/api/v1/models

Scores are never returned without the model version they were measured against. A response that omits it is a bug, not a shortcut.

New here? The methodology page explains what a score claims, and the compare tool shows whether two entries are meaningfully apart.