registrysummarization-fidelityplain-summary

Plain summary

Ask for a summary and hope. Baseline for the fidelity suite.

Compare

Prompt

// no system prompt // user Summarize: {{source}}

Score history

Model versionVersionScore · CIHoldoutState
llama-4.2-405bv158.0 48.8–66.856.3fresh
gemini-3.2-prov167.0 57.8–75.066.7fresh
claude-opus-4.8v170.5 61.5–78.268.8fresh
claude-fable-5v173.2 64.3–80.572.9fresh
mistral-large-3v163.4 54.2–71.762.5fresh
gpt-5.6-solv171.4 62.5–79.070.8fresh
current scorev1
58.0
95% CI 48.8 to 66.8 · n=112
Holdout score56.3
Judgemodel
Cost per run$0.221
Median latency2.3s
LicenseCC-BY-4.0
Author@system
0 endorsements

Sign in to endorse this entry.

Across models

llama-4.2-405b58.0fresh
gemini-3.2-pro67.0fresh
claude-opus-4.870.5fresh
claude-fable-573.2fresh
mistral-large-363.4fresh
gpt-5.6-sol71.4fresh

Versions

v1Baseline58.0