registryreasoning-chaindirect-answer

Direct answer

No scaffolding at all. The baseline for the reasoning suite.

Compare

Prompt

// no system prompt // user {{problem}}

Score history

Model versionVersionScore · CIHoldoutState
claude-opus-4.8v166.7 58.1–74.364.8fresh
llama-4.2-405bv154.8 46.1–63.253.7fresh
gemini-3.2-prov164.3 55.6–72.163.0fresh
gpt-5.6-solv168.3 59.7–75.766.7fresh
mistral-large-3v158.7 50.0–66.957.4fresh
claude-fable-5v169.0 60.5–76.568.5fresh
current scorev1
66.7
95% CI 58.1 to 74.3 · n=126
Holdout score64.8
Judgeprogrammatic
Cost per run$1.820
Median latency1.9s
LicenseCC-BY-4.0
Author@system
0 endorsements

Sign in to endorse this entry.

Across models

claude-opus-4.866.7fresh
llama-4.2-405b54.8fresh
gemini-3.2-pro64.3fresh
gpt-5.6-sol68.3fresh
mistral-large-358.7fresh
claude-fable-569.0fresh

Versions

v1Baseline66.7