registryreasoning-chainverify-then-answer

Verify then answer

Requires the model to state its answer, then check it against the constraints before committing. Catches arithmetic slips that pure chain of thought misses.

Compare

Prompt

// system Solve the problem. Before giving your final answer, restate each constraint and confirm your answer satisfies it. If any check fails, revise. // user {{problem}}

Score history

Model versionVersionScore · CIHoldoutState
mistral-large-3v278.6 70.6–84.877.8fresh
claude-fable-5v288.9 82.2–93.388.9fresh
llama-4.2-405bv274.6 66.4–81.474.1fresh
gpt-5.6-solv288.1 81.3–92.787.0fresh
gemini-3.2-prov283.3 75.9–88.883.3fresh
claude-opus-4.8v285.7 78.5–90.885.2fresh
claude-fable-4v180.2 72.3–86.279.6superseded
current scorev2
78.6
95% CI 70.6 to 84.8 · n=126
Holdout score77.8
Judgeprogrammatic
Cost per run$0.415
Median latency1.7s
LicenseCC-BY-4.0
Author@marek
0 endorsements

Sign in to endorse this entry.

Across models

mistral-large-378.6fresh
claude-fable-588.9fresh
llama-4.2-405b74.6fresh
gpt-5.6-sol88.1fresh
gemini-3.2-pro83.3fresh
claude-opus-4.885.7fresh

Versions

v2Made the check explicit per constraint78.6
v1First submission80.2