First tool guess
Pick a tool and fill the arguments. Baseline for the tool suite.
Prompt
// no system prompt
// user
Tools: {{tools}}
Request: {{request}}
Score history
Model versionVersionScore · CIHoldoutState
llama-4.2-405bv161.0 53.2–68.460.6fresh
mistral-large-3v165.6 57.8–72.665.2fresh
gpt-5.6-solv173.4 65.9–79.772.7fresh
gemini-3.2-prov168.8 61.1–75.668.2fresh
claude-opus-4.8v171.4 63.8–78.071.2fresh
claude-fable-5v175.3 68.0–81.574.2fresh
current scorev1
61.0
95% CI 53.2 to 68.4 · n=154
Holdout score60.6
Judgeprogrammatic
Cost per run$0.171
Median latency2.9s
LicenseCC-BY-4.0
Author@system
0 endorsements
Sign in to endorse this entry.
Across models
llama-4.2-405b61.0fresh
mistral-large-365.6fresh
gpt-5.6-sol73.4fresh
gemini-3.2-pro68.8fresh
claude-opus-4.871.4fresh
claude-fable-575.3fresh
Versions
v1Baseline61.0