registrystructured-json-extractionrole-then-format

Role then format

Establishes the extraction role first, then constrains the output shape. Strong on flat objects, loses ground once nesting appears.

Compare

Prompt

// system You are a precise data extraction assistant. Return the result as JSON only. Format: JSON matching the described shape. Rules: 1. Omit keys you cannot determine. 2. Never invent keys absent from the schema. // user Extract from: {{document}}

Score history

Model versionVersionScore · CIHoldoutState
claude-fable-5v291.7 86.5–95.090.3fresh
llama-4.2-405bv276.8 69.8–82.576.4fresh
mistral-large-3v281.5 75.0–86.780.6fresh
claude-opus-4.8v288.7 83.0–92.687.5fresh
gemini-3.2-prov285.7 79.6–90.284.7fresh
gpt-5.6-solv290.5 85.1–94.188.9fresh
claude-fable-4v184.5 78.3–89.283.3superseded
current scorev2
91.7
95% CI 86.5 to 95.0 · n=168
Holdout score90.3
Judgeprogrammatic
Cost per run$1.850
Median latency2.4s
LicenseCC-BY-4.0
Author@ilya
0 endorsements

Sign in to endorse this entry.

Across models

claude-fable-591.7fresh
llama-4.2-405b76.8fresh
mistral-large-381.5fresh
claude-opus-4.888.7fresh
gemini-3.2-pro85.7fresh
gpt-5.6-sol90.5fresh

Versions

v2Tightened the format clause91.7
v1First submission84.5