Competition Math (AIME)
90 tasks
Hard mathematical-reasoning benchmark built from AIME 2024, 2025, and 2026 (American Invitational Mathematics Examination). Every answer is an integer in 0..999, so scoring is fully DETERMINISTIC and un-gameable — we extract the model's final integer (`Final answer: N` sentinel, then \boxed{N}, then last integer) and compare for exact equality. No LLM judge, so no judge cost or judge variance poisoning the reasoning priors. Closes a frontier category (hard reasoning/math) the scorecard was previously blind to.
| Model | Success | $ / task | $ / success | Tasks |
|---|---|---|---|---|
codaidefaultbest | 93.3% | $0.0413 | $0.0442 | 60/60 |
claude-opus-4-8 | 91.7% | $0.0326 | $0.0356 | 60/60 |
codai,claude-opus-4-8 | 90.0% | $0.0260 | $0.0289 | 60/60 |