Please wait while we prepare your content.
The whole bench on one grid. Rows are models, columns are our five task types. Read down a column to pick a model for a job.
| Model | Code gen | Extraction | Summariz. | Agentic | Classif. | Avg cost | P50 lat. |
|---|---|---|---|---|---|---|---|
| Passed rubric | Passed rubric | Passed rubric | Passed rubric | Mixed — passed with edits | $0.028 | 8.8s | |
| Passed rubric | Passed rubric | Mixed — passed with edits | Passed rubric | Passed rubric | $0.010 | 13.1s | |
| Mixed — passed with edits | Passed rubric | Passed rubric | Passed rubric | Passed rubric | $0.008 | 11.4s | |
| Passed rubric | Passed rubric | Passed rubric | Failed | Passed rubric | $0.004 | 3.4s | |
| Passed rubric | Mixed — passed with edits | Failed | Passed rubric | Passed rubric | $0.016 | 32.0s | |
| Mixed — passed with edits | Passed rubric | Passed rubric | Failed | Passed rubric | $0.003 | 4.9s | |
| Failed | Passed rubric | Mixed — passed with edits | Failed | Passed rubric | $0.002 | 6.1s | |
| Failed | Passed rubric | Failed | Failed | Mixed — passed with edits | $0.006 | 8.2s | |
| Mixed — passed with edits | Passed rubric | Failed | Failed | Not run yet | $0.005 | 24.1s | |
| Passed rubric | Failed | Not run yet | Failed | Failed | $0.001 | 4.8s |
Each cell is the best of two attempts against the task rubric. Baselines set the bar — a cheaper model that matches a baseline column-for-column is the real finding.