Artificial Analysis Progress Index v1
The Artificial Analysis Progress Index v1 scores language models released since 2022 on one scale that stays consistent over time.
Score
Artificial Analysis Progress Index, Over Time
How It Works
The Progress Index v1 complements the Artificial Analysis Intelligence Index. We recommend the Intelligence Index for choosing a model today, and the Progress Index v1 for comparing models over longer time periods.
The Intelligence Index is a weighted average of scores on a fixed set of evaluations. We change that set with each version to reflect current trends in model usage and capability, so scores from different versions can't be compared. For example, in v5, we replaced Humanity's Last Exam with HLE-Diamond, among other changes. The Progress Index v1 instead incorporates results across retired evaluations and evaluations not included in the Intelligence Index, so its scale for scoring language models stays consistent over time.
We fit each task* to a difficulty and model to a score on the same scale using item response theory, a statistical method used to score standardized tests. A model whose score matches a task's difficulty has a 50% chance of solving it.
| Intelligence Index | Progress Index v1 | |
|---|---|---|
| Use Case | Choosing a model today | Comparing models over longer time periods |
| Evaluations | 9 evaluations | Current and retired evaluations |
| Consistency | A new version, not comparable with earlier ones | The same scale, so scores stay comparable |
| Score | Weighted average of evaluation scores | Estimated from task results |
* For some evaluations, such as HumanEval and MATH-500, we estimate one difficulty for the whole evaluation instead of one per task. See the methodology for details.
Example Tasks
Each chart shows the estimated probability that a model solves one task, based on its Progress Index v1 score. Dots mark the top model released in each year, and the table lists each year's top three.
Every morning Aya goes for a -kilometer-long walk and stops at a coffee shop afterwards. When she walks at a constant speed of kilometers per hour, the walk takes her 4 hours, including minutes spent in the coffee shop. When she walks kilometers per hour, the walk takes her 2 hours and 24 minutes, including minutes spent in the coffee shop. Suppose Aya walks at kilometers per hour. Find the number of minutes the walk takes her, including the minutes spent in the coffee shop.
AIME 2024 I, Problem 1
| Released | Model | Progress Index v1 | Chance of solving |
|---|---|---|---|
| 2022–2023 | Claude 2.0 | 128 | 9% |
| GPT-4 | 140 | 26% | |
| GPT-4 Turbo | 160 | 74% | |
| 2024 | o1-mini | 200 | 99% |
| o1-preview | 226 | >99% | |
| o1 | 228 | >99% | |
| 2025 | GPT-5.2 Codex | 290 | >99% |
| Gemini 3 Pro Preview | 291 | >99% | |
| GPT-5.2 | 295 | >99% | |
| 2026 | Gemini 4 Carbon | 351 | >99% |
| Claude Opus 5.5 | 351 | >99% | |
| Claude Fable 5.2 | 358 | >99% |