RankedAI
BetaCtrl+K
| Model | Size | Version | Released | LBCode | AiderPoly | Aider | SWEBench | HumEval | HumEval+ | ArenaCode | WebDevArena | Codeforces | LBReason | MMLU Pro | MMLU | GPQA | Aidan | Halluc. | MATH | LBMath | LBIF | LBData | LBLang | LBAvg | Arena | AA Score | In Cost | Out Cost | Cutoff | Context | Org | Type | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 |   | o1 | Latest | Dec 17, 2024 | 69.7% | 61.7% | 84.2% | 41.0% | 1334 | 1015.79 | 1891 | 91.6% | 76.0% | 3204 | 67.3% | 81.5% | 65.5% | 65.4% | 73.5% | 1359 | $15 | $60 | Oct 23 | 200K | OpenAI | Proprietary | ||||||||
2 |   | Claude 3.5 Sonnet | Latest | Oct 22, 2024 | 67.1% | 45.3% | 84.2% | 49.0% | 93.7% | 86.2% | 1311 | 1218.58 | 58.7% | 78.0% | 88.0% | 65.0% | 2691 | 4.6% | 78.3% | 51.3% | 69.3% | 52.8% | 53.8% | 60.7% | 1282 | 80 | $3 | $15 | Apr 24 | 200K | Anthropic | Proprietary | ||
3 |  | Gemini Exp 1206 | Latest | Dec 6, 2024 | 63.4% | 38.2% | 80.5% | 1313 | 1017.09 | 57.0% | 70.0% | 77.3% | 63.2% | 50.8% | 63.7% | 1379 | 2M | Proprietary | ||||||||||||||||
4 |  | Deepseek 3 | 671B | Latest | Dec 26, 2024 | 61.8% | 48.4% | 79.7% | 42.0% | 82.6% | 86.6% | 1281 | 960.01 | 56.8% | 75.9% | 88.5% | 59.1% | 60.5% | 75.3% | 60.9% | 47.5% | 60.5% | 1315 | 80 | $0.07 | $1.1 | 128K | DeepSeek | Open Source | |||||
5 | | Claude 3.5 Sonnet | Old | Jun 20, 2024 | 60.9% | 77.4% | 33.4% | 92.0% | 81.7% | 1281 | 58.7% | 76.1% | 88.7% | 59.4% | 1439 | 71.1% | 53.3% | 68.0% | 56.7% | 53.2% | 59.7% | 1268 | 76 | $3 | $15 | Apr 24 | 200K | Anthropic | Proprietary | |||||
6 |  | Qwen 2.5 | 72B | Latest | Sep 19, 2024 | 57.6% | 55.6% | 23.8% | 86.6% | 51.2% | 1247 | 46.0% | 71.1% | 86.8% | 49.0% | 4.7% | 83.1% | 52.4% | 64.4% | 48.4% | 35.0% | 51.4% | 1259 | 75 | $0.38 | $0.57 | Sep 24 | 128K | Alibaba | Open Source | ||||
7 | | Qwen 2.5 Coder | 32B | Latest | Nov 12, 2024 | 56.9% | 8.0% | 72.9% | 92.7% | 86.3% | 1227 | 909.17 | 47.3% | 79.0% | 41.0% | 72.0% | 46.0% | 58.7% | 43.4% | 23.3% | 45.6% | 1220 | 70 | $0.2 | $0.2 | Sep 24 | 128K | Alibaba | Open Source | |||||
8 | | Gemini 2.0 Flash Exp | Latest | Dec 11, 2024 | 54.4% | 22.2% | 69.9% | 51.8% | 91.0% | 1290 | 971.68 | 59.1% | 76.4% | 87.0% | 62.1% | 1895 | 1.3% | 89.7% | 59.4% | 81.9% | 59.7% | 33.5% | 58.0% | 1356 | 82 | Aug 24 | 1M | Proprietary | ||||||
9 | | Gemini 2 Flash Thinking | Latest | Dec 19, 2024 | 53.1% | 1283 | 1022.92 | 64.6% | 2755 | 69.0% | 79.3% | 68.1% | 36.8% | 61.8% | 1369 | 32K | Proprietary | |||||||||||||||||
10 | | Gemini Exp 1114 | Old | Nov 14, 2024 | 52.4% | 60.9% | 1324 | 54.7% | 54.9% | 77.1% | 57.5% | 38.7% | 56.4% | 1343 | 32K | Proprietary | ||||||||||||||||||
11 | | GPT 4o | Old | Aug 6, 2024 | 51.4% | 71.4% | 90.0% | 87.2% | 1251 | 54.7% | 74.7% | 89.0% | 51.0% | 1333 | 78.0% | 48.2% | 68.6% | 52.9% | 47.6% | 55.1% | 1265 | 77 | $2.5 | $10 | Oct 23 | 128K | OpenAI | Proprietary | ||||||
12 | | Claude 3.5 Haiku | Latest | Oct 22, 2024 | 51.4% | 28.0% | 75.2% | 40.6% | 88.1% | 1263 | 1137.96 | 29.3% | 65.0% | 81.0% | 41.6% | 904 | 4.9% | 73.0% | 35.5% | 61.9% | 42.4% | 35.4% | 47.8% | 1239 | 69 | $1 | $5 | Apr 24 | 200K | Anthropic | Proprietary | |||
13 | | o1 Preview | Old | Sep 12, 2024 | 50.8% | 79.7% | 41.0% | 92.4% | 89.0% | 1311 | 1258 | 68.0% | 91.0% | 73.3% | 1892 | 3.3% | 85.0% | 62.9% | 74.6% | 64.0% | 68.7% | 69.1% | 1333 | 85 | $15 | $60 | Oct 23 | 128K | OpenAI | Proprietary | ||||
14 | | Gemini Exp 1121 | Old | Nov 21, 2024 | 50.4% | 57.9% | 1341 | 45.3% | 62.7% | 80.2% | 57.0% | 40.0% | 59.0% | 1365 | 32K | Proprietary | ||||||||||||||||||
15 | | Gemini 1.5 Pro 002 | Latest | Sep 24, 2024 | 48.8% | 66.9% | 84.1% | 79.3% | 1253 | 899.08 | 46.0% | 75.8% | 86.0% | 59.1% | 1515 | 6.6% | 86.5% | 57.4% | 70.8% | 52.3% | 43.3% | 55.5% | 1301 | 80 | $1.25 | $2.5 | Nov 23 | 2M | Proprietary | |||||
16 | | o1 mini | Latest | Sep 12, 2024 | 48.0% | 32.9% | 70.7% | 35.0% | 92.4% | 89.0% | 1298 | 1065.1 | 1650 | 77.3% | 85.0% | 60.0% | 1577 | 1.4% | 90.0% | 59.2% | 65.4% | 54.1% | 40.9% | 53.4% | 1308 | 82 | $3 | $12 | Oct 23 | 128K | OpenAI | Proprietary | ||
17 |  | Mistral Large | 123B | Latest | Nov 18, 2024 | 47.1% | 65.4% | 90.0% | 1228 | 43.4% | 84.0% | 47.0% | 953 | 4.1% | 72.0% | 32.6% | 63.7% | 46.6% | 39.5% | 50.0% | 1251 | 74 | $2 | $6 | Jul 24 | 128K | Mistral | Proprietary | ||||||
18 | | Mistral Large | 123B | Old | Jul 24, 2024 | 47.1% | 60.2% | 91.3% | 62.2% | 1244 | 42.0% | 65.9% | 85.0% | 48.0% | 4.1% | 72.0% | 43.7% | 67.9% | 44.0% | 39.4% | 50.4% | 1250 | 74 | $2 | $6 | Jul 24 | 128K | Mistral | Proprietary | |||||
19 | | DeepSeek 2.5 | 236B | Old | Sep 12, 2024 | 46.3% | 17.8% | 72.2% | 89.0% | 83.5% | 1255 | 39.3% | 65.8% | 81.0% | 42.0% | 55.0% | 56.7% | 45.8% | 31.6% | 44.7% | 1221 | 66 | $0.14 | $0.28 | Nov 23 | 128K | DeepSeek | Proprietary | ||||||
20 | | GPT 4o | Latest | Nov 20, 2024 | 46.1% | 15.1% | 72.9% | 31.0% | 90.0% | 1305 | 964.35 | 53.3% | 74.7% | 86.0% | 39.0% | 1269 | 1.5% | 69.0% | 42.5% | 64.9% | 47.2% | 47.4% | 53.2% | 1361 | 71 | $2.5 | $10 | Oct 23 | 128K | OpenAI | Proprietary | |||
21 |   | Grok 2 | Latest | Aug 13, 2024 | 45.1% | 58.6% | 88.4% | 80.5% | 1253 | 36.3% | 75.5% | 87.5% | 56.0% | 826 | 4.6% | 76.1% | 45.8% | 69.6% | 50.3% | 43.2% | 54.4% | 1290 | 70 | $5 | $15 | Aug 24 | 128K | xAI | Proprietary | |||||
22 | | GPT 4o mini | Latest | Jul 18, 2024 | 43.1% | 3.6% | 55.6% | 87.2% | 83.5% | 1245 | 35.3% | 63.1% | 82.0% | 43.0% | 1.7% | 75.0% | 35.6% | 56.8% | 44.5% | 28.6% | 43.3% | 1272 | 71 | $0.15 | $0.6 | Oct 23 | 128K | OpenAI | Proprietary | |||||
23 |  | Llama 3.1 | 405B | Latest | Jul 23, 2024 | 42.6% | 66.2% | 24.5% | 89.0% | 1260 | 818.58 | 53.3% | 73.3% | 87.0% | 51.1% | 778 | 3.9% | 73.8% | 40.5% | 75.9% | 54.5% | 45.5% | 58.6% | 1266 | 72 | $5 | $15 | Dec 23 | 128K | Meta | Open Source | |||
24 | | Gemini 1.5 Flash 002 | Old | Sep 24, 2024 | 41.9% | 52.6% | 84.0% | 75.6% | 1210 | 50.0% | 67.3% | 81.0% | 51.0% | 1047 | 3.4% | 77.9% | 47.2% | 78.8% | 44.2% | 27.9% | 50.3% | 1269 | 73 | $0.075 | $0.3 | Nov 23 | 1M | Proprietary | ||||||
25 | | DeepSeek Coder 2 | 236B | Latest | Jul 22, 2024 | 41.5% | 72.9% | 90.2% | 82.3% | 1226 | 45.3% | 63.6% | 80.0% | 42.0% | 58.0% | 67.2% | 38.3% | 33.0% | 47.2% | 1213 | 67 | $0.14 | $0.28 | Nov 23 | 128K | DeepSeek | Proprietary | |||||||
26 | | Gemini 1.5 Pro 001 | Old | Aug 27, 2024 | 40.9% | 66.9% | 1244 | 49.3% | 69.0% | 9.1% | 56.1% | 69.3% | 50.8% | 46.2% | 55.4% | 1260 | Nov 23 | 2M | Proprietary | |||||||||||||||
27 | | Gemini 1.5 Flash 001 | Old | Aug 27, 2024 | 40.6% | 52.6% | 1208 | 47.3% | 59.1% | 6.6% | 73.0% | 47.9% | 29.6% | 50.1% | 1227 | Nov 23 | 1M | Proprietary | ||||||||||||||||
28 | | Claude 3 Opus | Latest | Mar 4, 2024 | 38.6% | 68.4% | 22.0% | 84.9% | 77.4% | 1236 | 41.3% | 68.5% | 84.0% | 50.4% | 908 | 10.1% | 60.1% | 43.4% | 63.9% | 54.3% | 50.4% | 56.2% | 1248 | 70 | $15 | $75 | Aug 23 | 200K | Anthropic | Proprietary | ||||
29 | | Qwen 2.5 | 7B | Latest | Sep 19, 2024 | 38.4% | 63.9% | 26.2% | 2.8% | 38.2% | 52.1% | 32.8% | 15.8% | 33.6% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||
30 | | Grok-2 mini | Latest | Aug 13, 2024 | 37.5% | 54.9% | 85.7% | 1230 | 42.0% | 72.0% | 86.2% | 51.0% | 73.0% | 69.7% | 44.3% | 39.5% | 51.1% | 1266 | Aug 24 | 128K | xAI | Proprietary | ||||||||||||
31 | | QwQ Preview | 32B | Experimental | Nov 28, 2024 | 37.2% | 57.7% | 75.0% | 65.2% | 12.9% | 50.8% | 35.6% | 3.2% | 19.6% | 28.5% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||
32 |   | Nova Pro | Latest | Dec 3, 2024 | 37.1% | 54.1% | 89.0% | 1228 | 32.6% | 84.0% | 46.9% | 76.6% | 39.1% | 67.1% | 43.6% | 37.0% | 49.2% | 75 | $0.8 | $3.2 | 300K | Amazon | Proprietary | |||||||||||
33 | | Llama 3.3 | 70B | Latest | Dec 6, 2024 | 36.6% | 59.4% | 88.4% | 87.6% | 1232 | 50.7% | 68.9% | 86.0% | 50.5% | 779 | 4.0% | 77.0% | 41.1% | 82.7% | 48.3% | 39.2% | 56.7% | 74 | $0.1 | $0.4 | Meta | Open Source | |||||||
34 | | Gemma 2 | 27B | Latest | Jun 27, 2024 | 35.9% | 51.8% | 1203 | 28.1% | 56.5% | 75.2% | 1028 | 42.3% | 26.2% | 58.1% | 43.6% | 32.6% | 37.4% | Jun 24 | 8K | Open Source | |||||||||||||
35 | | Llama 3.1 | 70B | Old | Jul 23, 2024 | 32.7% | 58.6% | 80.5% | 1209 | 40.7% | 66.4% | 86.0% | 43.0% | 5.0% | 60.0% | 34.4% | 69.0% | 49.1% | 34.6% | 50.9% | 1242 | 65 | $1 | $3 | Dec 23 | 128K | Meta | Open Source | ||||||
36 | | Gemini 1.5 Flash | 8B | Latest | Sep 24, 2024 | 28.7% | 38.3% | 1167 | 33.3% | 58.7% | 38.4% | 58.7% | 52.6% | 44.0% | 31.6% | 42.7% | 1177 | $0.0375 | $0.15 | Nov 23 | 1M | Proprietary | ||||||||||||
37 | | Mistral Nemo | 12B | Latest | Jul 24, 2024 | 28.7% | 33.1% | 25.3% | 44.8% | 68.0% | 33.0% | 11.2% | 40.0% | 16.9% | 51.8% | 33.4% | 14.1% | 32.0% | 52 | $0.15 | $0.15 | Apr 24 | 128K | Mistral | Proprietary | |||||||||
38 | | Nova Lite | Latest | Dec 3, 2024 | 27.5% | 85.4% | 1200 | 34.7% | 79.0% | 42.0% | 73.3% | 30.8% | 70 | $0.06 | $0.24 | 300K | Amazon | Proprietary | ||||||||||||||||
39 | | Qwen 2.5 | 32B | Latest | Sep 19, 2024 | 27.0% | 58.6% | 88.4% | 69.2% | 83.9% | 49.5% | 3.0% | 83.1% | Sep 24 | 128K | Alibaba | Open Source | |||||||||||||||||
40 | | Claude 3 Haiku | Old | Mar 7, 2024 | 24.5% | 47.4% | 75.9% | 1184 | 25.9% | 42.3% | 75.2% | 33.3% | 40.9% | 22.9% | 1179 | 54 | $0.25 | $1.25 | Aug 23 | 200K | Anthropic | Proprietary | ||||||||||||
41 | | Qwen 2.5 Coder | 14B | Latest | Nov 12, 2024 | 23.4% | 69.2% | 89.6% | 84.0% | Sep 24 | 128K | Alibaba | Open Source | |||||||||||||||||||||
42 | | Gemma 2 | 9B | Latest | Jun 27, 2024 | 22.5% | 40.2% | 1180 | 15.2% | 52.1% | 71.3% | 859 | 10.1% | 36.6% | 19.5% | 52.6% | 35.1% | 25.5% | 28.4% | Jun 24 | 8K | Open Source | ||||||||||||
43 | | Qwen 2.5 | 14B | Latest | Sep 19, 2024 | 20.9% | 83.5% | 63.7% | 80.0% | 45.5% | 4.2% | 80.0% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||
44 | | Qwen 2.5 Coder | 7B | Latest | Nov 12, 2024 | 20.3% | 63.9% | 88.4% | 81.9% | Sep 24 | 128K | Alibaba | Open Source | |||||||||||||||||||||
45 | | Nova Micro | Latest | Dec 3, 2024 | 20.2% | 81.1% | 1181 | 25.1% | 76.0% | 40.0% | 69.3% | 33.7% | 66 | $0.04 | $0.14 | 128K | Amazon | Proprietary | ||||||||||||||||
46 | | Llama 3.1 | 8B | Latest | Jul 23, 2024 | 19.7% | 37.6% | 72.6% | 62.8% | 1174 | 15.3% | 44.3% | 71.0% | 27.0% | 5.4% | 64.0% | 16.6% | 1171 | 53 | $0.3 | $0.6 | Dec 23 | 128K | Meta | Open Source | |||||||||
47 | | Qwen 2.5 Coder | 3B | Latest | Nov 12, 2024 | 10.8% | 39.1% | 84.1% | 75.2% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||||||||
48 | | Qwen 2.5 Coder | 1.5B | Latest | Nov 12, 2024 | 6.1% | 31.6% | 70.7% | 66.5% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||||||||
49 | | Qwen 2.5 | 1.5B | Latest | Sep 19, 2024 | 4.7% | 32.1% | 15.8% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||
50 | | Qwen 2.5 Coder | 0.5B | Latest | Nov 12, 2024 | 2.0% | 14.3% | 61.6% | 53.8% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||||||||
51 | | Qwen 2.5 | 0.5B | Latest | Sep 19, 2024 | 2.0% | 14.9% | 25.2% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||
52 | | Llama 3.2 | 3B | Latest | Sep 25, 2024 | 26.3% | 1048 | 22.2% | 63.4% | 21.0% | 7.9% | 49.0% | 1093 | 46 | $0.08 | $0.1 | Dec 23 | 128K | Meta | Open Source | ||||||||||||||
53 | | Pixtral Large | 123B | Latest | Nov 18, 2024 | 82.0% | 85.0% | 52.0% | 72.0% | 73 | $2 | $6 | 128K | Mistral | Proprietary | |||||||||||||||||||
54 | | Deepseek 2.5 | Old | Dec 10, 2024 | 1262 | Nov 23 | 128K | DeepSeek | Proprietary | |||||||||||||||||||||||||
55 | Sky T1 Preview | 32B | Latest | Jan 11, 2025 | 56.8% | NovaSky | Open Source | |||||||||||||||||||||||||||
56 | | Qwen 2.5 | 3B | Latest | Sep 19, 2024 | 43.7% | 7.0% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||||||||||
57 | | o3 mini low | Upcoming | 1697 | OpenAI | Proprietary | ||||||||||||||||||||||||||||
58 | | o3 mini medium | Upcoming | 1997 | OpenAI | Proprietary | ||||||||||||||||||||||||||||
59 | | o3 mini high | Upcoming | 2073 | OpenAI | Proprietary | ||||||||||||||||||||||||||||
60 | | o3 | Upcoming | 2727 | 87.7% | OpenAI | Proprietary | |||||||||||||||||||||||||||
61 |  | Phi 4 | 14B | Latest | Dec 12, 2024 | 82.6% | 82.8% | 70.4% | 84.8% | 56.1% | 80.4% | Microsoft | Open Source | |||||||||||||||||||||
62 | | o1 Pro mode | Latest | 79.0% | OpenAI | Proprietary | ||||||||||||||||||||||||||||
63 | | R1 Lite Preview | Latest | Nov 20, 2024 | 58.5% | 91.6% | DeepSeek | Proprietary | ||||||||||||||||||||||||||
64 | | Llama 3.2 | 1B | Latest | Sep 25, 2024 | 1009 | 11.9% | 49.3% | 14.0% | 20.7% | 27.0% | 1044 | 28 | $0.04 | $0.08 | Dec 23 | 128K | Meta | Open Source | |||||||||||||||
65 | | Codestral 25.01 | Latest | Jan 14, 2025 | 11.1% | 86.6% | 256K | Mistral | Proprietary | |||||||||||||||||||||||||
66 | | Llama 3.2 (Vision) | 90B | Latest | Sep 25, 2024 | 86.0% | 43.0% | 4.3% | 68.0% | 67 | $0.9 | $0.9 | Dec 23 | 128K | Meta | Open Source | ||||||||||||||||||
67 | | Llama 3.2 (Vision) | 11B | Latest | Sep 25, 2024 | 73.0% | 26.0% | 5.5% | 51.9% | 54 | $0.19 | $0.19 | Dec 23 | 128K | Meta | Open Source | ||||||||||||||||||
68 | | Gemma 2 | 2B | Latest | Jul 31, 2024 | 17.7% | 51.3% | 7.0% | 15.0% | Jun 24 | 8K | Open Source | ||||||||||||||||||||||
69 | | Qwen 2.5 Turbo | Latest | Nov 15, 2024 | Sep 24 | 1M | Alibaba | Proprietary |
69 models