GPSBench closed book benchmark leaderboard
Coordinate arithmetic and place knowledge, with no tools and no internet. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.
GPSBench closed book score
Share of questions answered correctly, 0 to 100 · Higher is better
How to read this chart
Rows. Each model family once, at its best model and reasoning level, with at least 75% of the questions scored; the table lists every model and level, with how many questions each has scored.
Data table: GPSBench closed book score
| Model | Score | Questions scored | Cost per question | Time per question (at 100 output tokens/s) |
|---|---|---|---|---|
| Step 5 Preview (Reasoning) | 93 | 297 of 297 | $0.033 | 859s |
| Nemotron 3.5 Super 120B (Reasoning) | 90 | 297 of 297 | $0.0033 | 57s |
| Nemotron 3.5 Super VL 120B (Reasoning) | 89 | 270 of 297 | $0.0038 | 69s |
| Mistral Large 4 (high) | 83 | 297 of 297 | $0.030 | 940s |
| Nemotron 3.5 Lightning 30B (Reasoning) | 78 | 297 of 297 | $0.0019 | 91s |
| Qwen 3.6 35B-A3B (Reasoning) | 74 | 297 of 297 | $0.0069 | 41s |
| Muse Glimmer 30B (high) | 73 | 297 of 297 | $0.0077 | 578s |
| Nemotron 3 Nano Omni 30B (Reasoning) | 64 | 297 of 297 | No data | 108s |
| Laguna XS 2.1 (Reasoning) | 43 | 297 of 297 | $0.00029 | 22s |
About this evaluation
297 items from the nine pure-GPS tasks of GPSBench's test split, such as converting a coordinate between formats or naming the place at a point. The model answers in one call with no tools and no internet.
The score is the share of items correct under GPSBench's own scorer. GPSBench is public, so a model may have seen its items in training.
Published by GPSBench authors. Licence: Code MIT, task files CC BY 4.0.
