Axis Spatial Spatial Intelligence Index

Search modelsMenu

GPSBench closed book benchmark leaderboard

Coordinate arithmetic and place knowledge, with no tools and no internet. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.

GPSBench closed book score

Share of questions answered correctly, 0 to 100 · Higher is better

GPSBench closed book scoreScores 0 to 100: Step 5 Preview (Reasoning): 93; Nemotron 3.5 Super 120B (Reasoning): 90; Nemotron 3.5 Super VL 120B (Reasoning): 89; Mistral Large 4 (high): 83; Nemotron 3.5 Lightning 30B (Reasoning): 78; Qwen 3.6 35B-A3B (Reasoning): 74; Muse Glimmer 30B (high): 73; Nemotron 3 Nano Omni 30B (Reasoning): 64; Laguna XS 2.1 (Reasoning): 430255075100Step 5 Preview (Reasoning)93Nemotron 3.5 Super 120B (Reasoning)90Nemotron 3.5 Super VL 120B (Reasoning)89Mistral Large 4 (high)83Nemotron 3.5 Lightning 30B (Reasoning)78Qwen 3.6 35B-A3B (Reasoning)74Muse Glimmer 30B (high)73Nemotron 3 Nano Omni 30B (Reasoning)64Laguna XS 2.1 (Reasoning)43
How to read this chart

Rows. Each model family once, at its best model and reasoning level, with at least 75% of the questions scored; the table lists every model and level, with how many questions each has scored.

Data table: GPSBench closed book score
GPSBench closed book score (share of questions answered correctly, 0 to 100)
ModelScoreQuestions scoredCost per questionTime per question (at 100 output tokens/s)
Step 5 Preview (Reasoning)93297 of 297$0.033859s
Nemotron 3.5 Super 120B (Reasoning)90297 of 297$0.003357s
Nemotron 3.5 Super VL 120B (Reasoning)89270 of 297$0.003869s
Mistral Large 4 (high)83297 of 297$0.030940s
Nemotron 3.5 Lightning 30B (Reasoning)78297 of 297$0.001991s
Qwen 3.6 35B-A3B (Reasoning)74297 of 297$0.006941s
Muse Glimmer 30B (high)73297 of 297$0.0077578s
Nemotron 3 Nano Omni 30B (Reasoning)64297 of 297No data108s
Laguna XS 2.1 (Reasoning)43297 of 297$0.0002922s

About this evaluation

297 items from the nine pure-GPS tasks of GPSBench's test split, such as converting a coordinate between formats or naming the place at a point. The model answers in one call with no tools and no internet.

The score is the share of items correct under GPSBench's own scorer. GPSBench is public, so a model may have seen its items in training.

Published by GPSBench authors. Licence: Code MIT, task files CC BY 4.0.

Related links