Place and coordinate tasks, closed book benchmark leaderboard
Geocoding, reverse geocoding, distance and direction, with no tools and no internet. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.
Place and coordinate tasks, closed book score
Share of questions answered correctly, 0 to 100 · Higher is better
How to read this chart
Rows. Each model family once, at its best model and reasoning level, with at least 75% of the questions scored; the table lists every model and level, with how many questions each has scored.
Data table: Place and coordinate tasks, closed book score
| Model | Score | Questions scored | Cost per question | Time per question (at 100 output tokens/s) |
|---|---|---|---|---|
| Claude Opus 5.5 (high) | 92 | 144 of 144 | $0.026 | 15s |
| Claude Sonnet 5.5 (high) | 85 | 144 of 144 | $0.015 | 27s |
| DeepSeek V4 Flash (max) | 72 | 144 of 144 | $0.016 | 139s |
| DeepSeek V4 Flash (high) | 71 | 144 of 144 | $0.015 | 116s |
| Claude Haiku 5.5 (high) | 69 | 144 of 144 | $0.0022 | 21s |
| Nemotron 3 Ultra (high) | 67 | 144 of 144 | $0.038 | 496s |
| DeepSeek V4.1 Flash (max) | 60 | 144 of 144 | $0.017 | 136s |
| Step 5 Preview (Reasoning) | 60 | 144 of 144 | $0.084 | 692s |
| GLM-5.3-Flash (low) | 55 | 144 of 144 | $0.0016 | 42s |
| GLM-5.3 (low) | 54 | 144 of 144 | $0.0092 | 17s |
| Muse Glimmer 30B (high) | 52 | 144 of 144 | $0.018 | 543s |
| Nemotron 3.5 Lightning 30B (Reasoning) | 45 | 144 of 144 | $0.0023 | 144s |
| Mistral Large 4 (high) | 42 | 138 of 144 | $0.062 | 1933s |
| Nemotron 3.5 Super VL 120B (Reasoning) | 42 | 140 of 144 | $0.017 | 322s |
| Qwen 3.6 35B-A3B (Reasoning) | 42 | 144 of 144 | $0.0096 | 83s |
| MiMo V2.6 Flash (Reasoning) | 38 | 144 of 144 | $0.0044 | 142s |
| GLM-5.3-Flash (max) | 34 | 144 of 144 | $0.019 | 282s |
| MiniMax M3 (Reasoning) | 33 | 144 of 144 | $0.027 | 130s |
| GLM-5.3 (max) | 29 | 144 of 144 | $0.13 | 192s |
| MiniMax M3.1 Flash (max) | 28 | 144 of 144 | $0.017 | 1580s |
| Laguna XS 2.1 (Reasoning) | 22 | 144 of 144 | $0.0020 | 138s |
| Qwen 3.8 27B (xhigh) | 21 | 144 of 144 | $0.12 | 2177s |
| Ministral 3 14B (Non-reasoning) | 10 | 144 of 144 | No data | 26s |
| Nemotron 3 Nano Omni 30B (Reasoning) | 10 | 144 of 144 | No data | 203s |
| Nemotron 3.5 Bolt E4B (Reasoning) | 8 | 144 of 144 | No data | 132s |
About this evaluation
48 items built by Axis Spatial on GeoNames and Natural Earth places: find a place's coordinates, name the place at a point, give a distance or a bearing. The model answers in one call with no tools and no internet.
The score is the share of items whose answer is within the task's checked threshold.
Published by Axis Spatial. Licence: GeoNames CC BY 4.0, Natural Earth public domain.
