Place and coordinate tasks benchmark leaderboard
Finding places by name and coordinates, and the distance and direction between them, from Place and coordinate tasks. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.
Score
All
Place and coordinate tasks score
Share of tasks passed, 0 to 100 · Higher is better
How to read this chart
What this metric means. The share of tasks the entry passed, 0 to 100.
Data table: Place and coordinate tasks score
| Model | Developer | Place and coordinate tasks score |
|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 100 |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 100 |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 100 |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 100 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 100 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 100 |
| Claude Opus 5.5 (high), Claude Code | Anthropic | 100 |
| Claude Sonnet 5.5 (high), Claude Code | Anthropic | 100 |
| Claude Haiku 5.5 (high), Claude Code | Anthropic | 100 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 98 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 83 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 83 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 60 |
Open weights
Place and coordinate tasks score
Share of tasks passed, 0 to 100 · Higher is better
How to read this chart
What this metric means. The share of tasks the entry passed, 0 to 100.
Data table: Place and coordinate tasks score
| Model | Developer | Place and coordinate tasks score |
|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 100 |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 100 |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 100 |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 100 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 100 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 100 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 98 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 83 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 83 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 60 |
All
Place and coordinate tasks: score vs. cost per task
Place and coordinate tasks score against average cost per task (USD) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher Place and coordinate tasks score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.
Data table: Place and coordinate tasks: score vs. cost per task
| Model | Place and coordinate tasks score | Cost per task (USD) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | $0.0044 | No |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | $0.0017 | Yes |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | $0.0062 | No |
| Muse Glimmer 30B (high), Terminus-2 | 100 | $0.018 | No |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | $0.013 | No |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | $0.024 | No |
| Claude Opus 5.5 (high), Claude Code | 100 | $0.063 | No |
| Claude Sonnet 5.5 (high), Claude Code | 100 | $0.030 | No |
| Claude Haiku 5.5 (high), Claude Code | 100 | $0.0028 | No |
| Mistral Large 4 (high), Terminus-2 | 98 | $0.0093 | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | $0.0052 | No |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | $0.010 | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | No data | No data |
Open weights
Place and coordinate tasks: score vs. cost per task
Place and coordinate tasks score against average cost per task (USD) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher Place and coordinate tasks score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.
Data table: Place and coordinate tasks: score vs. cost per task
| Model | Place and coordinate tasks score | Cost per task (USD) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | $0.0044 | No |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | $0.0017 | Yes |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | $0.0062 | No |
| Muse Glimmer 30B (high), Terminus-2 | 100 | $0.018 | No |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | $0.013 | No |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | $0.024 | No |
| Mistral Large 4 (high), Terminus-2 | 98 | $0.0093 | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | $0.0052 | No |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | $0.010 | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | No data | No data |
Token usage
All
Place and coordinate tasks: tokens per task
Average tokens per task by type · Lower is better
- Uncached input
- Cached input
- Output (reasoning included)
How to read this chart
What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.
Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.
Data table: Place and coordinate tasks: tokens per task
| Model | Developer | Uncached input | Cached input | Output (reasoning included) | Total |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 5k | 10k | 2k | 18k |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 5k | 11k | 2k | 19k |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 14k | 4k | 2k | 20k |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 7k | 16k | 3k | 26k |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 7k | 20k | 2k | 29k |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 4k | 40k | 5k | 49k |
| Claude Sonnet 5.5 (high), Claude Code | Anthropic | 5k | 44k | 523 | 49k |
| Claude Opus 5.5 (high), Claude Code | Anthropic | 5k | 43k | 581 | 49k |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 34k | 15k | 4k | 53k |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 23k | 63k | 5k | 91k |
| Claude Haiku 5.5 (high), Claude Code | Anthropic | 6k | 92k | 1k | 99k |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 15k | 90k | 10k | 114k |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 12k | 159k | 6k | 178k |
Open weights
Place and coordinate tasks: tokens per task
Average tokens per task by type · Lower is better
- Uncached input
- Cached input
- Output (reasoning included)
How to read this chart
What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.
Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.
Data table: Place and coordinate tasks: tokens per task
| Model | Developer | Uncached input | Cached input | Output (reasoning included) | Total |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 5k | 10k | 2k | 18k |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 5k | 11k | 2k | 19k |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 14k | 4k | 2k | 20k |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 7k | 16k | 3k | 26k |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 7k | 20k | 2k | 29k |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 4k | 40k | 5k | 49k |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 34k | 15k | 4k | 53k |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 23k | 63k | 5k | 91k |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 15k | 90k | 10k | 114k |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 12k | 159k | 6k | 178k |
Cost
All
Place and coordinate tasks: cost per task
Average cost per task (USD) by token type · Lower is better
- Uncached input
- Cached input
- Output
- Not split by token type
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.
Data table: Place and coordinate tasks: cost per task
| Model | Developer | Uncached input | Cached input | Output | Not split by token type | Total |
|---|---|---|---|---|---|---|
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | $0.00093 | $0.000056 | $0.00069 | $0.00 | $0.0017 |
| Claude Haiku 5.5 (high), Claude Code | Anthropic | $0.00063 | $0.00092 | $0.00058 | $0.00063 | $0.0028 |
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | $0.00077 | $0.000031 | $0.0014 | $0.0022 | $0.0044 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | $0.0014 | $0.0028 | $0.0011 | $0.00000027 | $0.0052 |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | $0.0021 | $0.00097 | $0.0031 | $0.00000027 | $0.0062 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | $0.0033 | $0.00079 | $0.0052 | $0.00 | $0.0093 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | $0.0012 | $0.0080 | $0.0012 | $0.00 | $0.010 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | $0.0084 | $0.00044 | $0.0047 | $0.00 | $0.013 |
| Muse Glimmer 30B (high), Terminus-2 | Meta | $0.012 | $0.00061 | $0.0057 | $0.00 | $0.018 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | $0.0043 | $0.0042 | $0.015 | $0.00000013 | $0.024 |
| Claude Sonnet 5.5 (high), Claude Code | Anthropic | $0.010 | $0.0044 | $0.0052 | $0.0099 | $0.030 |
| Claude Opus 5.5 (high), Claude Code | Anthropic | $0.021 | $0.0087 | $0.012 | $0.021 | $0.063 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | No data | No data | No data | No data | No data |
Open weights
Place and coordinate tasks: cost per task
Average cost per task (USD) by token type · Lower is better
- Uncached input
- Cached input
- Output
- Not split by token type
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.
Data table: Place and coordinate tasks: cost per task
| Model | Developer | Uncached input | Cached input | Output | Not split by token type | Total |
|---|---|---|---|---|---|---|
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | $0.00093 | $0.000056 | $0.00069 | $0.00 | $0.0017 |
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | $0.00077 | $0.000031 | $0.0014 | $0.0022 | $0.0044 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | $0.0014 | $0.0028 | $0.0011 | $0.00000027 | $0.0052 |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | $0.0021 | $0.00097 | $0.0031 | $0.00000027 | $0.0062 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | $0.0033 | $0.00079 | $0.0052 | $0.00 | $0.0093 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | $0.0012 | $0.0080 | $0.0012 | $0.00 | $0.010 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | $0.0084 | $0.00044 | $0.0047 | $0.00 | $0.013 |
| Muse Glimmer 30B (high), Terminus-2 | Meta | $0.012 | $0.00061 | $0.0057 | $0.00 | $0.018 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | $0.0043 | $0.0042 | $0.015 | $0.00000013 | $0.024 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | No data | No data | No data | No data | No data |
Speed
All
Place and coordinate tasks: time per task
Time per task (at 100 output tokens/s), minutes · Lower is better
How to read this chart
Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
Data table: Place and coordinate tasks: time per task
| Model | Developer | Time per task (min) |
|---|---|---|
| Claude Sonnet 5.5 (high), Claude Code | Anthropic | 0.2 min |
| Claude Opus 5.5 (high), Claude Code | Anthropic | 0.2 min |
| Claude Haiku 5.5 (high), Claude Code | Anthropic | 0.3 min |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 0.4 min |
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 0.5 min |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 0.7 min |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 0.7 min |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 0.8 min |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 0.8 min |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 1.2 min |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 1.3 min |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 1.4 min |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 2.2 min |
Open weights
Place and coordinate tasks: time per task
Time per task (at 100 output tokens/s), minutes · Lower is better
How to read this chart
Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
Data table: Place and coordinate tasks: time per task
| Model | Developer | Time per task (min) |
|---|---|---|
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 0.4 min |
| DeepSeek V4.1 Flash (max), Terminus-2 | DeepSeek | 0.5 min |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 0.7 min |
| MiniMax M3 (Reasoning), Terminus-2 | MiniMax | 0.7 min |
| Muse Glimmer 30B (high), Terminus-2 | Meta | 0.8 min |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | Xiaomi | 0.8 min |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 1.2 min |
| Laguna XS 2.1 (Reasoning), Terminus-2 | Poolside | 1.3 min |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 1.4 min |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 2.2 min |
All
Place and coordinate tasks: score vs. time per task
Place and coordinate tasks score against time per task (at 100 output tokens/s), minutes · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher Place and coordinate tasks score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.
Data table: Place and coordinate tasks: score vs. time per task
| Model | Place and coordinate tasks score | Time per task (minutes) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | 0.5 min | No |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | 0.8 min | No |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | 0.7 min | No |
| Muse Glimmer 30B (high), Terminus-2 | 100 | 0.8 min | No |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | 0.4 min | No |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | 1.2 min | No |
| Claude Opus 5.5 (high), Claude Code | 100 | 0.2 min | No |
| Claude Sonnet 5.5 (high), Claude Code | 100 | 0.2 min | Yes |
| Claude Haiku 5.5 (high), Claude Code | 100 | 0.3 min | No |
| Mistral Large 4 (high), Terminus-2 | 98 | 0.7 min | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | 1.4 min | No |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | 1.3 min | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | 2.2 min | No |
Open weights
Place and coordinate tasks: score vs. time per task
Place and coordinate tasks score against time per task (at 100 output tokens/s), minutes · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher Place and coordinate tasks score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.
Data table: Place and coordinate tasks: score vs. time per task
| Model | Place and coordinate tasks score | Time per task (minutes) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | 0.5 min | No |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | 0.8 min | No |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | 0.7 min | No |
| Muse Glimmer 30B (high), Terminus-2 | 100 | 0.8 min | No |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | 0.4 min | Yes |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | 1.2 min | No |
| Mistral Large 4 (high), Terminus-2 | 98 | 0.7 min | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | 1.4 min | No |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | 1.3 min | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | 2.2 min | No |
Score vs. release date
All
Place and coordinate tasks: score vs. release date
Place and coordinate tasks score against the model's public release date
How to read this chart
How to read this chart. Each point is one entry: across, the date its model was first released; up, its Place and coordinate tasks score. Models with no published release date are not plotted.
Data table: Place and coordinate tasks: score vs. release date
| Model | Place and coordinate tasks score | Release date |
|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | 10 September 2026 |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | No data |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | No data |
| Muse Glimmer 30B (high), Terminus-2 | 100 | No data |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | 4 June 2026 |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | No data |
| Claude Opus 5.5 (high), Claude Code | 100 | 22 September 2026 |
| Claude Sonnet 5.5 (high), Claude Code | 100 | 28 September 2026 |
| Claude Haiku 5.5 (high), Claude Code | 100 | 7 October 2026 |
| Mistral Large 4 (high), Terminus-2 | 98 | No data |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | 11 August 2026 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | 2 July 2026 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | 28 April 2026 |
Open weights
Place and coordinate tasks: score vs. release date
Place and coordinate tasks score against the model's public release date
How to read this chart
How to read this chart. Each point is one entry: across, the date its model was first released; up, its Place and coordinate tasks score. Models with no published release date are not plotted.
Data table: Place and coordinate tasks: score vs. release date
| Model | Place and coordinate tasks score | Release date |
|---|---|---|
| DeepSeek V4.1 Flash (max), Terminus-2 | 100 | 10 September 2026 |
| MiMo V2.6 Flash (Reasoning), Terminus-2 | 100 | No data |
| MiniMax M3 (Reasoning), Terminus-2 | 100 | No data |
| Muse Glimmer 30B (high), Terminus-2 | 100 | No data |
| Nemotron 3 Ultra (high), Terminus-2 | 100 | 4 June 2026 |
| Step 5 Preview (Reasoning), Terminus-2 | 100 | No data |
| Mistral Large 4 (high), Terminus-2 | 98 | No data |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 83 | 11 August 2026 |
| Laguna XS 2.1 (Reasoning), Terminus-2 | 83 | 2 July 2026 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 60 | 28 April 2026 |
Background
Tasks we built, in four families: find a place's coordinates from its name, name the place at a coordinate, measure the distance between two places, and give the direction from one to another. Every continent is covered in each family, with planted hard cases: places that share a name, old names, routes across the 180th meridian and long routes where a straight line on the map gives the wrong bearing.
Licence: GeoNames data CC BY 4.0; Natural Earth public domain.
Published by us, from GeoNames and Natural Earth data.
Related links
Explore evaluations
- Geospatial Intelligence IndexAgent tasks, vision, guided workflows and geospatial knowledge, per model
- Geospatial Agent IndexEarth-Bench V2, GeoBenchX and GS-QA2; Text + vision adds Vision agents
- Earth-Bench V2Earth-Bench V2 poses Earth observation questions over satellite products (indices, temperatures, land cover) that are answered by computing over the data.
- GeoBenchXGeoBenchX asks for maps, numbers and lists from country-level and city-level geospatial data.
- GS-QA2GS-QA2 asks questions about places and terrain in the United States over OpenStreetMap vector layers and ASTER elevation data: nearest features, directions, distances, counts, elevations, slopes and features that meet terrain conditions.
- GeoAgentBenchGeoAgentBench gives an agent a geospatial question and the data to answer it, and expects the result files and a map.
- GeoAnalystBenchGeoAnalystBench collects GIS analyst workflows (suitability analysis, interpolation, hot spots, network and overlay work).
- OGD4AllOGD4All asks everyday questions about the City of Zurich's open government data: trees, buildings, transport, schools and more, from about 430 public datasets in GeoPackage and CSV form.
- GeoNatureAgentGeoNatureAgent asks for figures from environmental rasters over Spanish provinces and regions: here, the share of land suitable for CO₂ storage, by province, with follow-up questions that compare, rank or break the figures down.
- MultiGlobeQAMultiGlobeQA asks spatial questions about places all over the world: what lies within, next to or nearest to what, how far apart things are, and how many there are.
- GeoAgentEvalGeoAgentEval tests whether an agent can build, audit and repair spatial data in a PostGIS database, and decline honestly when a request cannot be done: flood zones, wards, routes, coordinate systems and stale records in one made-up town.
- ImpactMeshImpactMesh pairs Sentinel-2 optical and Sentinel-1 radar images, taken at four dates around a wildfire or a flood, with the burn scars and flood extents mapped by the Copernicus Emergency Management Service.
- OpenEarthAgent MapsOpenEarthAgent Maps asks questions about real places over OpenStreetMap: which features lie beyond a distance or a travel time from the nearest school, hospital or fire station, which pairs are closest, and averages over them.
- OpenEarthAgent Sentinel-2OpenEarthAgent Sentinel-2 asks how a place changed between two dates: vegetation loss and gain, urban growth, or burn severity.
- GS-QA2 FullGS-QA2 Full asks the same questions as GS-QA2, over the same OpenStreetMap vector layers and ASTER elevation data, but runs every question that passed our admission checks, not a balanced sample, across almost all of the release's question templates.
- InSAR Regional Snow MappingOur first tasks on synthetic aperture radar (SAR): 64 Sentinel-1 interferogram windows from January 2021 to March 2023 over the Trentino Alps, with phase, coherence and displacement layers, two weather layers and modelled snow depth.
- DLPU phase unwrappingA radar interferogram records ground height and movement as a phase that wraps around every cycle.
- GeoChronoGeoChrono asks about long-term change in a place seen over many years: what the land cover is, whether an area grew or shrank, when an object appeared and how things changed in order.
- GeoChrono larger setThe larger GeoChrono set is 1,000 of the 17,688 ChronoBench questions that have a keyed answer, 250 for each of GeoChrono's four levels, chosen with a fixed seed.
- BigEarthNet.txtBigEarthNet.txt asks about 1.2 km image patches over Europe: which land cover is present, how much, how many patches of it, what is next to what, and the country, season and climate zone.
- DVL-BenchDVL-Bench asks about change in a place seen over several years: which kinds of ground cover changed, which change was least, whether vegetation or built-up surface grew or shrank, when building expansion stopped, in which period the most change happened, and by how much.
- ThinkGeo drawn (judged)ThinkGeo asks an agent to find objects in an aerial or radar image and show the answer on the image: draw a box on the smallest plane, label each aircraft with a number, write the count of ships, or plot a chart.
- Geospatial knowledgeClosed-book multiple-choice questions on geospatial knowledge
- VisionQuestions answered from images
- Earth-Bench V2 imagesEarth-Bench V2 images are the Earth-Bench V2 questions that need the model to look at RGB satellite pictures: multiple choice on counting objects or scene types, ranking pictures by count or area, and comparing what changed.
- ThinkGeo imagesThinkGeo asks for one number about aerial and satellite images: a count, an area, a distance or a heading, accepted within a range.
- Vision agentsAgents that look at aerial images and use code
