Axis Spatial Spatial Intelligence Index

Search modelsMenu

OpenEarthAgent Maps benchmark leaderboard

Distances and routes between places on OpenStreetMap, from OpenEarthAgent Maps. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.

Score

All

OpenEarthAgent Maps score

Share of tasks passed, 0 to 100 · Higher is better

OpenEarthAgent Maps scoreScores 0 to 100: Claude Sonnet 5.5 (xhigh), Claude Code: 100; Mistral Large 4 (high), Terminus-2: 100; Claude Opus 5.5 (medium), Claude Code: 100; Claude Sonnet 5.5 (high), Claude Code: 100; Claude Opus 5.5 (xhigh), Claude Code: 100; Claude Opus 5.5 (max), Claude Code: 100; Claude Haiku 5.5 (max), Claude Code: 100; Claude Opus 5.5 (high), Claude Code: 100; Claude Haiku 5.5 (high), Claude Code: 100; Step 5 Preview (Reasoning), Terminus-2: 99; Claude Sonnet 5.5 (medium), Claude Code: 99; Claude Haiku 5.5 (xhigh), Claude Code: 99; Claude Opus 5.5 (low), Claude Code: 99; Claude Sonnet 5.5 (low), Claude Code: 99; Claude Opus 5 (high), Claude Code: 99; Claude Opus 4.8 (high), Claude Code: 98; Claude Haiku 5.5 (low), Claude Code: 97; Claude Opus 4.7 (high), Claude Code: 97; Claude Haiku 5.5 (medium), Claude Code: 97; Claude Sonnet 5 (high), Claude Code: 96; Nemotron 3 Ultra (high), Terminus-2: 95; Claude Haiku 4.5 (Reasoning), Claude Code: 93; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 84; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 330255075100Claude Sonnet 5.5 (xhigh)Claude Code100Mistral Large 4 (high)Terminus-2100Claude Opus 5.5 (medium)Claude Code100Claude Sonnet 5.5 (high)Claude Code100Claude Opus 5.5 (xhigh)Claude Code100Claude Opus 5.5 (max)Claude Code100Claude Haiku 5.5 (max)Claude Code100Claude Opus 5.5 (high)Claude Code100Claude Haiku 5.5 (high)Claude Code100Step 5 Preview (Reasoning)Terminus-299Claude Sonnet 5.5 (medium)Claude Code99Claude Haiku 5.5 (xhigh)Claude Code99Claude Opus 5.5 (low)Claude Code99Claude Sonnet 5.5 (low)Claude Code99Claude Opus 5 (high)Claude Code99Claude Opus 4.8 (high)Claude Code98Claude Haiku 5.5 (low)Claude Code97Claude Opus 4.7 (high)Claude Code97Claude Haiku 5.5 (medium)Claude Code97Claude Sonnet 5 (high)Claude Code96Nemotron 3 Ultra (high)Terminus-295Claude Haiku 4.5 (Reasoning)Claude Code93Nemotron 3.5 Super VL 120B (Reasoning)Terminus-285Nemotron 3.5 Lightning 30B (Reasoning)Terminus-284Nemotron 3 Nano Omni 30B (Reasoning)Terminus-233
How to read this chart

What this metric means. The share of tasks the entry passed, 0 to 100.

Data table: OpenEarthAgent Maps score

Open weights

OpenEarthAgent Maps score

Share of tasks passed, 0 to 100 · Higher is better

OpenEarthAgent Maps scoreScores 0 to 100: Mistral Large 4 (high), Terminus-2: 100; Step 5 Preview (Reasoning), Terminus-2: 99; Nemotron 3 Ultra (high), Terminus-2: 95; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 84; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 330255075100Mistral Large 4 (high)Terminus-2100Step 5 Preview (Reasoning)Terminus-299Nemotron 3 Ultra (high)Terminus-295Nemotron 3.5 Super VL 120B (Reasoning)Terminus-285Nemotron 3.5 Lightning 30B (Reasoning)Terminus-284Nemotron 3 Nano Omni 30B (Reasoning)Terminus-233
How to read this chart

What this metric means. The share of tasks the entry passed, 0 to 100.

Data table: OpenEarthAgent Maps score

All

OpenEarthAgent Maps: score vs. cost per task

OpenEarthAgent Maps score against average cost per task (USD) · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
OpenEarthAgent Maps: score vs. cost per taskOpenEarthAgent Maps score (0 to 100) against cost per task in US dollars, log scale: Mistral Large 4 (high), Terminus-2: 99.76, $0.020; Nemotron 3 Ultra (high), Terminus-2: 94.83, $0.024; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, $0.0079; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, $0.0045; Step 5 Preview (Reasoning), Terminus-2: 99.28, $0.044; Claude Haiku 4.5 (Reasoning), Claude Code: 93.03, $0.043; Claude Opus 5.5 (medium), Claude Code: 99.76, $0.067; Claude Opus 5.5 (high), Claude Code: 99.52, $0.072; Claude Sonnet 5.5 (medium), Claude Code: 99.28, $0.033; Claude Sonnet 5.5 (high), Claude Code: 99.76, $0.034; Claude Haiku 5.5 (medium), Claude Code: 96.88, $0.0024; Claude Haiku 5.5 (high), Claude Code: 99.52, $0.0029; Claude Opus 5.5 (xhigh), Claude Code: 99.76, $0.088; Claude Sonnet 5.5 (xhigh), Claude Code: 100, $0.042; Claude Haiku 5.5 (xhigh), Claude Code: 99.04, $0.0045; Claude Opus 5.5 (low), Claude Code: 98.8, $0.065; Claude Sonnet 5.5 (low), Claude Code: 98.8, $0.032; Claude Haiku 5.5 (low), Claude Code: 97.36, $0.0023; Claude Opus 5.5 (max), Claude Code: 99.76, $0.29; Claude Haiku 5.5 (max), Claude Code: 99.76, $0.0099; Claude Opus 5 (high), Claude Code: 98.8, $0.12; Claude Sonnet 5 (high), Claude Code: 96.15, $0.055; Claude Opus 4.8 (high), Claude Code: 97.6, $0.12; Claude Opus 4.7 (high), Claude Code: 97.36, $0.18Most attractive quadrant828486889092949698100$0.002$0.005$0.01$0.02$0.05$0.10$0.20$0.50Cost per task (USD, log scale)OpenEarthAgent Maps scoreClaude Sonnet 5.5 (xhigh), Claude CodeMistral Large 4 (high), Terminus-2Claude Haiku 5.5 (max), Claude CodeClaude Haiku 5.5 (high), Claude CodeClaude Haiku 5.5 (low), Claude CodeClaude Opus 5.5 (medium), Claude CodeClaude Sonnet 5.5 (high), Claude CodeClaude Opus 5.5 (xhigh), Claude CodeClaude Opus 5.5 (max), Claude CodeClaude Opus 5.5 (high), Claude CodeStep 5 Preview (Reasoning), Terminus-2Claude Sonnet 5.5 (medium), Claude CodeClaude Haiku 5.5 (xhigh), Claude CodeClaude Opus 5.5 (low), Claude CodeClaude Sonnet 5.5 (low), Claude CodeClaude Opus 5 (high), Claude CodeClaude Opus 4.8 (high), Claude CodeClaude Opus 4.7 (high), Claude CodeClaude Haiku 5.5 (medium), Claude CodeClaude Sonnet 5 (high), Claude CodeNemotron 3 Ultra (high), Terminus-2Claude Haiku 4.5 (Reasoning), Claude CodeNemotron 3.5 Super VL 120B (Reasoning), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher OpenEarthAgent Maps score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.

Data table: OpenEarthAgent Maps: score vs. cost per task

Open weights

OpenEarthAgent Maps: score vs. cost per task

OpenEarthAgent Maps score against average cost per task (USD) · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
OpenEarthAgent Maps: score vs. cost per taskOpenEarthAgent Maps score (0 to 100) against cost per task in US dollars, log scale: Mistral Large 4 (high), Terminus-2: 99.76, $0.020; Nemotron 3 Ultra (high), Terminus-2: 94.83, $0.024; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, $0.0079; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, $0.0045; Step 5 Preview (Reasoning), Terminus-2: 99.28, $0.044Most attractive quadrant828486889092949698100$0.002$0.005$0.01$0.02$0.05Cost per task (USD, log scale)OpenEarthAgent Maps scoreMistral Large 4 (high), Terminus-2Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2Step 5 Preview (Reasoning), Terminus-2Nemotron 3 Ultra (high), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher OpenEarthAgent Maps score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.

Data table: OpenEarthAgent Maps: score vs. cost per task
OpenEarthAgent Maps: score vs. cost per task
ModelOpenEarthAgent Maps scoreCost per task (USD)On the Pareto line
Mistral Large 4 (high), Terminus-2100$0.020Yes
Step 5 Preview (Reasoning), Terminus-299$0.044No
Nemotron 3 Ultra (high), Terminus-295$0.024No
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-285$0.0045Yes
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-284$0.0079No
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-233No dataNo data

Token usage

All

OpenEarthAgent Maps: tokens per task

Average tokens per task by type · Lower is better

  • Uncached input
  • Cached input
  • Output (reasoning included)
OpenEarthAgent Maps: tokens per taskNemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 35k; Nemotron 3 Ultra (high), Terminus-2: 46k; Claude Sonnet 5.5 (low), Claude Code: 52k; Claude Opus 5.5 (low), Claude Code: 52k; Claude Sonnet 5.5 (medium), Claude Code: 53k; Claude Sonnet 5.5 (high), Claude Code: 54k; Mistral Large 4 (high), Terminus-2: 56k; Claude Opus 5.5 (medium), Claude Code: 56k; Claude Opus 5.5 (high), Claude Code: 57k; Claude Sonnet 5.5 (xhigh), Claude Code: 66k; Claude Opus 5.5 (xhigh), Claude Code: 68k; Claude Opus 4.8 (high), Claude Code: 69k; Claude Opus 5 (high), Claude Code: 73k; Claude Haiku 5.5 (medium), Claude Code: 91k; Claude Haiku 5.5 (low), Claude Code: 91k; Step 5 Preview (Reasoning), Terminus-2: 100k; Claude Haiku 5.5 (high), Claude Code: 102k; Claude Sonnet 5 (high), Claude Code: 107k; Claude Haiku 4.5 (Reasoning), Claude Code: 139k; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 147k; Claude Haiku 5.5 (xhigh), Claude Code: 154k; Claude Opus 4.7 (high), Claude Code: 157k; Claude Opus 5.5 (max), Claude Code: 176k; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 192k; Claude Haiku 5.5 (max), Claude Code: 256k0100k200k300k400kNemotron 3.5 Super VL 120B (Reasoning)Terminus-235kNemotron 3 Ultra (high)Terminus-246kClaude Sonnet 5.5 (low)Claude Code52kClaude Opus 5.5 (low)Claude Code52kClaude Sonnet 5.5 (medium)Claude Code53kClaude Sonnet 5.5 (high)Claude Code54kMistral Large 4 (high)Terminus-256kClaude Opus 5.5 (medium)Claude Code56kClaude Opus 5.5 (high)Claude Code57kClaude Sonnet 5.5 (xhigh)Claude Code66kClaude Opus 5.5 (xhigh)Claude Code68kClaude Opus 4.8 (high)Claude Code69kClaude Opus 5 (high)Claude Code73kClaude Haiku 5.5 (medium)Claude Code91kClaude Haiku 5.5 (low)Claude Code91kStep 5 Preview (Reasoning)Terminus-2100kClaude Haiku 5.5 (high)Claude Code102kClaude Sonnet 5 (high)Claude Code107kClaude Haiku 4.5 (Reasoning)Claude Code139kNemotron 3.5 Lightning 30B (Reasoning)Terminus-2147kClaude Haiku 5.5 (xhigh)Claude Code154kClaude Opus 4.7 (high)Claude Code157kClaude Opus 5.5 (max)Claude Code176kNemotron 3 Nano Omni 30B (Reasoning)Terminus-2192kClaude Haiku 5.5 (max)Claude Code256k
How to read this chart

What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Data table: OpenEarthAgent Maps: tokens per task
OpenEarthAgent Maps: tokens per task
ModelDeveloperUncached inputCached inputOutput (reasoning included)Total
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2NVIDIA19k13k3k35k
Nemotron 3 Ultra (high), Terminus-2NVIDIA22k21k3k46k
Claude Sonnet 5.5 (low), Claude CodeAnthropic5k46k68152k
Claude Opus 5.5 (low), Claude CodeAnthropic5k46k70652k
Claude Sonnet 5.5 (medium), Claude CodeAnthropic5k47k70353k
Claude Sonnet 5.5 (high), Claude CodeAnthropic5k47k75354k
Mistral Large 4 (high), Terminus-2Mistral AI11k40k5k56k
Claude Opus 5.5 (medium), Claude CodeAnthropic5k50k80156k
Claude Opus 5.5 (high), Claude CodeAnthropic6k51k85157k
Claude Sonnet 5.5 (xhigh), Claude CodeAnthropic7k58k95866k
Claude Opus 5.5 (xhigh), Claude CodeAnthropic7k60k1k68k
Claude Opus 4.8 (high), Claude CodeAnthropic6k62k1k69k
Claude Opus 5 (high), Claude CodeAnthropic7k66k96673k
Claude Haiku 5.5 (medium), Claude CodeAnthropic5k84k1k91k
Claude Haiku 5.5 (low), Claude CodeAnthropic5k85k91091k
Step 5 Preview (Reasoning), Terminus-2StepFun12k81k8k100k
Claude Haiku 5.5 (high), Claude CodeAnthropic7k94k1k102k
Claude Sonnet 5 (high), Claude CodeAnthropic6k100k939107k
Claude Haiku 4.5 (Reasoning), Claude CodeAnthropic9k128k3k139k
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2NVIDIA31k111k5k147k
Claude Haiku 5.5 (xhigh), Claude CodeAnthropic10k142k2k154k
Claude Opus 4.7 (high), Claude CodeAnthropic7k148k1k157k
Claude Opus 5.5 (max), Claude CodeAnthropic17k153k6k176k
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2NVIDIA23k156k13k192k
Claude Haiku 5.5 (max), Claude CodeAnthropic17k230k8k256k

Open weights

OpenEarthAgent Maps: tokens per task

Average tokens per task by type · Lower is better

  • Uncached input
  • Cached input
  • Output (reasoning included)
OpenEarthAgent Maps: tokens per taskNemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 35k; Nemotron 3 Ultra (high), Terminus-2: 46k; Mistral Large 4 (high), Terminus-2: 56k; Step 5 Preview (Reasoning), Terminus-2: 100k; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 147k; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 192k050k100k150k200kNemotron 3.5 Super VL 120B (Reasoning)Terminus-235kNemotron 3 Ultra (high)Terminus-246kMistral Large 4 (high)Terminus-256kStep 5 Preview (Reasoning)Terminus-2100kNemotron 3.5 Lightning 30B (Reasoning)Terminus-2147kNemotron 3 Nano Omni 30B (Reasoning)Terminus-2192k
How to read this chart

What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Data table: OpenEarthAgent Maps: tokens per task
OpenEarthAgent Maps: tokens per task
ModelDeveloperUncached inputCached inputOutput (reasoning included)Total
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2NVIDIA19k13k3k35k
Nemotron 3 Ultra (high), Terminus-2NVIDIA22k21k3k46k
Mistral Large 4 (high), Terminus-2Mistral AI11k40k5k56k
Step 5 Preview (Reasoning), Terminus-2StepFun12k81k8k100k
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2NVIDIA31k111k5k147k
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2NVIDIA23k156k13k192k

Cost

All

OpenEarthAgent Maps: cost per task

Average cost per task (USD) by token type · Lower is better

  • Uncached input
  • Cached input
  • Output
  • Not split by token type
OpenEarthAgent Maps: cost per taskValues in US dollars: Claude Haiku 5.5 (low), Claude Code: $0.0023; Claude Haiku 5.5 (medium), Claude Code: $0.0024; Claude Haiku 5.5 (high), Claude Code: $0.0029; Claude Haiku 5.5 (xhigh), Claude Code: $0.0045; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: $0.0045; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: $0.0079; Claude Haiku 5.5 (max), Claude Code: $0.0099; Mistral Large 4 (high), Terminus-2: $0.020; Nemotron 3 Ultra (high), Terminus-2: $0.024; Claude Sonnet 5.5 (low), Claude Code: $0.032; Claude Sonnet 5.5 (medium), Claude Code: $0.033; Claude Sonnet 5.5 (high), Claude Code: $0.034; Claude Sonnet 5.5 (xhigh), Claude Code: $0.042; Claude Haiku 4.5 (Reasoning), Claude Code: $0.043; Step 5 Preview (Reasoning), Terminus-2: $0.044; Claude Sonnet 5 (high), Claude Code: $0.055; Claude Opus 5.5 (low), Claude Code: $0.065; Claude Opus 5.5 (medium), Claude Code: $0.067; Claude Opus 5.5 (high), Claude Code: $0.072; Claude Opus 5.5 (xhigh), Claude Code: $0.088; Claude Opus 4.8 (high), Claude Code: $0.12; Claude Opus 5 (high), Claude Code: $0.12; Claude Opus 4.7 (high), Claude Code: $0.18; Claude Opus 5.5 (max), Claude Code: $0.29$0$0.10$0.20$0.30$0.40Claude Haiku 5.5 (low)Claude Code$0.0023Claude Haiku 5.5 (medium)Claude Code$0.0024Claude Haiku 5.5 (high)Claude Code$0.0029Claude Haiku 5.5 (xhigh)Claude Code$0.0045Nemotron 3.5 Super VL 120B (Reasoning)Terminus-2$0.0045Nemotron 3.5 Lightning 30B (Reasoning)Terminus-2$0.0079Claude Haiku 5.5 (max)Claude Code$0.0099Mistral Large 4 (high)Terminus-2$0.020Nemotron 3 Ultra (high)Terminus-2$0.024Claude Sonnet 5.5 (low)Claude Code$0.032Claude Sonnet 5.5 (medium)Claude Code$0.033Claude Sonnet 5.5 (high)Claude Code$0.034Claude Sonnet 5.5 (xhigh)Claude Code$0.042Claude Haiku 4.5 (Reasoning)Claude Code$0.043Step 5 Preview (Reasoning)Terminus-2$0.044Claude Sonnet 5 (high)Claude Code$0.055Claude Opus 5.5 (low)Claude Code$0.065Claude Opus 5.5 (medium)Claude Code$0.067Claude Opus 5.5 (high)Claude Code$0.072Claude Opus 5.5 (xhigh)Claude Code$0.088Claude Opus 4.8 (high)Claude Code$0.12Claude Opus 5 (high)Claude Code$0.12Claude Opus 4.7 (high)Claude Code$0.18Claude Opus 5.5 (max)Claude Code$0.29
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.

Data table: OpenEarthAgent Maps: cost per task
OpenEarthAgent Maps: cost per task
ModelDeveloperUncached inputCached inputOutputNot split by token typeTotal
Claude Haiku 5.5 (low), Claude CodeAnthropic$0.00051$0.00085$0.00046$0.00051$0.0023
Claude Haiku 5.5 (medium), Claude CodeAnthropic$0.00051$0.00084$0.00051$0.00051$0.0024
Claude Haiku 5.5 (high), Claude CodeAnthropic$0.00067$0.00094$0.00064$0.00067$0.0029
Claude Haiku 5.5 (xhigh), Claude CodeAnthropic$0.00098$0.0014$0.0011$0.00098$0.0045
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2NVIDIA$0.0029$0.00025$0.0014$0.00000031$0.0045
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2NVIDIA$0.0018$0.0050$0.0010$0.00000044$0.0079
Claude Haiku 5.5 (max), Claude CodeAnthropic$0.0017$0.0023$0.0042$0.0017$0.0099
Mistral Large 4 (high), Terminus-2Mistral AI$0.0080$0.0028$0.0095$0.00000020$0.020
Nemotron 3 Ultra (high), Terminus-2NVIDIA$0.013$0.0025$0.0079$0.00000012$0.024
Claude Sonnet 5.5 (low), Claude CodeAnthropic$0.010$0.0046$0.0068$0.010$0.032
Claude Sonnet 5.5 (medium), Claude CodeAnthropic$0.011$0.0047$0.0070$0.011$0.033
Claude Sonnet 5.5 (high), Claude CodeAnthropic$0.011$0.0047$0.0075$0.011$0.034
Claude Sonnet 5.5 (xhigh), Claude CodeAnthropic$0.014$0.0058$0.0096$0.014$0.042
Claude Haiku 4.5 (Reasoning), Claude CodeAnthropic$0.0087$0.013$0.013$0.0087$0.043
Step 5 Preview (Reasoning), Terminus-2StepFun$0.012$0.0085$0.023$0.00$0.044
Claude Sonnet 5 (high), Claude CodeAnthropic$0.013$0.020$0.0094$0.013$0.055
Claude Opus 5.5 (low), Claude CodeAnthropic$0.021$0.0093$0.014$0.021$0.065
Claude Opus 5.5 (medium), Claude CodeAnthropic$0.021$0.010$0.016$0.021$0.067
Claude Opus 5.5 (high), Claude CodeAnthropic$0.022$0.010$0.017$0.022$0.072
Claude Opus 5.5 (xhigh), Claude CodeAnthropic$0.027$0.012$0.021$0.027$0.088
Claude Opus 4.8 (high), Claude CodeAnthropic$0.029$0.031$0.028$0.029$0.12
Claude Opus 5 (high), Claude CodeAnthropic$0.033$0.033$0.024$0.033$0.12
Claude Opus 4.7 (high), Claude CodeAnthropic$0.037$0.074$0.031$0.037$0.18
Claude Opus 5.5 (max), Claude CodeAnthropic$0.066$0.031$0.12$0.066$0.29
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2NVIDIANo dataNo dataNo dataNo dataNo data

Open weights

OpenEarthAgent Maps: cost per task

Average cost per task (USD) by token type · Lower is better

  • Uncached input
  • Cached input
  • Output
  • Not split by token type
OpenEarthAgent Maps: cost per taskValues in US dollars: Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: $0.0045; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: $0.0079; Mistral Large 4 (high), Terminus-2: $0.020; Nemotron 3 Ultra (high), Terminus-2: $0.024; Step 5 Preview (Reasoning), Terminus-2: $0.044$0$0.015$0.03$0.045$0.06Nemotron 3.5 Super VL 120B (Reasoning)Terminus-2$0.0045Nemotron 3.5 Lightning 30B (Reasoning)Terminus-2$0.0079Mistral Large 4 (high)Terminus-2$0.020Nemotron 3 Ultra (high)Terminus-2$0.024Step 5 Preview (Reasoning)Terminus-2$0.044
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.

Data table: OpenEarthAgent Maps: cost per task
OpenEarthAgent Maps: cost per task
ModelDeveloperUncached inputCached inputOutputNot split by token typeTotal
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2NVIDIA$0.0029$0.00025$0.0014$0.00000031$0.0045
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2NVIDIA$0.0018$0.0050$0.0010$0.00000044$0.0079
Mistral Large 4 (high), Terminus-2Mistral AI$0.0080$0.0028$0.0095$0.00000020$0.020
Nemotron 3 Ultra (high), Terminus-2NVIDIA$0.013$0.0025$0.0079$0.00000012$0.024
Step 5 Preview (Reasoning), Terminus-2StepFun$0.012$0.0085$0.023$0.00$0.044
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2NVIDIANo dataNo dataNo dataNo dataNo data

Speed

All

OpenEarthAgent Maps: time per task

Time per task (at 100 output tokens/s), minutes · Lower is better

OpenEarthAgent Maps: time per taskOpenEarthAgent Maps: time per task: Claude Sonnet 5.5 (high), Claude Code: 0.6 min; Claude Sonnet 5.5 (low), Claude Code: 0.6 min; Claude Opus 5.5 (low), Claude Code: 0.6 min; Claude Haiku 5.5 (low), Claude Code: 0.6 min; Claude Opus 5.5 (medium), Claude Code: 0.6 min; Claude Haiku 5.5 (medium), Claude Code: 0.6 min; Claude Sonnet 5.5 (xhigh), Claude Code: 0.6 min; Claude Sonnet 5 (high), Claude Code: 0.6 min; Claude Opus 5.5 (xhigh), Claude Code: 0.6 min; Claude Sonnet 5.5 (medium), Claude Code: 0.6 min; Claude Haiku 5.5 (high), Claude Code: 0.7 min; Claude Opus 5.5 (high), Claude Code: 0.7 min; Claude Opus 5 (high), Claude Code: 0.7 min; Claude Opus 4.8 (high), Claude Code: 0.9 min; Claude Opus 4.7 (high), Claude Code: 0.9 min; Claude Haiku 5.5 (xhigh), Claude Code: 0.9 min; Claude Haiku 4.5 (Reasoning), Claude Code: 1.1 min; Claude Opus 5.5 (max), Claude Code: 1.9 min; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 2.1 min; Claude Haiku 5.5 (max), Claude Code: 2.2 min; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 2.8 min; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 3.0 min; Mistral Large 4 (high), Terminus-2: 3.3 min; Step 5 Preview (Reasoning), Terminus-2: 3.8 min; Nemotron 3 Ultra (high), Terminus-2: 4.8 min0 min1.5 min3 min4.5 min6 minClaude Sonnet 5.5 (high)Claude Code0.6 minClaude Sonnet 5.5 (low)Claude Code0.6 minClaude Opus 5.5 (low)Claude Code0.6 minClaude Haiku 5.5 (low)Claude Code0.6 minClaude Opus 5.5 (medium)Claude Code0.6 minClaude Haiku 5.5 (medium)Claude Code0.6 minClaude Sonnet 5.5 (xhigh)Claude Code0.6 minClaude Sonnet 5 (high)Claude Code0.6 minClaude Opus 5.5 (xhigh)Claude Code0.6 minClaude Sonnet 5.5 (medium)Claude Code0.6 minClaude Haiku 5.5 (high)Claude Code0.7 minClaude Opus 5.5 (high)Claude Code0.7 minClaude Opus 5 (high)Claude Code0.7 minClaude Opus 4.8 (high)Claude Code0.9 minClaude Opus 4.7 (high)Claude Code0.9 minClaude Haiku 5.5 (xhigh)Claude Code0.9 minClaude Haiku 4.5 (Reasoning)Claude Code1.1 minClaude Opus 5.5 (max)Claude Code1.9 minNemotron 3.5 Lightning 30B (Reasoning)Terminus-22.1 minClaude Haiku 5.5 (max)Claude Code2.2 minNemotron 3.5 Super VL 120B (Reasoning)Terminus-22.8 minNemotron 3 Nano Omni 30B (Reasoning)Terminus-23.0 minMistral Large 4 (high)Terminus-23.3 minStep 5 Preview (Reasoning)Terminus-23.8 minNemotron 3 Ultra (high)Terminus-24.8 min
How to read this chart

Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

Data table: OpenEarthAgent Maps: time per task

Open weights

OpenEarthAgent Maps: time per task

Time per task (at 100 output tokens/s), minutes · Lower is better

OpenEarthAgent Maps: time per taskOpenEarthAgent Maps: time per task: Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 2.1 min; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 2.8 min; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 3.0 min; Mistral Large 4 (high), Terminus-2: 3.3 min; Step 5 Preview (Reasoning), Terminus-2: 3.8 min; Nemotron 3 Ultra (high), Terminus-2: 4.8 min0 min1.5 min3 min4.5 min6 minNemotron 3.5 Lightning 30B (Reasoning)Terminus-22.1 minNemotron 3.5 Super VL 120B (Reasoning)Terminus-22.8 minNemotron 3 Nano Omni 30B (Reasoning)Terminus-23.0 minMistral Large 4 (high)Terminus-23.3 minStep 5 Preview (Reasoning)Terminus-23.8 minNemotron 3 Ultra (high)Terminus-24.8 min
How to read this chart

Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

Data table: OpenEarthAgent Maps: time per task

All

OpenEarthAgent Maps: score vs. time per task

OpenEarthAgent Maps score against time per task (at 100 output tokens/s), minutes · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
OpenEarthAgent Maps: score vs. time per taskOpenEarthAgent Maps score (0 to 100) against time per task (minutes), log scale: Mistral Large 4 (high), Terminus-2: 99.76, 3.3 min; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 32.69, 3.0 min; Nemotron 3 Ultra (high), Terminus-2: 94.83, 4.8 min; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, 2.1 min; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, 2.8 min; Step 5 Preview (Reasoning), Terminus-2: 99.28, 3.8 min; Claude Haiku 4.5 (Reasoning), Claude Code: 93.03, 1.1 min; Claude Opus 5.5 (medium), Claude Code: 99.76, 0.6 min; Claude Opus 5.5 (high), Claude Code: 99.52, 0.7 min; Claude Sonnet 5.5 (medium), Claude Code: 99.28, 0.6 min; Claude Sonnet 5.5 (high), Claude Code: 99.76, 0.6 min; Claude Haiku 5.5 (medium), Claude Code: 96.88, 0.6 min; Claude Haiku 5.5 (high), Claude Code: 99.52, 0.7 min; Claude Opus 5.5 (xhigh), Claude Code: 99.76, 0.6 min; Claude Sonnet 5.5 (xhigh), Claude Code: 100, 0.6 min; Claude Haiku 5.5 (xhigh), Claude Code: 99.04, 0.9 min; Claude Opus 5.5 (low), Claude Code: 98.8, 0.6 min; Claude Sonnet 5.5 (low), Claude Code: 98.8, 0.6 min; Claude Haiku 5.5 (low), Claude Code: 97.36, 0.6 min; Claude Opus 5.5 (max), Claude Code: 99.76, 1.9 min; Claude Haiku 5.5 (max), Claude Code: 99.76, 2.2 min; Claude Opus 5 (high), Claude Code: 98.8, 0.7 min; Claude Sonnet 5 (high), Claude Code: 96.15, 0.6 min; Claude Opus 4.8 (high), Claude Code: 97.6, 0.9 min; Claude Opus 4.7 (high), Claude Code: 97.36, 0.9 minMost attractive quadrant20304050607080901000.5 min1 min2 min5 min10 minTime per task (minutes, log scale)OpenEarthAgent Maps scoreClaude Sonnet 5.5 (xhigh), Claude CodeClaude Opus 5.5 (medium), Claude CodeClaude Sonnet 5.5 (high), Claude CodeMistral Large 4 (high), Terminus-2Claude Opus 5.5 (xhigh), Claude CodeClaude Opus 5.5 (max), Claude CodeClaude Haiku 5.5 (max), Claude CodeClaude Opus 5.5 (high), Claude CodeClaude Haiku 5.5 (high), Claude CodeStep 5 Preview (Reasoning), Terminus-2Claude Sonnet 5.5 (medium), Claude CodeClaude Haiku 5.5 (xhigh), Claude CodeClaude Opus 5.5 (low), Claude CodeClaude Sonnet 5.5 (low), Claude CodeClaude Opus 5 (high), Claude CodeClaude Opus 4.8 (high), Claude CodeClaude Haiku 5.5 (low), Claude CodeClaude Opus 4.7 (high), Claude CodeClaude Haiku 5.5 (medium), Claude CodeClaude Sonnet 5 (high), Claude CodeNemotron 3 Ultra (high), Terminus-2Claude Haiku 4.5 (Reasoning), Claude CodeNemotron 3.5 Super VL 120B (Reasoning), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2
How to read this chart

Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher OpenEarthAgent Maps score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.

Data table: OpenEarthAgent Maps: score vs. time per task

Open weights

OpenEarthAgent Maps: score vs. time per task

OpenEarthAgent Maps score against time per task (at 100 output tokens/s), minutes · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
OpenEarthAgent Maps: score vs. time per taskOpenEarthAgent Maps score (0 to 100) against time per task (minutes), log scale: Mistral Large 4 (high), Terminus-2: 99.76, 3.3 min; Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 32.69, 3.0 min; Nemotron 3 Ultra (high), Terminus-2: 94.83, 4.8 min; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, 2.1 min; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, 2.8 min; Step 5 Preview (Reasoning), Terminus-2: 99.28, 3.8 minMost attractive quadrant20304050607080901001 min2 min5 min10 minTime per task (minutes, log scale)OpenEarthAgent Maps scoreMistral Large 4 (high), Terminus-2Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2Step 5 Preview (Reasoning), Terminus-2Nemotron 3 Ultra (high), Terminus-2Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2
How to read this chart

Time per task. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher OpenEarthAgent Maps score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.

Data table: OpenEarthAgent Maps: score vs. time per task
OpenEarthAgent Maps: score vs. time per task
ModelOpenEarthAgent Maps scoreTime per task (minutes)On the Pareto line
Mistral Large 4 (high), Terminus-21003.3 minYes
Step 5 Preview (Reasoning), Terminus-2993.8 minNo
Nemotron 3 Ultra (high), Terminus-2954.8 minNo
Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2852.8 minYes
Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2842.1 minYes
Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2333.0 minNo

Score vs. release date

All

OpenEarthAgent Maps: score vs. release date

OpenEarthAgent Maps score against the model's public release date

OpenEarthAgent Maps: score vs. release dateOpenEarthAgent Maps score (0 to 100) against release date: Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 32.69, 28 April 2026; Nemotron 3 Ultra (high), Terminus-2: 94.83, 4 June 2026; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, 11 August 2026; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, 15 October 2026; Claude Haiku 4.5 (Reasoning), Claude Code: 93.03, 15 October 2025; Claude Opus 5.5 (medium), Claude Code: 99.76, 22 September 2026; Claude Opus 5.5 (high), Claude Code: 99.52, 22 September 2026; Claude Sonnet 5.5 (medium), Claude Code: 99.28, 28 September 2026; Claude Sonnet 5.5 (high), Claude Code: 99.76, 28 September 2026; Claude Haiku 5.5 (medium), Claude Code: 96.88, 7 October 2026; Claude Haiku 5.5 (high), Claude Code: 99.52, 7 October 2026; Claude Opus 5.5 (xhigh), Claude Code: 99.76, 22 September 2026; Claude Sonnet 5.5 (xhigh), Claude Code: 100, 28 September 2026; Claude Haiku 5.5 (xhigh), Claude Code: 99.04, 7 October 2026; Claude Opus 5.5 (low), Claude Code: 98.8, 22 September 2026; Claude Sonnet 5.5 (low), Claude Code: 98.8, 28 September 2026; Claude Haiku 5.5 (low), Claude Code: 97.36, 7 October 2026; Claude Opus 5.5 (max), Claude Code: 99.76, 22 September 2026; Claude Haiku 5.5 (max), Claude Code: 99.76, 7 October 2026; Claude Opus 5 (high), Claude Code: 98.8, 24 July 2026; Claude Sonnet 5 (high), Claude Code: 96.15, 30 June 2026; Claude Opus 4.8 (high), Claude Code: 97.6, 28 May 2026; Claude Opus 4.7 (high), Claude Code: 97.36, 16 April 20262030405060708090100Oct 2025Jan 2026Apr 2026Jul 2026Oct 2026Jan 2027Release dateOpenEarthAgent Maps scoreClaude Sonnet 5.5 (xhigh), Claude CodeClaude Opus 5.5 (medium), Claude CodeClaude Sonnet 5.5 (high), Claude CodeClaude Opus 5.5 (xhigh), Claude CodeClaude Opus 5.5 (max), Claude CodeClaude Haiku 5.5 (max), Claude CodeClaude Opus 5.5 (high), Claude CodeClaude Haiku 5.5 (high), Claude CodeClaude Sonnet 5.5 (medium), Claude CodeClaude Haiku 5.5 (xhigh), Claude CodeClaude Opus 5.5 (low), Claude CodeClaude Sonnet 5.5 (low), Claude CodeClaude Opus 5 (high), Claude CodeClaude Opus 4.8 (high), Claude CodeClaude Haiku 5.5 (low), Claude CodeClaude Opus 4.7 (high), Claude CodeClaude Haiku 5.5 (medium), Claude CodeClaude Sonnet 5 (high), Claude CodeNemotron 3 Ultra (high), Terminus-2Claude Haiku 4.5 (Reasoning), Claude CodeNemotron 3.5 Super VL 120B (Reasoning), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2
How to read this chart

How to read this chart. Each point is one entry: across, the date its model was first released; up, its OpenEarthAgent Maps score. Models with no published release date are not plotted.

Data table: OpenEarthAgent Maps: score vs. release date

Open weights

OpenEarthAgent Maps: score vs. release date

OpenEarthAgent Maps score against the model's public release date

OpenEarthAgent Maps: score vs. release dateOpenEarthAgent Maps score (0 to 100) against release date: Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2: 32.69, 28 April 2026; Nemotron 3 Ultra (high), Terminus-2: 94.83, 4 June 2026; Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2: 83.89, 11 August 2026; Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2: 85.1, 15 October 20262030405060708090100Mar 2026May 2026Jul 2026Sep 2026Nov 2026Release dateOpenEarthAgent Maps scoreNemotron 3 Ultra (high), Terminus-2Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2
How to read this chart

How to read this chart. Each point is one entry: across, the date its model was first released; up, its OpenEarthAgent Maps score. Models with no published release date are not plotted.

Data table: OpenEarthAgent Maps: score vs. release date

Background

OpenEarthAgent Maps asks questions about real places over OpenStreetMap: which features lie beyond a distance or a travel time from the nearest school, hospital or fire station, which pairs are closest, and averages over them. Its source agent called live map services; here the agent gets a dated map snapshot (19 January 2026) and offline copies of the source's own map tools, with no internet access.

Licence: dataset Apache 2.0 as its card declares. Map data © OpenStreetMap contributors, ODbL 1.0.

Published by MBZUAI.

Related links

Explore evaluations