Axis Spatial Spatial Intelligence Index

Search modelsMenu

GeoChrono larger set benchmark leaderboard

Change over time in aerial image series, 1,000 questions, answered from the images alone, with no tools. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.

Score

GeoChrono larger set score

Share of questions answered correctly, 0 to 100 · Higher is better

GeoChrono larger set scoreScores 0 to 100: MiniMax M3.1 Flash (max): 75; Qwen 3.8 27B (xhigh): 69; Mistral Large 4 (high): 67; Muse Glimmer 30B (high): 53; Nemotron 3.5 Super VL 120B (Reasoning): 53; Step 5 Preview (Reasoning): 50; Qwen 3.6 35B-A3B (Reasoning): 34; Nemotron 3 Nano Omni 30B (Reasoning): 310255075100MiniMax M3.1 Flash (max)75Qwen 3.8 27B (xhigh)69Mistral Large 4 (high)67Muse Glimmer 30B (high)53Nemotron 3.5 Super VL 120B (Reasoning)53Step 5 Preview (Reasoning)50Qwen 3.6 35B-A3B (Reasoning)34Nemotron 3 Nano Omni 30B (Reasoning)31
How to read this chart

What this metric means. The share of questions the model answered correctly from the images alone, with no tools.

Rows. Default: each developer's best model, then the best of each other model family (such as Opus, Sonnet and Haiku), then the best of the rest, up to 16, or one per family where there are more. Tables keep every entry.

Data table: GeoChrono larger set score
GeoChrono larger set score
ModelGeoChrono larger set score
MiniMax M3.1 Flash (max)75
Qwen 3.8 27B (xhigh)69
Mistral Large 4 (high)67
Muse Glimmer 30B (high)53
Nemotron 3.5 Super VL 120B (Reasoning)53
Step 5 Preview (Reasoning)50
Qwen 3.6 35B-A3B (Reasoning)34
Nemotron 3 Nano Omni 30B (Reasoning)31

GeoChrono larger set: score vs. cost per question

GeoChrono larger set score against average cost per question (USD) · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
GeoChrono larger set: score vs. cost per questionGeoChrono larger set score (0 to 100) against cost per question in US dollars, log scale: MiniMax M3.1 Flash (max): 75, $0.0023; Qwen 3.8 27B (xhigh): 69.08, $0.024; Mistral Large 4 (high): 66.92, $0.019; Muse Glimmer 30B (high): 53.38, $0.0084; Nemotron 3.5 Super VL 120B (Reasoning): 52.89, $0.0040; Step 5 Preview (Reasoning): 49.68, $0.028; Qwen 3.6 35B-A3B (Reasoning): 33.5, $0.043Most attractive quadrant20304050607080$0.002$0.005$0.01$0.02$0.05Cost per question (USD, log scale)GeoChrono larger set scoreMiniMax M3.1 Flash (max)Muse Glimmer 30B (high)Nemotron 3.5 Super VL 120B (Reasoning)Qwen 3.8 27B (xhigh)Mistral Large 4 (high)Step 5 Preview (Reasoning)Qwen 3.6 35B-A3B (Reasoning)
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one entry. Further left means a lower average cost per question (log scale); higher means a higher GeoChrono larger set score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.

Data table: GeoChrono larger set: score vs. cost per question
GeoChrono larger set: score vs. cost per question
ModelGeoChrono larger set scoreCost per question (USD)On the Pareto line
MiniMax M3.1 Flash (max)75$0.0023Yes
Qwen 3.8 27B (xhigh)69$0.024No
Mistral Large 4 (high)67$0.019No
Muse Glimmer 30B (high)53$0.0084No
Nemotron 3.5 Super VL 120B (Reasoning)53$0.0040No
Step 5 Preview (Reasoning)50$0.028No
Qwen 3.6 35B-A3B (Reasoning)34$0.043No
Nemotron 3 Nano Omni 30B (Reasoning)31No dataNo data

Token usage

GeoChrono larger set: tokens per question

Average tokens per question by type · Lower is better

  • Uncached input
  • Cached input
  • Output (reasoning included)
GeoChrono larger set: tokens per questionMiniMax M3.1 Flash (max): 7k; Step 5 Preview (Reasoning): 12k; Muse Glimmer 30B (high): 13k; Nemotron 3.5 Super VL 120B (Reasoning): 14k; Qwen 3.8 27B (xhigh): 14k; Nemotron 3 Nano Omni 30B (Reasoning): 15k; Mistral Large 4 (high): 15k; Qwen 3.6 35B-A3B (Reasoning): 36k010k20k30k40kMiniMax M3.1 Flash (max)7kStep 5 Preview (Reasoning)12kMuse Glimmer 30B (high)13kNemotron 3.5 Super VL 120B (Reasoning)14kQwen 3.8 27B (xhigh)14kNemotron 3 Nano Omni 30B (Reasoning)15kMistral Large 4 (high)15kQwen 3.6 35B-A3B (Reasoning)36k
How to read this chart

What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Data table: GeoChrono larger set: tokens per question
GeoChrono larger set: tokens per question
ModelDeveloperUncached inputCached inputOutput (reasoning included)Total
MiniMax M3.1 Flash (max)MiniMax4k1283k7k
Step 5 Preview (Reasoning)StepFun4k1928k12k
Muse Glimmer 30B (high)Meta9k933k13k
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA8kNo data6k14k
Qwen 3.8 27B (xhigh)Alibaba (Qwen)8kNo data6k14k
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIA7k9777k15k
Mistral Large 4 (high)Mistral AI9k1446k15k
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)8k028k36k

Cost

GeoChrono larger set: cost per question

Average cost per question (USD) by token type · Lower is better

  • Uncached input
  • Cached input
  • Output
  • Not split by token type
GeoChrono larger set: cost per questionValues in US dollars: MiniMax M3.1 Flash (max): $0.0023; Nemotron 3.5 Super VL 120B (Reasoning): $0.0040; Muse Glimmer 30B (high): $0.0084; Mistral Large 4 (high): $0.019; Qwen 3.8 27B (xhigh): $0.024; Step 5 Preview (Reasoning): $0.028; Qwen 3.6 35B-A3B (Reasoning): $0.043$0$0.015$0.03$0.045$0.06MiniMax M3.1 Flash (max)$0.0023Nemotron 3.5 Super VL 120B (Reasoning)$0.0040Muse Glimmer 30B (high)$0.0084Mistral Large 4 (high)$0.019Qwen 3.8 27B (xhigh)$0.024Step 5 Preview (Reasoning)$0.028Qwen 3.6 35B-A3B (Reasoning)$0.043
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.

Data table: GeoChrono larger set: cost per question
GeoChrono larger set: cost per question
ModelDeveloperUncached inputCached inputOutputNot split by token typeTotal
MiniMax M3.1 Flash (max)MiniMax$0.00067$0.0000019$0.0017$0.00000052$0.0023
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA$0.0013$0.00$0.0028$0.00$0.0040
Muse Glimmer 30B (high)Meta$0.0032$0.0000037$0.0052$0.00$0.0084
Mistral Large 4 (high)Mistral AI$0.0065$0.000010$0.012$0.00000019$0.019
Qwen 3.8 27B (xhigh)Alibaba (Qwen)$0.0037$0.00$0.019$0.0013$0.024
Step 5 Preview (Reasoning)StepFun$0.0044$0.000020$0.024$0.00$0.028
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)$0.0020$0.00$0.041$0.00000033$0.043
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIANo dataNo dataNo dataNo dataNo data

Speed

GeoChrono larger set: time per question

Time per question (at 100 output tokens/s), seconds · Lower is better

GeoChrono larger set: time per questionGeoChrono larger set: time per question: MiniMax M3.1 Flash (max): 38s; Muse Glimmer 30B (high): 38s; Nemotron 3.5 Super VL 120B (Reasoning): 59s; Mistral Large 4 (high): 61s; Qwen 3.8 27B (xhigh): 67s; Nemotron 3 Nano Omni 30B (Reasoning): 71s; Step 5 Preview (Reasoning): 84s; Qwen 3.6 35B-A3B (Reasoning): 285s0s100s200s300s400sMiniMax M3.1 Flash (max)38sMuse Glimmer 30B (high)38sNemotron 3.5 Super VL 120B (Reasoning)59sMistral Large 4 (high)61sQwen 3.8 27B (xhigh)67sNemotron 3 Nano Omni 30B (Reasoning)71sStep 5 Preview (Reasoning)84sQwen 3.6 35B-A3B (Reasoning)285s
How to read this chart

Time per question. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

Data table: GeoChrono larger set: time per question
GeoChrono larger set: time per question
ModelDeveloperTime per question (s)
MiniMax M3.1 Flash (max)MiniMax38s
Muse Glimmer 30B (high)Meta38s
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA59s
Mistral Large 4 (high)Mistral AI61s
Qwen 3.8 27B (xhigh)Alibaba (Qwen)67s
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIA71s
Step 5 Preview (Reasoning)StepFun84s
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)285s

GeoChrono larger set: score vs. time per question

GeoChrono larger set score against time per question (at 100 output tokens/s), seconds · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
GeoChrono larger set: score vs. time per questionGeoChrono larger set score (0 to 100) against time per question (seconds), log scale: MiniMax M3.1 Flash (max): 75, 38s; Qwen 3.8 27B (xhigh): 69.08, 67s; Mistral Large 4 (high): 66.92, 61s; Muse Glimmer 30B (high): 53.38, 38s; Nemotron 3.5 Super VL 120B (Reasoning): 52.89, 59s; Step 5 Preview (Reasoning): 49.68, 84s; Qwen 3.6 35B-A3B (Reasoning): 33.5, 285s; Nemotron 3 Nano Omni 30B (Reasoning): 30.83, 71sMost attractive quadrant2030405060708020s50s100s200s500sTime per question (seconds, log scale)GeoChrono larger set scoreMiniMax M3.1 Flash (max)Qwen 3.8 27B (xhigh)Mistral Large 4 (high)Muse Glimmer 30B (high)Nemotron 3.5 Super VL 120B (Reasoning)Step 5 Preview (Reasoning)Qwen 3.6 35B-A3B (Reasoning)Nemotron 3 Nano Omni 30B (Reasoning)
How to read this chart

Time per question. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

How to read this chart. Each point is one entry. Further left means a lower average time per question (log scale); higher means a higher GeoChrono larger set score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.

Data table: GeoChrono larger set: score vs. time per question
GeoChrono larger set: score vs. time per question
ModelGeoChrono larger set scoreTime per question (seconds)On the Pareto line
MiniMax M3.1 Flash (max)7538sYes
Qwen 3.8 27B (xhigh)6967sNo
Mistral Large 4 (high)6761sNo
Muse Glimmer 30B (high)5338sNo
Nemotron 3.5 Super VL 120B (Reasoning)5359sNo
Step 5 Preview (Reasoning)5084sNo
Qwen 3.6 35B-A3B (Reasoning)34285sNo
Nemotron 3 Nano Omni 30B (Reasoning)3171sNo

Score vs. release date

GeoChrono larger set: score vs. release date

GeoChrono larger set score against the model's public release date

GeoChrono larger set: score vs. release dateGeoChrono larger set score (0 to 100) against release date: MiniMax M3.1 Flash (max): 75, 19 October 2026; Qwen 3.8 27B (xhigh): 69.08, 14 August 2026; Nemotron 3.5 Super VL 120B (Reasoning): 52.89, 15 October 2026; Nemotron 3 Nano Omni 30B (Reasoning): 30.83, 28 April 202620304050607080Mar 2026May 2026Jul 2026Sep 2026Nov 2026Release dateGeoChrono larger set scoreMiniMax M3.1 Flash (max)Qwen 3.8 27B (xhigh)Nemotron 3.5 Super VL 120B (Reasoning)Nemotron 3 Nano Omni 30B (Reasoning)
How to read this chart

How to read this chart. Each point is one entry: across, the date its model was first released; up, its GeoChrono larger set score. Models with no published release date are not plotted.

Data table: GeoChrono larger set: score vs. release date
GeoChrono larger set: score vs. release date
ModelGeoChrono larger set scoreRelease date
MiniMax M3.1 Flash (max)7519 October 2026
Qwen 3.8 27B (xhigh)6914 August 2026
Mistral Large 4 (high)67No data
Muse Glimmer 30B (high)53No data
Nemotron 3.5 Super VL 120B (Reasoning)5315 October 2026
Step 5 Preview (Reasoning)50No data
Qwen 3.6 35B-A3B (Reasoning)34No data
Nemotron 3 Nano Omni 30B (Reasoning)3128 April 2026

Background

The larger GeoChrono set is 1,000 of the 17,688 ChronoBench questions that have a keyed answer, 250 for each of GeoChrono's four levels, chosen with a fixed seed. It contains the 300 questions of the GeoChrono sample, which keeps its own scores. A partial evaluation: models are added as they finish, so a model's score covers the questions it has answered so far, and scores on different numbers of questions are not directly comparable.

Licence: questions Apache-2.0; images from DVL-Suite (Apache-2.0), aerial photographs of the USDA National Agriculture Imagery Program (public domain).

Published by the GeoChrono authors (IntelliSensing).

Related links

Explore evaluations