Axis Spatial Spatial Intelligence Index

Search modelsMenu

DVL-Bench benchmark leaderboard

Change in aerial image series: what changed, when and by how much, answered from the images alone, with no tools. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.

Score

DVL-Bench score

Share of questions answered correctly, 0 to 100 · Higher is better

DVL-Bench scoreScores 0 to 100: MiniMax M3.1 Flash (max): 31; Step 5 Preview (Reasoning): 29; Mistral Large 4 (high): 28; Nemotron 3.5 Super VL 120B (Reasoning): 26; Muse Glimmer 30B (high): 25; Qwen 3.6 35B-A3B (Reasoning): 25; Qwen 3.8 27B (xhigh): 23; Nemotron 3 Nano Omni 30B (Reasoning): 200255075100MiniMax M3.1 Flash (max)31Step 5 Preview (Reasoning)29Mistral Large 4 (high)28Nemotron 3.5 Super VL 120B (Reasoning)26Muse Glimmer 30B (high)25Qwen 3.6 35B-A3B (Reasoning)25Qwen 3.8 27B (xhigh)23Nemotron 3 Nano Omni 30B (Reasoning)20
How to read this chart

What this metric means. The share of questions the model answered correctly from the images alone, with no tools.

Rows. Default: each developer's best model, then the best of each other model family (such as Opus, Sonnet and Haiku), then the best of the rest, up to 16, or one per family where there are more. Tables keep every entry.

Data table: DVL-Bench score
DVL-Bench score
ModelDVL-Bench score
MiniMax M3.1 Flash (max)31
Step 5 Preview (Reasoning)29
Mistral Large 4 (high)28
Nemotron 3.5 Super VL 120B (Reasoning)26
Muse Glimmer 30B (high)25
Qwen 3.6 35B-A3B (Reasoning)25
Qwen 3.8 27B (xhigh)23
Nemotron 3 Nano Omni 30B (Reasoning)20

DVL-Bench: score vs. cost per question

DVL-Bench score against average cost per question (USD) · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
DVL-Bench: score vs. cost per questionDVL-Bench score (0 to 100) against cost per question in US dollars, log scale: MiniMax M3.1 Flash (max): 31.49, $0.0046; Step 5 Preview (Reasoning): 28.88, $0.035; Mistral Large 4 (high): 28.39, $0.020; Nemotron 3.5 Super VL 120B (Reasoning): 25.75, $0.0038; Muse Glimmer 30B (high): 25.21, $0.0070; Qwen 3.6 35B-A3B (Reasoning): 24.75, $0.032; Qwen 3.8 27B (xhigh): 22.74, $0.036Most attractive quadrant2022242628303234$0.002$0.005$0.01$0.02$0.05Cost per question (USD, log scale)DVL-Bench scoreMiniMax M3.1 Flash (max)Nemotron 3.5 Super VL 120B (Reasoning)Step 5 Preview (Reasoning)Mistral Large 4 (high)Muse Glimmer 30B (high)Qwen 3.6 35B-A3B (Reasoning)Qwen 3.8 27B (xhigh)
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one entry. Further left means a lower average cost per question (log scale); higher means a higher DVL-Bench score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no cheaper one beats on score. Only exact costs are plotted; the table lists every entry.

Data table: DVL-Bench: score vs. cost per question
DVL-Bench: score vs. cost per question
ModelDVL-Bench scoreCost per question (USD)On the Pareto line
MiniMax M3.1 Flash (max)31$0.0046Yes
Step 5 Preview (Reasoning)29$0.035No
Mistral Large 4 (high)28$0.020No
Nemotron 3.5 Super VL 120B (Reasoning)26$0.0038Yes
Muse Glimmer 30B (high)25$0.0070No
Qwen 3.6 35B-A3B (Reasoning)25$0.032No
Qwen 3.8 27B (xhigh)23$0.036No
Nemotron 3 Nano Omni 30B (Reasoning)20No dataNo data

Token usage

DVL-Bench: tokens per question

Average tokens per question by type · Lower is better

  • Uncached input
  • Cached input
  • Output (reasoning included)
DVL-Bench: tokens per questionMiniMax M3.1 Flash (max): 11k; Muse Glimmer 30B (high): 12k; Nemotron 3.5 Super VL 120B (Reasoning): 13k; Step 5 Preview (Reasoning): 15k; Nemotron 3 Nano Omni 30B (Reasoning): 15k; Mistral Large 4 (high): 15k; Qwen 3.8 27B (xhigh): 18k; Qwen 3.6 35B-A3B (Reasoning): 28k010k20k30k40kMiniMax M3.1 Flash (max)11kMuse Glimmer 30B (high)12kNemotron 3.5 Super VL 120B (Reasoning)13kStep 5 Preview (Reasoning)15kNemotron 3 Nano Omni 30B (Reasoning)15kMistral Large 4 (high)15kQwen 3.8 27B (xhigh)18kQwen 3.6 35B-A3B (Reasoning)28k
How to read this chart

What the parts mean. Uncached input: input tokens read at the full price. Cached input: input tokens served from the provider's cache at the lower cached price. Output: the tokens the model wrote, its reasoning included, as reported by the provider.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Data table: DVL-Bench: tokens per question
DVL-Bench: tokens per question
ModelDeveloperUncached inputCached inputOutput (reasoning included)Total
MiniMax M3.1 Flash (max)MiniMax4k2587k11k
Muse Glimmer 30B (high)Meta9k533k12k
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA8kNo data5k13k
Step 5 Preview (Reasoning)StepFun4k010k15k
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIA8k2787k15k
Mistral Large 4 (high)Mistral AI9k06k15k
Qwen 3.8 27B (xhigh)Alibaba (Qwen)7kNo data10k18k
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)7k2120k28k

Cost

DVL-Bench: cost per question

Average cost per question (USD) by token type · Lower is better

  • Uncached input
  • Cached input
  • Output
  • Not split by token type
DVL-Bench: cost per questionValues in US dollars: Nemotron 3.5 Super VL 120B (Reasoning): $0.0038; MiniMax M3.1 Flash (max): $0.0046; Muse Glimmer 30B (high): $0.0070; Mistral Large 4 (high): $0.020; Qwen 3.6 35B-A3B (Reasoning): $0.032; Step 5 Preview (Reasoning): $0.035; Qwen 3.8 27B (xhigh): $0.036$0$0.01$0.02$0.03$0.04Nemotron 3.5 Super VL 120B (Reasoning)$0.0038MiniMax M3.1 Flash (max)$0.0046Muse Glimmer 30B (high)$0.0070Mistral Large 4 (high)$0.020Qwen 3.6 35B-A3B (Reasoning)$0.032Step 5 Preview (Reasoning)$0.035Qwen 3.8 27B (xhigh)$0.036
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How the cost is split. Each part is the entry's average tokens of that type times its list price. The rest, mostly tokens written to the cache, is shown as not split by token type.

Data table: DVL-Bench: cost per question
DVL-Bench: cost per question
ModelDeveloperUncached inputCached inputOutputNot split by token typeTotal
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA$0.0012$0.00$0.0026$0.00000028$0.0038
MiniMax M3.1 Flash (max)MiniMax$0.00063$0.0000039$0.0040$0.00000042$0.0046
Muse Glimmer 30B (high)Meta$0.0031$0.0000021$0.0039$0.00$0.0070
Mistral Large 4 (high)Mistral AI$0.0064$0.00$0.013$0.00$0.020
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)$0.0018$0.0000021$0.030$0.00$0.032
Step 5 Preview (Reasoning)StepFun$0.0043$0.00$0.031$0.00$0.035
Qwen 3.8 27B (xhigh)Alibaba (Qwen)$0.0034$0.00$0.030$0.0020$0.036
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIANo dataNo dataNo dataNo dataNo data

Speed

DVL-Bench: time per question

Time per question (at 100 output tokens/s), seconds · Lower is better

DVL-Bench: time per questionDVL-Bench: time per question: Muse Glimmer 30B (high): 30s; Nemotron 3.5 Super VL 120B (Reasoning): 55s; Mistral Large 4 (high): 66s; Nemotron 3 Nano Omni 30B (Reasoning): 76s; MiniMax M3.1 Flash (max): 82s; Qwen 3.8 27B (xhigh): 106s; Step 5 Preview (Reasoning): 110s; Qwen 3.6 35B-A3B (Reasoning): 210s0s100s200s300s400sMuse Glimmer 30B (high)30sNemotron 3.5 Super VL 120B (Reasoning)55sMistral Large 4 (high)66sNemotron 3 Nano Omni 30B (Reasoning)76sMiniMax M3.1 Flash (max)82sQwen 3.8 27B (xhigh)106sStep 5 Preview (Reasoning)110sQwen 3.6 35B-A3B (Reasoning)210s
How to read this chart

Time per question. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

Data table: DVL-Bench: time per question
DVL-Bench: time per question
ModelDeveloperTime per question (s)
Muse Glimmer 30B (high)Meta30s
Nemotron 3.5 Super VL 120B (Reasoning)NVIDIA55s
Mistral Large 4 (high)Mistral AI66s
Nemotron 3 Nano Omni 30B (Reasoning)NVIDIA76s
MiniMax M3.1 Flash (max)MiniMax82s
Qwen 3.8 27B (xhigh)Alibaba (Qwen)106s
Step 5 Preview (Reasoning)StepFun110s
Qwen 3.6 35B-A3B (Reasoning)Alibaba (Qwen)210s

DVL-Bench: score vs. time per question

DVL-Bench score against time per question (at 100 output tokens/s), seconds · Higher and further left is better

  • Most attractive quadrant
  • Pareto line
DVL-Bench: score vs. time per questionDVL-Bench score (0 to 100) against time per question (seconds), log scale: MiniMax M3.1 Flash (max): 31.49, 82s; Step 5 Preview (Reasoning): 28.88, 110s; Mistral Large 4 (high): 28.39, 66s; Nemotron 3.5 Super VL 120B (Reasoning): 25.75, 55s; Muse Glimmer 30B (high): 25.21, 30s; Qwen 3.6 35B-A3B (Reasoning): 24.75, 210s; Qwen 3.8 27B (xhigh): 22.74, 106s; Nemotron 3 Nano Omni 30B (Reasoning): 19.73, 76sMost attractive quadrant18202224262830323420s50s100s200s500sTime per question (seconds, log scale)DVL-Bench scoreMiniMax M3.1 Flash (max)Mistral Large 4 (high)Nemotron 3.5 Super VL 120B (Reasoning)Muse Glimmer 30B (high)Step 5 Preview (Reasoning)Qwen 3.6 35B-A3B (Reasoning)Qwen 3.8 27B (xhigh)Nemotron 3 Nano Omni 30B (Reasoning)
How to read this chart

Time per question. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.

How to read this chart. Each point is one entry. Further left means a lower average time per question (log scale); higher means a higher DVL-Bench score. Both axes fit the points shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot: a reading aid, not a fixed bar. The dotted Pareto line joins the points that no faster one beats on score.

Data table: DVL-Bench: score vs. time per question
DVL-Bench: score vs. time per question
ModelDVL-Bench scoreTime per question (seconds)On the Pareto line
MiniMax M3.1 Flash (max)3182sYes
Step 5 Preview (Reasoning)29110sNo
Mistral Large 4 (high)2866sYes
Nemotron 3.5 Super VL 120B (Reasoning)2655sYes
Muse Glimmer 30B (high)2530sYes
Qwen 3.6 35B-A3B (Reasoning)25210sNo
Qwen 3.8 27B (xhigh)23106sNo
Nemotron 3 Nano Omni 30B (Reasoning)2076sNo

Score vs. release date

DVL-Bench: score vs. release date

DVL-Bench score against the model's public release date

DVL-Bench: score vs. release dateDVL-Bench score (0 to 100) against release date: MiniMax M3.1 Flash (max): 31.49, 19 October 2026; Nemotron 3.5 Super VL 120B (Reasoning): 25.75, 15 October 2026; Qwen 3.8 27B (xhigh): 22.74, 14 August 2026; Nemotron 3 Nano Omni 30B (Reasoning): 19.73, 28 April 2026182022242628303234Mar 2026May 2026Jul 2026Sep 2026Nov 2026Release dateDVL-Bench scoreMiniMax M3.1 Flash (max)Nemotron 3.5 Super VL 120B (Reasoning)Qwen 3.8 27B (xhigh)Nemotron 3 Nano Omni 30B (Reasoning)
How to read this chart

How to read this chart. Each point is one entry: across, the date its model was first released; up, its DVL-Bench score. Models with no published release date are not plotted.

Data table: DVL-Bench: score vs. release date
DVL-Bench: score vs. release date
ModelDVL-Bench scoreRelease date
MiniMax M3.1 Flash (max)3119 October 2026
Step 5 Preview (Reasoning)29No data
Mistral Large 4 (high)28No data
Nemotron 3.5 Super VL 120B (Reasoning)2615 October 2026
Muse Glimmer 30B (high)25No data
Qwen 3.6 35B-A3B (Reasoning)25No data
Qwen 3.8 27B (xhigh)2314 August 2026
Nemotron 3 Nano Omni 30B (Reasoning)2028 April 2026

Background

DVL-Bench asks about change in a place seen over several years: which kinds of ground cover changed, which change was least, whether vegetation or built-up surface grew or shrank, when building expansion stopped, in which period the most change happened, and by how much. Each question shows 4 to 10 aerial photographs of the same place, one per year, labelled with the year. 189 questions have one correct option and 110 ask for every correct option.

Licence: dataset and code Apache-2.0; images from the USDA National Agriculture Imagery Program (public domain).

Published by Xuan et al. (DynamicVL).

Related links

Explore evaluations