gpt-oss-20b (high) vs Kimi K3 (Reasoning)
Results updated 31 October 2026 · In Terminus-2 and Kimi Code, so part of any gap can come from the harness · How we benchmark
Verdict
Kimi K3 (Reasoning) clearly beats gpt-oss-20b (high) by 42 points, and is cheaper and faster.
- Cost per task
- $0.017 vs $0.00
- Time per task
- 7.2 min vs 1.5 min (estimated)
- Tokens per attempt
- 67k vs 345k
- Image input
- No vs Yes
Kimi K3 (Reasoning), Kimi Code scores higher than gpt-oss-20b (high), Terminus-2 on all 3 evaluations scored for both. Kimi K3 (Reasoning), Kimi Code is cheaper and faster: $0.00 against $0.017 per task, and 1.5 min against 7.2 min.
All
Geospatial Agent Index: Text only
Equal average of Earth-Bench V2, GeoBenchX and GS-QA2, 0 to 100 · Higher is better
Data table: Geospatial Agent Index: Text only
Open weights
Geospatial Agent Index: Text only
Equal average of Earth-Bench V2, GeoBenchX and GS-QA2, 0 to 100 · Higher is better
Data table: Geospatial Agent Index: Text only
Where they differ most
By kind of work, share of tasks passed, 0 to 100 · Higher is better
| Kind of work | gpt-oss-20b (high) | Kimi K3 (Reasoning) |
|---|---|---|
| Making maps | 21 | 82 |
| GIS analysis | 26 | 84 |
| Remote sensing and raster analysis | 33 | 81 |
| Climate and time series | 31 | 78 |
Where the tasks are set: each kind of work links to its tasks and scoresIntelligence
All
Geospatial Intelligence Index: Text only
Weighted average of agent tasks, guided workflows and geospatial knowledge, 0 to 100 · Higher is better
How to read this chart
What this metric means. A weighted average of group scores, each 0 to 100: Agent tasks 4/6, Guided workflows 1/6, Geospatial knowledge 1/6.
Data table: Geospatial Intelligence Index: Text only
Open weights
Geospatial Intelligence Index: Text only
Weighted average of agent tasks, guided workflows and geospatial knowledge, 0 to 100 · Higher is better
How to read this chart
What this metric means. A weighted average of group scores, each 0 to 100: Agent tasks 4/6, Guided workflows 1/6, Geospatial knowledge 1/6.
Data table: Geospatial Intelligence Index: Text only
| Model | Developer | Geospatial Intelligence Index: Text only |
|---|---|---|
| Kimi K3 (Reasoning) Open weights | Moonshot AI | 81.5 |
| DeepSeek V4 Pro (max) Open weights | DeepSeek | 79.0 |
| GLM-5.2 (max) Open weights | Z.ai | 78.4 |
| DeepSeek V4 Flash (max) Open weights | DeepSeek | 77.2 |
| GLM-5.3 (max) Open weights | Z.ai | 77.1 |
| Qwen 3.8 27B (xhigh) Open weights | Alibaba (Qwen) | 75.1 |
| Nemotron 3 Ultra (high) Open weights | NVIDIA | 73.7 |
| GLM-5.3-Flash (max) Open weights | Z.ai | 71.0 |
| Gemma 4 26B A4B (Reasoning) Open weights | 64.4 | |
| Step 5 Preview (Reasoning) Open weights | StepFun | 63.8 |
| MiniMax M3.1 Flash (max) Open weights | MiniMax | 60.5 |
| Nemotron 3.5 Super VL 120B (Reasoning) Open weights | NVIDIA | 59.0 |
| gpt-oss-120b (high) Open weights | OpenAI | 51.9 |
| GLM-4.7-Flash (Reasoning) Open weights | Z.ai | 49.6 |
| Mistral Large 3 (Non-reasoning) Open weights | Mistral AI | 44.5 |
| Nemotron 3 Nano Omni 30B (Reasoning) Open weights | NVIDIA | 31.9 |
| Nemotron 3.5 Lightning 30B (Reasoning) Open weights | NVIDIA | 27.4 |
| gpt-oss-20b (high) Open weights | OpenAI | 22.1 |
| Llama 3.3 70B (Non-reasoning) Open weights | Meta | 19.9 |
By evaluation
All
Earth-Bench V2
higher is better
Data table: Earth-Bench V2
GeoBenchX
higher is better
Data table: GeoBenchX
GS-QA2
higher is better
Data table: GS-QA2
GeoAgentBench
higher is better
Data table: GeoAgentBench
GeoAnalystBench
higher is better
Data table: GeoAnalystBench
Earth-Bench V2 images
higher is better
Data table: Earth-Bench V2 images
ThinkGeo images
higher is better
Data table: ThinkGeo images
| Model | Developer | ThinkGeo images |
|---|---|---|
| Claude Opus 5.5 (xhigh) | Anthropic | 47.7 |
| Claude Sonnet 5.5 (xhigh) | Anthropic | 46.5 |
| GPT-6.1 Sol (xhigh) | OpenAI | 40.7 |
| GPT-6 Sol (high) | OpenAI | 40.7 |
| GPT-5.6 Sol (high) | OpenAI | 40.7 |
| Claude Opus 5 (high) | Anthropic | 38.4 |
| GPT-5.6 Luna (xhigh) | OpenAI | 37.2 |
| Qwen 3.8 27B (xhigh) Open weights | Alibaba (Qwen) | 36.7 |
| Claude Haiku 5.5 (xhigh) | Anthropic | 34.9 |
| GPT-6 Luna (xhigh) | OpenAI | 34.9 |
| GLM-5.3-Flash (max) Open weights | Z.ai | 34.9 |
| Claude Opus 4.8 (high) | Anthropic | 33.7 |
| GPT-5.6 Terra (high) | OpenAI | 33.7 |
| Gemini 3.8 Flash (low) | 33.7 | |
| Kimi K2.7 Code (Reasoning) Open weights | Moonshot AI | 31.4 |
| Kimi K3 (Reasoning) Open weights | Moonshot AI | 30.2 |
| Claude Sonnet 5 (high) | Anthropic | 27.9 |
| Kimi K2.6 (Reasoning) Open weights | Moonshot AI | 27.9 |
| Claude Opus 4.7 (high) | Anthropic | 26.7 |
| Gemini 3.1 Flash-Lite (high) | 24.4 | |
| Gemma 4 26B A4B (Reasoning) Open weights | 22.4 | |
| Claude Haiku 4.5 (Reasoning) | Anthropic | 18.6 |
| Mistral Large 3 (Non-reasoning) Open weights | Mistral AI | 15.1 |
| Mistral Small 3.1 24B (Non-reasoning) Open weights | Mistral AI | 11.6 |
Geospatial knowledge
higher is better
Data table: Geospatial knowledge
OGD4All
higher is better
Data table: OGD4All
GeoNatureAgent
higher is better
Data table: GeoNatureAgent
MultiGlobeQA
higher is better
Data table: MultiGlobeQA
GeoAgentEval
higher is better
Data table: GeoAgentEval
Open weights
Earth-Bench V2
higher is better
Data table: Earth-Bench V2
GeoBenchX
higher is better
Data table: GeoBenchX
GS-QA2
higher is better
Data table: GS-QA2
GeoAgentBench
higher is better
Data table: GeoAgentBench
GeoAnalystBench
higher is better
Data table: GeoAnalystBench
Earth-Bench V2 images
higher is better
Data table: Earth-Bench V2 images
| Model | Developer | Earth-Bench V2 images |
|---|---|---|
| Qwen 3.8 27B (xhigh) Open weights | Alibaba (Qwen) | 69.4 |
| Kimi K3 (Reasoning) Open weights | Moonshot AI | 66.0 |
| Gemma 4 26B A4B (Reasoning) Open weights | 64.0 | |
| Kimi K2.6 (Reasoning) Open weights | Moonshot AI | 60.0 |
| Kimi K2.7 Code (Reasoning) Open weights | Moonshot AI | 56.0 |
| Mistral Small 3.1 24B (Non-reasoning) Open weights | Mistral AI | 54.0 |
| GLM-5.3-Flash (max) Open weights | Z.ai | 52.8 |
| Mistral Large 3 (Non-reasoning) Open weights | Mistral AI | 51.5 |
ThinkGeo images
higher is better
Data table: ThinkGeo images
| Model | Developer | ThinkGeo images |
|---|---|---|
| Qwen 3.8 27B (xhigh) Open weights | Alibaba (Qwen) | 36.7 |
| GLM-5.3-Flash (max) Open weights | Z.ai | 34.9 |
| Kimi K2.7 Code (Reasoning) Open weights | Moonshot AI | 31.4 |
| Kimi K3 (Reasoning) Open weights | Moonshot AI | 30.2 |
| Kimi K2.6 (Reasoning) Open weights | Moonshot AI | 27.9 |
| Gemma 4 26B A4B (Reasoning) Open weights | 22.4 | |
| Mistral Large 3 (Non-reasoning) Open weights | Mistral AI | 15.1 |
| Mistral Small 3.1 24B (Non-reasoning) Open weights | Mistral AI | 11.6 |
Geospatial knowledge
higher is better
Data table: Geospatial knowledge
| Model | Developer | Geospatial knowledge |
|---|---|---|
| Kimi K2.7 Code (Reasoning) Open weights | Moonshot AI | 92.1 |
| Kimi K3 (Reasoning) Open weights | Moonshot AI | 92.0 |
| DeepSeek V4 Pro (max) Open weights | DeepSeek | 91.9 |
| Nemotron 3.5 Super VL 120B (Reasoning) Open weights | NVIDIA | 91.9 |
| GLM-5.3 (max) Open weights | Z.ai | 91.8 |
| Kimi K2.6 (Reasoning) Open weights | Moonshot AI | 91.8 |
| GLM-5.3-Flash (max) Open weights | Z.ai | 91.8 |
| Step 5 Preview (Reasoning) Open weights | StepFun | 91.5 |
| DeepSeek V4 Flash (max) Open weights | DeepSeek | 91.5 |
| Qwen 3.8 27B (xhigh) Open weights | Alibaba (Qwen) | 91.1 |
| Nemotron 3 Ultra (high) Open weights | NVIDIA | 90.8 |
| gpt-oss-120b (high) Open weights | OpenAI | 89.8 |
| Nemotron 3.5 Lightning 30B (Reasoning) Open weights | NVIDIA | 89.8 |
| MiniMax M3.1 Flash (max) Open weights | MiniMax | 89.0 |
| GLM-5.2 (max) Open weights | Z.ai | 89.0 |
| Mistral Large 3 (Non-reasoning) Open weights | Mistral AI | 88.7 |
| Gemma 4 26B A4B (Reasoning) Open weights | 88.7 | |
| GLM-4.7-Flash (Reasoning) Open weights | Z.ai | 85.6 |
| gpt-oss-20b (high) Open weights | OpenAI | 84.8 |
| Nemotron 3 Nano Omni 30B (Reasoning) Open weights | NVIDIA | 84.8 |
| Llama 3.3 70B (Non-reasoning) Open weights | Meta | 82.5 |
| Mistral Small 3.1 24B (Non-reasoning) Open weights | Mistral AI | 81.8 |
OGD4All
higher is better
Data table: OGD4All
| Model | Developer | OGD4All |
|---|---|---|
| Kimi K3 (Reasoning), Kimi Code | Moonshot AI | 97 |
| GLM-5.3 (max), Claude Code | Z.ai | 96 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 96 |
| Qwen 3.8 27B (xhigh), Qwen Code | Alibaba (Qwen) | 95 |
| GLM-5.3-Flash (max), Claude Code | Z.ai | 93 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 93 |
| Qwen 3.8 27B (xhigh), Terminus-2 | Alibaba (Qwen) | 92 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 92 |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | NVIDIA | 91 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 90 |
| MiniMax M3.1 Flash (max), Terminus-2 | MiniMax | 88 |
| DeepSeek V4 Flash (high), Claude Code | DeepSeek | 70 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 47 |
GeoNatureAgent
higher is better
Data table: GeoNatureAgent
| Model | Developer | GeoNatureAgent |
|---|---|---|
| MiniMax M3.1 Flash (max), Terminus-2 | MiniMax | 100 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 100 |
| Qwen 3.8 27B (xhigh), Terminus-2 | Alibaba (Qwen) | 100 |
| GLM-5.3-Flash (max), Claude Code | Z.ai | 100 |
| GLM-5.3 (max), Claude Code | Z.ai | 100 |
| Kimi K3 (Reasoning), Kimi Code | Moonshot AI | 100 |
| Qwen 3.8 27B (xhigh), Qwen Code | Alibaba (Qwen) | 100 |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | NVIDIA | 92 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 92 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 88 |
| DeepSeek V4 Flash (high), Claude Code | DeepSeek | 71 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 64 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 0 |
MultiGlobeQA
higher is better
Data table: MultiGlobeQA
| Model | Developer | MultiGlobeQA |
|---|---|---|
| Kimi K3 (Reasoning), Kimi Code | Moonshot AI | 77 |
| GLM-5.3 (max), Claude Code | Z.ai | 74 |
| Qwen 3.8 27B (xhigh), Qwen Code | Alibaba (Qwen) | 74 |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | NVIDIA | 73 |
| GLM-5.3-Flash (max), Claude Code | Z.ai | 73 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 72 |
| Qwen 3.8 27B (xhigh), Terminus-2 | Alibaba (Qwen) | 71 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 70 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 68 |
| MiniMax M3.1 Flash (max), Terminus-2 | MiniMax | 65 |
| DeepSeek V4 Flash (high), Claude Code | DeepSeek | 64 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 62 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 48 |
GeoAgentEval
higher is better
Data table: GeoAgentEval
| Model | Developer | GeoAgentEval |
|---|---|---|
| GLM-5.3-Flash (max), Claude Code | Z.ai | 96 |
| Kimi K3 (Reasoning), Kimi Code | Moonshot AI | 96 |
| Qwen 3.8 27B (xhigh), Terminus-2 | Alibaba (Qwen) | 92 |
| Nemotron 3 Ultra (high), Terminus-2 | NVIDIA | 92 |
| Step 5 Preview (Reasoning), Terminus-2 | StepFun | 91 |
| GLM-5.3 (max), Claude Code | Z.ai | 88 |
| Qwen 3.8 27B (xhigh), Qwen Code | Alibaba (Qwen) | 88 |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | NVIDIA | 80 |
| DeepSeek V4 Flash (high), Claude Code | DeepSeek | 80 |
| MiniMax M3.1 Flash (max), Terminus-2 | MiniMax | 67 |
| Mistral Large 4 (high), Terminus-2 | Mistral AI | 60 |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | NVIDIA | 48 |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | NVIDIA | 36 |
gpt-oss-20b (high): No image input: the developer's documentation says the model cannot take images. It is ranked in the Text only view and never given 0.
Cost
All
Cost per task
Average cost per agent task (USD) · Lower is better
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Lower is better. Costs are pay-per-token list prices, not any discount or credit a customer might have.
Data table: Cost per task
Open weights
Cost per task
Average cost per agent task (USD) · Lower is better
How to read this chart
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Lower is better. Costs are pay-per-token list prices, not any discount or credit a customer might have.
Data table: Cost per task
All
Geospatial Agent Index vs. cost per task
Geospatial Agent Index, Text only; cost per task (USD) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Why a log scale. Costs and times differ by a factor of ten or more between models, so equal steps on the axis are equal ratios.
Not yet ranked. Entries with too few tasks run so far are not plotted; the table lists them, and every entry is on the models page.
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.
Data table: Geospatial Agent Index vs. cost per task
Open weights
Geospatial Agent Index vs. cost per task
Geospatial Agent Index, Text only; cost per task (USD) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Why a log scale. Costs and times differ by a factor of ten or more between models, so equal steps on the axis are equal ratios.
What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.
How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.
Data table: Geospatial Agent Index vs. cost per task
| Model | Geospatial Agent Index | Cost per task (USD) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4 Pro (max), Terminus-2 | 71 | $0.35 | No |
| Gemma 4 26B A4B (Reasoning), Terminus-2 | 56 | $0.021 | No |
| gpt-oss-120b (high), Terminus-2 | 46 | $0.026 | No |
| gpt-oss-20b (high), Terminus-2 | 33 | $0.017 | No |
| Llama 3.3 70B (Non-reasoning), Terminus-2 | 8 | $0.023 | No |
| MiniMax M3.1 Flash (max), Terminus-2 | 52 | $0.018 | No |
| Mistral Large 4 (high), Terminus-2 | 59 | $0.10 | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 21 | $0.13 | No |
| Nemotron 3 Ultra (high), Terminus-2 | 68 | $0.067 | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 13 | $0.013 | No |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | 52 | $0.0096 | No |
| Step 5 Preview (Reasoning), Terminus-2 | 51 | $0.13 | No |
| GLM-5.3-Flash (max), Claude Code | 72 | $0.093 | No |
| GLM-5.3 (max), Claude Code | 73 | $0.48 | No |
| DeepSeek V4.1 Flash (max), Codex | 71 | $0.067 | No |
| Kimi K2.7 Code (Reasoning), Kimi Code | 72 | $0.18 | No |
| Kimi K3 (Reasoning), Kimi Code | 75 | $0.00 | Yes |
| Qwen 3.8 27B (xhigh), Qwen Code | 68 | $0.22 | No |
Time
All
Time per task
Average minutes per agent task, at 100 output tokens/s · Lower is better
How to read this chart
What this metric means. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
Data table: Time per task
Open weights
Time per task
Average minutes per agent task, at 100 output tokens/s · Lower is better
How to read this chart
What this metric means. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
Data table: Time per task
All
Geospatial Agent Index vs. time per task
Geospatial Agent Index, Text only; time per task (minutes) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Why a log scale. Costs and times differ by a factor of ten or more between models, so equal steps on the axis are equal ratios.
What this metric means. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
Not yet ranked. Entries with too few tasks run so far are not plotted; the table lists them, and every entry is on the models page.
How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no faster shown model beats on score.
Data table: Geospatial Agent Index vs. time per task
Open weights
Geospatial Agent Index vs. time per task
Geospatial Agent Index, Text only; time per task (minutes) · Higher and further left is better
- Most attractive quadrant
- Pareto line
How to read this chart
Why a log scale. Costs and times differ by a factor of ten or more between models, so equal steps on the axis are equal ratios.
What this metric means. Time as if every model wrote 100 output tokens per second: output tokens divided by 100, plus the time the agent's commands took. Serving speed differs between providers and is not ranked.
How to read this chart. Each point is one entry. Further left means a lower average time per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no faster shown model beats on score.
Data table: Geospatial Agent Index vs. time per task
| Model | Geospatial Agent Index | Time per task (minutes) | On the Pareto line |
|---|---|---|---|
| DeepSeek V4 Pro (max), Terminus-2 | 71 | 4.9 min | No |
| Gemma 4 26B A4B (Reasoning), Terminus-2 | 56 | 5.6 min | No |
| gpt-oss-120b (high), Terminus-2 | 46 | 3.0 min | No |
| gpt-oss-20b (high), Terminus-2 | 33 | 7.2 min | No |
| Llama 3.3 70B (Non-reasoning), Terminus-2 | 8 | 2.1 min | No |
| MiniMax M3.1 Flash (max), Terminus-2 | 52 | 8.7 min | No |
| Mistral Large 4 (high), Terminus-2 | 59 | 12.0 min | No |
| Nemotron 3 Nano Omni 30B (Reasoning), Terminus-2 | 21 | 4.6 min | No |
| Nemotron 3 Ultra (high), Terminus-2 | 68 | 2.4 min | No |
| Nemotron 3.5 Lightning 30B (Reasoning), Terminus-2 | 13 | 4.8 min | No |
| Nemotron 3.5 Super VL 120B (Reasoning), Terminus-2 | 52 | 3.8 min | No |
| Step 5 Preview (Reasoning), Terminus-2 | 51 | 13.1 min | No |
| GLM-5.3-Flash (max), Claude Code | 72 | 6.5 min | No |
| GLM-5.3 (max), Claude Code | 73 | 4.3 min | No |
| DeepSeek V4.1 Flash (max), Codex | 71 | 8.6 min | No |
| Kimi K2.7 Code (Reasoning), Kimi Code | 72 | 1.9 min | No |
| Kimi K3 (Reasoning), Kimi Code | 75 | 1.5 min | Yes |
| Qwen 3.8 27B (xhigh), Qwen Code | 68 | 4.7 min | No |
Tokens
All
Output tokens per task
Average output tokens per agent task, reasoning included · Lower is better
How to read this chart
Output tokens per task. The average number of tokens the model wrote per attempt, its reasoning included, as reported by the provider.
Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.
Data table: Output tokens per task
Open weights
Output tokens per task
Average output tokens per agent task, reasoning included · Lower is better
How to read this chart
Output tokens per task. The average number of tokens the model wrote per attempt, its reasoning included, as reported by the provider.
Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.
Data table: Output tokens per task
All
Turns per task
Average agent turns per task · Lower is better
How to read this chart
What this metric means. A turn is one model reply plus the commands it ran.
Data table: Turns per task
Open weights
Turns per task
Average agent turns per task · Lower is better
How to read this chart
What this metric means. A turn is one model reply plus the commands it ran.
Data table: Turns per task
Side by side
| Metric | gpt-oss-20b (high), Terminus-2 | Kimi K3 (Reasoning), Kimi Code |
|---|---|---|
| Geospatial Agent Index: Text only | 33 | 75 |
| Geospatial Agent Index: Text + vision | No image input | No data |
| Earth-Bench V2 score | 32 | 74 |
| GeoBenchX score | 38 | 65 |
| GS-QA2 score | 30 | 85 |
| Vision agents score | No image input | No data |
| GeoAgentBench score (Guided workflows; not in the Agent Index) | 53 | 96 |
| GeoAnalystBench score (Guided workflows; not in the Agent Index) | 37 | 100 |
| OGD4All score (standalone; in no index) | No data | 97 |
| GeoNatureAgent score (standalone; in no index) | No data | 100 |
| MultiGlobeQA score (standalone; in no index) | No data | 77 |
| GeoAgentEval score (standalone; in no index) | No data | 96 |
| Remote sensing and raster analysis score | 33 | 81 |
| GIS analysis score | 26 | 84 |
| Climate and time series score | 31 | 78 |
| Spatial statistics score | 0 | 38 |
| Making maps score | 21 | 82 |
| Recognising infeasible requests score | 67 | 52 |
| Cost per task | $0.017 | $0.00 |
| Time per task (at 100 output tokens/s) | 7.2 min | 1.5 min (estimated) |
| Tokens per attempt | 67k | 345k |
| Turns per attempt | 5.2 | No data |
| Input price per 1M tokens | $0.20 | $3.00 |
| Output price per 1M tokens | $0.30 | $15.00 |
| Context window | 128,000 | 1,000,000 |
| Image input | No | Yes |
| Developer | OpenAI | Moonshot AI |