Axis Spatial

Geospatial Agent Index

Compare agents

Which agent setup should you use for geospatial (GIS and Earth observation) tasks? Compare models, agent harnesses and reasoning settings by task success, cost and execution time. For which model is best overall, see the Geospatial Intelligence Index, where this index is the agents group.

48 model configurations · 8 developers · 634 tasks · 2 index evaluations · 2 guided workflows · GS-QA2 joining · Full leaderboard · Compare models

Ongoing evaluation: results update as runs complete. Last updated 12 October 2026.

Axis Spatial Geospatial Agent Index

Composite index of 2 hard evaluations, 320 tasks in all:

Each evaluation score averages pass@1 across its tasks. The index gives equal weight to its 2 components; GS-QA2 is first averaged within each question template. Each configuration is planned for 3 attempts per task; an entry without a first attempt on every index task is listed apart as partial coverage and not ranked. See methodology 0.30 for scoring details and version history.

Cost, execution time and token usage are reported separately from the same attempts, on the index's tasks. GeoAgentBench, GeoAnalystBench are published separately as guided GIS workflows. Breakdowns by task format and kind of geospatial work do not change the index.

GS-QA2 is joining the index. GS-QA2 Balanced-245 asks questions about places and terrain in the United States over map and elevation data, answered by querying a database in the sandbox and checked by code against the source's reference query. The 245 questions are a balanced sample: up to 5 from each of 49 question templates, drawn with a fixed seed (20261011), and the score averages over templates first. 31 of 48 entries have results on it so far, 27 with a first attempt on every question. It will count as the third component of the Geospatial Agent Index, equally weighted with the other two, once the remaining runs are complete; entries without it will then be listed as partial coverage. Until then its scores are published on its own page and do not change the index.

Headline results

Geospatial Agent Index

Equal-weight average of the evaluation scores, 0 to 100 · Higher is better

  • Planned attempts still to come
Geospatial Agent IndexClaude Code – Claude Opus 5.5 (xhigh): 69* (partial coverage); Claude Code – Claude Opus 5.5 (high): 68* (partial coverage); Claude Code – Claude Opus 5.5 (medium): 65* (partial coverage); Codex – GPT-6.1 Sol (medium): 65* (partial coverage); Codex – GPT-6.1 Sol (xhigh): 65* (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 65* (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 64* (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 63* (partial coverage); Codex – GPT-6.1 Sol (high): 63* (partial coverage); Claude Code – Claude Haiku 5.5 (high): 60* (partial coverage); Codex – GPT-6 Luna (xhigh): 60* (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 58* (partial coverage); Terminus-2 – Kimi K3 (always on): 58* (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 58* (partial coverage); Terminus-2 – GLM-5.2 (high): 56* (partial coverage); Terminus-2 – GLM-5.2 (max): 55* (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 55* (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 54* (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 53* (partial coverage); Terminus-2 – Kimi K2.6 (always on): 52* (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 52* (partial coverage); Terminus-2 – GLM-5.2 (none): 43* (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 40* (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 39* (partial coverage); Terminus-2 – GLM-5.3 (low): 38* (partial coverage); Terminus-2 – GLM-5.3 (max, default): 38* (partial coverage); Terminus-2 – gpt-oss-120b (high): 34* (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 29* (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 27* (partial coverage); Terminus-2 – gpt-oss-120b (low): 25* (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 7* (partial coverage)025507510069*Claude Opus 5.5 (xhigh)68*Claude Opus 5.5 (high)65*Claude Opus 5.5 (medium)65*GPT-6.1 Sol (medium)65*GPT-6.1 Sol (xhigh)65*Claude Sonnet 5.5 (high)64*Claude Sonnet 5.5 (xhigh)63*Claude Haiku 5.5 (xhigh)63*GPT-6.1 Sol (high)60*Claude Haiku 5.5 (high)60*GPT-6 Luna (xhigh)58*Claude Haiku 5.5 (medium)58*Kimi K3 (always on)58*Claude Sonnet 5.5 (medium)56*GLM-5.2 (high)55*GLM-5.2 (max)55*Kimi K2.7 Code (always on)54*DeepSeek V4 Pro (high)53*DeepSeek V4 Flash (max)52*Kimi K2.6 (always on)52*DeepSeek V4 Flash (high)43*GLM-5.2 (none)40*GLM-5.3-Flash (low)39*Gemma 4 26B A4B (thinking on)38*GLM-5.3 (low)38*GLM-5.3 (max, default)34*gpt-oss-120b (high)29*GLM-4.7-Flash (thinking on)27*GLM-5.3-Flash (max)25*gpt-oss-120b (low)7*Llama 3.3 70B (Non-reasoning)
Data table: Geospatial Agent Index
Geospatial Agent Index
ModelCreatorGeospatial Agent IndexRange while attempts are pendingBenchmarks coveredCoverage (attempts)
Claude Code – Claude Opus 5.5 (xhigh)Anthropic69*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic68*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic64*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic63*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI63*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic60*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI60*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic58*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI58*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic58*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai56*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai55*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI55*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek54*No pending judgements; coverage incomplete; 2 trace reviews declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek53*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI52*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek52*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai43*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai40*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google39*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai38*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai38*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI34*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai29*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai27*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI25*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta7*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts

Time per task

Average agent wall time per task · Lower is better

  • Planned attempts still to come
Time per taskClaude Code – Claude Sonnet 5.5 (medium): 0.6 min (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 0.8 min (partial coverage); Claude Code – Claude Opus 5.5 (medium): 0.8 min (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 0.8 min (partial coverage); Terminus-2 – gpt-oss-120b (low): 0.9 min (partial coverage); Claude Code – Claude Opus 5.5 (high): 1.1 min (partial coverage); Claude Code – Claude Haiku 5.5 (high): 1.1 min (partial coverage); Codex – GPT-6.1 Sol (medium): 1.4 min (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 1.6 min (partial coverage); Codex – GPT-6.1 Sol (high): 1.8 min (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 2.2 min (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 2.4 min (partial coverage); Codex – GPT-6.1 Sol (xhigh): 3.0 min (partial coverage); Codex – GPT-6 Luna (xhigh): 3.9 min (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 3.9 min (partial coverage); Terminus-2 – Kimi K3 (always on): 3.9 min (partial coverage); Terminus-2 – GLM-5.2 (none): 4.1 min (partial coverage); Terminus-2 – gpt-oss-120b (high): 5.2 min (partial coverage); Terminus-2 – GLM-5.2 (high): 8.3 min (partial coverage); Terminus-2 – GLM-5.3 (low): 8.7 min (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 9.4 min (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 10.6 min (partial coverage); Terminus-2 – Kimi K2.6 (always on): 10.6 min (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 10.9 min (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 11.1 min (partial coverage); Terminus-2 – GLM-5.2 (max): 11.8 min (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 11.8 min (partial coverage); Terminus-2 – GLM-5.3 (max, default): 12.4 min (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 12.8 min (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 13.3 min (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 19.9 min (partial coverage)0m5m10m15m20m0.6 minClaude Sonnet 5.5 (medium)0.8 minClaude Sonnet 5.5 (high)0.8 minClaude Opus 5.5 (medium)0.8 minClaude Haiku 5.5 (medium)0.9 mingpt-oss-120b (low)1.1 minClaude Opus 5.5 (high)1.1 minClaude Haiku 5.5 (high)1.4 minGPT-6.1 Sol (medium)1.6 minClaude Sonnet 5.5 (xhigh)1.8 minGPT-6.1 Sol (high)2.2 minClaude Haiku 5.5 (xhigh)2.4 minClaude Opus 5.5 (xhigh)3.0 minGPT-6.1 Sol (xhigh)3.9 minGPT-6 Luna (xhigh)3.9 minLlama 3.3 70B (Non-reasoning)3.9 minKimi K3 (always on)4.1 minGLM-5.2 (none)5.2 mingpt-oss-120b (high)8.3 minGLM-5.2 (high)8.7 minGLM-5.3 (low)9.4 minKimi K2.7 Code (always on)10.6 minDeepSeek V4 Pro (high)10.6 minKimi K2.6 (always on)10.9 minGemma 4 26B A4B (thinking on)11.1 minGLM-4.7-Flash (thinking on)11.8 minGLM-5.2 (max)11.8 minDeepSeek V4 Flash (max)12.4 minGLM-5.3 (max, default)12.8 minDeepSeek V4 Flash (high)13.3 minGLM-5.3-Flash (low)19.9 minGLM-5.3-Flash (max)
How to read this chart

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Time per task
Time per task
ModelCreatorTime per taskCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek11.8 min320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek12.8 min320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek10.6 min320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google10.9 min320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai11.1 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai8.3 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai4.1 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai11.8 min320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai13.3 min320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai19.9 min320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai8.7 min320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai12.4 min320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI0.9 min320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI5.2 min320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI10.6 min320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI9.4 min320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI3.9 min320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta3.9 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic1.1 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic0.6 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic1.1 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic2.4 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic1.6 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic2.2 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI1.4 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI1.8 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI3.0 min320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI3.9 min320 of 614 planned attempts

Cost per task

Average cost per task (USD) · Lower is better

  • Planned attempts still to come
Cost per taskTerminus-2 – gpt-oss-120b (low): $0.0061 (partial coverage); Claude Code – Claude Haiku 5.5 (medium): $0.0080 (partial coverage); Codex – GPT-6 Luna (xhigh): $0.010 (partial coverage); Claude Code – Claude Haiku 5.5 (high): $0.011 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): $0.018 (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): $0.024 (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): $0.026 (partial coverage); Terminus-2 – gpt-oss-120b (high): $0.033 (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): $0.035 (partial coverage); Terminus-2 – GLM-5.3-Flash (low): $0.051 (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): $0.058 (partial coverage); Terminus-2 – GLM-5.3-Flash (max): $0.060 (partial coverage); Codex – GPT-6.1 Sol (medium): $0.061 (partial coverage); Claude Code – Claude Sonnet 5.5 (high): $0.072 (partial coverage); Codex – GPT-6.1 Sol (high): $0.081 (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): $0.086 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): $0.087 (partial coverage); Terminus-2 – Kimi K2.6 (always on): $0.109 (partial coverage); Codex – GPT-6.1 Sol (xhigh): $0.126 (partial coverage); Claude Code – Claude Opus 5.5 (medium): $0.149 (partial coverage); Terminus-2 – GLM-5.2 (none): $0.151 (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): $0.172 (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): $0.183 (partial coverage); Claude Code – Claude Opus 5.5 (high): $0.186 (partial coverage); Terminus-2 – GLM-5.2 (high): $0.319 (partial coverage); Terminus-2 – Kimi K3 (always on): $0.336 (partial coverage); Terminus-2 – GLM-5.3 (low): $0.407 (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): $0.411 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): $0.428 (partial coverage); Terminus-2 – GLM-5.2 (max): $0.443 (partial coverage); Terminus-2 – GLM-5.3 (max, default): $0.525 (partial coverage)$0$0.150$0.300$0.450$0.600$0.0061gpt-oss-120b (low)$0.0080Claude Haiku 5.5 (medium)$0.010GPT-6 Luna (xhigh)$0.011Claude Haiku 5.5 (high)$0.018Gemma 4 26B A4B (thinking on)$0.024Llama 3.3 70B (Non-reasoning)$0.026Claude Haiku 5.5 (xhigh)$0.033gpt-oss-120b (high)$0.035GLM-4.7-Flash (thinking on)$0.051GLM-5.3-Flash (low)$0.058Claude Sonnet 5.5 (medium)$0.060GLM-5.3-Flash (max)$0.061GPT-6.1 Sol (medium)$0.072Claude Sonnet 5.5 (high)$0.081GPT-6.1 Sol (high)$0.086DeepSeek V4 Flash (max)$0.087DeepSeek V4 Flash (high)$0.109Kimi K2.6 (always on)$0.126GPT-6.1 Sol (xhigh)$0.149Claude Opus 5.5 (medium)$0.151GLM-5.2 (none)$0.172Claude Sonnet 5.5 (xhigh)$0.183Kimi K2.7 Code (always on)$0.186Claude Opus 5.5 (high)$0.319GLM-5.2 (high)$0.336Kimi K3 (always on)$0.407GLM-5.3 (low)$0.411Claude Opus 5.5 (xhigh)$0.428DeepSeek V4 Pro (high)$0.443GLM-5.2 (max)$0.525GLM-5.3 (max, default)
How to read this chart

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Cost per task
Cost per task
ModelCreatorCost per taskCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek$0.086320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek$0.087320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek$0.428320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google$0.018320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai$0.035320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai$0.319320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai$0.151320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai$0.443320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai$0.051320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai$0.060320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai$0.407320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai$0.525320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI$0.0061320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI$0.033320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI$0.109320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI$0.183320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI$0.336320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta$0.024320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic$0.149320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic$0.186320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic$0.058320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic$0.072320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic$0.0080320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic$0.011320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic$0.411320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic$0.172320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic$0.026320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI$0.061320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI$0.081320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI$0.126320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI$0.010320 of 614 planned attempts

Scope. Results come from file-based geospatial analysis tasks: the agent reads rasters and vectors with code, computes, and makes maps. Image interpretation, where the model has to look at an image to answer, is not covered.

Performance

Geospatial Agent Index

Geospatial Agent Index

Equal-weight average of the evaluation scores, 0 to 100 · Higher is better

  • Planned attempts still to come
Geospatial Agent IndexClaude Code – Claude Opus 5.5 (xhigh): 69* (partial coverage); Claude Code – Claude Opus 5.5 (high): 68* (partial coverage); Claude Code – Claude Opus 5.5 (medium): 65* (partial coverage); Codex – GPT-6.1 Sol (medium): 65* (partial coverage); Codex – GPT-6.1 Sol (xhigh): 65* (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 65* (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 64* (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 63* (partial coverage); Codex – GPT-6.1 Sol (high): 63* (partial coverage); Claude Code – Claude Haiku 5.5 (high): 60* (partial coverage); Codex – GPT-6 Luna (xhigh): 60* (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 58* (partial coverage); Terminus-2 – Kimi K3 (always on): 58* (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 58* (partial coverage); Terminus-2 – GLM-5.2 (high): 56* (partial coverage); Terminus-2 – GLM-5.2 (max): 55* (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 55* (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 54* (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 53* (partial coverage); Terminus-2 – Kimi K2.6 (always on): 52* (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 52* (partial coverage); Terminus-2 – GLM-5.2 (none): 43* (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 40* (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 39* (partial coverage); Terminus-2 – GLM-5.3 (low): 38* (partial coverage); Terminus-2 – GLM-5.3 (max, default): 38* (partial coverage); Terminus-2 – gpt-oss-120b (high): 34* (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 29* (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 27* (partial coverage); Terminus-2 – gpt-oss-120b (low): 25* (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 7* (partial coverage)025507510069*Claude CodeClaude Opus5.5 (xhigh)68*Claude CodeClaude Opus5.5 (high)65*Claude CodeClaude Opus5.5 (medium)65*CodexGPT-6.1 Sol(medium)65*CodexGPT-6.1 Sol(xhigh)65*Claude CodeClaude Sonnet5.5 (high)64*Claude CodeClaude Sonnet5.5 (xhigh)63*Claude CodeClaude Haiku5.5 (xhigh)63*CodexGPT-6.1 Sol(high)60*Claude CodeClaude Haiku5.5 (high)60*CodexGPT-6 Luna(xhigh)58*Claude CodeClaude Haiku5.5 (medium)58*Terminus-2Kimi K3(always on)58*Claude CodeClaude Sonnet5.5 (medium)56*Terminus-2GLM-5.2(high)55*Terminus-2GLM-5.2 (max)55*Terminus-2Kimi K2.7Code (alwayson)54*Terminus-2DeepSeek V4Pro (high)53*Terminus-2DeepSeek V4Flash (max)52*Terminus-2Kimi K2.6(always on)52*Terminus-2DeepSeek V4Flash (high)43*Terminus-2GLM-5.2(none)40*Terminus-2GLM-5.3-Flash(low)39*Terminus-2Gemma 4 26BA4B (thinkingon)38*Terminus-2GLM-5.3 (low)38*Terminus-2GLM-5.3 (max,default)34*Terminus-2gpt-oss-120b(high)29*Terminus-2GLM-4.7-Flash(thinking on)27*Terminus-2GLM-5.3-Flash(max)25*Terminus-2gpt-oss-120b(low)7*Terminus-2Llama 3.3 70B(Non-reasoning)
How to read this chart

What this metric means. The equal-weight average of the 2 evaluation scores. Each evaluation score is the average pass@1 over that benchmark's tasks, from 0 to 100.

Partial coverage entries. An entry is ranked only when it has a first attempt on every task of every index component. Entries without that are marked partial coverage: they are listed apart with their coverage, for example "36 of 147 Earth-Bench V2 tasks", and are left out of the ranking, the index charts, the Pareto lines and the highlights. An entry becomes ranked as soon as its first attempts are complete.

Coverage hatch. A hatched bar, and an asterisk in tables, marks an entry that does not yet have results for every planned attempt. While attempts await review, the table gives the range the score could still reach.

First attempt only. Scores currently use each entry's first attempt on every task. The full protocol is 3 attempts; averages over 3 will replace these once every entry has them. An entry without a first attempt on every task yet is marked "partial: attempt 1 on n of N tasks".

Not shown: partial coverage or being added. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.1 Flash-Lite (low) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.1 Flash-Lite (minimal) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.1 Flash-Lite (high) (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks); Terminus-2 – Gemini 3.1 Pro (preview) (low) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.1 Pro (preview) (high) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.8 Flash (high) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – Gemini 3.8 Flash (low) (36 of 147 Earth-Bench V2 tasks); Terminus-2 – gpt-oss-20b (high) (172 of 173 GeoBenchX tasks); Terminus-2 – Qwen 3.8 27B (xhigh) (36 of 147 Earth-Bench V2 tasks; 0 of 173 GeoBenchX tasks); Claude Code – Claude Haiku 4.5 (thinking 32k) (47 of 147 Earth-Bench V2 tasks; 47 of 173 GeoBenchX tasks); Kimi Code – Kimi K2.6 (always on) (11 of 147 Earth-Bench V2 tasks; 61 of 173 GeoBenchX tasks); Kimi Code – Kimi K2.7 Code (always on) (12 of 147 Earth-Bench V2 tasks; 70 of 173 GeoBenchX tasks); Claude Code – Claude Opus 5 (high) (112 of 147 Earth-Bench V2 tasks; 48 of 173 GeoBenchX tasks); Claude Code – Claude Sonnet 5 (high) (41 of 147 Earth-Bench V2 tasks; 63 of 173 GeoBenchX tasks); Claude Code – Claude Opus 4.8 (high) (49 of 147 Earth-Bench V2 tasks; 56 of 173 GeoBenchX tasks); Claude Code – Claude Opus 4.7 (high) (49 of 147 Earth-Bench V2 tasks; 60 of 173 GeoBenchX tasks).

Data table: Geospatial Agent Index
Geospatial Agent Index
ModelCreatorGeospatial Agent IndexRange while attempts are pendingBenchmarks coveredCoverage (attempts)
Claude Code – Claude Opus 5.5 (xhigh)Anthropic69*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic68*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic65*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic64*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic63*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI63*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic60*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI60*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic58*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI58*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic58*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai56*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai55*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI55*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek54*No pending judgements; coverage incomplete; 2 trace reviews declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek53*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI52*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek52*No pending judgements; coverage incomplete; 1 trace review declined by the judge2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai43*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai40*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google39*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai38*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai38*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI34*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai29*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai27*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI25*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta7*No pending judgements; coverage incomplete2 of 2320 of 614 planned attempts

Score by benchmark

Score by benchmark (0 to 100) and the Geospatial Agent Index
ModelEarth-Bench V2GeoBenchXGeospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)667269*
Codex – GPT-6.1 Sol (xhigh)616965*
Claude Code – Claude Sonnet 5.5 (xhigh)577164*
Claude Code – Claude Haiku 5.5 (xhigh)586863*
Codex – GPT-6 Luna (xhigh)566460*
Terminus-2 – Kimi K3 (always on)615558*
Terminus-2 – GLM-5.2 (max)525855*
Terminus-2 – Kimi K2.7 Code (always on)565455*
Terminus-2 – DeepSeek V4 Pro (high)525654*
Terminus-2 – DeepSeek V4 Flash (max)525453*
Terminus-2 – Kimi K2.6 (always on)495552*
Terminus-2 – Gemma 4 26B A4B (thinking on)294939*
Terminus-2 – GLM-5.3 (max, default)334238*
Terminus-2 – gpt-oss-120b (high)323634*
Terminus-2 – GLM-4.7-Flash (thinking on)253229*
Terminus-2 – GLM-5.3-Flash (max)282727*
Terminus-2 – Llama 3.3 70B (Non-reasoning)767*
Claude Code – Claude Opus 5 (high)636263* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)565656* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)555756* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)584954* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)515553* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)475551* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)285943* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)314940* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)394040* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)284536* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)284034* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)243128*
Terminus-2 – Qwen 3.8 27B (xhigh)28No data28* (partial: attempt 1 on 36 of 320 tasks)

Each model and harness is shown at its highest reasoning level, as Artificial Analysis does; other levels are under 'Show every reasoning level'.

Show every reasoning level
Score by benchmark (0 to 100) and the Geospatial Agent Index, every reasoning level
ModelEarth-Bench V2GeoBenchXGeospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)667269*
Claude Code – Claude Opus 5.5 (high)686868*
Claude Code – Claude Opus 5.5 (medium)626965*
Codex – GPT-6.1 Sol (medium)616965*
Codex – GPT-6.1 Sol (xhigh)616965*
Claude Code – Claude Sonnet 5.5 (high)587165*
Claude Code – Claude Sonnet 5.5 (xhigh)577164*
Claude Code – Claude Haiku 5.5 (xhigh)586863*
Codex – GPT-6.1 Sol (high)586763*
Claude Code – Claude Haiku 5.5 (high)586260*
Codex – GPT-6 Luna (xhigh)566460*
Claude Code – Claude Haiku 5.5 (medium)556158*
Terminus-2 – Kimi K3 (always on)615558*
Claude Code – Claude Sonnet 5.5 (medium)496658*
Terminus-2 – GLM-5.2 (high)545856*
Terminus-2 – GLM-5.2 (max)525855*
Terminus-2 – Kimi K2.7 Code (always on)565455*
Terminus-2 – DeepSeek V4 Pro (high)525654*
Terminus-2 – DeepSeek V4 Flash (max)525453*
Terminus-2 – Kimi K2.6 (always on)495552*
Terminus-2 – DeepSeek V4 Flash (high)495552*
Terminus-2 – GLM-5.2 (none)404643*
Terminus-2 – GLM-5.3-Flash (low)374340*
Terminus-2 – Gemma 4 26B A4B (thinking on)294939*
Terminus-2 – GLM-5.3 (low)334338*
Terminus-2 – GLM-5.3 (max, default)334238*
Terminus-2 – gpt-oss-120b (high)323634*
Terminus-2 – GLM-4.7-Flash (thinking on)253229*
Terminus-2 – GLM-5.3-Flash (max)282727*
Terminus-2 – gpt-oss-120b (low)203125*
Terminus-2 – Llama 3.3 70B (Non-reasoning)767*
Claude Code – Claude Opus 5 (high)636263* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)565656* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)555756* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)584954* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)515553* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)475551* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (low)366249* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)286044* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)285943* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)314940* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)394040* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)314337* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)284536* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)284034* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)193929* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)243128*
Terminus-2 – Qwen 3.8 27B (xhigh)28No data28* (partial: attempt 1 on 36 of 320 tasks)

By task format

Score by task format (0 to 100) and the Geospatial Agent Index
ModelMulti-step analysis (82 tasks)Single-answer questions (159 tasks)Recognising infeasible requests (79 tasks)Geospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)80676269*
Claude Code – Claude Opus 5.5 (high)76695968*
Claude Code – Claude Opus 5.5 (medium)76636165*
Codex – GPT-6.1 Sol (medium)70627065*
Codex – GPT-6.1 Sol (xhigh)73606765*
Claude Code – Claude Sonnet 5.5 (high)79606265*
Claude Code – Claude Sonnet 5.5 (xhigh)79576364*
Claude Code – Claude Haiku 5.5 (xhigh)73586563*
Codex – GPT-6.1 Sol (high)68586763*
Claude Code – Claude Haiku 5.5 (high)70585660*
Codex – GPT-6 Luna (xhigh)67576260*
Claude Code – Claude Haiku 5.5 (medium)66565658*
Terminus-2 – Kimi K3 (always on)72603858*
Claude Code – Claude Sonnet 5.5 (medium)73515858*
Terminus-2 – GLM-5.2 (high)74554156*
Terminus-2 – GLM-5.2 (max)78533755*
Terminus-2 – Kimi K2.7 Code (always on)65554355*
Terminus-2 – DeepSeek V4 Pro (high)67514754*
Terminus-2 – DeepSeek V4 Flash (max)67524153*
Terminus-2 – Kimi K2.6 (always on)66504352*
Terminus-2 – DeepSeek V4 Flash (high)67494352*
Terminus-2 – GLM-5.2 (none)56423443*
Terminus-2 – GLM-5.3-Flash (low)49383540*
Terminus-2 – Gemma 4 26B A4B (thinking on)46334739*
Terminus-2 – GLM-5.3 (low)50333838*
Terminus-2 – GLM-5.3 (max, default)46343838*
Terminus-2 – gpt-oss-120b (high)28334434*
Terminus-2 – GLM-4.7-Flash (thinking on)30263329*
Terminus-2 – GLM-5.3-Flash (max)26272927*
Terminus-2 – gpt-oss-120b (low)13214725*
Terminus-2 – Llama 3.3 70B (Non-reasoning)17137*
Claude Code – Claude Opus 5 (high)77635063* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)75534156* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)80504056* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)69563154* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)69543853* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)64523851* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (low)45358449* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)46407244* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)46337343* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)46434740* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)50422940* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)40334737* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)55332636* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)18256734* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)30195129* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)12255128*
Terminus-2 – Qwen 3.8 27B (xhigh)No data28No data28* (partial: attempt 1 on 36 of 320 tasks)

Multi-step analysis: carry out a workflow, deliver data files and maps. Single-answer questions: one number, choice or list. Recognising infeasible requests: decline when the data cannot answer. More on task formats.

By kind of work

Score by kind of work (0 to 100) and the Geospatial Agent Index
ModelRaster and remote sensing (94 tasks)Vector and overlay (6 tasks)Networks and routing (0 tasks)Climate and time series (68 tasks)Spatial statistics and interpolation (14 tasks)Mapping and cartography (59 tasks)Recognising infeasible requests (79 tasks)Geospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)6967No data6657856269*
Claude Code – Claude Opus 5.5 (high)7250No data6550835968*
Claude Code – Claude Opus 5.5 (medium)6350No data6643836165*
Codex – GPT-6.1 Sol (medium)5967No data6829787065*
Codex – GPT-6.1 Sol (xhigh)5650No data6936816765*
Claude Code – Claude Sonnet 5.5 (high)6167No data6357816265*
Claude Code – Claude Sonnet 5.5 (xhigh)6050No data6043856364*
Claude Code – Claude Haiku 5.5 (xhigh)5633No data6243836563*
Codex – GPT-6.1 Sol (high)5450No data6629786763*
Claude Code – Claude Haiku 5.5 (high)5633No data6629785660*
Codex – GPT-6 Luna (xhigh)5550No data6343696260*
Claude Code – Claude Haiku 5.5 (medium)5650No data5721765658*
Terminus-2 – Kimi K3 (always on)5933No data6650783858*
Claude Code – Claude Sonnet 5.5 (medium)4967No data5657765858*
Terminus-2 – GLM-5.2 (high)5450No data6243784156*
Terminus-2 – GLM-5.2 (max)5650No data5657803755*
Terminus-2 – Kimi K2.7 Code (always on)5150No data6336714355*
Terminus-2 – DeepSeek V4 Pro (high)5417No data5121784754*
Terminus-2 – DeepSeek V4 Flash (max)5133No data5921764153*
Terminus-2 – Kimi K2.6 (always on)5150No data5414754352*
Terminus-2 – DeepSeek V4 Flash (high)5217No data5136734352*
Terminus-2 – GLM-5.2 (none)4033No data440713443*
Terminus-2 – GLM-5.3-Flash (low)3650No data440583540*
Terminus-2 – Gemma 4 26B A4B (thinking on)3167No data347564739*
Terminus-2 – GLM-5.3 (low)3117No data380633838*
Terminus-2 – GLM-5.3 (max, default)3150No data380583838*
Terminus-2 – gpt-oss-120b (high)2950No data380324434*
Terminus-2 – GLM-4.7-Flash (thinking on)2650No data260373329*
Terminus-2 – GLM-5.3-Flash (max)2433No data310312927*
Terminus-2 – gpt-oss-120b (low)1650No data240194725*
Terminus-2 – Llama 3.3 70B (Non-reasoning)40No data1002137*
Claude Code – Claude Opus 5 (high)62No dataNo data6550875063* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)520No data6750804156* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)600No data067834056* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)670No data017813154* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)560No data5533793853* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)50100No data5017833851* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (low)380No data4229518449* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)3867No data4229497244* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)2633No data5821547343* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)3950No data500584740* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)3833No data580632940* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)3333No data427474737* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)33No dataNo data3340602636* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)2317No data257226734* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)2333No data170365129* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)2433No data240165128*
Terminus-2 – Qwen 3.8 27B (xhigh)25No dataNo data33No dataNo dataNo data28* (partial: attempt 1 on 36 of 320 tasks)

Capability pages explain each kind of work.

By data type

Score by data type (0 to 100) and the Geospatial Agent Index
ModelOptical multispectral (114 tasks)Climate or gridded time series (10 tasks)Other gridded data (50 tasks)Vector only (77 tasks)Geospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)61100787869*
Claude Code – Claude Opus 5.5 (high)6580767468*
Claude Code – Claude Opus 5.5 (medium)58100727365*
Codex – GPT-6.1 Sol (medium)5870666965*
Codex – GPT-6.1 Sol (xhigh)5770687165*
Claude Code – Claude Sonnet 5.5 (high)54100707565*
Claude Code – Claude Sonnet 5.5 (xhigh)53100687464*
Claude Code – Claude Haiku 5.5 (xhigh)5570647163*
Codex – GPT-6.1 Sol (high)5460666863*
Claude Code – Claude Haiku 5.5 (high)5490666560*
Codex – GPT-6 Luna (xhigh)5380626460*
Claude Code – Claude Haiku 5.5 (medium)5290606458*
Terminus-2 – Kimi K3 (always on)5580726958*
Claude Code – Claude Sonnet 5.5 (medium)4390707158*
Terminus-2 – GLM-5.2 (high)47100686956*
Terminus-2 – GLM-5.2 (max)48100607355*
Terminus-2 – Kimi K2.7 Code (always on)5180586255*
Terminus-2 – DeepSeek V4 Pro (high)4580606454*
Terminus-2 – DeepSeek V4 Flash (max)46100626253*
Terminus-2 – Kimi K2.6 (always on)4390626252*
Terminus-2 – DeepSeek V4 Flash (high)4090666152*
Terminus-2 – GLM-5.2 (none)3650525643*
Terminus-2 – GLM-5.3-Flash (low)3070544740*
Terminus-2 – Gemma 4 26B A4B (thinking on)2360464839*
Terminus-2 – GLM-5.3 (low)2540524838*
Terminus-2 – GLM-5.3 (max, default)2450544738*
Terminus-2 – gpt-oss-120b (high)2940362934*
Terminus-2 – GLM-4.7-Flash (thinking on)1950363229*
Terminus-2 – GLM-5.3-Flash (max)1850422627*
Terminus-2 – gpt-oss-120b (low)1720261825*
Terminus-2 – Llama 3.3 70B (Non-reasoning)601217*
Claude Code – Claude Opus 5 (high)59100677563* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)54100566956* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)44100677556* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)50100716354* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)49100676453* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)450756751* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (low)2344754349* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)1767754744* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)2344504743* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)2456754740* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)3356585140* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)2356503937* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)24No data755836* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)1322501934* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)1056253029* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)2130281428*
Terminus-2 – Qwen 3.8 27B (xhigh)13100100No data28* (partial: attempt 1 on 36 of 320 tasks)

A task counts under every data type it reads. More on data types.

By task length

Score by task length (0 to 100) and the Geospatial Agent Index
ModelShort tasks (70 tasks)Medium tasks (91 tasks)Long tasks (159 tasks)Geospatial Agent Index
Claude Code – Claude Opus 5.5 (xhigh)99765269*
Claude Code – Claude Opus 5.5 (high)99735268*
Claude Code – Claude Opus 5.5 (medium)99714865*
Codex – GPT-6.1 Sol (medium)99704865*
Codex – GPT-6.1 Sol (xhigh)99684965*
Claude Code – Claude Sonnet 5.5 (high)97754665*
Claude Code – Claude Sonnet 5.5 (xhigh)99744464*
Claude Code – Claude Haiku 5.5 (xhigh)99684563*
Codex – GPT-6.1 Sol (high)99704363*
Claude Code – Claude Haiku 5.5 (high)97664260*
Codex – GPT-6 Luna (xhigh)97664260*
Claude Code – Claude Haiku 5.5 (medium)97604058*
Terminus-2 – Kimi K3 (always on)87604358*
Claude Code – Claude Sonnet 5.5 (medium)97604058*
Terminus-2 – GLM-5.2 (high)84634056*
Terminus-2 – GLM-5.2 (max)81594255*
Terminus-2 – Kimi K2.7 Code (always on)87554055*
Terminus-2 – DeepSeek V4 Pro (high)91593554*
Terminus-2 – DeepSeek V4 Flash (max)87543853*
Terminus-2 – Kimi K2.6 (always on)89573452*
Terminus-2 – DeepSeek V4 Flash (high)89593252*
Terminus-2 – GLM-5.2 (none)77512543*
Terminus-2 – GLM-5.3-Flash (low)79481940*
Terminus-2 – Gemma 4 26B A4B (thinking on)86411939*
Terminus-2 – GLM-5.3 (low)77491538*
Terminus-2 – GLM-5.3 (max, default)83481338*
Terminus-2 – gpt-oss-120b (high)69371834*
Terminus-2 – GLM-4.7-Flash (thinking on)70301129*
Terminus-2 – GLM-5.3-Flash (max)7426727*
Terminus-2 – gpt-oss-120b (low)7121825*
Terminus-2 – Llama 3.3 70B (Non-reasoning)23327*
Claude Code – Claude Opus 5 (high)100764563* (partial: attempt 1 on 160 of 320 tasks)
Claude Code – Claude Sonnet 5 (high)100624456* (partial: attempt 1 on 104 of 320 tasks)
Kimi Code – Kimi K2.6 (always on)100534456* (partial: attempt 1 on 72 of 320 tasks)
Kimi Code – Kimi K2.7 Code (always on)86384554* (partial: attempt 1 on 82 of 320 tasks)
Claude Code – Claude Opus 4.7 (high)100524653* (partial: attempt 1 on 109 of 320 tasks)
Claude Code – Claude Opus 4.8 (high)100643951* (partial: attempt 1 on 105 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (low)100583849* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)98603244* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)93603243* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)83532440* (partial: attempt 1 on 205 of 320 tasks)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)65442740* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)87392237* (partial: attempt 1 on 209 of 320 tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)100352736* (partial: attempt 1 on 94 of 320 tasks)
Terminus-2 – Gemini 3.8 Flash (high)96351434* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)85321529* (partial: attempt 1 on 209 of 320 tasks)
Terminus-2 – gpt-oss-20b (high)70231328*
Terminus-2 – Qwen 3.8 27B (xhigh)10050428* (partial: attempt 1 on 36 of 320 tasks)

Thirds of the tasks by median agent time. More on task length.

Earth-Bench V2

Earth-Bench V2 score

Average pass@1 over 147 tasks, 0 to 100 · Higher is better

  • Planned attempts still to come
Earth-Bench V2 scoreClaude Code – Claude Opus 5.5 (high): 68 (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 66 (partial coverage); Claude Code – Claude Opus 5 (high): 63 (partial coverage); Claude Code – Claude Opus 5.5 (medium): 62 (partial coverage); Codex – GPT-6.1 Sol (medium): 61 (partial coverage); Terminus-2 – Kimi K3 (always on): 61 (partial coverage); Codex – GPT-6.1 Sol (xhigh): 61 (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 58 (partial coverage); Claude Code – Claude Haiku 5.5 (high): 58 (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 58 (partial coverage); Codex – GPT-6.1 Sol (high): 58 (partial coverage); Kimi Code – Kimi K2.7 Code (always on): 58 (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 57 (partial coverage); Codex – GPT-6 Luna (xhigh): 56 (partial coverage); Claude Code – Claude Sonnet 5 (high): 56 (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 56 (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 55 (partial coverage); Kimi Code – Kimi K2.6 (always on): 55 (partial coverage); Terminus-2 – GLM-5.2 (high): 54 (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 52 (partial coverage); Terminus-2 – GLM-5.2 (max): 52 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 52 (partial coverage); Claude Code – Claude Opus 4.7 (high): 51 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 49 (partial coverage); Terminus-2 – Kimi K2.6 (always on): 49 (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 49 (partial coverage); Claude Code – Claude Opus 4.8 (high): 47 (partial coverage); Terminus-2 – GLM-5.2 (none): 40 (partial coverage); Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high): 39 (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 37 (partial coverage); Terminus-2 – Gemini 3.8 Flash (low): 36 (partial coverage); Terminus-2 – GLM-5.3 (max, default): 33 (partial coverage); Terminus-2 – GLM-5.3 (low): 33 (partial coverage); Terminus-2 – gpt-oss-120b (high): 32 (partial coverage); Terminus-2 – Gemini 3.1 Flash-Lite (high): 31 (partial coverage); Terminus-2 – Gemini 3.1 Flash-Lite (low): 31 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 29 (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 28 (partial coverage); Terminus-2 – Gemini 3.1 Pro (preview) (low): 28 (partial coverage); Terminus-2 – Gemini 3.1 Pro (preview) (high): 28 (partial coverage); Terminus-2 – Gemini 3.8 Flash (high): 28 (partial coverage); Terminus-2 – Qwen 3.8 27B (xhigh): 28 (partial coverage); Claude Code – Claude Haiku 4.5 (thinking 32k): 28 (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 25 (partial coverage); Terminus-2 – gpt-oss-20b (high): 24 (partial coverage); Terminus-2 – gpt-oss-120b (low): 20 (partial coverage); Terminus-2 – Gemini 3.1 Flash-Lite (minimal): 19 (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 7 (partial coverage)02040608068Claude CodeClaude Opus5.5 (high)66Claude CodeClaude Opus5.5 (xhigh)63Claude CodeClaude Opus 5(high)62Claude CodeClaude Opus5.5 (medium)61CodexGPT-6.1 Sol(medium)61Terminus-2Kimi K3(always on)61CodexGPT-6.1 Sol(xhigh)58Claude CodeClaude Sonnet5.5 (high)58Claude CodeClaude Haiku5.5 (high)58Claude CodeClaude Haiku5.5 (xhigh)58CodexGPT-6.1 Sol(high)58Kimi CodeKimi K2.7Code (alwayson)57Claude CodeClaude Sonnet5.5 (xhigh)56CodexGPT-6 Luna(xhigh)56Claude CodeClaude Sonnet5 (high)56Terminus-2Kimi K2.7Code (alwayson)55Claude CodeClaude Haiku5.5 (medium)55Kimi CodeKimi K2.6(always on)54Terminus-2GLM-5.2(high)52Terminus-2DeepSeek V4Flash (max)52Terminus-2GLM-5.2 (max)52Terminus-2DeepSeek V4Pro (high)51Claude CodeClaude Opus4.7 (high)49Terminus-2DeepSeek V4Flash (high)49Terminus-2Kimi K2.6(always on)49Claude CodeClaude Sonnet5.5 (medium)47Claude CodeClaude Opus4.8 (high)40Terminus-2GLM-5.2(none)39Terminus-2Claude Sonnet4.6(adaptive,37Terminus-2GLM-5.3-Flash(low)36Terminus-2Gemini 3.8Flash (low)33Terminus-2GLM-5.3 (max,default)33Terminus-2GLM-5.3 (low)32Terminus-2gpt-oss-120b(high)31Terminus-2Gemini 3.1Flash-Lite(high)31Terminus-2Gemini 3.1Flash-Lite(low)29Terminus-2Gemma 4 26BA4B (thinkingon)28Terminus-2GLM-5.3-Flash(max)28Terminus-2Gemini 3.1Pro (preview)(low)28Terminus-2Gemini 3.1Pro (preview)(high)28Terminus-2Gemini 3.8Flash (high)28Terminus-2Qwen 3.8 27B(xhigh)28Claude CodeClaude Haiku4.5 (thinking32k)25Terminus-2GLM-4.7-Flash(thinking on)24Terminus-2gpt-oss-20b(high)20Terminus-2gpt-oss-120b(low)19Terminus-2Gemini 3.1Flash-Lite(minimal)7Terminus-2Llama 3.3 70B(Non-reasoning)
How to read this chart

About this benchmark. Earth observation data analysis. Earth-Bench V2 results and background.

Data table: Earth-Bench V2 score
Earth-Bench V2 score
ModelCreatorEarth-Bench V2 scoreRange while attempts are pendingCoverage (attempts)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)Anthropic39No pending judgements; coverage incomplete36 of 441 planned attempts (19 excluded)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek52No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek49No pending judgements; coverage incomplete; 1 trace review declined by the judge147 of 441 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek52No pending judgements; coverage incomplete; 2 trace reviews declined by the judge147 of 441 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (low)Google31No pending judgements; coverage incomplete36 of 441 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)Google19No pending judgements; coverage incomplete36 of 441 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (high)Google31No pending judgements; coverage incomplete35 of 441 planned attempts (1 excluded)
Terminus-2 – Gemini 3.1 Pro (preview) (low)Google28No pending judgements; coverage incomplete36 of 441 planned attempts (2 excluded)
Terminus-2 – Gemini 3.1 Pro (preview) (high)Google28No pending judgements; coverage incomplete36 of 441 planned attempts
Terminus-2 – Gemini 3.8 Flash (high)Google28No pending judgements; coverage incomplete36 of 441 planned attempts
Terminus-2 – Gemini 3.8 Flash (low)Google36No pending judgements; coverage incomplete36 of 441 planned attempts (12 excluded)
Terminus-2 – Gemma 4 26B A4B (thinking on)Google29No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai25No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai54No pending judgements; coverage incomplete; 1 trace review declined by the judge147 of 441 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai40No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai52No pending judgements; coverage incomplete; 1 trace review declined by the judge147 of 441 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai37No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai28No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai33No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai33No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI20No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI32No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – gpt-oss-20b (high)OpenAI24No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI49No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI56No pending judgements; coverage incomplete; 1 trace review declined by the judge147 of 441 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI61No pending judgements; coverage incomplete; 1 trace review declined by the judge147 of 441 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta7No pending judgements; coverage incomplete147 of 441 planned attempts
Terminus-2 – Qwen 3.8 27B (xhigh)Alibaba (Qwen)28No pending judgements; coverage incomplete36 of 441 planned attempts
Claude Code – Claude Haiku 4.5 (thinking 32k)Anthropic28No pending judgements; coverage incomplete47 of 441 planned attempts
Kimi Code – Kimi K2.6 (always on)Moonshot AI55No pending judgements; coverage incomplete; 6 results awaiting review11 of 441 planned attempts
Kimi Code – Kimi K2.7 Code (always on)Moonshot AI58No pending judgements; coverage incomplete; 7 results awaiting review12 of 441 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic62No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic68No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic49No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic58No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic55No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic58No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic66No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic57No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic58No pending judgements; coverage incomplete147 of 441 planned attempts
Claude Code – Claude Opus 5 (high)Anthropic63No pending judgements; coverage incomplete; 18 results awaiting review112 of 441 planned attempts
Claude Code – Claude Sonnet 5 (high)Anthropic56No pending judgements; coverage incomplete41 of 441 planned attempts
Claude Code – Claude Opus 4.8 (high)Anthropic47No pending judgements; coverage incomplete49 of 441 planned attempts
Claude Code – Claude Opus 4.7 (high)Anthropic51No pending judgements; coverage incomplete49 of 441 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI61No pending judgements; coverage incomplete147 of 441 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI58No pending judgements; coverage incomplete147 of 441 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI61No pending judgements; coverage incomplete147 of 441 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI56No pending judgements; coverage incomplete147 of 441 planned attempts

GeoBenchX

GeoBenchX score

Average pass@1 over 173 tasks, 0 to 100 · Higher is better

  • Planned attempts still to come
GeoBenchX scoreClaude Code – Claude Opus 5.5 (xhigh): 72; Claude Code – Claude Sonnet 5.5 (high): 71; Claude Code – Claude Sonnet 5.5 (xhigh): 71; Codex – GPT-6.1 Sol (medium): 69; Codex – GPT-6.1 Sol (xhigh): 69; Claude Code – Claude Opus 5.5 (medium): 69; Claude Code – Claude Opus 5.5 (high): 68; Claude Code – Claude Haiku 5.5 (xhigh): 68; Codex – GPT-6.1 Sol (high): 67; Claude Code – Claude Sonnet 5.5 (medium): 66; Codex – GPT-6 Luna (xhigh): 64; Claude Code – Claude Opus 5 (high): 62 (partial coverage); Claude Code – Claude Haiku 5.5 (high): 62; Terminus-2 – Gemini 3.8 Flash (low): 62; Claude Code – Claude Haiku 5.5 (medium): 61; Terminus-2 – Gemini 3.1 Pro (preview) (low): 60; Terminus-2 – Gemini 3.1 Pro (preview) (high): 59; Terminus-2 – GLM-5.2 (max): 58; Terminus-2 – GLM-5.2 (high): 58; Kimi Code – Kimi K2.6 (always on): 57 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 56; Claude Code – Claude Sonnet 5 (high): 56 (partial coverage); Terminus-2 – Kimi K2.6 (always on): 55; Terminus-2 – Kimi K3 (always on): 55; Claude Code – Claude Opus 4.8 (high): 55 (partial coverage); Claude Code – Claude Opus 4.7 (high): 55 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 55; Terminus-2 – DeepSeek V4 Flash (max): 54; Terminus-2 – Kimi K2.7 Code (always on): 54; Kimi Code – Kimi K2.7 Code (always on): 49 (partial coverage); Terminus-2 – Gemini 3.1 Flash-Lite (high): 49 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 49; Terminus-2 – GLM-5.2 (none): 46; Claude Code – Claude Haiku 4.5 (thinking 32k): 45 (partial coverage); Terminus-2 – Gemini 3.1 Flash-Lite (low): 43; Terminus-2 – GLM-5.3 (low): 43; Terminus-2 – GLM-5.3-Flash (low): 43; Terminus-2 – GLM-5.3 (max, default): 42; Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high): 40; Terminus-2 – Gemini 3.8 Flash (high): 40; Terminus-2 – Gemini 3.1 Flash-Lite (minimal): 39; Terminus-2 – gpt-oss-120b (high): 36; Terminus-2 – GLM-4.7-Flash (thinking on): 32; Terminus-2 – gpt-oss-20b (high): 31 (partial coverage); Terminus-2 – gpt-oss-120b (low): 31; Terminus-2 – GLM-5.3-Flash (max): 27; Terminus-2 – Llama 3.3 70B (Non-reasoning): 6; Terminus-2 – Qwen 3.8 27B (xhigh): No data02040608072Claude CodeClaude Opus5.5 (xhigh)71Claude CodeClaude Sonnet5.5 (high)71Claude CodeClaude Sonnet5.5 (xhigh)69CodexGPT-6.1 Sol(medium)69CodexGPT-6.1 Sol(xhigh)69Claude CodeClaude Opus5.5 (medium)68Claude CodeClaude Opus5.5 (high)68Claude CodeClaude Haiku5.5 (xhigh)67CodexGPT-6.1 Sol(high)66Claude CodeClaude Sonnet5.5 (medium)64CodexGPT-6 Luna(xhigh)62Claude CodeClaude Opus 5(high)62Claude CodeClaude Haiku5.5 (high)62Terminus-2Gemini 3.8Flash (low)61Claude CodeClaude Haiku5.5 (medium)60Terminus-2Gemini 3.1Pro (preview)(low)59Terminus-2Gemini 3.1Pro (preview)(high)58Terminus-2GLM-5.2 (max)58Terminus-2GLM-5.2(high)57Kimi CodeKimi K2.6(always on)56Terminus-2DeepSeek V4Pro (high)56Claude CodeClaude Sonnet5 (high)55Terminus-2Kimi K2.6(always on)55Terminus-2Kimi K3(always on)55Claude CodeClaude Opus4.8 (high)55Claude CodeClaude Opus4.7 (high)55Terminus-2DeepSeek V4Flash (high)54Terminus-2DeepSeek V4Flash (max)54Terminus-2Kimi K2.7Code (alwayson)49Kimi CodeKimi K2.7Code (alwayson)49Terminus-2Gemini 3.1Flash-Lite(high)49Terminus-2Gemma 4 26BA4B (thinkingon)46Terminus-2GLM-5.2(none)45Claude CodeClaude Haiku4.5 (thinking32k)43Terminus-2Gemini 3.1Flash-Lite(low)43Terminus-2GLM-5.3 (low)43Terminus-2GLM-5.3-Flash(low)42Terminus-2GLM-5.3 (max,default)40Terminus-2Claude Sonnet4.6(adaptive,40Terminus-2Gemini 3.8Flash (high)39Terminus-2Gemini 3.1Flash-Lite(minimal)36Terminus-2gpt-oss-120b(high)32Terminus-2GLM-4.7-Flash(thinking on)31Terminus-2gpt-oss-20b(high)31Terminus-2gpt-oss-120b(low)27Terminus-2GLM-5.3-Flash(max)6Terminus-2Llama 3.3 70B(Non-reasoning)No dataTerminus-2Qwen 3.8 27B(xhigh)
How to read this chart

About this benchmark. Multi-step spatial execution, including tasks to reject. GeoBenchX results and background.

Data table: GeoBenchX score
GeoBenchX score
ModelCreatorGeoBenchX scoreRange while attempts are pendingCoverage (attempts)
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)Anthropic40No pending judgements173 of 173 planned attempts (72 excluded)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek54No pending judgements; 1 trace review declined by the judge173 of 173 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek55No pending judgements173 of 173 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek56No pending judgements173 of 173 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (low)Google43No pending judgements173 of 173 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)Google39No pending judgements173 of 173 planned attempts
Terminus-2 – Gemini 3.1 Flash-Lite (high)Google49No pending judgements; coverage incomplete170 of 173 planned attempts (3 excluded)
Terminus-2 – Gemini 3.1 Pro (preview) (low)Google60No pending judgements173 of 173 planned attempts (15 excluded)
Terminus-2 – Gemini 3.1 Pro (preview) (high)Google59No pending judgements173 of 173 planned attempts
Terminus-2 – Gemini 3.8 Flash (high)Google40No pending judgements173 of 173 planned attempts
Terminus-2 – Gemini 3.8 Flash (low)Google62No pending judgements173 of 173 planned attempts (42 excluded)
Terminus-2 – Gemma 4 26B A4B (thinking on)Google49No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai32No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai58No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai46No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai58No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai43No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai27No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai43No pending judgements173 of 173 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai42No pending judgements173 of 173 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI31No pending judgements173 of 173 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI36No pending judgements173 of 173 planned attempts
Terminus-2 – gpt-oss-20b (high)OpenAI31No pending judgements; coverage incomplete172 of 173 planned attempts (1 excluded)
Terminus-2 – Kimi K2.6 (always on)Moonshot AI55No pending judgements173 of 173 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI54No pending judgements173 of 173 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI55No pending judgements173 of 173 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta6No pending judgements173 of 173 planned attempts
Terminus-2 – Qwen 3.8 27B (xhigh)Alibaba (Qwen)No dataNo data0 of 173 planned attempts
Claude Code – Claude Haiku 4.5 (thinking 32k)Anthropic45No pending judgements; coverage incomplete47 of 173 planned attempts
Kimi Code – Kimi K2.6 (always on)Moonshot AI5756 to 57; 34 results awaiting review60 of 173 planned attempts
Kimi Code – Kimi K2.7 Code (always on)Moonshot AI4947 to 51; 33 results awaiting review67 of 173 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic69No pending judgements173 of 173 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic68No pending judgements173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic66No pending judgements173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic71No pending judgements173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic61No pending judgements173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic62No pending judgements173 of 173 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic72No pending judgements173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic71No pending judgements173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic68No pending judgements173 of 173 planned attempts
Claude Code – Claude Opus 5 (high)Anthropic62No pending judgements; coverage incomplete48 of 173 planned attempts
Claude Code – Claude Sonnet 5 (high)Anthropic56No pending judgements; coverage incomplete63 of 173 planned attempts
Claude Code – Claude Opus 4.8 (high)Anthropic55No pending judgements; coverage incomplete56 of 173 planned attempts
Claude Code – Claude Opus 4.7 (high)Anthropic55No pending judgements; coverage incomplete60 of 173 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI69No pending judgements173 of 173 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI67No pending judgements173 of 173 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI69No pending judgements173 of 173 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI64No pending judgements173 of 173 planned attempts

Geospatial Agent Index vs. cost per task

Higher and further left is better

  • Planned attempts still to come
  • Most attractive quadrant
  • Pareto line
Geospatial Agent Index vs. cost per taskTerminus-2 – DeepSeek V4 Flash (max): Geospatial Agent Index 53.07, cost per task (usd) $0.086; Terminus-2 – DeepSeek V4 Flash (high): Geospatial Agent Index 51.94, cost per task (usd) $0.087; Terminus-2 – DeepSeek V4 Pro (high): Geospatial Agent Index 53.89, cost per task (usd) $0.428; Terminus-2 – Gemma 4 26B A4B (thinking on): Geospatial Agent Index 38.9, cost per task (usd) $0.018; Terminus-2 – GLM-4.7-Flash (thinking on): Geospatial Agent Index 28.77, cost per task (usd) $0.035; Terminus-2 – GLM-5.2 (high): Geospatial Agent Index 56.11, cost per task (usd) $0.319; Terminus-2 – GLM-5.2 (none): Geospatial Agent Index 43.19, cost per task (usd) $0.151; Terminus-2 – GLM-5.2 (max): Geospatial Agent Index 55.38, cost per task (usd) $0.443; Terminus-2 – GLM-5.3-Flash (low): Geospatial Agent Index 40.09, cost per task (usd) $0.051; Terminus-2 – GLM-5.3-Flash (max): Geospatial Agent Index 27.24, cost per task (usd) $0.060; Terminus-2 – GLM-5.3 (low): Geospatial Agent Index 38, cost per task (usd) $0.407; Terminus-2 – GLM-5.3 (max, default): Geospatial Agent Index 37.77, cost per task (usd) $0.525; Terminus-2 – gpt-oss-120b (low): Geospatial Agent Index 25.19, cost per task (usd) $0.0061; Terminus-2 – gpt-oss-120b (high): Geospatial Agent Index 34.2, cost per task (usd) $0.033; Terminus-2 – Kimi K2.6 (always on): Geospatial Agent Index 52.23, cost per task (usd) $0.109; Terminus-2 – Kimi K2.7 Code (always on): Geospatial Agent Index 54.77, cost per task (usd) $0.183; Terminus-2 – Kimi K3 (always on): Geospatial Agent Index 58.02, cost per task (usd) $0.336; Terminus-2 – Llama 3.3 70B (Non-reasoning): Geospatial Agent Index 6.92, cost per task (usd) $0.024; Claude Code – Claude Opus 5.5 (medium): Geospatial Agent Index 65.34, cost per task (usd) $0.149; Claude Code – Claude Opus 5.5 (high): Geospatial Agent Index 68.12, cost per task (usd) $0.186; Claude Code – Claude Sonnet 5.5 (medium): Geospatial Agent Index 57.72, cost per task (usd) $0.058; Claude Code – Claude Sonnet 5.5 (high): Geospatial Agent Index 64.8, cost per task (usd) $0.072; Claude Code – Claude Haiku 5.5 (medium): Geospatial Agent Index 58.19, cost per task (usd) $0.0080; Claude Code – Claude Haiku 5.5 (high): Geospatial Agent Index 60.47, cost per task (usd) $0.011; Claude Code – Claude Opus 5.5 (xhigh): Geospatial Agent Index 68.84, cost per task (usd) $0.411; Claude Code – Claude Sonnet 5.5 (xhigh): Geospatial Agent Index 63.83, cost per task (usd) $0.172; Claude Code – Claude Haiku 5.5 (xhigh): Geospatial Agent Index 63.06, cost per task (usd) $0.026; Codex – GPT-6.1 Sol (medium): Geospatial Agent Index 65.29, cost per task (usd) $0.061; Codex – GPT-6.1 Sol (high): Geospatial Agent Index 62.77, cost per task (usd) $0.081; Codex – GPT-6.1 Sol (xhigh): Geospatial Agent Index 64.95, cost per task (usd) $0.126; Codex – GPT-6 Luna (xhigh): Geospatial Agent Index 60.31, cost per task (usd) $0.010Most attractive quadrant01020304050607080$0.005$0.01$0.02$0.05$0.10$0.20$0.50$1.00Cost per task (USD) (log scale)Geospatial Agent IndexClaude Code – Claude Opus 5.5 (xhigh)Claude Code – Claude Opus 5.5 (high)Claude Code – Claude Opus 5.5 (medium)Codex – GPT-6.1 Sol (medium)Codex – GPT-6.1 Sol (xhigh)Claude Code – Claude Sonnet 5.5 (high)Claude Code – Claude Sonnet 5.5 (xhigh)Claude Code – Claude Haiku 5.5 (xhigh)Codex – GPT-6.1 Sol (high)Claude Code – Claude Haiku 5.5 (high)Codex – GPT-6 Luna (xhigh)Claude Code – Claude Haiku 5.5 (medium)Terminus-2 – Kimi K3 (always on)Claude Code – Claude Sonnet 5.5 (medium)Terminus-2 – GLM-5.2 (high)Terminus-2 – GLM-5.2 (max)Terminus-2 – Kimi K2.7 Code (always on)Terminus-2 – DeepSeek V4 Pro (high)Terminus-2 – DeepSeek V4 Flash (max)Terminus-2 – Kimi K2.6 (always on)Terminus-2 – DeepSeek V4 Flash (high)Terminus-2 – GLM-5.2 (none)Terminus-2 – GLM-5.3-Flash (low)Terminus-2 – Gemma 4 26B A4B (thinking on)Terminus-2 – GLM-5.3 (low)Terminus-2 – GLM-5.3 (max, default)Terminus-2 – gpt-oss-120b (high)Terminus-2 – GLM-4.7-Flash (thinking on)Terminus-2 – GLM-5.3-Flash (max)Terminus-2 – gpt-oss-120b (low)Terminus-2 – Llama 3.3 70B (Non-reasoning)
How to read this chart

How to read this chart. Each point is a model. Up is a higher index, left is cheaper. The dashed Pareto line joins the models that no other model beats on both at once. The shaded corner holds models better than the median on both.

Why a log scale. Costs differ by a factor of ten or more between models, so equal steps on the cost axis are equal ratios.

Partial coverage. Not plotted and not on the Pareto line; listed in the table: Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one model configuration. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.

Data table: Geospatial Agent Index vs. cost per task
Geospatial Agent Index vs. cost per task
ModelGeospatial Agent IndexCost per task (USD)On the Pareto line
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)40* (partial: attempt 1 on 209 of 320 tasks)$0.291Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – DeepSeek V4 Flash (max)53*$0.086No
Terminus-2 – DeepSeek V4 Flash (high)52*$0.087No
Terminus-2 – DeepSeek V4 Pro (high)54*$0.428No
Terminus-2 – Gemini 3.1 Flash-Lite (low)37* (partial: attempt 1 on 209 of 320 tasks)$0.013Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)29* (partial: attempt 1 on 209 of 320 tasks)$0.0057Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)40* (partial: attempt 1 on 205 of 320 tasks)$0.021Not plotted: partial coverage (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)44* (partial: attempt 1 on 209 of 320 tasks)$0.061Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)43* (partial: attempt 1 on 209 of 320 tasks)$0.275Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.8 Flash (high)34* (partial: attempt 1 on 209 of 320 tasks)$0.336Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.8 Flash (low)49* (partial: attempt 1 on 209 of 320 tasks)$0.147Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemma 4 26B A4B (thinking on)39*$0.018No
Terminus-2 – GLM-4.7-Flash (thinking on)29*$0.035No
Terminus-2 – GLM-5.2 (high)56*$0.319No
Terminus-2 – GLM-5.2 (none)43*$0.151No
Terminus-2 – GLM-5.2 (max)55*$0.443No
Terminus-2 – GLM-5.3-Flash (low)40*$0.051No
Terminus-2 – GLM-5.3-Flash (max)27*$0.060No
Terminus-2 – GLM-5.3 (low)38*$0.407No
Terminus-2 – GLM-5.3 (max, default)38*$0.525No
Terminus-2 – gpt-oss-120b (low)25*$0.0061Yes
Terminus-2 – gpt-oss-120b (high)34*$0.033No
Terminus-2 – gpt-oss-20b (high)28*$0.016Not plotted: partial coverage (172 of 173 GeoBenchX tasks)
Terminus-2 – Kimi K2.6 (always on)52*$0.109No
Terminus-2 – Kimi K2.7 Code (always on)55*$0.183No
Terminus-2 – Kimi K3 (always on)58*$0.336No
Terminus-2 – Llama 3.3 70B (Non-reasoning)7*$0.024No
Terminus-2 – Qwen 3.8 27B (xhigh)28* (partial: attempt 1 on 36 of 320 tasks)$0.051Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; 0 of 173 GeoBenchX tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)36* (partial: attempt 1 on 94 of 320 tasks)$0.155Not plotted: partial coverage (47 of 147 Earth-Bench V2 tasks; 47 of 173 GeoBenchX tasks)
Kimi Code – Kimi K2.6 (always on)56* (partial: attempt 1 on 72 of 320 tasks)$0.219Not plotted: partial coverage (11 of 147 Earth-Bench V2 tasks; 61 of 173 GeoBenchX tasks)
Kimi Code – Kimi K2.7 Code (always on)54* (partial: attempt 1 on 82 of 320 tasks)$0.227Not plotted: partial coverage (12 of 147 Earth-Bench V2 tasks; 70 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 5.5 (medium)65*$0.149Yes
Claude Code – Claude Opus 5.5 (high)68*$0.186Yes
Claude Code – Claude Sonnet 5.5 (medium)58*$0.058No
Claude Code – Claude Sonnet 5.5 (high)65*$0.072No
Claude Code – Claude Haiku 5.5 (medium)58*$0.0080Yes
Claude Code – Claude Haiku 5.5 (high)60*$0.011Yes
Claude Code – Claude Opus 5.5 (xhigh)69*$0.411Yes
Claude Code – Claude Sonnet 5.5 (xhigh)64*$0.172No
Claude Code – Claude Haiku 5.5 (xhigh)63*$0.026Yes
Claude Code – Claude Opus 5 (high)63* (partial: attempt 1 on 160 of 320 tasks)$0.522Not plotted: partial coverage (112 of 147 Earth-Bench V2 tasks; 48 of 173 GeoBenchX tasks)
Claude Code – Claude Sonnet 5 (high)56* (partial: attempt 1 on 104 of 320 tasks)$0.331Not plotted: partial coverage (41 of 147 Earth-Bench V2 tasks; 63 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 4.8 (high)51* (partial: attempt 1 on 105 of 320 tasks)$0.447Not plotted: partial coverage (49 of 147 Earth-Bench V2 tasks; 56 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 4.7 (high)53* (partial: attempt 1 on 109 of 320 tasks)$0.521Not plotted: partial coverage (49 of 147 Earth-Bench V2 tasks; 60 of 173 GeoBenchX tasks)
Codex – GPT-6.1 Sol (medium)65*$0.061Yes
Codex – GPT-6.1 Sol (high)63*$0.081No
Codex – GPT-6.1 Sol (xhigh)65*$0.126No
Codex – GPT-6 Luna (xhigh)60*$0.010Yes

Token usage

Token consumption across the Geospatial Agent Index, as reported by the provider.

Token usage

Token usage

Total tokens used across the Geospatial Agent Index · Lower is better

  • Planned attempts still to come
Token usageTerminus-2 – gpt-oss-120b (low): 4.84M (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 20.89M (partial coverage); Terminus-2 – Kimi K3 (always on): 24.40M (partial coverage); Terminus-2 – gpt-oss-120b (high): 25.28M (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 28.13M (partial coverage); Codex – GPT-6.1 Sol (medium): 35.86M (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 36.28M (partial coverage); Claude Code – Claude Opus 5.5 (medium): 36.86M (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 44.67M (partial coverage); Claude Code – Claude Opus 5.5 (high): 44.76M (partial coverage); Codex – GPT-6.1 Sol (high): 47.57M (partial coverage); Terminus-2 – Kimi K2.6 (always on): 64.10M (partial coverage); Codex – GPT-6.1 Sol (xhigh): 67.20M (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 73.48M (partial coverage); Codex – GPT-6 Luna (xhigh): 79.18M (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 80.59M (partial coverage); Terminus-2 – GLM-5.2 (none): 87.71M (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 87.80M (partial coverage); Claude Code – Claude Haiku 5.5 (high): 109.20M (partial coverage); Terminus-2 – GLM-5.3 (low): 109.39M (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 111.09M (partial coverage); Terminus-2 – GLM-5.3 (max, default): 123.20M (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 136.26M (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 149.21M (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 154.76M (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 159.31M (partial coverage); Terminus-2 – GLM-5.2 (high): 163.71M (partial coverage); Terminus-2 – GLM-5.2 (max): 175.80M (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 181.98M (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 187.33M (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 209.12M (partial coverage)062.50M125.00M187.50M250.00M4.84MTerminus-2gpt-oss-120b(low)20.89MTerminus-2Llama 3.3 70B(Non-reasoning)24.40MTerminus-2Kimi K3(always on)25.28MTerminus-2gpt-oss-120b(high)28.13MClaude CodeClaude Sonnet5.5 (medium)35.86MCodexGPT-6.1 Sol(medium)36.28MClaude CodeClaude Sonnet5.5 (high)36.86MClaude CodeClaude Opus5.5 (medium)44.67MTerminus-2Gemma 4 26BA4B (thinkingon)44.76MClaude CodeClaude Opus5.5 (high)47.57MCodexGPT-6.1 Sol(high)64.10MTerminus-2Kimi K2.6(always on)67.20MCodexGPT-6.1 Sol(xhigh)73.48MClaude CodeClaude Sonnet5.5 (xhigh)79.18MCodexGPT-6 Luna(xhigh)80.59MClaude CodeClaude Haiku5.5 (medium)87.71MTerminus-2GLM-5.2(none)87.80MClaude CodeClaude Opus5.5 (xhigh)109.20MClaude CodeClaude Haiku5.5 (high)109.39MTerminus-2GLM-5.3 (low)111.09MTerminus-2DeepSeek V4Pro (high)123.20MTerminus-2GLM-5.3 (max,default)136.26MTerminus-2DeepSeek V4Flash (max)149.21MTerminus-2Kimi K2.7Code (alwayson)154.76MTerminus-2GLM-4.7-Flash(thinking on)159.31MTerminus-2DeepSeek V4Flash (high)163.71MTerminus-2GLM-5.2(high)175.80MTerminus-2GLM-5.2 (max)181.98MTerminus-2GLM-5.3-Flash(low)187.33MTerminus-2GLM-5.3-Flash(max)209.12MClaude CodeClaude Haiku5.5 (xhigh)
How to read this chart

What this metric means. All input and output tokens, including reasoning, summed over every attempt at the index's tasks. Entries with fewer attempts run so far have lower totals.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Token usage
Token usage
ModelCreatorToken usageCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek136.26M320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek159.31M320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek111.09M320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google44.67M320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai154.76M320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai163.71M320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai87.71M320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai175.80M320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai181.98M320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai187.33M320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai109.39M320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai123.20M320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI4.84M320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI25.28M320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI64.10M320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI149.21M320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI24.40M320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta20.89M320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic36.86M320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic44.76M320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic28.13M320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic36.28M320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic80.59M320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic109.20M320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic87.80M320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic73.48M320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic209.12M320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI35.86M320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI47.57M320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI67.20M320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI79.18M320 of 614 planned attempts

Token distribution

Token distribution per task

Average tokens per task, spread across tasks · Lower is better

  • Planned attempts still to come
Token distribution per taskTerminus-2 – gpt-oss-120b (low): minimum 3k, first quartile 7k, median 9k, third quartile 14k, maximum 215k; Terminus-2 – gpt-oss-120b (high): minimum 6k, first quartile 16k, median 35k, third quartile 90k, maximum 773k; Terminus-2 – Kimi K3 (always on): minimum 7k, first quartile 26k, median 45k, third quartile 82k, maximum 681k; Terminus-2 – Llama 3.3 70B (Non-reasoning): minimum 6k, first quartile 28k, median 46k, third quartile 86k, maximum 321k; Claude Code – Claude Sonnet 5.5 (medium): minimum 37k, first quartile 64k, median 81k, third quartile 106k, maximum 264k; Codex – GPT-6.1 Sol (medium): minimum 36k, first quartile 58k, median 89k, third quartile 131k, maximum 1.44M; Claude Code – Claude Sonnet 5.5 (high): minimum 37k, first quartile 71k, median 99k, third quartile 141k, maximum 445k; Claude Code – Claude Opus 5.5 (medium): minimum 38k, first quartile 69k, median 104k, third quartile 137k, maximum 522k; Terminus-2 – Gemma 4 26B A4B (thinking on): minimum 16k, first quartile 62k, median 105k, third quartile 176k, maximum 886k; Claude Code – Claude Opus 5.5 (high): minimum 38k, first quartile 74k, median 114k, third quartile 177k, maximum 923k; Codex – GPT-6.1 Sol (high): minimum 42k, first quartile 73k, median 117k, third quartile 191k, maximum 1.16M; Terminus-2 – Kimi K2.6 (always on): minimum 11k, first quartile 62k, median 132k, third quartile 246k, maximum 1.46M; Terminus-2 – GLM-5.2 (none): minimum 12k, first quartile 67k, median 140k, third quartile 302k, maximum 3.25M; Terminus-2 – GLM-5.3 (low): minimum 7k, first quartile 63k, median 162k, third quartile 402k, maximum 3.00M; Codex – GPT-6.1 Sol (xhigh): minimum 55k, first quartile 88k, median 172k, third quartile 299k, maximum 893k; Claude Code – Claude Sonnet 5.5 (xhigh): minimum 58k, first quartile 103k, median 177k, third quartile 294k, maximum 2.30M; Terminus-2 – DeepSeek V4 Pro (high): minimum 12k, first quartile 72k, median 179k, third quartile 411k, maximum 2.29M; Codex – GPT-6 Luna (xhigh): minimum 25k, first quartile 102k, median 185k, third quartile 333k, maximum 1.18M; Claude Code – Claude Opus 5.5 (xhigh): minimum 50k, first quartile 100k, median 201k, third quartile 345k, maximum 2.45M; Terminus-2 – Kimi K2.7 Code (always on): minimum 7k, first quartile 79k, median 213k, third quartile 584k, maximum 3.66M; Claude Code – Claude Haiku 5.5 (medium): minimum 97k, first quartile 156k, median 216k, third quartile 314k, maximum 1.03M; Terminus-2 – GLM-5.3-Flash (low): minimum 7k, first quartile 89k, median 241k, third quartile 632k, maximum 5.19M; Terminus-2 – DeepSeek V4 Flash (max): minimum 17k, first quartile 116k, median 243k, third quartile 571k, maximum 2.97M; Terminus-2 – GLM-5.3 (max, default): minimum 8k, first quartile 82k, median 260k, third quartile 534k, maximum 3.30M; Terminus-2 – DeepSeek V4 Flash (high): minimum 17k, first quartile 112k, median 269k, third quartile 746k, maximum 2.35M; Terminus-2 – GLM-5.2 (high): minimum 8k, first quartile 112k, median 271k, third quartile 570k, maximum 6.47M; Claude Code – Claude Haiku 5.5 (high): minimum 105k, first quartile 194k, median 291k, third quartile 419k, maximum 1.47M; Terminus-2 – GLM-4.7-Flash (thinking on): minimum 14k, first quartile 157k, median 330k, third quartile 680k, maximum 2.95M; Terminus-2 – GLM-5.3-Flash (max): minimum 7k, first quartile 139k, median 351k, third quartile 754k, maximum 5.71M; Terminus-2 – GLM-5.2 (max): minimum 9k, first quartile 155k, median 358k, third quartile 755k, maximum 4.12M; Claude Code – Claude Haiku 5.5 (xhigh): minimum 111k, first quartile 279k, median 485k, third quartile 839k, maximum 3.36M02.00M4.00M6.00M8.00M9kTerminus-2gpt-oss-120b(low)35kTerminus-2gpt-oss-120b(high)45kTerminus-2Kimi K3(always on)46kTerminus-2Llama 3.3 70B(Non-reasoning)81kClaude CodeClaude Sonnet5.5 (medium)89kCodexGPT-6.1 Sol(medium)99kClaude CodeClaude Sonnet5.5 (high)104kClaude CodeClaude Opus5.5 (medium)105kTerminus-2Gemma 4 26BA4B (thinkingon)114kClaude CodeClaude Opus5.5 (high)117kCodexGPT-6.1 Sol(high)132kTerminus-2Kimi K2.6(always on)140kTerminus-2GLM-5.2(none)162kTerminus-2GLM-5.3 (low)172kCodexGPT-6.1 Sol(xhigh)177kClaude CodeClaude Sonnet5.5 (xhigh)179kTerminus-2DeepSeek V4Pro (high)185kCodexGPT-6 Luna(xhigh)201kClaude CodeClaude Opus5.5 (xhigh)213kTerminus-2Kimi K2.7Code (alwayson)216kClaude CodeClaude Haiku5.5 (medium)241kTerminus-2GLM-5.3-Flash(low)243kTerminus-2DeepSeek V4Flash (max)260kTerminus-2GLM-5.3 (max,default)269kTerminus-2DeepSeek V4Flash (high)271kTerminus-2GLM-5.2(high)291kClaude CodeClaude Haiku5.5 (high)330kTerminus-2GLM-4.7-Flash(thinking on)351kTerminus-2GLM-5.3-Flash(max)358kTerminus-2GLM-5.2 (max)485kClaude CodeClaude Haiku5.5 (xhigh)
How to read this chart

How to read this chart. Each box spans the middle half of a model's tasks, from the first to the third quartile of average tokens per task; the line inside is the median and the whiskers reach the lightest and heaviest task. A tall box means token use varies a lot from task to task.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Token distribution per task
Token distribution per task
ModelTasksMinimumFirst quartileMedianThird quartileMaximum
Terminus-2 – gpt-oss-120b (low)3203k7k9k14k215k
Terminus-2 – gpt-oss-120b (high)3206k16k35k90k773k
Terminus-2 – Kimi K3 (always on)3207k26k45k82k681k
Terminus-2 – Llama 3.3 70B (Non-reasoning)3206k28k46k86k321k
Claude Code – Claude Sonnet 5.5 (medium)32037k64k81k106k264k
Codex – GPT-6.1 Sol (medium)32036k58k89k131k1.44M
Claude Code – Claude Sonnet 5.5 (high)32037k71k99k141k445k
Claude Code – Claude Opus 5.5 (medium)32038k69k104k137k522k
Terminus-2 – Gemma 4 26B A4B (thinking on)32016k62k105k176k886k
Claude Code – Claude Opus 5.5 (high)32038k74k114k177k923k
Codex – GPT-6.1 Sol (high)32042k73k117k191k1.16M
Terminus-2 – Kimi K2.6 (always on)32011k62k132k246k1.46M
Terminus-2 – GLM-5.2 (none)32012k67k140k302k3.25M
Terminus-2 – GLM-5.3 (low)3207k63k162k402k3.00M
Codex – GPT-6.1 Sol (xhigh)32055k88k172k299k893k
Claude Code – Claude Sonnet 5.5 (xhigh)32058k103k177k294k2.30M
Terminus-2 – DeepSeek V4 Pro (high)32012k72k179k411k2.29M
Codex – GPT-6 Luna (xhigh)32025k102k185k333k1.18M
Claude Code – Claude Opus 5.5 (xhigh)32050k100k201k345k2.45M
Terminus-2 – Kimi K2.7 Code (always on)3207k79k213k584k3.66M
Claude Code – Claude Haiku 5.5 (medium)32097k156k216k314k1.03M
Terminus-2 – GLM-5.3-Flash (low)3207k89k241k632k5.19M
Terminus-2 – DeepSeek V4 Flash (max)32017k116k243k571k2.97M
Terminus-2 – GLM-5.3 (max, default)3208k82k260k534k3.30M
Terminus-2 – DeepSeek V4 Flash (high)32017k112k269k746k2.35M
Terminus-2 – GLM-5.2 (high)3208k112k271k570k6.47M
Claude Code – Claude Haiku 5.5 (high)320105k194k291k419k1.47M
Terminus-2 – GLM-4.7-Flash (thinking on)32014k157k330k680k2.95M
Terminus-2 – GLM-5.3-Flash (max)3207k139k351k754k5.71M
Terminus-2 – GLM-5.2 (max)3209k155k358k755k4.12M
Claude Code – Claude Haiku 5.5 (xhigh)320111k279k485k839k3.36M

Cache hit rate

Cache hit rate

Cached input as a share of all input tokens · Higher is better

  • Planned attempts still to come
Cache hit rateClaude Code – Claude Haiku 5.5 (high): 94% (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 94% (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 94% (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 93% (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 92% (partial coverage); Claude Code – Claude Opus 5.5 (high): 92% (partial coverage); Claude Code – Claude Opus 5.5 (medium): 92% (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 91% (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 91% (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 91% (partial coverage); Terminus-2 – GLM-5.2 (high): 90% (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 89% (partial coverage); Codex – GPT-6.1 Sol (xhigh): 88% (partial coverage); Codex – GPT-6 Luna (xhigh): 87% (partial coverage); Codex – GPT-6.1 Sol (high): 87% (partial coverage); Terminus-2 – Kimi K2.6 (always on): 87% (partial coverage); Codex – GPT-6.1 Sol (medium): 86% (partial coverage); Terminus-2 – GLM-5.2 (none): 84% (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 84% (partial coverage); Terminus-2 – GLM-5.2 (max): 77% (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 63% (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 57% (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 42% (partial coverage); Terminus-2 – GLM-5.3 (low): 32% (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 25% (partial coverage); Terminus-2 – GLM-5.3 (max, default): 25% (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): No data; Terminus-2 – gpt-oss-120b (low): No data; Terminus-2 – gpt-oss-120b (high): No data; Terminus-2 – Kimi K3 (always on): No data; Terminus-2 – Llama 3.3 70B (Non-reasoning): No data0%25%50%75%100%94%Claude CodeClaude Haiku5.5 (high)94%Claude CodeClaude Haiku5.5 (xhigh)94%Claude CodeClaude Haiku5.5 (medium)93%Terminus-2Kimi K2.7Code (alwayson)92%Claude CodeClaude Sonnet5.5 (high)92%Claude CodeClaude Opus5.5 (high)92%Claude CodeClaude Opus5.5 (medium)91%Claude CodeClaude Opus5.5 (xhigh)91%Claude CodeClaude Sonnet5.5 (xhigh)91%Claude CodeClaude Sonnet5.5 (medium)90%Terminus-2GLM-5.2(high)89%Terminus-2DeepSeek V4Flash (high)88%CodexGPT-6.1 Sol(xhigh)87%CodexGPT-6 Luna(xhigh)87%CodexGPT-6.1 Sol(high)87%Terminus-2Kimi K2.6(always on)86%CodexGPT-6.1 Sol(medium)84%Terminus-2GLM-5.2(none)84%Terminus-2DeepSeek V4Flash (max)77%Terminus-2GLM-5.2 (max)63%Terminus-2GLM-5.3-Flash(low)57%Terminus-2GLM-5.3-Flash(max)42%Terminus-2Gemma 4 26BA4B (thinkingon)32%Terminus-2GLM-5.3 (low)25%Terminus-2DeepSeek V4Pro (high)25%Terminus-2GLM-5.3 (max,default)No dataTerminus-2GLM-4.7-Flash(thinking on)No dataTerminus-2gpt-oss-120b(low)No dataTerminus-2gpt-oss-120b(high)No dataTerminus-2Kimi K3(always on)No dataTerminus-2Llama 3.3 70B(Non-reasoning)
How to read this chart

What this metric means. The share of input tokens served from the provider's prompt cache. Cached input is cheaper, so a higher rate lowers cost per task.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Cache hit rate
Cache hit rate
ModelCreatorCache hit rateCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek84%320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek89%320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek25%320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google42%320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.aiNo data320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai90%320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai84%320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai77%320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai63%320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai57%320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai32%320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai25%320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAINo data320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAINo data320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI87%320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI93%320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AINo data320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)MetaNo data320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic92%320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic92%320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic91%320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic92%320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic94%320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic94%320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic91%320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic91%320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic94%320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI86%320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI87%320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI88%320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI87%320 of 614 planned attempts

Input vs output

Input vs output tokens per attempt

Average tokens per attempt · Lower is better

  • Input (not cached)
  • Cached input
  • Output, including reasoning
  • Partial coverage
Input vs output tokens per attemptClaude Code – Claude Haiku 5.5 (xhigh): Input (not cached) 39k, Cached input 595k, Output, including reasoning 20k; Terminus-2 – GLM-5.3-Flash (max): Input (not cached) 240k, Cached input 316k, Output, including reasoning 30k; Terminus-2 – GLM-5.3-Flash (low): Input (not cached) 203k, Cached input 346k, Output, including reasoning 20k; Terminus-2 – GLM-5.2 (max): Input (not cached) 116k, Cached input 393k, Output, including reasoning 40k; Terminus-2 – GLM-5.2 (high): Input (not cached) 48k, Cached input 432k, Output, including reasoning 32k; Terminus-2 – DeepSeek V4 Flash (high): Input (not cached) 51k, Cached input 402k, Output, including reasoning 45k; Terminus-2 – GLM-4.7-Flash (thinking on): Input (not cached) 467k, Cached input No data, Output, including reasoning 17k; Terminus-2 – Kimi K2.7 Code (always on): Input (not cached) 31k, Cached input 416k, Output, including reasoning 19k; Terminus-2 – DeepSeek V4 Flash (max): Input (not cached) 62k, Cached input 323k, Output, including reasoning 41k; Terminus-2 – GLM-5.3 (max, default): Input (not cached) 268k, Cached input 88k, Output, including reasoning 29k; Terminus-2 – DeepSeek V4 Pro (high): Input (not cached) 239k, Cached input 81k, Output, including reasoning 28k; Terminus-2 – GLM-5.3 (low): Input (not cached) 221k, Cached input 104k, Output, including reasoning 16k; Claude Code – Claude Haiku 5.5 (high): Input (not cached) 20k, Cached input 313k, Output, including reasoning 8k; Claude Code – Claude Opus 5.5 (xhigh): Input (not cached) 23k, Cached input 243k, Output, including reasoning 9k; Terminus-2 – GLM-5.2 (none): Input (not cached) 42k, Cached input 225k, Output, including reasoning 8k; Claude Code – Claude Haiku 5.5 (medium): Input (not cached) 16k, Cached input 231k, Output, including reasoning 5k; Codex – GPT-6 Luna (xhigh): Input (not cached) 31k, Cached input 207k, Output, including reasoning 10k; Claude Code – Claude Sonnet 5.5 (xhigh): Input (not cached) 20k, Cached input 202k, Output, including reasoning 7k; Codex – GPT-6.1 Sol (xhigh): Input (not cached) 25k, Cached input 179k, Output, including reasoning 6k; Terminus-2 – Kimi K2.6 (always on): Input (not cached) 24k, Cached input 161k, Output, including reasoning 15k; Codex – GPT-6.1 Sol (high): Input (not cached) 19k, Cached input 127k, Output, including reasoning 3k; Claude Code – Claude Opus 5.5 (high): Input (not cached) 11k, Cached input 125k, Output, including reasoning 3k; Terminus-2 – Gemma 4 26B A4B (thinking on): Input (not cached) 70k, Cached input 50k, Output, including reasoning 19k; Claude Code – Claude Opus 5.5 (medium): Input (not cached) 9k, Cached input 103k, Output, including reasoning 3k; Claude Code – Claude Sonnet 5.5 (high): Input (not cached) 9k, Cached input 102k, Output, including reasoning 3k; Codex – GPT-6.1 Sol (medium): Input (not cached) 16k, Cached input 95k, Output, including reasoning 2k; Claude Code – Claude Sonnet 5.5 (medium): Input (not cached) 8k, Cached input 78k, Output, including reasoning 2k; Terminus-2 – gpt-oss-120b (high): Input (not cached) 65k, Cached input No data, Output, including reasoning 14k; Terminus-2 – Kimi K3 (always on): Input (not cached) 67k, Cached input No data, Output, including reasoning 9k; Terminus-2 – Llama 3.3 70B (Non-reasoning): Input (not cached) 63k, Cached input No data, Output, including reasoning 3k; Terminus-2 – gpt-oss-120b (low): Input (not cached) 13k, Cached input No data, Output, including reasoning 2k0200k400k600k800k654kClaude CodeClaude Haiku5.5 (xhigh)585kTerminus-2GLM-5.3-Flash(max)569kTerminus-2GLM-5.3-Flash(low)549kTerminus-2GLM-5.2 (max)512kTerminus-2GLM-5.2(high)498kTerminus-2DeepSeek V4Flash (high)484kTerminus-2GLM-4.7-Flash(thinking on)466kTerminus-2Kimi K2.7Code (alwayson)426kTerminus-2DeepSeek V4Flash (max)385kTerminus-2GLM-5.3 (max,default)347kTerminus-2DeepSeek V4Pro (high)342kTerminus-2GLM-5.3 (low)341kClaude CodeClaude Haiku5.5 (high)274kClaude CodeClaude Opus5.5 (xhigh)274kTerminus-2GLM-5.2(none)252kClaude CodeClaude Haiku5.5 (medium)247kCodexGPT-6 Luna(xhigh)230kClaude CodeClaude Sonnet5.5 (xhigh)210kCodexGPT-6.1 Sol(xhigh)200kTerminus-2Kimi K2.6(always on)149kCodexGPT-6.1 Sol(high)140kClaude CodeClaude Opus5.5 (high)140kTerminus-2Gemma 4 26BA4B (thinkingon)115kClaude CodeClaude Opus5.5 (medium)113kClaude CodeClaude Sonnet5.5 (high)112kCodexGPT-6.1 Sol(medium)88kClaude CodeClaude Sonnet5.5 (medium)79kTerminus-2gpt-oss-120b(high)76kTerminus-2Kimi K3(always on)65kTerminus-2Llama 3.3 70B(Non-reasoning)15kTerminus-2gpt-oss-120b(low)
How to read this chart

What this metric means. Average tokens per attempt, as reported by the provider: input the model read (split into new and cached input) and output it wrote, which includes its reasoning.

Compare with care. Models use different tokenisers, so the same text is a different number of tokens for different models. Use token counts to see how much work each model did, and cost to compare spending.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Input vs output tokens per attempt
Input vs output tokens per attempt
ModelInput (not cached)Cached inputOutput, including reasoningTotalCoverage (attempts)
Claude Code – Claude Haiku 5.5 (xhigh)39k595k20k654k320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)240k316k30k585k320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)203k346k20k569k320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)116k393k40k549k320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)48k432k32k512k320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)51k402k45k498k320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)467kNo data17k484k320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)31k416k19k466k320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (max)62k323k41k426k320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)268k88k29k385k320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)239k81k28k347k320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)221k104k16k342k320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)20k313k8k341k320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)23k243k9k274k320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)42k225k8k274k320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)16k231k5k252k320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)31k207k10k247k320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)20k202k7k230k320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)25k179k6k210k320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)24k161k15k200k320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)19k127k3k149k320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)11k125k3k140k320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)70k50k19k140k320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)9k103k3k115k320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)9k102k3k113k320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)16k95k2k112k320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)8k78k2k88k320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)65kNo data14k79k320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)67kNo data9k76k320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)63kNo data3k65k320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)13kNo data2k15k320 of 614 planned attempts

Tokens by benchmark

Earth-Bench V2: tokens per attempt

Average tokens per attempt · Lower is better

  • Planned attempts still to come
Earth-Bench V2: tokens per attemptTerminus-2 – gpt-oss-120b (low): 12k (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 48k (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 100k (partial coverage); Terminus-2 – Kimi K3 (always on): 103k (partial coverage); Terminus-2 – gpt-oss-120b (high): 108k (partial coverage); Codex – GPT-6.1 Sol (medium): 111k (partial coverage); Claude Code – Claude Opus 5.5 (medium): 121k (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 128k (partial coverage); Codex – GPT-6.1 Sol (high): 135k (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 135k (partial coverage); Claude Code – Claude Opus 5.5 (high): 148k (partial coverage); Codex – GPT-6.1 Sol (xhigh): 188k (partial coverage); Codex – GPT-6 Luna (xhigh): 205k (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 226k (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 241k (partial coverage); Terminus-2 – Kimi K2.6 (always on): 272k (partial coverage); Claude Code – Claude Haiku 5.5 (high): 291k (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 307k (partial coverage); Terminus-2 – GLM-5.2 (none): 378k (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 464k (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 497k (partial coverage); Terminus-2 – GLM-5.3 (low): 510k (partial coverage); Terminus-2 – GLM-5.3 (max, default): 511k (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 532k (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 549k (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 698k (partial coverage); Terminus-2 – GLM-5.2 (max): 709k (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 717k (partial coverage); Terminus-2 – GLM-5.2 (high): 758k (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 791k (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 915k (partial coverage)0250k500k750k1.00M12kgpt-oss-120b (low)48kLlama 3.3 70B (Non-reasoning)100kClaude Sonnet 5.5 (medium)103kKimi K3 (always on)108kgpt-oss-120b (high)111kGPT-6.1 Sol (medium)121kClaude Opus 5.5 (medium)128kClaude Sonnet 5.5 (high)135kGPT-6.1 Sol (high)135kGemma 4 26B A4B (thinking on)148kClaude Opus 5.5 (high)188kGPT-6.1 Sol (xhigh)205kGPT-6 Luna (xhigh)226kClaude Haiku 5.5 (medium)241kClaude Sonnet 5.5 (xhigh)272kKimi K2.6 (always on)291kClaude Haiku 5.5 (high)307kClaude Opus 5.5 (xhigh)378kGLM-5.2 (none)464kDeepSeek V4 Pro (high)497kClaude Haiku 5.5 (xhigh)510kGLM-5.3 (low)511kGLM-5.3 (max, default)532kGLM-4.7-Flash (thinking on)549kDeepSeek V4 Flash (max)698kDeepSeek V4 Flash (high)709kGLM-5.2 (max)717kKimi K2.7 Code (always on)758kGLM-5.2 (high)791kGLM-5.3-Flash (max)915kGLM-5.3-Flash (low)
How to read this chart

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Earth-Bench V2: tokens per attempt
Earth-Bench V2: tokens per attempt
ModelCreatorEarth-Bench V2: tokens per attemptCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek549k147 of 441 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek698k147 of 441 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek464k147 of 441 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google135k147 of 441 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai532k147 of 441 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai758k147 of 441 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai378k147 of 441 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai709k147 of 441 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai915k147 of 441 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai791k147 of 441 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai510k147 of 441 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai511k147 of 441 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI12k147 of 441 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI108k147 of 441 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI272k147 of 441 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI717k147 of 441 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI103k147 of 441 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta48k147 of 441 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic121k147 of 441 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic148k147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic100k147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic128k147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic226k147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic291k147 of 441 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic307k147 of 441 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic241k147 of 441 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic497k147 of 441 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI111k147 of 441 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI135k147 of 441 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI188k147 of 441 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI205k147 of 441 planned attempts

GeoBenchX: tokens per attempt

Average tokens per attempt · Lower is better

GeoBenchX: tokens per attemptTerminus-2 – gpt-oss-120b (low): 18k; Terminus-2 – Kimi K3 (always on): 53k; Terminus-2 – gpt-oss-120b (high): 54k; Claude Code – Claude Sonnet 5.5 (medium): 78k; Terminus-2 – Llama 3.3 70B (Non-reasoning): 80k; Claude Code – Claude Sonnet 5.5 (high): 101k; Claude Code – Claude Opus 5.5 (medium): 111k; Codex – GPT-6.1 Sol (medium): 113k; Claude Code – Claude Opus 5.5 (high): 133k; Terminus-2 – Kimi K2.6 (always on): 140k; Terminus-2 – Gemma 4 26B A4B (thinking on): 143k; Codex – GPT-6.1 Sol (high): 160k; Terminus-2 – GLM-5.2 (none): 186k; Terminus-2 – GLM-5.3 (low): 199k; Claude Code – Claude Sonnet 5.5 (xhigh): 220k; Codex – GPT-6.1 Sol (xhigh): 228k; Claude Code – Claude Opus 5.5 (xhigh): 247k; Terminus-2 – DeepSeek V4 Pro (high): 248k; Terminus-2 – Kimi K2.7 Code (always on): 253k; Claude Code – Claude Haiku 5.5 (medium): 274k; Terminus-2 – GLM-5.3-Flash (low): 275k; Terminus-2 – GLM-5.3 (max, default): 278k; Codex – GPT-6 Luna (xhigh): 284k; Terminus-2 – GLM-5.2 (high): 302k; Terminus-2 – DeepSeek V4 Flash (max): 321k; Terminus-2 – DeepSeek V4 Flash (high): 328k; Claude Code – Claude Haiku 5.5 (high): 384k; Terminus-2 – GLM-5.3-Flash (max): 411k; Terminus-2 – GLM-5.2 (max): 414k; Terminus-2 – GLM-4.7-Flash (thinking on): 442k; Claude Code – Claude Haiku 5.5 (xhigh): 786k0200k400k600k800k18kgpt-oss-120b (low)53kKimi K3 (always on)54kgpt-oss-120b (high)78kClaude Sonnet 5.5 (medium)80kLlama 3.3 70B (Non-reasoning)101kClaude Sonnet 5.5 (high)111kClaude Opus 5.5 (medium)113kGPT-6.1 Sol (medium)133kClaude Opus 5.5 (high)140kKimi K2.6 (always on)143kGemma 4 26B A4B (thinking on)160kGPT-6.1 Sol (high)186kGLM-5.2 (none)199kGLM-5.3 (low)220kClaude Sonnet 5.5 (xhigh)228kGPT-6.1 Sol (xhigh)247kClaude Opus 5.5 (xhigh)248kDeepSeek V4 Pro (high)253kKimi K2.7 Code (always on)274kClaude Haiku 5.5 (medium)275kGLM-5.3-Flash (low)278kGLM-5.3 (max, default)284kGPT-6 Luna (xhigh)302kGLM-5.2 (high)321kDeepSeek V4 Flash (max)328kDeepSeek V4 Flash (high)384kClaude Haiku 5.5 (high)411kGLM-5.3-Flash (max)414kGLM-5.2 (max)442kGLM-4.7-Flash (thinking on)786kClaude Haiku 5.5 (xhigh)
How to read this chart

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: GeoBenchX: tokens per attempt
GeoBenchX: tokens per attempt
ModelCreatorGeoBenchX: tokens per attemptCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek321k173 of 173 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek328k173 of 173 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek248k173 of 173 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google143k173 of 173 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai442k173 of 173 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai302k173 of 173 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai186k173 of 173 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai414k173 of 173 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai275k173 of 173 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai411k173 of 173 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai199k173 of 173 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai278k173 of 173 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI18k173 of 173 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI54k173 of 173 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI140k173 of 173 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI253k173 of 173 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI53k173 of 173 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta80k173 of 173 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic111k173 of 173 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic133k173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic78k173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic101k173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic274k173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic384k173 of 173 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic247k173 of 173 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic220k173 of 173 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic786k173 of 173 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI113k173 of 173 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI160k173 of 173 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI228k173 of 173 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI284k173 of 173 planned attempts

Cost

Cost per task

Cost per task

Average cost per attempt (USD) · Lower is better

  • Planned attempts still to come
Cost per taskTerminus-2 – gpt-oss-120b (low): $0.0061 (partial coverage); Claude Code – Claude Haiku 5.5 (medium): $0.0080 (partial coverage); Codex – GPT-6 Luna (xhigh): $0.010 (partial coverage); Claude Code – Claude Haiku 5.5 (high): $0.011 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): $0.018 (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): $0.024 (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): $0.026 (partial coverage); Terminus-2 – gpt-oss-120b (high): $0.033 (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): $0.035 (partial coverage); Terminus-2 – GLM-5.3-Flash (low): $0.051 (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): $0.058 (partial coverage); Terminus-2 – GLM-5.3-Flash (max): $0.060 (partial coverage); Codex – GPT-6.1 Sol (medium): $0.061 (partial coverage); Claude Code – Claude Sonnet 5.5 (high): $0.072 (partial coverage); Codex – GPT-6.1 Sol (high): $0.081 (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): $0.086 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): $0.087 (partial coverage); Terminus-2 – Kimi K2.6 (always on): $0.109 (partial coverage); Codex – GPT-6.1 Sol (xhigh): $0.126 (partial coverage); Claude Code – Claude Opus 5.5 (medium): $0.149 (partial coverage); Terminus-2 – GLM-5.2 (none): $0.151 (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): $0.172 (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): $0.183 (partial coverage); Claude Code – Claude Opus 5.5 (high): $0.186 (partial coverage); Terminus-2 – GLM-5.2 (high): $0.319 (partial coverage); Terminus-2 – Kimi K3 (always on): $0.336 (partial coverage); Terminus-2 – GLM-5.3 (low): $0.407 (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): $0.411 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): $0.428 (partial coverage); Terminus-2 – GLM-5.2 (max): $0.443 (partial coverage); Terminus-2 – GLM-5.3 (max, default): $0.525 (partial coverage)$0$0.150$0.300$0.450$0.600$0.0061Terminus-2gpt-oss-120b(low)$0.0080Claude CodeClaude Haiku5.5 (medium)$0.010CodexGPT-6 Luna(xhigh)$0.011Claude CodeClaude Haiku5.5 (high)$0.018Terminus-2Gemma 4 26BA4B (thinkingon)$0.024Terminus-2Llama 3.3 70B(Non-reasoning)$0.026Claude CodeClaude Haiku5.5 (xhigh)$0.033Terminus-2gpt-oss-120b(high)$0.035Terminus-2GLM-4.7-Flash(thinking on)$0.051Terminus-2GLM-5.3-Flash(low)$0.058Claude CodeClaude Sonnet5.5 (medium)$0.060Terminus-2GLM-5.3-Flash(max)$0.061CodexGPT-6.1 Sol(medium)$0.072Claude CodeClaude Sonnet5.5 (high)$0.081CodexGPT-6.1 Sol(high)$0.086Terminus-2DeepSeek V4Flash (max)$0.087Terminus-2DeepSeek V4Flash (high)$0.109Terminus-2Kimi K2.6(always on)$0.126CodexGPT-6.1 Sol(xhigh)$0.149Claude CodeClaude Opus5.5 (medium)$0.151Terminus-2GLM-5.2(none)$0.172Claude CodeClaude Sonnet5.5 (xhigh)$0.183Terminus-2Kimi K2.7Code (alwayson)$0.186Claude CodeClaude Opus5.5 (high)$0.319Terminus-2GLM-5.2(high)$0.336Terminus-2Kimi K3(always on)$0.407Terminus-2GLM-5.3 (low)$0.411Claude CodeClaude Opus5.5 (xhigh)$0.428Terminus-2DeepSeek V4Pro (high)$0.443Terminus-2GLM-5.2 (max)$0.525Terminus-2GLM-5.3 (max,default)
How to read this chart

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Lower is better. Costs are pay-per-token list prices, not any discount or credit a customer might have.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Cost per task
Cost per task
ModelCreatorCost per taskCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek$0.086320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek$0.087320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek$0.428320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google$0.018320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai$0.035320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai$0.319320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai$0.151320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai$0.443320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai$0.051320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai$0.060320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai$0.407320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai$0.525320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI$0.0061320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI$0.033320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI$0.109320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI$0.183320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI$0.336320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta$0.024320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic$0.149320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic$0.186320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic$0.058320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic$0.072320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic$0.0080320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic$0.011320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic$0.411320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic$0.172320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic$0.026320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI$0.061320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI$0.081320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI$0.126320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI$0.010320 of 614 planned attempts

Total cost

Total cost of attempts

Sum over every scored attempt (USD)

  • Planned attempts still to come
Total cost of attemptsTerminus-2 – gpt-oss-120b (low): $1.95 (partial coverage); Claude Code – Claude Haiku 5.5 (medium): $2.56 (partial coverage); Codex – GPT-6 Luna (xhigh): $3.26 (partial coverage); Claude Code – Claude Haiku 5.5 (high): $3.59 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): $5.71 (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): $7.84 (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): $8.29 (partial coverage); Terminus-2 – gpt-oss-120b (high): $10.70 (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): $11.18 (partial coverage); Terminus-2 – GLM-5.3-Flash (low): $16.21 (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): $18.56 (partial coverage); Terminus-2 – GLM-5.3-Flash (max): $19.30 (partial coverage); Codex – GPT-6.1 Sol (medium): $19.40 (partial coverage); Claude Code – Claude Sonnet 5.5 (high): $23.18 (partial coverage); Codex – GPT-6.1 Sol (high): $25.99 (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): $27.42 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): $27.87 (partial coverage); Terminus-2 – Kimi K2.6 (always on): $35.01 (partial coverage); Codex – GPT-6.1 Sol (xhigh): $40.32 (partial coverage); Claude Code – Claude Opus 5.5 (medium): $47.62 (partial coverage); Terminus-2 – GLM-5.2 (none): $48.22 (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): $54.97 (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): $58.55 (partial coverage); Claude Code – Claude Opus 5.5 (high): $59.61 (partial coverage); Terminus-2 – GLM-5.2 (high): $101.95 (partial coverage); Terminus-2 – Kimi K3 (always on): $107.67 (partial coverage); Terminus-2 – GLM-5.3 (low): $130.40 (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): $131.53 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): $137.04 (partial coverage); Terminus-2 – GLM-5.2 (max): $141.81 (partial coverage); Terminus-2 – GLM-5.3 (max, default): $168.12 (partial coverage)$0$50$100$150$200$1.95Terminus-2gpt-oss-120b(low)$2.56Claude CodeClaude Haiku5.5 (medium)$3.26CodexGPT-6 Luna(xhigh)$3.59Claude CodeClaude Haiku5.5 (high)$5.71Terminus-2Gemma 4 26BA4B (thinkingon)$7.84Terminus-2Llama 3.3 70B(Non-reasoning)$8.29Claude CodeClaude Haiku5.5 (xhigh)$10.70Terminus-2gpt-oss-120b(high)$11.18Terminus-2GLM-4.7-Flash(thinking on)$16.21Terminus-2GLM-5.3-Flash(low)$18.56Claude CodeClaude Sonnet5.5 (medium)$19.30Terminus-2GLM-5.3-Flash(max)$19.40CodexGPT-6.1 Sol(medium)$23.18Claude CodeClaude Sonnet5.5 (high)$25.99CodexGPT-6.1 Sol(high)$27.42Terminus-2DeepSeek V4Flash (max)$27.87Terminus-2DeepSeek V4Flash (high)$35.01Terminus-2Kimi K2.6(always on)$40.32CodexGPT-6.1 Sol(xhigh)$47.62Claude CodeClaude Opus5.5 (medium)$48.22Terminus-2GLM-5.2(none)$54.97Claude CodeClaude Sonnet5.5 (xhigh)$58.55Terminus-2Kimi K2.7Code (alwayson)$59.61Claude CodeClaude Opus5.5 (high)$101.95Terminus-2GLM-5.2(high)$107.67Terminus-2Kimi K3(always on)$130.40Terminus-2GLM-5.3 (low)$131.53Claude CodeClaude Opus5.5 (xhigh)$137.04Terminus-2DeepSeek V4Pro (high)$141.81Terminus-2GLM-5.2 (max)$168.12Terminus-2GLM-5.3 (max,default)
How to read this chart

What this metric means. What it cost to run every scored attempt of this model, at list prices.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Total cost of attempts
Total cost of attempts
ModelCreatorTotal cost of attemptsCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek$27.42320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek$27.87320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek$137.04320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google$5.71320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai$11.18320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai$101.95320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai$48.22320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai$141.81320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai$16.21320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai$19.30320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai$130.40320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai$168.12320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI$1.95320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI$10.70320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI$35.01320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI$58.55320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI$107.67320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta$7.84320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic$47.62320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic$59.61320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic$18.56320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic$23.18320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic$2.56320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic$3.59320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic$131.53320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic$54.97320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic$8.29320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI$19.40320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI$25.99320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI$40.32320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI$3.26320 of 614 planned attempts

Execution time

Execution time

Execution time

Average active agent time per task · Lower is better

  • Planned attempts still to come
Execution timeClaude Code – Claude Sonnet 5.5 (medium): 0.6 min (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 0.8 min (partial coverage); Claude Code – Claude Opus 5.5 (medium): 0.8 min (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 0.8 min (partial coverage); Terminus-2 – gpt-oss-120b (low): 0.9 min (partial coverage); Claude Code – Claude Opus 5.5 (high): 1.1 min (partial coverage); Claude Code – Claude Haiku 5.5 (high): 1.1 min (partial coverage); Codex – GPT-6.1 Sol (medium): 1.4 min (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 1.6 min (partial coverage); Codex – GPT-6.1 Sol (high): 1.8 min (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 2.2 min (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 2.4 min (partial coverage); Codex – GPT-6.1 Sol (xhigh): 3.0 min (partial coverage); Codex – GPT-6 Luna (xhigh): 3.9 min (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 3.9 min (partial coverage); Terminus-2 – Kimi K3 (always on): 3.9 min (partial coverage); Terminus-2 – GLM-5.2 (none): 4.1 min (partial coverage); Terminus-2 – gpt-oss-120b (high): 5.2 min (partial coverage); Terminus-2 – GLM-5.2 (high): 8.3 min (partial coverage); Terminus-2 – GLM-5.3 (low): 8.7 min (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 9.4 min (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 10.6 min (partial coverage); Terminus-2 – Kimi K2.6 (always on): 10.6 min (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 10.9 min (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 11.1 min (partial coverage); Terminus-2 – GLM-5.2 (max): 11.8 min (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 11.8 min (partial coverage); Terminus-2 – GLM-5.3 (max, default): 12.4 min (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 12.8 min (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 13.3 min (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 19.9 min (partial coverage)0m5m10m15m20m0.6 minClaude CodeClaude Sonnet5.5 (medium)0.8 minClaude CodeClaude Sonnet5.5 (high)0.8 minClaude CodeClaude Opus5.5 (medium)0.8 minClaude CodeClaude Haiku5.5 (medium)0.9 minTerminus-2gpt-oss-120b(low)1.1 minClaude CodeClaude Opus5.5 (high)1.1 minClaude CodeClaude Haiku5.5 (high)1.4 minCodexGPT-6.1 Sol(medium)1.6 minClaude CodeClaude Sonnet5.5 (xhigh)1.8 minCodexGPT-6.1 Sol(high)2.2 minClaude CodeClaude Haiku5.5 (xhigh)2.4 minClaude CodeClaude Opus5.5 (xhigh)3.0 minCodexGPT-6.1 Sol(xhigh)3.9 minCodexGPT-6 Luna(xhigh)3.9 minTerminus-2Llama 3.3 70B(Non-reasoning)3.9 minTerminus-2Kimi K3(always on)4.1 minTerminus-2GLM-5.2(none)5.2 minTerminus-2gpt-oss-120b(high)8.3 minTerminus-2GLM-5.2(high)8.7 minTerminus-2GLM-5.3 (low)9.4 minTerminus-2Kimi K2.7Code (alwayson)10.6 minTerminus-2DeepSeek V4Pro (high)10.6 minTerminus-2Kimi K2.6(always on)10.9 minTerminus-2Gemma 4 26BA4B (thinkingon)11.1 minTerminus-2GLM-4.7-Flash(thinking on)11.8 minTerminus-2GLM-5.2 (max)11.8 minTerminus-2DeepSeek V4Flash (max)12.4 minTerminus-2GLM-5.3 (max,default)12.8 minTerminus-2DeepSeek V4Flash (high)13.3 minTerminus-2GLM-5.3-Flash(low)19.9 minTerminus-2GLM-5.3-Flash(max)
How to read this chart

What execution time is measuring. Agent wall time per attempt, from the agent's first model call to its last action: model thinking, waiting for the provider and running commands. Sandbox start-up and grading are not included.

How to read this chart. Lower is better. An attempt that reaches the task's time limit scores 0 and counts at the limit.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Execution time
Execution time
ModelCreatorExecution timeCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek11.8 min320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek12.8 min320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek10.6 min320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google10.9 min320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai11.1 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai8.3 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai4.1 min320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai11.8 min320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai13.3 min320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai19.9 min320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai8.7 min320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai12.4 min320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI0.9 min320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI5.2 min320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI10.6 min320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI9.4 min320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI3.9 min320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta3.9 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic1.1 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic0.6 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic0.8 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic1.1 min320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic2.4 min320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic1.6 min320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic2.2 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI1.4 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI1.8 min320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI3.0 min320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI3.9 min320 of 614 planned attempts

Turns

Turns

Average number of agent turns per task · Lower is better

  • Planned attempts still to come
TurnsTerminus-2 – gpt-oss-120b (low): 3.1 (partial coverage); Claude Code – Claude Sonnet 5.5 (medium): 4.5 (partial coverage); Claude Code – Claude Sonnet 5.5 (high): 5.4 (partial coverage); Claude Code – Claude Opus 5.5 (medium): 5.5 (partial coverage); Terminus-2 – gpt-oss-120b (high): 6.2 (partial coverage); Claude Code – Claude Opus 5.5 (high): 6.2 (partial coverage); Codex – GPT-6.1 Sol (medium): 6.5 (partial coverage); Claude Code – Claude Haiku 5.5 (medium): 6.8 (partial coverage); Terminus-2 – Kimi K3 (always on): 7.7 (partial coverage); Codex – GPT-6.1 Sol (high): 7.8 (partial coverage); Claude Code – Claude Sonnet 5.5 (xhigh): 8.2 (partial coverage); Claude Code – Claude Haiku 5.5 (high): 8.3 (partial coverage); Claude Code – Claude Opus 5.5 (xhigh): 8.9 (partial coverage); Codex – GPT-6.1 Sol (xhigh): 9.1 (partial coverage); Codex – GPT-6 Luna (xhigh): 10.2 (partial coverage); Terminus-2 – Gemma 4 26B A4B (thinking on): 10.3 (partial coverage); Claude Code – Claude Haiku 5.5 (xhigh): 12.1 (partial coverage); Terminus-2 – GLM-5.3 (max, default): 12.2 (partial coverage); Terminus-2 – Kimi K2.6 (always on): 13.2 (partial coverage); Terminus-2 – GLM-5.3 (low): 13.3 (partial coverage); Terminus-2 – GLM-5.2 (none): 13.4 (partial coverage); Terminus-2 – GLM-5.2 (max): 14.8 (partial coverage); Terminus-2 – Llama 3.3 70B (Non-reasoning): 14.9 (partial coverage); Terminus-2 – GLM-5.2 (high): 14.9 (partial coverage); Terminus-2 – DeepSeek V4 Flash (max): 16.1 (partial coverage); Terminus-2 – DeepSeek V4 Pro (high): 16.2 (partial coverage); Terminus-2 – GLM-5.3-Flash (max): 16.2 (partial coverage); Terminus-2 – GLM-5.3-Flash (low): 17.0 (partial coverage); Terminus-2 – DeepSeek V4 Flash (high): 17.8 (partial coverage); Terminus-2 – Kimi K2.7 Code (always on): 17.9 (partial coverage); Terminus-2 – GLM-4.7-Flash (thinking on): 18.8 (partial coverage)051015203.1Terminus-2gpt-oss-120b(low)4.5Claude CodeClaude Sonnet5.5 (medium)5.4Claude CodeClaude Sonnet5.5 (high)5.5Claude CodeClaude Opus5.5 (medium)6.2Terminus-2gpt-oss-120b(high)6.2Claude CodeClaude Opus5.5 (high)6.5CodexGPT-6.1 Sol(medium)6.8Claude CodeClaude Haiku5.5 (medium)7.7Terminus-2Kimi K3(always on)7.8CodexGPT-6.1 Sol(high)8.2Claude CodeClaude Sonnet5.5 (xhigh)8.3Claude CodeClaude Haiku5.5 (high)8.9Claude CodeClaude Opus5.5 (xhigh)9.1CodexGPT-6.1 Sol(xhigh)10.2CodexGPT-6 Luna(xhigh)10.3Terminus-2Gemma 4 26BA4B (thinkingon)12.1Claude CodeClaude Haiku5.5 (xhigh)12.2Terminus-2GLM-5.3 (max,default)13.2Terminus-2Kimi K2.6(always on)13.3Terminus-2GLM-5.3 (low)13.4Terminus-2GLM-5.2(none)14.8Terminus-2GLM-5.2 (max)14.9Terminus-2Llama 3.3 70B(Non-reasoning)14.9Terminus-2GLM-5.2(high)16.1Terminus-2DeepSeek V4Flash (max)16.2Terminus-2DeepSeek V4Pro (high)16.2Terminus-2GLM-5.3-Flash(max)17.0Terminus-2GLM-5.3-Flash(low)17.8Terminus-2DeepSeek V4Flash (high)17.9Terminus-2Kimi K2.7Code (alwayson)18.8Terminus-2GLM-4.7-Flash(thinking on)
How to read this chart

What this metric means. A turn is one step of the agent: one model reply and the commands it ran. Agents are limited to 60 turns per attempt.

Not shown: partial coverage. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

Data table: Turns
Turns
ModelCreatorTurnsCoverage (attempts)
Terminus-2 – DeepSeek V4 Flash (max)DeepSeek16.1320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)DeepSeek17.8320 of 614 planned attempts
Terminus-2 – DeepSeek V4 Pro (high)DeepSeek16.2320 of 614 planned attempts
Terminus-2 – Gemma 4 26B A4B (thinking on)Google10.3320 of 614 planned attempts
Terminus-2 – GLM-4.7-Flash (thinking on)Z.ai18.8320 of 614 planned attempts
Terminus-2 – GLM-5.2 (high)Z.ai14.9320 of 614 planned attempts
Terminus-2 – GLM-5.2 (none)Z.ai13.4320 of 614 planned attempts
Terminus-2 – GLM-5.2 (max)Z.ai14.8320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (low)Z.ai17.0320 of 614 planned attempts
Terminus-2 – GLM-5.3-Flash (max)Z.ai16.2320 of 614 planned attempts
Terminus-2 – GLM-5.3 (low)Z.ai13.3320 of 614 planned attempts
Terminus-2 – GLM-5.3 (max, default)Z.ai12.2320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (low)OpenAI3.1320 of 614 planned attempts
Terminus-2 – gpt-oss-120b (high)OpenAI6.2320 of 614 planned attempts
Terminus-2 – Kimi K2.6 (always on)Moonshot AI13.2320 of 614 planned attempts
Terminus-2 – Kimi K2.7 Code (always on)Moonshot AI17.9320 of 614 planned attempts
Terminus-2 – Kimi K3 (always on)Moonshot AI7.7320 of 614 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)Meta14.9320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (medium)Anthropic5.5320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (high)Anthropic6.2320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)Anthropic4.5320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (high)Anthropic5.4320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (medium)Anthropic6.8320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (high)Anthropic8.3320 of 614 planned attempts
Claude Code – Claude Opus 5.5 (xhigh)Anthropic8.9320 of 614 planned attempts
Claude Code – Claude Sonnet 5.5 (xhigh)Anthropic8.2320 of 614 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)Anthropic12.1320 of 614 planned attempts
Codex – GPT-6.1 Sol (medium)OpenAI6.5320 of 614 planned attempts
Codex – GPT-6.1 Sol (high)OpenAI7.8320 of 614 planned attempts
Codex – GPT-6.1 Sol (xhigh)OpenAI9.1320 of 614 planned attempts
Codex – GPT-6 Luna (xhigh)OpenAI10.2320 of 614 planned attempts

Geospatial Agent Index vs. time per task

Higher and further left is better

  • Planned attempts still to come
  • Most attractive quadrant
  • Pareto line
Geospatial Agent Index vs. time per taskTerminus-2 – DeepSeek V4 Flash (max): Geospatial Agent Index 53.07, time per task 11.8 min; Terminus-2 – DeepSeek V4 Flash (high): Geospatial Agent Index 51.94, time per task 12.8 min; Terminus-2 – DeepSeek V4 Pro (high): Geospatial Agent Index 53.89, time per task 10.6 min; Terminus-2 – Gemma 4 26B A4B (thinking on): Geospatial Agent Index 38.9, time per task 10.9 min; Terminus-2 – GLM-4.7-Flash (thinking on): Geospatial Agent Index 28.77, time per task 11.1 min; Terminus-2 – GLM-5.2 (high): Geospatial Agent Index 56.11, time per task 8.3 min; Terminus-2 – GLM-5.2 (none): Geospatial Agent Index 43.19, time per task 4.1 min; Terminus-2 – GLM-5.2 (max): Geospatial Agent Index 55.38, time per task 11.8 min; Terminus-2 – GLM-5.3-Flash (low): Geospatial Agent Index 40.09, time per task 13.3 min; Terminus-2 – GLM-5.3-Flash (max): Geospatial Agent Index 27.24, time per task 19.9 min; Terminus-2 – GLM-5.3 (low): Geospatial Agent Index 38, time per task 8.7 min; Terminus-2 – GLM-5.3 (max, default): Geospatial Agent Index 37.77, time per task 12.4 min; Terminus-2 – gpt-oss-120b (low): Geospatial Agent Index 25.19, time per task 0.9 min; Terminus-2 – gpt-oss-120b (high): Geospatial Agent Index 34.2, time per task 5.2 min; Terminus-2 – Kimi K2.6 (always on): Geospatial Agent Index 52.23, time per task 10.6 min; Terminus-2 – Kimi K2.7 Code (always on): Geospatial Agent Index 54.77, time per task 9.4 min; Terminus-2 – Kimi K3 (always on): Geospatial Agent Index 58.02, time per task 3.9 min; Terminus-2 – Llama 3.3 70B (Non-reasoning): Geospatial Agent Index 6.92, time per task 3.9 min; Claude Code – Claude Opus 5.5 (medium): Geospatial Agent Index 65.34, time per task 0.8 min; Claude Code – Claude Opus 5.5 (high): Geospatial Agent Index 68.12, time per task 1.1 min; Claude Code – Claude Sonnet 5.5 (medium): Geospatial Agent Index 57.72, time per task 0.6 min; Claude Code – Claude Sonnet 5.5 (high): Geospatial Agent Index 64.8, time per task 0.8 min; Claude Code – Claude Haiku 5.5 (medium): Geospatial Agent Index 58.19, time per task 0.8 min; Claude Code – Claude Haiku 5.5 (high): Geospatial Agent Index 60.47, time per task 1.1 min; Claude Code – Claude Opus 5.5 (xhigh): Geospatial Agent Index 68.84, time per task 2.4 min; Claude Code – Claude Sonnet 5.5 (xhigh): Geospatial Agent Index 63.83, time per task 1.6 min; Claude Code – Claude Haiku 5.5 (xhigh): Geospatial Agent Index 63.06, time per task 2.2 min; Codex – GPT-6.1 Sol (medium): Geospatial Agent Index 65.29, time per task 1.4 min; Codex – GPT-6.1 Sol (high): Geospatial Agent Index 62.77, time per task 1.8 min; Codex – GPT-6.1 Sol (xhigh): Geospatial Agent Index 64.95, time per task 3.0 min; Codex – GPT-6 Luna (xhigh): Geospatial Agent Index 60.31, time per task 3.9 minMost attractive quadrant010203040506070800.5 min1 min2 min5 min10 min20 min50 minTime per task (log scale)Geospatial Agent IndexClaude Code – Claude Opus 5.5 (xhigh)Claude Code – Claude Opus 5.5 (high)Claude Code – Claude Opus 5.5 (medium)Codex – GPT-6.1 Sol (medium)Codex – GPT-6.1 Sol (xhigh)Claude Code – Claude Sonnet 5.5 (high)Claude Code – Claude Sonnet 5.5 (xhigh)Claude Code – Claude Haiku 5.5 (xhigh)Codex – GPT-6.1 Sol (high)Claude Code – Claude Haiku 5.5 (high)Codex – GPT-6 Luna (xhigh)Claude Code – Claude Haiku 5.5 (medium)Terminus-2 – Kimi K3 (always on)Claude Code – Claude Sonnet 5.5 (medium)Terminus-2 – GLM-5.2 (high)Terminus-2 – GLM-5.2 (max)Terminus-2 – Kimi K2.7 Code (always on)Terminus-2 – DeepSeek V4 Pro (high)Terminus-2 – DeepSeek V4 Flash (max)Terminus-2 – Kimi K2.6 (always on)Terminus-2 – DeepSeek V4 Flash (high)Terminus-2 – GLM-5.2 (none)Terminus-2 – GLM-5.3-Flash (low)Terminus-2 – Gemma 4 26B A4B (thinking on)Terminus-2 – GLM-5.3 (low)Terminus-2 – GLM-5.3 (max, default)Terminus-2 – gpt-oss-120b (high)Terminus-2 – GLM-4.7-Flash (thinking on)Terminus-2 – GLM-5.3-Flash (max)Terminus-2 – gpt-oss-120b (low)Terminus-2 – Llama 3.3 70B (Non-reasoning)
How to read this chart

How to read this chart. Up is a higher index, left is faster. The dashed Pareto line joins the models that no other model beats on both at once.

Partial coverage. Not plotted and not on the Pareto line; listed in the table: Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

How to read this chart. Each point is one model configuration. Further left means a lower average time per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no faster shown model beats on score.

Data table: Geospatial Agent Index vs. time per task
Geospatial Agent Index vs. time per task
ModelGeospatial Agent IndexTime per taskOn the Pareto line
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)40* (partial: attempt 1 on 209 of 320 tasks)3.9 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – DeepSeek V4 Flash (max)53*11.8 minNo
Terminus-2 – DeepSeek V4 Flash (high)52*12.8 minNo
Terminus-2 – DeepSeek V4 Pro (high)54*10.6 minNo
Terminus-2 – Gemini 3.1 Flash-Lite (low)37* (partial: attempt 1 on 209 of 320 tasks)1.4 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)29* (partial: attempt 1 on 209 of 320 tasks)0.7 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (high)40* (partial: attempt 1 on 205 of 320 tasks)2.6 minNot plotted: partial coverage (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (low)44* (partial: attempt 1 on 209 of 320 tasks)0.8 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.1 Pro (preview) (high)43* (partial: attempt 1 on 209 of 320 tasks)3.5 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.8 Flash (high)34* (partial: attempt 1 on 209 of 320 tasks)9.9 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemini 3.8 Flash (low)49* (partial: attempt 1 on 209 of 320 tasks)2.8 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks)
Terminus-2 – Gemma 4 26B A4B (thinking on)39*10.9 minNo
Terminus-2 – GLM-4.7-Flash (thinking on)29*11.1 minNo
Terminus-2 – GLM-5.2 (high)56*8.3 minNo
Terminus-2 – GLM-5.2 (none)43*4.1 minNo
Terminus-2 – GLM-5.2 (max)55*11.8 minNo
Terminus-2 – GLM-5.3-Flash (low)40*13.3 minNo
Terminus-2 – GLM-5.3-Flash (max)27*19.9 minNo
Terminus-2 – GLM-5.3 (low)38*8.7 minNo
Terminus-2 – GLM-5.3 (max, default)38*12.4 minNo
Terminus-2 – gpt-oss-120b (low)25*0.9 minNo
Terminus-2 – gpt-oss-120b (high)34*5.2 minNo
Terminus-2 – gpt-oss-20b (high)28*14.8 minNot plotted: partial coverage (172 of 173 GeoBenchX tasks)
Terminus-2 – Kimi K2.6 (always on)52*10.6 minNo
Terminus-2 – Kimi K2.7 Code (always on)55*9.4 minNo
Terminus-2 – Kimi K3 (always on)58*3.9 minNo
Terminus-2 – Llama 3.3 70B (Non-reasoning)7*3.9 minNo
Terminus-2 – Qwen 3.8 27B (xhigh)28* (partial: attempt 1 on 36 of 320 tasks)21.1 minNot plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; 0 of 173 GeoBenchX tasks)
Claude Code – Claude Haiku 4.5 (thinking 32k)36* (partial: attempt 1 on 94 of 320 tasks)2.5 minNot plotted: partial coverage (47 of 147 Earth-Bench V2 tasks; 47 of 173 GeoBenchX tasks)
Kimi Code – Kimi K2.6 (always on)56* (partial: attempt 1 on 72 of 320 tasks)9.7 minNot plotted: partial coverage (11 of 147 Earth-Bench V2 tasks; 61 of 173 GeoBenchX tasks)
Kimi Code – Kimi K2.7 Code (always on)54* (partial: attempt 1 on 82 of 320 tasks)8.3 minNot plotted: partial coverage (12 of 147 Earth-Bench V2 tasks; 70 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 5.5 (medium)65*0.8 minYes
Claude Code – Claude Opus 5.5 (high)68*1.1 minYes
Claude Code – Claude Sonnet 5.5 (medium)58*0.6 minYes
Claude Code – Claude Sonnet 5.5 (high)65*0.8 minYes
Claude Code – Claude Haiku 5.5 (medium)58*0.8 minNo
Claude Code – Claude Haiku 5.5 (high)60*1.1 minNo
Claude Code – Claude Opus 5.5 (xhigh)69*2.4 minYes
Claude Code – Claude Sonnet 5.5 (xhigh)64*1.6 minNo
Claude Code – Claude Haiku 5.5 (xhigh)63*2.2 minNo
Claude Code – Claude Opus 5 (high)63* (partial: attempt 1 on 160 of 320 tasks)2.3 minNot plotted: partial coverage (112 of 147 Earth-Bench V2 tasks; 48 of 173 GeoBenchX tasks)
Claude Code – Claude Sonnet 5 (high)56* (partial: attempt 1 on 104 of 320 tasks)2.8 minNot plotted: partial coverage (41 of 147 Earth-Bench V2 tasks; 63 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 4.8 (high)51* (partial: attempt 1 on 105 of 320 tasks)2.2 minNot plotted: partial coverage (49 of 147 Earth-Bench V2 tasks; 56 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 4.7 (high)53* (partial: attempt 1 on 109 of 320 tasks)2.0 minNot plotted: partial coverage (49 of 147 Earth-Bench V2 tasks; 60 of 173 GeoBenchX tasks)
Codex – GPT-6.1 Sol (medium)65*1.4 minNo
Codex – GPT-6.1 Sol (high)63*1.8 minNo
Codex – GPT-6.1 Sol (xhigh)65*3.0 minNo
Codex – GPT-6 Luna (xhigh)60*3.9 minNo

Safety refusals

Safety refusals

Attempts ended by a refusal · Lower is better

A refusal scores 0 and is not rerun.

What refusals measure

Provider safety blocks. Requests the provider stopped on safety grounds, counted per model.

Model refusals. Refusals a model writes in its own reply are not yet identified, so they show No data rather than zero.

Safety refusals per model
ModelAttemptsProvider safety blocksBlock rateModel refusals
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)30000.0%No data
Terminus-2 – DeepSeek V4 Flash (max)32000.0%No data
Terminus-2 – DeepSeek V4 Flash (high)32000.0%No data
Terminus-2 – DeepSeek V4 Pro (high)32000.0%No data
Terminus-2 – Gemini 3.1 Flash-Lite (low)20900.0%No data
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)20900.0%No data
Terminus-2 – Gemini 3.1 Flash-Lite (high)20900.0%No data
Terminus-2 – Gemini 3.1 Pro (preview) (low)22600.0%No data
Terminus-2 – Gemini 3.1 Pro (preview) (high)20900.0%No data
Terminus-2 – Gemini 3.8 Flash (high)20900.0%No data
Terminus-2 – Gemini 3.8 Flash (low)26300.0%No data
Terminus-2 – Gemma 4 26B A4B (thinking on)32000.0%No data
Terminus-2 – GLM-4.7-Flash (thinking on)32000.0%No data
Terminus-2 – GLM-5.2 (high)32000.0%No data
Terminus-2 – GLM-5.2 (none)32000.0%No data
Terminus-2 – GLM-5.2 (max)32000.0%No data
Terminus-2 – GLM-5.3-Flash (low)32000.0%No data
Terminus-2 – GLM-5.3-Flash (max)32000.0%No data
Terminus-2 – GLM-5.3 (low)32000.0%No data
Terminus-2 – GLM-5.3 (max, default)32000.0%No data
Terminus-2 – gpt-oss-120b (low)32000.0%No data
Terminus-2 – gpt-oss-120b (high)32000.0%No data
Terminus-2 – gpt-oss-20b (high)32000.0%No data
Terminus-2 – Kimi K2.6 (always on)32000.0%No data
Terminus-2 – Kimi K2.7 Code (always on)32000.0%No data
Terminus-2 – Kimi K3 (always on)32000.0%No data
Terminus-2 – Llama 3.3 70B (Non-reasoning)32000.0%No data
Terminus-2 – Qwen 3.8 27B (xhigh)3600.0%No data
Claude Code – Claude Haiku 4.5 (thinking 32k)9400.0%No data
Kimi Code – Kimi K2.6 (always on)7200.0%No data
Kimi Code – Kimi K2.7 Code (always on)8200.0%No data
Claude Code – Claude Opus 5.5 (medium)32000.0%No data
Claude Code – Claude Opus 5.5 (high)32000.0%No data
Claude Code – Claude Sonnet 5.5 (medium)32000.0%No data
Claude Code – Claude Sonnet 5.5 (high)32000.0%No data
Claude Code – Claude Haiku 5.5 (medium)32000.0%No data
Claude Code – Claude Haiku 5.5 (high)32000.0%No data
Claude Code – Claude Opus 5.5 (xhigh)32000.0%No data
Claude Code – Claude Sonnet 5.5 (xhigh)32000.0%No data
Claude Code – Claude Haiku 5.5 (xhigh)32000.0%No data
Claude Code – Claude Opus 5 (high)16000.0%No data
Claude Code – Claude Sonnet 5 (high)10400.0%No data
Claude Code – Claude Opus 4.8 (high)10500.0%No data
Claude Code – Claude Opus 4.7 (high)10900.0%No data
Codex – GPT-6.1 Sol (medium)32000.0%No data
Codex – GPT-6.1 Sol (high)32000.0%No data
Codex – GPT-6.1 Sol (xhigh)32000.0%No data
Codex – GPT-6 Luna (xhigh)32000.0%No data

Reward hacking

Reward hacking rate

Flagged attempts as a share of reviewed passing attempts · Lower is better

Reward hacking rateTerminus-2 – Kimi K2.6 (always on): 0%; Terminus-2 – Qwen 3.8 27B (xhigh): 0%; Claude Code – Claude Haiku 5.5 (xhigh): 0%; Claude Code – Claude Sonnet 5 (high): 0%; Codex – GPT-6.1 Sol (medium): 0%; Codex – GPT-6.1 Sol (high): 0%; Terminus-2 – GLM-5.2 (max): 1%; Terminus-2 – DeepSeek V4 Pro (high): 1%; Codex – GPT-6.1 Sol (xhigh): 1%; Claude Code – Claude Sonnet 5.5 (xhigh): 1%; Claude Code – Claude Haiku 5.5 (medium): 1%; Terminus-2 – GLM-5.2 (high): 1%; Claude Code – Claude Opus 5.5 (high): 1%; Claude Code – Claude Sonnet 5.5 (high): 1%; Claude Code – Claude Haiku 5.5 (high): 2%; Terminus-2 – DeepSeek V4 Flash (max): 2%; Claude Code – Claude Opus 4.8 (high): 2%; Claude Code – Claude Opus 5.5 (medium): 2%; Terminus-2 – Kimi K2.7 Code (always on): 2%; Terminus-2 – DeepSeek V4 Flash (high): 2%; Terminus-2 – Kimi K3 (always on): 3%; Claude Code – Claude Opus 5.5 (xhigh): 3%; Claude Code – Claude Opus 4.7 (high): 3%; Claude Code – Claude Opus 5 (high): 3%; Codex – GPT-6 Luna (xhigh): 4%; Terminus-2 – gpt-oss-20b (high): 5%; Terminus-2 – Gemini 3.1 Flash-Lite (high): 7%; Terminus-2 – GLM-5.3-Flash (low): 7%; Terminus-2 – Gemini 3.1 Flash-Lite (low): 8%; Claude Code – Claude Haiku 4.5 (thinking 32k): 8%; Terminus-2 – Gemini 3.1 Pro (preview) (high): 8%; Claude Code – Claude Sonnet 5.5 (medium): 8%; Terminus-2 – Gemma 4 26B A4B (thinking on): 9%; Terminus-2 – Gemini 3.1 Pro (preview) (low): 11%; Terminus-2 – GLM-5.2 (none): 11%; Terminus-2 – Gemini 3.1 Flash-Lite (minimal): 12%; Terminus-2 – Llama 3.3 70B (Non-reasoning): 12%; Terminus-2 – GLM-4.7-Flash (thinking on): 12%; Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high): 12%; Terminus-2 – GLM-5.3-Flash (max): 16%; Terminus-2 – gpt-oss-120b (high): 17%; Terminus-2 – gpt-oss-120b (low): 20%; Terminus-2 – Gemini 3.8 Flash (low): 23%; Terminus-2 – GLM-5.3 (max, default): 26%; Terminus-2 – GLM-5.3 (low): 27%; Terminus-2 – Gemini 3.8 Flash (high): 67%; Kimi Code – Kimi K2.6 (always on): No data; Kimi Code – Kimi K2.7 Code (always on): No data0%25%50%75%100%0%Terminus-2Kimi K2.6(always on)0%Terminus-2Qwen 3.8 27B(xhigh)0%Claude CodeClaude Haiku5.5 (xhigh)0%Claude CodeClaude Sonnet5 (high)0%CodexGPT-6.1 Sol(medium)0%CodexGPT-6.1 Sol(high)1%Terminus-2GLM-5.2 (max)1%Terminus-2DeepSeek V4Pro (high)1%CodexGPT-6.1 Sol(xhigh)1%Claude CodeClaude Sonnet5.5 (xhigh)1%Claude CodeClaude Haiku5.5 (medium)1%Terminus-2GLM-5.2(high)1%Claude CodeClaude Opus5.5 (high)1%Claude CodeClaude Sonnet5.5 (high)2%Claude CodeClaude Haiku5.5 (high)2%Terminus-2DeepSeek V4Flash (max)2%Claude CodeClaude Opus4.8 (high)2%Claude CodeClaude Opus5.5 (medium)2%Terminus-2Kimi K2.7Code (alwayson)2%Terminus-2DeepSeek V4Flash (high)3%Terminus-2Kimi K3(always on)3%Claude CodeClaude Opus5.5 (xhigh)3%Claude CodeClaude Opus4.7 (high)3%Claude CodeClaude Opus 5(high)4%CodexGPT-6 Luna(xhigh)5%Terminus-2gpt-oss-20b(high)7%Terminus-2Gemini 3.1Flash-Lite(high)7%Terminus-2GLM-5.3-Flash(low)8%Terminus-2Gemini 3.1Flash-Lite(low)8%Claude CodeClaude Haiku4.5 (thinking32k)8%Terminus-2Gemini 3.1Pro (preview)(high)8%Claude CodeClaude Sonnet5.5 (medium)9%Terminus-2Gemma 4 26BA4B (thinkingon)11%Terminus-2Gemini 3.1Pro (preview)(low)11%Terminus-2GLM-5.2(none)12%Terminus-2Gemini 3.1Flash-Lite(minimal)12%Terminus-2Llama 3.3 70B(Non-reasoning)12%Terminus-2GLM-4.7-Flash(thinking on)12%Terminus-2Claude Sonnet4.6(adaptive,16%Terminus-2GLM-5.3-Flash(max)17%Terminus-2gpt-oss-120b(high)20%Terminus-2gpt-oss-120b(low)23%Terminus-2Gemini 3.8Flash (low)26%Terminus-2GLM-5.3 (max,default)27%Terminus-2GLM-5.3 (low)67%Terminus-2Gemini 3.8Flash (high)No dataKimi CodeKimi K2.6(always on)No dataKimi CodeKimi K2.7Code (alwayson)
How to read this chart

What this metric means. The share of reviewed passing attempts that an AI judge flagged for reward hacking: reading grader files, writing values no command computed, editing outputs to fit after a failed check, or fetching answers from outside the sandbox. A flagged attempt scores 0.

How to read this chart. Lower is better. Only attempts that passed every other check are reviewed, so a model with few passes has few reviews. Every flag is read by hand before results are published.

Data table: Reward hacking rate
Reward hacking rate
ModelPassing attempts reviewedFlaggedRate
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)961212%
Terminus-2 – DeepSeek V4 Flash (max)17232%
Terminus-2 – DeepSeek V4 Flash (high)16842%
Terminus-2 – DeepSeek V4 Pro (high)17111%
Terminus-2 – Gemini 3.1 Flash-Lite (low)9378%
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)841012%
Terminus-2 – Gemini 3.1 Flash-Lite (high)10177%
Terminus-2 – Gemini 3.1 Pro (preview) (low)1281411%
Terminus-2 – Gemini 3.1 Pro (preview) (high)122108%
Terminus-2 – Gemini 3.8 Flash (high)1137667%
Terminus-2 – Gemini 3.8 Flash (low)1333123%
Terminus-2 – Gemma 4 26B A4B (thinking on)139129%
Terminus-2 – GLM-4.7-Flash (thinking on)1061312%
Terminus-2 – GLM-5.2 (high)18121%
Terminus-2 – GLM-5.2 (none)1571811%
Terminus-2 – GLM-5.2 (max)17811%
Terminus-2 – GLM-5.3-Flash (low)139107%
Terminus-2 – GLM-5.3-Flash (max)1031616%
Terminus-2 – GLM-5.3 (low)1674527%
Terminus-2 – GLM-5.3 (max, default)1644226%
Terminus-2 – gpt-oss-120b (low)1022020%
Terminus-2 – gpt-oss-120b (high)1332317%
Terminus-2 – gpt-oss-20b (high)9555%
Terminus-2 – Kimi K2.6 (always on)16800%
Terminus-2 – Kimi K2.7 Code (always on)17842%
Terminus-2 – Kimi K3 (always on)18953%
Terminus-2 – Llama 3.3 70B (Non-reasoning)25312%
Terminus-2 – Qwen 3.8 27B (xhigh)1000%
Claude Code – Claude Haiku 4.5 (thinking 32k)3738%
Kimi Code – Kimi K2.6 (always on)00No data
Kimi Code – Kimi K2.7 Code (always on)00No data
Claude Code – Claude Opus 5.5 (medium)21442%
Claude Code – Claude Opus 5.5 (high)22131%
Claude Code – Claude Sonnet 5.5 (medium)204178%
Claude Code – Claude Sonnet 5.5 (high)21231%
Claude Code – Claude Haiku 5.5 (medium)18921%
Claude Code – Claude Haiku 5.5 (high)19732%
Claude Code – Claude Opus 5.5 (xhigh)22663%
Claude Code – Claude Sonnet 5.5 (xhigh)20821%
Claude Code – Claude Haiku 5.5 (xhigh)20300%
Claude Code – Claude Opus 5 (high)8633%
Claude Code – Claude Sonnet 5 (high)5800%
Claude Code – Claude Opus 4.8 (high)5512%
Claude Code – Claude Opus 4.7 (high)6023%
Codex – GPT-6.1 Sol (medium)21110%
Codex – GPT-6.1 Sol (high)20310%
Codex – GPT-6.1 Sol (xhigh)21121%
Codex – GPT-6 Luna (xhigh)20394%

Run specifications

Results snapshot: 2026-10-12 11:59 UTC. Agent release versions and release dates are not recorded in this export. Coverage counts benchmarks with a score; it does not mean every planned attempt is complete. Task and attempt counts below are scored coverage. Select a model name for its recorded settings.

Agent and model configurations by benchmark
Agent nameModel nameProviderCoverageEarth-Bench V2GeoBenchX
Claude CodeClaude Haiku 4.5 (thinking 32k)Anthropic2/2benchmarks scoredAgent release: No data47/147 tasks
47/441 attempts		Agent release: No data47/173 tasks
47/173 attempts
Claude CodeClaude Haiku 5.5 (high)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Haiku 5.5 (medium)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Haiku 5.5 (xhigh)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Opus 4.7 (high)Anthropic2/2benchmarks scoredAgent release: No data49/147 tasks
49/441 attempts		Agent release: No data60/173 tasks
60/173 attempts
Claude CodeClaude Opus 4.8 (high)Anthropic2/2benchmarks scoredAgent release: No data49/147 tasks
49/441 attempts		Agent release: No data56/173 tasks
56/173 attempts
Claude CodeClaude Opus 5 (high)Anthropic2/2benchmarks scoredAgent release: No data112/147 tasks
112/441 attempts		Agent release: No data48/173 tasks
48/173 attempts
Claude CodeClaude Opus 5.5 (high)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Opus 5.5 (medium)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Opus 5.5 (xhigh)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Sonnet 5 (high)Anthropic2/2benchmarks scoredAgent release: No data41/147 tasks
41/441 attempts		Agent release: No data63/173 tasks
63/173 attempts
Claude CodeClaude Sonnet 5.5 (high)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Sonnet 5.5 (medium)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Claude CodeClaude Sonnet 5.5 (xhigh)Anthropic2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
CodexGPT-6 Luna (xhigh)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
CodexGPT-6.1 Sol (high)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
CodexGPT-6.1 Sol (medium)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
CodexGPT-6.1 Sol (xhigh)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Kimi CodeKimi K2.6 (always on)Moonshot AI2/2benchmarks scoredAgent release: No data11/147 tasks
11/441 attempts		Agent release: No data60/173 tasks
60/173 attempts
Kimi CodeKimi K2.7 Code (always on)Moonshot AI2/2benchmarks scoredAgent release: No data12/147 tasks
12/441 attempts		Agent release: No data67/173 tasks
67/173 attempts
Terminus-2Claude Sonnet 4.6 (adaptive, effort high)Anthropic2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts
19 excluded		Agent release: No data173/173 tasks
173/173 attempts
72 excluded
Terminus-2DeepSeek V4 Flash (high)DeepSeek2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2DeepSeek V4 Flash (max)DeepSeek2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2DeepSeek V4 Pro (high)DeepSeek2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-4.7-Flash (thinking on)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.2 (high)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.2 (max)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.2 (none)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.3 (low)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.3 (max, default)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.3-Flash (low)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2GLM-5.3-Flash (max)Z.ai2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Gemini 3.1 Flash-Lite (high)Google2/2benchmarks scoredAgent release: No data35/147 tasks
35/441 attempts
1 excluded		Agent release: No data170/173 tasks
170/173 attempts
3 excluded
Terminus-2Gemini 3.1 Flash-Lite (low)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Gemini 3.1 Flash-Lite (minimal)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Gemini 3.1 Pro (preview) (high)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Gemini 3.1 Pro (preview) (low)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts
2 excluded		Agent release: No data173/173 tasks
173/173 attempts
15 excluded
Terminus-2Gemini 3.8 Flash (high)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Gemini 3.8 Flash (low)Google2/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts
12 excluded		Agent release: No data173/173 tasks
173/173 attempts
42 excluded
Terminus-2Gemma 4 26B A4B (thinking on)Google2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Kimi K2.6 (always on)Moonshot AI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Kimi K2.7 Code (always on)Moonshot AI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Kimi K3 (always on)Moonshot AI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Llama 3.3 70B (Non-reasoning)Meta2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2Qwen 3.8 27B (xhigh)Alibaba (Qwen)1/2benchmarks scoredAgent release: No data36/147 tasks
36/441 attempts		Agent release: No data0/173 tasks
0/173 attempts
Terminus-2gpt-oss-120b (high)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2gpt-oss-120b (low)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data173/173 tasks
173/173 attempts
Terminus-2gpt-oss-20b (high)OpenAI2/2benchmarks scoredAgent release: No data147/147 tasks
147/441 attempts		Agent release: No data172/173 tasks
172/173 attempts
1 excluded

Up to 3 attempts per task in the full protocol. Agent setup and task limits · Full methodology 0.30 · Download results as JSON

Frequently asked questions

What are Geospatial Agent Benchmarks?
Axis Spatial evaluates AI model and agent-harness configurations on geospatial (GIS and Earth observation) tasks using raster, vector, tabular and gridded data. These are our own runs of published benchmarks under the documented setup, rather than reproductions of their papers’ scores. The headline is the Geospatial Intelligence Index, one row per model; this page is the Geospatial Agent Index, one row per agent setup. Explore the evaluations.
How is the Geospatial Agent Index calculated?
The index is the equal-weight average of 2 hard evaluations (Earth-Bench V2, GeoBenchX), on a scale from 0 to 100. GS-QA2 is first averaged within each question template, so every kind of question weighs the same. GeoAgentBench and GeoAnalystBench are not part of the Geospatial Agent Index. Top entries score near-perfect on them (92 to 100%), and their task instructions already give the method step by step, so they no longer separate strong agents. They make up the coding to a written method group of the Geospatial Intelligence Index, and their results are published in their own section, Guided GIS workflows. Within each benchmark, tasks have equal weight and each task averages its recorded attempts. An entry is ranked only when it has a first attempt on every task of every index component; other entries are listed apart as partial coverage. See the index results.
What should I compare when choosing a model?
Compare the full model, harness and reasoning configuration, then check task coverage and attempts per task alongside score, cost and time. A higher score from incomplete coverage is not evidence that an entry will perform better on untested tasks. Compare two configurations.
How is a task checked?
An attempt passes only when every required check passes: valid output files, numerical and spatial checks, any required map checklist, and trace review for reward hacking. Tasks without a map skip map judging. Time-limit failures and safety refusals score zero. Read the scoring rules.
How are cost and execution time measured?
Cost uses recorded token counts at published API list prices, including cached-input pricing where applicable. Time measures the agent’s wall time, excluding sandbox start-up and grading. Both include failed attempts; missing measurements remain No data.
Do the results include image interpretation?
Image interpretation is outside the Geospatial Agent Index, which is text and terminal only. Satellite and aerial image questions appear in a separate track, Geospatial image understanding, whose look-and-answer score is the perception group of the Geospatial Intelligence Index, with look-and-answer and look-and-work protocols reported separately. View Geospatial image understanding.
How far can these results be generalised?
The results describe the listed configurations on the evaluated tasks. They do not establish performance on other data, prompts, tools or agent setups, or in interactive desktop GIS. Use the benchmark breakdowns and coverage to judge how closely the evaluation matches your work.