Claude Code – GLM-4.7-Flash (thinking on): geospatial agent results

Summary

Geospatial Agent Index: 27* (partial: attempt 1 on 52 of 565 tasks) (partial coverage, not ranked: 10 of 147 Earth-Bench V2 tasks; 42 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Cost per task: $0.089
Time per task: 13.6 min
Tokens per attempt: 1.37M
Turns per attempt: 35.5
Attempts decided: 49
Judged by: Claude Sonnet 5.5 (46 counted attempts); 1 result awaiting review

Geospatial Intelligence Index

GLM-4.7-Flash at each reasoning level we run, with its four group scores. Each model is shown at its highest reasoning level, as Artificial Analysis does; other levels are under 'Show every reasoning level'. See the Geospatial Intelligence Index.

GLM-4.7-Flash: Geospatial Intelligence Index at each reasoning level
Model	Geospatial Intelligence Index	Rank (every reasoning level)	Text-only view	Agents (3/9)	Coding to a written method (2/9)	Earth science (2/9)	Perception (2/9)	Scored in
GLM-4.7-Flash (thinking on) (default view)	Not ranked yet	Not ranked yet	Not ranked yet	39*	39	86	0 (cannot take images)	Terminus-2

Not ranked yet: GLM-4.7-Flash (thinking on): Agents: attempt 1 on 235 of 245 GS-QA2 tasks.

Geospatial Agent Index configurations

Every harness and reasoning level of GLM-4.7-Flash we run. See the Geospatial Agent Index.

GLM-4.7-Flash: Geospatial Agent Index configurations
Model	Harness	Geospatial Agent Index	Rank	Cost per task	Time per task
Terminus-2 – GLM-4.7-Flash (thinking on)	Terminus-2	39* (partial: attempt 1 on 555 of 565 tasks)	Not ranked yet	$0.038	10.7 min
Claude Code – GLM-4.7-Flash (thinking on) (this page)	Claude Code	27* (partial: attempt 1 on 52 of 565 tasks)	Not ranked yet	$0.089	13.6 min

Score by benchmark

Claude Code – GLM-4.7-Flash (thinking on): score by benchmark

Average pass@1, 0 to 100 · Higher is better

Data table: Claude Code – GLM-4.7-Flash (thinking on): score by benchmark

Claude Code – GLM-4.7-Flash (thinking on): by benchmark
Benchmark	Score	Tasks with a decided attempt	Cost per task	Time per task
Earth-Bench V2	20	10 of 147	$0.035	8.0 min
GeoBenchX	33	39 of 173	$0.102	14.9 min
GS-QA2	No data	0 of 245	No data	No data

Score by task format and kind of work

Claude Code – GLM-4.7-Flash (thinking on): score by task format

Average pass@1 over the tasks of each task format, 0 to 100 · Higher is better

How to read this chart

What this shows. The same attempts as the index, grouped by task format. Groups with few tasks are less certain. About these breakdowns.

Data table: Claude Code – GLM-4.7-Flash (thinking on): score by task format

Claude Code – GLM-4.7-Flash (thinking on): by task format
Task format	Score	Tasks with a decided attempt
Multi-step analysis	50	16 of 82
Single-answer questions	14	14 of 404
Recognising infeasible requests	26	19 of 79

Claude Code – GLM-4.7-Flash (thinking on): score by kind of work

Average pass@1 over the tasks of each kind of work, 0 to 100 · Higher is better

How to read this chart

What this shows. The same attempts as the index, grouped by kind of work. Groups with few tasks are less certain. About these breakdowns.

Data table: Claude Code – GLM-4.7-Flash (thinking on): score by kind of work

Claude Code – GLM-4.7-Flash (thinking on): by kind of work
Kind of work	Score	Tasks with a decided attempt
Raster and remote sensing	15	13 of 209
Vector and overlay	No data	0 of 136
Networks and routing	No data	0 of 0
Climate and time series	0	1 of 68
Spatial statistics and interpolation	20	5 of 14
Mapping and cartography	64	11 of 59
Recognising infeasible requests	26	19 of 79

Claude Code – GLM-4.7-Flash (thinking on): score by data type

Average pass@1 over the tasks of each data type, 0 to 100 · Higher is better

How to read this chart

What this shows. The same attempts as the index, grouped by data type. Groups with few tasks are less certain. About these breakdowns.

Data table: Claude Code – GLM-4.7-Flash (thinking on): score by data type

Claude Code – GLM-4.7-Flash (thinking on): by data type
Data type	Score	Tasks with a decided attempt
Optical multispectral	0	8 of 114
Elevation or terrain	No data	0 of 115
Climate or gridded time series	No data	0 of 10
Other gridded data	33	6 of 50
Vector only	47	15 of 207

Claude Code – GLM-4.7-Flash (thinking on): score by task length

Average pass@1 over the tasks of each task length, 0 to 100 · Higher is better

How to read this chart

What this shows. The same attempts as the index, grouped by task length. Groups with few tasks are less certain. About these breakdowns.

Data table: Claude Code – GLM-4.7-Flash (thinking on): score by task length

Claude Code – GLM-4.7-Flash (thinking on): by task length
Task length	Score	Tasks with a decided attempt
Short tasks	100	7 of 202
Medium tasks	18	11 of 172
Long tasks	19	31 of 191

Compared with other models

Geospatial Agent Index

Equal-weight average of the evaluation scores, 0 to 100 · Higher is better

Planned attempts still to come

How to read this chart

What this metric means. The equal-weight average of the 3 evaluation scores. Each evaluation score is the average pass@1 over that benchmark's tasks, from 0 to 100.

Partial coverage entries. An entry is ranked only when it has a first attempt on every task of every index component. Entries without that are marked partial coverage: they are listed apart with their coverage, for example "36 of 147 Earth-Bench V2 tasks", and are left out of the ranking, the index charts, the Pareto lines and the highlights. An entry becomes ranked as soon as its first attempts are complete.

Coverage hatch. A hatched bar, and an asterisk in tables, marks an entry that does not yet have results for every planned attempt. While attempts await review, the table gives the range the score could still reach.

First attempt only. Scores currently use each entry's first attempt on every task. The full protocol is 3 attempts; averages over 3 will replace these once every entry has them. An entry without a first attempt on every task yet is marked "partial: attempt 1 on n of N tasks".

Not shown: partial coverage or being added. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – DeepSeek V4 Pro (high) (239 of 245 GS-QA2 tasks); Terminus-2 – Gemini 3.1 Flash-Lite (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Flash-Lite (minimal) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Flash-Lite (high) (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Pro (preview) (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Pro (preview) (high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.8 Flash (high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.8 Flash (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemma 4 26B A4B (thinking on) (214 of 245 GS-QA2 tasks); Terminus-2 – GLM-4.7-Flash (thinking on) (235 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.2 (high) (244 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.2 (max) (238 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3-Flash (low) (188 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3-Flash (max) (144 of 245 GS-QA2 tasks); Terminus-2 – gpt-oss-120b (high) (241 of 245 GS-QA2 tasks); Terminus-2 – gpt-oss-20b (high) (172 of 173 GeoBenchX tasks; 230 of 245 GS-QA2 tasks); Terminus-2 – Kimi K2.6 (always on) (242 of 245 GS-QA2 tasks); Terminus-2 – Kimi K2.7 Code (always on) (240 of 245 GS-QA2 tasks); Terminus-2 – Qwen 3.8 27B (xhigh) (36 of 147 Earth-Bench V2 tasks; GeoBenchX incomplete; GS-QA2 incomplete); Claude Code – Claude Haiku 4.5 (thinking 32k) (102 of 147 Earth-Bench V2 tasks; 106 of 173 GeoBenchX tasks); Claude Code – GLM-4.7-Flash (thinking on) (10 of 147 Earth-Bench V2 tasks; 42 of 173 GeoBenchX tasks; GS-QA2 incomplete); Claude Code – GLM-5.2 (max) (140 of 147 Earth-Bench V2 tasks; 165 of 173 GeoBenchX tasks; 35 of 245 GS-QA2 tasks); Claude Code – GLM-5.3-Flash (max) (102 of 147 Earth-Bench V2 tasks; 171 of 173 GeoBenchX tasks; GS-QA2 incomplete); Claude Code – GLM-5.3 (max, default) (21 of 147 Earth-Bench V2 tasks; 96 of 173 GeoBenchX tasks; GS-QA2 incomplete); Codex – DeepSeek V4 Flash (high) (73 of 147 Earth-Bench V2 tasks; 101 of 173 GeoBenchX tasks; 29 of 245 GS-QA2 tasks); Codex – DeepSeek V4 Pro (high) (8 of 147 Earth-Bench V2 tasks; 40 of 173 GeoBenchX tasks; GS-QA2 incomplete); Codex – gpt-oss-120b (high) (8 of 147 Earth-Bench V2 tasks; 41 of 173 GeoBenchX tasks; GS-QA2 incomplete); Kimi Code – Kimi K2.6 (always on) (131 of 147 Earth-Bench V2 tasks); Kimi Code – Kimi K2.7 Code (always on) (146 of 147 Earth-Bench V2 tasks; 243 of 245 GS-QA2 tasks); Claude Code – Claude Opus 5.5 (high) (146 of 147 Earth-Bench V2 tasks); Claude Code – Claude Sonnet 5.5 (high) (244 of 245 GS-QA2 tasks); Claude Code – Claude Opus 5.5 (xhigh) (146 of 147 Earth-Bench V2 tasks); Claude Code – Claude Opus 5 (high) (127 of 147 Earth-Bench V2 tasks; 79 of 173 GeoBenchX tasks); Claude Code – Claude Sonnet 5 (high) (83 of 147 Earth-Bench V2 tasks; 111 of 173 GeoBenchX tasks); Claude Code – Claude Opus 4.8 (high) (135 of 147 Earth-Bench V2 tasks; 119 of 173 GeoBenchX tasks; 244 of 245 GS-QA2 tasks); Claude Code – Claude Opus 4.7 (high) (129 of 147 Earth-Bench V2 tasks; 122 of 173 GeoBenchX tasks).

Data table: Geospatial Agent Index

Geospatial Agent Index
Model	Creator	Geospatial Agent Index	Range while attempts are pending	Benchmarks covered	Coverage (attempts)
Codex – GPT-6.1 Sol (xhigh)	OpenAI	72*	No pending judgements; coverage incomplete; 211 results awaiting review	3 of 3	565 of 859 planned attempts
Codex – GPT-6.1 Sol (medium)	OpenAI	72*	No pending judgements; coverage incomplete; 208 results awaiting review	3 of 3	565 of 859 planned attempts
Claude Code – Claude Opus 5.5 (medium)	Anthropic	71*	No pending judgements; coverage incomplete; 201 results awaiting review	3 of 3	565 of 859 planned attempts
Codex – GPT-6.1 Sol (high)	OpenAI	70*	No pending judgements; coverage incomplete; 207 results awaiting review	3 of 3	565 of 859 planned attempts (1 excluded)
Claude Code – Claude Sonnet 5.5 (xhigh)	Anthropic	69*	No pending judgements; coverage incomplete; 196 results awaiting review	3 of 3	565 of 859 planned attempts
Claude Code – Claude Haiku 5.5 (xhigh)	Anthropic	67*	No pending judgements; coverage incomplete; 184 results awaiting review	3 of 3	565 of 859 planned attempts
Claude Code – Claude Haiku 5.5 (high)	Anthropic	66*	No pending judgements; coverage incomplete; 187 results awaiting review	3 of 3	565 of 859 planned attempts
Codex – GPT-6 Luna (xhigh)	OpenAI	64*	No pending judgements; coverage incomplete; 177 results awaiting review	3 of 3	565 of 859 planned attempts
Terminus-2 – Kimi K3 (always on)	Moonshot AI	64*	No pending judgements; coverage incomplete; 183 results awaiting review; 1 trace review declined by the judge	3 of 3	565 of 859 planned attempts
Claude Code – Claude Haiku 5.5 (medium)	Anthropic	63*	No pending judgements; coverage incomplete; 177 results awaiting review	3 of 3	565 of 859 planned attempts
Terminus-2 – DeepSeek V4 Flash (high)	DeepSeek	63*	No pending judgements; coverage incomplete; 176 results awaiting review; 4 trace reviews declined by the judge	3 of 3	565 of 859 planned attempts
Claude Code – Claude Sonnet 5.5 (medium)	Anthropic	63*	No pending judgements; coverage incomplete; 178 results awaiting review	3 of 3	565 of 859 planned attempts
Terminus-2 – GLM-5.3 (max, default)	Z.ai	62*	No pending judgements; coverage incomplete; 185 results awaiting review; 7 trace reviews declined by the judge	3 of 3	565 of 859 planned attempts
Terminus-2 – DeepSeek V4 Flash (max)	DeepSeek	61*	No pending judgements; coverage incomplete; 177 results awaiting review; 1 trace review declined by the judge	3 of 3	565 of 859 planned attempts
Terminus-2 – DeepSeek V4 Pro (max)	DeepSeek	59*	No pending judgements; coverage incomplete; 172 results awaiting review; 1 trace review declined by the judge	3 of 3	565 of 859 planned attempts
Terminus-2 – GLM-5.3 (low)	Z.ai	58*	No pending judgements; coverage incomplete; 180 results awaiting review; 5 trace reviews declined by the judge	3 of 3	565 of 859 planned attempts
Terminus-2 – GLM-5.2 (none)	Z.ai	55*	No pending judgements; coverage incomplete; 161 results awaiting review	3 of 3	565 of 859 planned attempts
Terminus-2 – gpt-oss-120b (low)	OpenAI	28*	No pending judgements; coverage incomplete; 65 results awaiting review	3 of 3	565 of 859 planned attempts
Terminus-2 – Llama 3.3 70B (Non-reasoning)	Meta	8*	No pending judgements; coverage incomplete; 24 results awaiting review	3 of 3	565 of 859 planned attempts

Geospatial Agent Index vs. cost per task

Higher and further left is better

Planned attempts still to come
Most attractive quadrant
Pareto line

How to read this chart

Partial coverage. Not plotted and not on the Pareto line; listed in the table: Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – DeepSeek V4 Pro (high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – Gemma 4 26B A4B (thinking on), Terminus-2 – GLM-4.7-Flash (thinking on), Terminus-2 – GLM-5.2 (high), Terminus-2 – GLM-5.2 (max), Terminus-2 – GLM-5.3-Flash (low), Terminus-2 – GLM-5.3-Flash (max), Terminus-2 – gpt-oss-120b (high), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Kimi K2.6 (always on), Terminus-2 – Kimi K2.7 Code (always on), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – Claude Haiku 4.5 (thinking 32k), Claude Code – GLM-4.7-Flash (thinking on), Claude Code – GLM-5.2 (max), Claude Code – GLM-5.3-Flash (max), Claude Code – GLM-5.3 (max, default), Codex – DeepSeek V4 Flash (high), Codex – DeepSeek V4 Pro (high), Codex – gpt-oss-120b (high), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5.5 (high), Claude Code – Claude Sonnet 5.5 (high), Claude Code – Claude Opus 5.5 (xhigh), Claude Code – Claude Opus 5 (high), Claude Code – Claude Sonnet 5 (high), Claude Code – Claude Opus 4.8 (high), Claude Code – Claude Opus 4.7 (high).

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one model configuration. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.

Data table: Geospatial Agent Index vs. cost per task

Geospatial Agent Index vs. cost per task
Model	Geospatial Agent Index	Cost per task (USD)	On the Pareto line
Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high)	47* (partial: attempt 1 on 209 of 565 tasks)	$0.291	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – DeepSeek V4 Flash (max)	61*	$0.067	No
Terminus-2 – DeepSeek V4 Flash (high)	63*	$0.076	No
Terminus-2 – DeepSeek V4 Pro (max)	59*	$0.341	No
Terminus-2 – DeepSeek V4 Pro (high)	63* (partial: attempt 1 on 559 of 565 tasks)	$0.354	Not plotted: partial coverage (239 of 245 GS-QA2 tasks)
Terminus-2 – Gemini 3.1 Flash-Lite (low)	39* (partial: attempt 1 on 209 of 565 tasks)	$0.013	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.1 Flash-Lite (minimal)	35* (partial: attempt 1 on 209 of 565 tasks)	$0.0057	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.1 Flash-Lite (high)	46* (partial: attempt 1 on 205 of 565 tasks)	$0.021	Not plotted: partial coverage (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.1 Pro (preview) (low)	48* (partial: attempt 1 on 209 of 565 tasks)	$0.061	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.1 Pro (preview) (high)	54* (partial: attempt 1 on 209 of 565 tasks)	$0.275	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.8 Flash (high)	39* (partial: attempt 1 on 209 of 565 tasks)	$0.336	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemini 3.8 Flash (low)	57* (partial: attempt 1 on 209 of 565 tasks)	$0.147	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete)
Terminus-2 – Gemma 4 26B A4B (thinking on)	50* (partial: attempt 1 on 534 of 565 tasks)	$0.020	Not plotted: partial coverage (214 of 245 GS-QA2 tasks)
Terminus-2 – GLM-4.7-Flash (thinking on)	39* (partial: attempt 1 on 555 of 565 tasks)	$0.038	Not plotted: partial coverage (235 of 245 GS-QA2 tasks)
Terminus-2 – GLM-5.2 (high)	62*	$0.250	Not plotted: partial coverage (244 of 245 GS-QA2 tasks)
Terminus-2 – GLM-5.2 (none)	55*	$0.112	No
Terminus-2 – GLM-5.2 (max)	64* (partial: attempt 1 on 558 of 565 tasks)	$0.346	Not plotted: partial coverage (238 of 245 GS-QA2 tasks)
Terminus-2 – GLM-5.3-Flash (low)	51* (partial: attempt 1 on 508 of 565 tasks)	$0.039	Not plotted: partial coverage (188 of 245 GS-QA2 tasks)
Terminus-2 – GLM-5.3-Flash (max)	52* (partial: attempt 1 on 464 of 565 tasks)	$0.077	Not plotted: partial coverage (144 of 245 GS-QA2 tasks)
Terminus-2 – GLM-5.3 (low)	58*	$0.311	No
Terminus-2 – GLM-5.3 (max, default)	62*	$0.489	No
Terminus-2 – gpt-oss-120b (low)	28*	$0.0048	Yes
Terminus-2 – gpt-oss-120b (high)	40* (partial: attempt 1 on 561 of 565 tasks)	$0.030	Not plotted: partial coverage (241 of 245 GS-QA2 tasks)
Terminus-2 – gpt-oss-20b (high)	29* (partial: attempt 1 on 549 of 565 tasks)	$0.017	Not plotted: partial coverage (172 of 173 GeoBenchX tasks; 230 of 245 GS-QA2 tasks)
Terminus-2 – Kimi K2.6 (always on)	61* (partial: attempt 1 on 562 of 565 tasks)	$0.108	Not plotted: partial coverage (242 of 245 GS-QA2 tasks)
Terminus-2 – Kimi K2.7 Code (always on)	63* (partial: attempt 1 on 560 of 565 tasks)	$0.131	Not plotted: partial coverage (240 of 245 GS-QA2 tasks)
Terminus-2 – Kimi K3 (always on)	64*	$0.252	No
Terminus-2 – Llama 3.3 70B (Non-reasoning)	8*	$0.022	No
Terminus-2 – Qwen 3.8 27B (xhigh)	28* (partial: attempt 1 on 36 of 565 tasks)	$0.051	Not plotted: partial coverage (36 of 147 Earth-Bench V2 tasks; GeoBenchX incomplete; GS-QA2 incomplete)
Claude Code – Claude Haiku 4.5 (thinking 32k)	49* (partial: attempt 1 on 453 of 565 tasks)	$0.099	Not plotted: partial coverage (102 of 147 Earth-Bench V2 tasks; 106 of 173 GeoBenchX tasks)
Claude Code – GLM-4.7-Flash (thinking on)	27* (partial: attempt 1 on 52 of 565 tasks)	$0.089	Not plotted: partial coverage (10 of 147 Earth-Bench V2 tasks; 42 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Claude Code – GLM-5.2 (max)	64* (partial: attempt 1 on 340 of 565 tasks)	$0.468	Not plotted: partial coverage (140 of 147 Earth-Bench V2 tasks; 165 of 173 GeoBenchX tasks; 35 of 245 GS-QA2 tasks)
Claude Code – GLM-5.3-Flash (max)	50* (partial: attempt 1 on 273 of 565 tasks)	$0.184	Not plotted: partial coverage (102 of 147 Earth-Bench V2 tasks; 171 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Claude Code – GLM-5.3 (max, default)	64* (partial: attempt 1 on 117 of 565 tasks)	$0.597	Not plotted: partial coverage (21 of 147 Earth-Bench V2 tasks; 96 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Codex – DeepSeek V4 Flash (high)	57* (partial: attempt 1 on 203 of 565 tasks)	$0.049	Not plotted: partial coverage (73 of 147 Earth-Bench V2 tasks; 101 of 173 GeoBenchX tasks; 29 of 245 GS-QA2 tasks)
Codex – DeepSeek V4 Pro (high)	42* (partial: attempt 1 on 48 of 565 tasks)	$0.745	Not plotted: partial coverage (8 of 147 Earth-Bench V2 tasks; 40 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Codex – gpt-oss-120b (high)	47* (partial: attempt 1 on 49 of 565 tasks)	$0.236	Not plotted: partial coverage (8 of 147 Earth-Bench V2 tasks; 41 of 173 GeoBenchX tasks; GS-QA2 incomplete)
Kimi Code – Kimi K2.6 (always on)	60* (partial: attempt 1 on 549 of 565 tasks)	$0.278	Not plotted: partial coverage (131 of 147 Earth-Bench V2 tasks)
Kimi Code – Kimi K2.7 Code (always on)	63* (partial: attempt 1 on 562 of 565 tasks)	$0.210	Not plotted: partial coverage (146 of 147 Earth-Bench V2 tasks; 243 of 245 GS-QA2 tasks)
Claude Code – Claude Opus 5.5 (medium)	71*	$0.118	No
Claude Code – Claude Opus 5.5 (high)	73*	$0.142	Not plotted: partial coverage (146 of 147 Earth-Bench V2 tasks)
Claude Code – Claude Sonnet 5.5 (medium)	63*	$0.048	No
Claude Code – Claude Sonnet 5.5 (high)	69*	$0.058	Not plotted: partial coverage (244 of 245 GS-QA2 tasks)
Claude Code – Claude Haiku 5.5 (medium)	63*	$0.0065	Yes
Claude Code – Claude Haiku 5.5 (high)	66*	$0.0089	Yes
Claude Code – Claude Opus 5.5 (xhigh)	74*	$0.283	Not plotted: partial coverage (146 of 147 Earth-Bench V2 tasks)
Claude Code – Claude Sonnet 5.5 (xhigh)	69*	$0.123	No
Claude Code – Claude Haiku 5.5 (xhigh)	67*	$0.018	Yes
Claude Code – Claude Opus 5 (high)	69* (partial: attempt 1 on 451 of 565 tasks)	$0.334	Not plotted: partial coverage (127 of 147 Earth-Bench V2 tasks; 79 of 173 GeoBenchX tasks)
Claude Code – Claude Sonnet 5 (high)	64* (partial: attempt 1 on 439 of 565 tasks)	$0.206	Not plotted: partial coverage (83 of 147 Earth-Bench V2 tasks; 111 of 173 GeoBenchX tasks)
Claude Code – Claude Opus 4.8 (high)	67* (partial: attempt 1 on 498 of 565 tasks)	$0.290	Not plotted: partial coverage (135 of 147 Earth-Bench V2 tasks; 119 of 173 GeoBenchX tasks; 244 of 245 GS-QA2 tasks)
Claude Code – Claude Opus 4.7 (high)	65* (partial: attempt 1 on 496 of 565 tasks)	$0.379	Not plotted: partial coverage (129 of 147 Earth-Bench V2 tasks; 122 of 173 GeoBenchX tasks)
Codex – GPT-6.1 Sol (medium)	72*	$0.049	Yes
Codex – GPT-6.1 Sol (high)	70*	$0.062	No
Codex – GPT-6.1 Sol (xhigh)	72*	$0.091	Yes
Codex – GPT-6 Luna (xhigh)	64*	$0.0076	Yes

Head-to-head comparisons

Specification and settings

Developer: Z.ai
Context window: 131,072 tokens
Image input: No
Reasoning setting: thinking on
Temperature: 0.6
Maximum output tokens: 65,536
Input price per 1M tokens: $0.06
Cached input price per 1M tokens: Not offered
Output price per 1M tokens: $0.40

Why this model was chosen: model selection.