Codex – gpt-oss-20b (high): geospatial agent results

What this shows. The same attempts as the index, grouped by task format. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by kind of work. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by data type. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by task length. Groups with few tasks are less certain. About these breakdowns.

What this metric means. The equal-weight average of the 3 evaluation scores. Each evaluation score is the average pass@1 over that benchmark's tasks, from 0 to 100.

Partial coverage entries. An entry is ranked only when it has a first attempt on every task of every index component. Entries without that are marked partial coverage: they are listed apart with their coverage, for example "36 of 147 Earth-Bench V2 tasks", and are left out of the ranking, the index charts, the Pareto lines and the highlights. An entry becomes ranked as soon as its first attempts are complete.

Coverage hatch. A hatched bar, and an asterisk in tables, marks an entry that does not yet have results for every planned attempt. While attempts await review, the table gives the range the score could still reach.

First attempt only. Scores currently use each entry's first attempt on every task. The full protocol is 3 attempts; averages over 3 will replace these once every entry has them. An entry without a first attempt on every task yet is marked "partial: attempt 1 on n of N tasks".

Not shown: partial coverage or being added. Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – DeepSeek V4 Flash (max) (242 of 245 GS-QA2 tasks); Terminus-2 – DeepSeek V4 Flash (high) (236 of 245 GS-QA2 tasks); Terminus-2 – DeepSeek V4 Pro (high) (239 of 245 GS-QA2 tasks); Terminus-2 – Gemini 3.1 Flash-Lite (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Flash-Lite (minimal) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Flash-Lite (high) (35 of 147 Earth-Bench V2 tasks; 170 of 173 GeoBenchX tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Pro (preview) (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.1 Pro (preview) (high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.8 Flash (high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemini 3.8 Flash (low) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Terminus-2 – Gemma 4 26B A4B (thinking on) (210 of 245 GS-QA2 tasks); Terminus-2 – GLM-4.7-Flash (thinking on) (235 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.2 (high) (244 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.2 (max) (238 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3-Flash (low) (244 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3-Flash (max) (200 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3 (low) (241 of 245 GS-QA2 tasks); Terminus-2 – GLM-5.3 (max, default) (240 of 245 GS-QA2 tasks); Terminus-2 – gpt-oss-120b (high) (241 of 245 GS-QA2 tasks); Terminus-2 – gpt-oss-20b (high) (172 of 173 GeoBenchX tasks; 230 of 245 GS-QA2 tasks); Terminus-2 – Kimi K2.6 (always on) (242 of 245 GS-QA2 tasks); Terminus-2 – Kimi K2.7 Code (always on) (240 of 245 GS-QA2 tasks); Terminus-2 – Qwen 3.8 27B (xhigh) (36 of 147 Earth-Bench V2 tasks; GeoBenchX incomplete; GS-QA2 incomplete); Claude Code – GLM-4.7-Flash (thinking on) (127 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Claude Code – GLM-5.2 (max) (140 of 147 Earth-Bench V2 tasks; 165 of 173 GeoBenchX tasks); Claude Code – GLM-5.3-Flash (max) (141 of 147 Earth-Bench V2 tasks; 171 of 173 GeoBenchX tasks; 185 of 245 GS-QA2 tasks); Claude Code – GLM-5.3 (max, default) (66 of 147 Earth-Bench V2 tasks; 171 of 173 GeoBenchX tasks; GS-QA2 incomplete); Codex – DeepSeek V4 Flash (high) (73 of 147 Earth-Bench V2 tasks; 101 of 173 GeoBenchX tasks; 237 of 245 GS-QA2 tasks); Codex – DeepSeek V4 Pro (high) (8 of 147 Earth-Bench V2 tasks; 40 of 173 GeoBenchX tasks; GS-QA2 incomplete); Codex – gpt-oss-120b (high) (36 of 147 Earth-Bench V2 tasks; GS-QA2 incomplete); Codex – gpt-oss-20b (high) (32 of 147 Earth-Bench V2 tasks; 141 of 173 GeoBenchX tasks; GS-QA2 incomplete); Kimi Code – Kimi K2.6 (always on) (131 of 147 Earth-Bench V2 tasks); Kimi Code – Kimi K2.7 Code (always on) (146 of 147 Earth-Bench V2 tasks; 243 of 245 GS-QA2 tasks); Claude Code – Claude Opus 5.5 (high) (146 of 147 Earth-Bench V2 tasks); Claude Code – Claude Sonnet 5.5 (high) (244 of 245 GS-QA2 tasks); Claude Code – Claude Opus 5.5 (xhigh) (146 of 147 Earth-Bench V2 tasks); Claude Code – Claude Opus 5 (high) (146 of 147 Earth-Bench V2 tasks; 162 of 173 GeoBenchX tasks); Claude Code – Claude Opus 4.8 (high) (244 of 245 GS-QA2 tasks).

Partial coverage. Not plotted and not on the Pareto line; listed in the table: Terminus-2 – Claude Sonnet 4.6 (adaptive, effort high), Terminus-2 – DeepSeek V4 Flash (max), Terminus-2 – DeepSeek V4 Flash (high), Terminus-2 – DeepSeek V4 Pro (high), Terminus-2 – Gemini 3.1 Flash-Lite (low), Terminus-2 – Gemini 3.1 Flash-Lite (minimal), Terminus-2 – Gemini 3.1 Flash-Lite (high), Terminus-2 – Gemini 3.1 Pro (preview) (low), Terminus-2 – Gemini 3.1 Pro (preview) (high), Terminus-2 – Gemini 3.8 Flash (high), Terminus-2 – Gemini 3.8 Flash (low), Terminus-2 – Gemma 4 26B A4B (thinking on), Terminus-2 – GLM-4.7-Flash (thinking on), Terminus-2 – GLM-5.2 (high), Terminus-2 – GLM-5.2 (max), Terminus-2 – GLM-5.3-Flash (low), Terminus-2 – GLM-5.3-Flash (max), Terminus-2 – GLM-5.3 (low), Terminus-2 – GLM-5.3 (max, default), Terminus-2 – gpt-oss-120b (high), Terminus-2 – gpt-oss-20b (high), Terminus-2 – Kimi K2.6 (always on), Terminus-2 – Kimi K2.7 Code (always on), Terminus-2 – Qwen 3.8 27B (xhigh), Claude Code – GLM-4.7-Flash (thinking on), Claude Code – GLM-5.2 (max), Claude Code – GLM-5.3-Flash (max), Claude Code – GLM-5.3 (max, default), Codex – DeepSeek V4 Flash (high), Codex – DeepSeek V4 Pro (high), Codex – gpt-oss-120b (high), Codex – gpt-oss-20b (high), Kimi Code – Kimi K2.6 (always on), Kimi Code – Kimi K2.7 Code (always on), Claude Code – Claude Opus 5.5 (high), Claude Code – Claude Sonnet 5.5 (high), Claude Code – Claude Opus 5.5 (xhigh), Claude Code – Claude Opus 5 (high), Claude Code – Claude Opus 4.8 (high).

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one model configuration. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.