Alibaba's Qwen 3.8 27B, running in Alibaba's own coding agent Qwen Code, scores 69 out of 100 on Earth-Bench V2, our benchmark of AI agents analysing satellite and Earth observation data. Claude Opus 4.7 at high reasoning, in Anthropic's own coding agent Claude Code, also scores 69. Qwen 3.8 27B gets there at just over half of Claude Opus 4.7's cost per task.
Each model runs in its lab's own coding agent. Qwen 3.8 27B is an open weights model. Results as of 16 November 2026.
On Earth observation data, Qwen 3.8 27B in Qwen Code is level with Claude Opus 4.7
- Claude Opus 5.5 (max)Claude Code80
- Kimi K3 (Reasoning)Kimi Code75
- GPT-6 Sol (high)Codex75
- Kimi K2.7 Code (Reasoning)Kimi Code73
- Qwen 3.8 27B (Reasoning)Qwen Code69
- Claude Opus 4.7 (high)Claude Code69
- MiMo V2.6 Flash (Reasoning)MiMo Code68
- MiniMax M3 (Reasoning)MiniMax Code68
- Kimi K2.6 (Reasoning)Kimi Code66
- Qwen 3.6 35B-A3B (Reasoning)Terminus-262
- Qwen 3.6 35B-A3B (Reasoning)Qwen Code59
- Qwen 3.8 27B (xhigh)Terminus-254
- Claude Haiku 4.5 (Reasoning)Claude Code51
Which models we ran, and on what
We benchmark two Alibaba models, both open weights: Qwen 3.8 27B and Qwen 3.6 35B-A3B. Each runs in two harnesses. One is Qwen Code. The other is Terminus-2, Terminal-Bench's open harness, our default. Each entry gets one scored attempt per task, in a sandbox with no internet.
The headline score is the Geospatial Intelligence Index, a score out of 100 across three groups of work:
- Agent tasks (4 of 6 parts): Earth-Bench V2 (satellite and Earth observation data), GeoBenchX (GIS tasks that end in a map, a number or a list) and GS-QA2 (questions about places and terrain, answered by querying a database of OpenStreetMap features and elevation).
- Guided workflows (1 of 6): GeoAgentBench and GeoAnalystBench, GIS workflows where the task gives the steps and the model writes and runs the code.
- Geospatial knowledge (1 of 6): questions about GIS and Earth observation, answered with no tools.
41 models are ranked on the index. Qwen 3.8 27B scores 75.8, the highest of the Qwen models, and Qwen 3.6 35B-A3B scores 62.3.
Geospatial Intelligence Index, Text only (0 to 100, higher is better)
- Kimi K3 (Reasoning)81.7
- Claude Opus 4.7 (high)80.8
- DeepSeek V4 Pro (max)79.7
- Gemini 3.8 Flash (low)79.6
- GLM-5.2 (high)78.6
- Qwen 3.8 27B (xhigh)75.8
- Nemotron 3 Ultra (high)73.7
- Mistral Large 4 (high)71.3
- Step 5 Preview (Reasoning)64.6
- Gemma 4 26B A4B (Reasoning)64.4
- Qwen 3.6 35B-A3B (Reasoning)62.3
- MiniMax M3.1 Flash (max)61.3
Where Qwen 3.8 27B is strong
Earth observation data, in Qwen Code. On Earth-Bench V2 the agent works through satellite rasters: vegetation indices, land surface temperature, land cover. In Qwen Code, Qwen 3.8 27B scores 69. Claude Sonnet 5 at high reasoning in Claude Code scores 68 at the same cost per task. Gaps under about 3 points are within our noise.
Following a given workflow. On GeoAgentBench, where the steps are spelled out, it scores 94 in Qwen Code. Claude Opus 4.7 scores 96.
Knowledge. It scores 91 on geospatial knowledge, against 93 for Claude Opus 4.7. The best score of any model is 95.
The harness matters for Qwen
The same model scores differently in the two harnesses. In Qwen Code it scores 69 on Earth-Bench V2; in Terminus-2, 54. On GeoAgentBench the gap is 94 against 62. In the other direction, Terminus-2 scores higher on GeoBenchX, GS-QA2 and GeoAnalystBench, by 4 to 10 points.
Qwen 3.8 27B in Qwen Code against Qwen 3.8 27B in Terminus-2 (0 to 100)
Qwen Code (Reasoning)Terminus-2 (xhigh)
- Earth-Bench V26954
- GeoBenchX6367
- GS-QA27579
- GeoAgentBench9462
- GeoAnalystBench7989
The two entries do not run at the same reasoning setting: Qwen Code runs the model with reasoning on, and Terminus-2 at xhigh. So part of each gap may come from the setting rather than the harness. On the Agent Index, the average of the three agent benchmarks, the two come out close: 69 in Qwen Code and 67 in Terminus-2. Qwen Code costs nearly twice as much per task for those 2 points.
We cover what changes when only the harness changes in Same model, different harness.
Where it falls behind
Knowing when a task cannot be done. GeoBenchX mixes solvable tasks with tasks that cannot be done with the data given, and a correct answer to those is to say so. In Qwen Code, Qwen 3.8 27B answers 38% of the impossible tasks anyway. Claude Opus 4.7 answers 30%. On the solvable tasks Qwen scores 64 against 74, so its GeoBenchX total is 63 against 73.
Spatial questions over a database. On GS-QA2 it scores 75 in Qwen Code and 79 in Terminus-2. Claude Opus 4.7 scores 79. Gemini 3.8 Flash, the best entry in Terminus-2, scores 86.
Qwen 3.6 35B-A3B
Qwen 3.6 35B-A3B is the earlier of the two models. In Terminus-2 it scores 60 on the Agent Index, 7 points below Qwen 3.8 27B in the same harness, at about half its cost per task.
On Earth-Bench V2 in Terminus-2 it scores 62, ahead of Qwen 3.8 27B's 54 in that harness. It is weaker on GIS work: 51 on GeoBenchX, where it answers 63% of the impossible tasks. On the index it scores 56 on guided workflows, against 86 for Qwen 3.8 27B.
What it costs
Cost per task is what one attempt would cost at the provider's list prices, failed attempts included.
- On Earth-Bench V2, Qwen 3.8 27B in Qwen Code costs just over half of Claude Opus 4.7 for the same score, and the same as Claude Sonnet 5 for 1 point more.
- On the index, it costs about two thirds of Claude Opus 4.7 per agent task, 5 points behind it (75.8 against 80.8).
- In Terminus-2 it costs a little over half of what it costs in Qwen Code.
Both Qwen models are open weights, so you can also run them on your own servers. Our costs use the list prices of hosted providers; on your own hardware the cost depends on that hardware.
What this means if you are choosing a model
- Satellite and Earth observation analysis on an open model: Qwen 3.8 27B in Qwen Code is level with Claude Opus 4.7 in Claude Code, at just over half the cost. Put it on your shortlist.
- Choosing the harness: run Qwen 3.8 27B in both Qwen Code and your own harness on your tasks. The gap between them is large on some benchmarks and goes both ways.
- Tasks where a wrong answer is worse than none: Qwen answers more impossible GeoBenchX tasks than Claude Opus 4.7. Add a check before its answers are used.
Every score for every model is on the Geospatial Intelligence Index. If you want to know how these models do on a task your team runs today, send it to us and we will benchmark it.
