Skip to main content

Lab results · Z.ai

How Z.ai's GLM models do on geospatial work

We benchmarked four Z.ai GLM models on GIS and Earth observation work, in two harnesses. Here is where they are strong, where they fall behind, and what they cost.

Research

Z.ai's GLM-5.3 scores 77 out of 100 on Earth-Bench V2, our benchmark of AI agents analysing satellite and Earth observation data. It runs at max reasoning in Claude Code, Anthropic's coding agent. In the same harness at the same level, Claude Opus 5.5 scores 80. Only Claude Opus 5.5, at max and at high reasoning, scores more than GLM-5.3 on this benchmark.

GLM-5.3 gets its 77 at about three fifths of Claude Opus 5.5's cost per task at max reasoning. It is an open weights model, so you can also run it on your own servers. Results as of 16 November 2026.

On Earth observation data, GLM-5.3 is in the top group in Claude Code

  1. Claude Opus 5.5 (max)Claude Code80
  2. Claude Opus 5.5 (high)Claude Code79
  3. GLM-5.3 (max)Claude Code77
  4. DeepSeek V4.1 Flash (max)Claude Code77
  5. Claude Opus 5.5 (xhigh)Claude Code77
  6. Kimi K3 (Reasoning)Kimi Code75
  7. GPT-6 Sol (high)Codex75
  8. GLM-5.2 (max)Claude Code71
  9. GLM-5.3-Flash (max)Claude Code69
  10. Qwen 3.8 27B (Reasoning)Qwen Code69
  11. MiMo V2.6 Flash (Reasoning)MiMo Code68
  12. MiniMax M3 (Reasoning)MiniMax Code68
  13. GLM-5.3 (max)Terminus-267
  14. GLM-4.7-Flash (Reasoning)Claude Code38
Earth-Bench V2 score, entries in coding agents, with every Z.ai model and GLM-5.3 in Terminus-2. Results as of 16 November 2026. Source: Earth-Bench V2.

Which models we ran, and on what

We benchmark four Z.ai models, all open weights: GLM-5.3, GLM-5.2, GLM-5.3-Flash and GLM-4.7-Flash. Each runs in two harnesses. One is Terminus-2, Terminal-Bench's open harness, our default. Z.ai has no coding agent of its own in our index, so the other is Claude Code, Anthropic's coding agent. Each entry gets one scored attempt per task, in a sandbox with no internet.

The headline score is the Geospatial Intelligence Index, a score out of 100 across three groups of work:

  • Agent tasks (4 of 6 parts): Earth-Bench V2 (satellite and Earth observation data), GeoBenchX (GIS tasks that end in a map, a number or a list) and GS-QA2 (questions about places and terrain, answered by querying a database of OpenStreetMap features and elevation).
  • Guided workflows (1 of 6): GeoAgentBench and GeoAnalystBench, GIS workflows where the task gives the steps and the model writes and runs the code.
  • Geospatial knowledge (1 of 6): questions about GIS and Earth observation, answered with no tools.

41 models are ranked on the index. GLM-5.2 at high reasoning scores 78.6 and GLM-5.3 at max reasoning 78.4, within noise of each other and of DeepSeek V4 Pro (79.7). These index scores use the Terminus-2 entries.

Geospatial Intelligence Index, Text only (0 to 100, higher is better)

  1. Claude Opus 5.5 (xhigh)88.7
  2. Kimi K3 (Reasoning)81.7
  3. DeepSeek V4 Pro (max)79.7
  4. GLM-5.2 (high)78.6
  5. GLM-5.3 (max)78.4
  6. Qwen 3.8 27B (xhigh)75.8
  7. Nemotron 3 Ultra (high)73.7
  8. GLM-5.3-Flash (low)72.1
  9. Mistral Large 4 (high)71.3
  10. MiniMax M3.1 Flash (max)61.3
  11. gpt-oss-120b (high)51.9
  12. GLM-4.7-Flash (Reasoning)49.6
Every Z.ai model, with the leader and the best open weights model from other labs near it. Each model at its best ranked level. Results as of 16 November 2026. Source: Geospatial Intelligence Index.

Where GLM is strong

Earth observation data, in Claude Code. GLM-5.3 at 77 is level with DeepSeek V4.1 Flash and with Claude Opus 5.5 at xhigh reasoning, all three in Claude Code. It is ahead of GPT-6 Sol in Codex (75) and of every other open weights model in any harness.

Guided workflows. When the task spells out the steps, GLM models are close to the top. GLM-5.3 in Claude Code scores 98 on GeoAgentBench and 95 on GeoAnalystBench, against 100 and 94 for Claude Opus 5.5 at max. GLM-5.2 at high reasoning in Terminus-2 scores 100 on GeoAnalystBench, and 97 for the guided workflow group as a whole, the best of any Z.ai model on the index.

The Flash model, for cost. GLM-5.3-Flash at max reasoning in Claude Code scores 72 on the Geospatial Agent Index, 2 points below GLM-5.3 in the same harness, at about a fifth of its cost per task. On GeoBenchX it scores 71, ahead of GLM-5.3's 65.

GLM-5.3 against Claude Opus 5.5, both at max reasoning in Claude Code (0 to 100)

GLM-5.3 (max)Claude Opus 5.5 (max)

  1. Earth-Bench V27780
  2. GeoBenchX6584
  3. GS-QA28194
  4. GeoAgentBench98100
  5. GeoAnalystBench9594
The three agent benchmarks, then the two guided workflow benchmarks. Same harness, same reasoning level. Results as of 16 November 2026. Source: GLM-5.3 (max) in Claude Code, every benchmark.

Where it falls behind

GIS tasks with a trap in them. GLM-5.3 scores 65 on GeoBenchX, 19 points behind Claude Opus 5.5 at max. GeoBenchX mixes solvable tasks with tasks that cannot be done with the data given, and the right answer to those is to say so. GLM-5.3 in Claude Code answers 40% of the impossible tasks anyway; Claude Opus 5.5 answers 15%. It is also behind on the solvable tasks, 69 against 84. The other GLM entries answer between 27% and 55% of them, depending on model, harness and level; GLM-5.3-Flash in Claude Code is the most careful, at 27%.

Spatial questions over a database. On GS-QA2, GLM-5.3 in Claude Code scores 81, against 94 for Claude Opus 5.5 at max. This is the benchmark where the closed models lead most clearly, and where GLM-5.3 does a little better in Terminus-2 (83).

Claude Code or Terminus-2

The harness changes GLM's result most on Earth observation work:

Model (reasoning level)Earth-Bench V2: Claude Code / Terminus-2Agent Index: Claude Code / Terminus-2
GLM-5.3 (max)77 / 6774 / 72
GLM-5.2 (max)71 / 6874 / 72
GLM-5.3-Flash (max)69 / 5172 / 63
GLM-4.7-Flash (Reasoning)38 / 4441 / 43

For GLM-5.3 and GLM-5.2 the index moves by 2 points, which is within noise. Earth-Bench V2 moves by 10 points for GLM-5.3 and 18 for GLM-5.3-Flash, both in favour of Claude Code. The older GLM-4.7-Flash does slightly better in Terminus-2 and costs about 7 times as much per task in Claude Code. For GLM-5.3, Claude Code costs about a quarter more per task than Terminus-2. We compare harnesses for every open model in Same model, different harness.

What it costs

Cost per task is what one attempt would cost at the provider's list prices, failed attempts included.

  • GLM-5.3 at max in Claude Code costs about two fifths of Claude Opus 5.5 at max per task on the Agent Index, for 12 points less (74 against 86).
  • On Earth-Bench V2 it costs about three fifths of Claude Opus 5.5 at max, for 3 points less. Claude Opus 5.5 at xhigh scores the same 77 at about two fifths of GLM-5.3's cost, so at that level the closed model is the cheaper way to the same score.
  • GLM-5.3-Flash in Claude Code scores 69 on Earth-Bench V2, level with Claude Opus 4.7 at high reasoning in the same harness, at about a third of its cost per task.

What this means if you are choosing a model

  • Earth observation analysis on an open model: GLM-5.3 in Claude Code is level with the best open model and with Claude Opus 5.5 at xhigh. Run it in Claude Code, not only in a general-purpose harness.
  • GIS work with incomplete or wrong data: check how often the model answers a task it should refuse. GLM models do this more often than Claude Opus 5.5.
  • A lower bill: GLM-5.3-Flash keeps most of GLM-5.3's agent score at about a fifth of its cost per task.

Every score for every model is on the Geospatial Intelligence Index. If you want to know how these models do on a task your team runs today, send it to us and we will benchmark it.