StepFun's Step 5 Preview scores 79 out of 100 on GS-QA2, our benchmark of AI agents answering questions about places and terrain. Qwen 3.8 27B at xhigh reasoning and Mistral Large 4 at high reasoning, in the same harness, also score 79.
The harness is Terminus-2, Terminal-Bench's open harness, our default. The best entry there, Gemini 3.8 Flash at low reasoning, scores 86. Results as of 16 November 2026.
On spatial questions, Step 5 Preview is level with Qwen 3.8 27B in the same harness
- Gemini 3.8 Flash (low)Terminus-286
- GLM-5.3 (max)Terminus-283
- Kimi K2.7 Code (Reasoning)Terminus-282
- DeepSeek V4 Pro (max)Terminus-280
- Kimi K3 (Reasoning)Terminus-280
- Qwen 3.8 27B (xhigh)Terminus-279
- Step 5 Preview (Reasoning)Terminus-279
- Mistral Large 4 (high)Terminus-279
- MiniMax M3.1 Flash (max)Terminus-271
- Nemotron 3 Ultra (high)Terminus-270
- Muse Glimmer 30B (high)Terminus-270
- Gemma 4 31B (Reasoning)Terminus-261
- gpt-oss-120b (high)Terminus-242
Which model we ran, and on what
We benchmark one StepFun model, Step 5 Preview, with reasoning on. It is open weights. StepFun has no coding agent of its own, so it runs in Terminus-2, with one scored attempt per task in a sandbox with no internet.
The headline score is the Geospatial Intelligence Index, a score out of 100 across three groups of work:
- Agent tasks (4 of 6 parts): Earth-Bench V2 (satellite and Earth observation data), GeoBenchX (GIS tasks that end in a map, a number or a list) and GS-QA2 (questions about places and terrain, answered by querying a database of OpenStreetMap features and elevation).
- Guided workflows (1 of 6): GeoAgentBench and GeoAnalystBench, GIS workflows where the task gives the steps and the model writes and runs the code.
- Geospatial knowledge (1 of 6): questions about GIS and Earth observation, answered with no tools.
41 models are ranked on the index. Step 5 Preview scores 64.6. One benchmark pulls that down a long way, which we come to below.
Where Step 5 Preview is strong
Spatial questions over a database. On GS-QA2 the agent answers questions such as the nearest park to a landmark or the slope at a point, by writing queries against a PostGIS database. Step 5 Preview's 79 is level with Qwen 3.8 27B and Mistral Large 4, and one point behind Kimi K3 and DeepSeek V4 Pro at max reasoning. Gaps under about 3 points are within our noise.
Following a given workflow. On GeoAgentBench it scores 84, against 62 for Qwen 3.8 27B and 77 for Mistral Large 4 in the same harness. On GeoAnalystBench it scores 89, the same as Qwen. Its guided workflow score of 87 puts it ahead of Qwen 3.8 27B and Mistral Large 4, at 86 each.
Guided workflows, the part of the Intelligence Index from GeoAgentBench and GeoAnalystBench (0 to 100)
- Kimi K3 (Reasoning)98
- DeepSeek V4 Pro (max)98
- GLM-5.2 (high)97
- DeepSeek V4 Flash (high)92
- Step 5 Preview (Reasoning)87
- Qwen 3.8 27B (xhigh)86
- Mistral Large 4 (high)86
- Nemotron 3 Ultra (high)80
- Gemma 4 26B A4B (Reasoning)75
- MiniMax M3.1 Flash (max)68
- gpt-oss-120b (high)50
Earth observation data. On Earth-Bench V2 it scores 60, ahead of Qwen 3.8 27B (54) and Mistral Large 4 (50) in Terminus-2, and level with Muse Glimmer 30B.
Knowledge. It scores 92 on geospatial knowledge, against 91 for Qwen 3.8 27B and Mistral Large 4. The best score of any model is 95.
Step 5 Preview against Qwen 3.8 27B, both in Terminus-2 (0 to 100)
Step 5 Preview (Reasoning)Qwen 3.8 27B (xhigh)
- Earth-Bench V26054
- GeoBenchX1867
- GS-QA27979
- GeoAgentBench8462
- GeoAnalystBench8989
Where it falls behind: GeoBenchX
On GeoBenchX Step 5 Preview scores 18. Qwen 3.8 27B scores 67 and Mistral Large 4 59 in the same harness. GeoBenchX tasks end in a map, a number or a list built from GIS data, and some of them cannot be done with the data given, where a correct answer is to say so.
Step 5 Preview scores 14 on the solvable tasks. On the impossible ones it scores 23: it answers 5% of them anyway, and on most of the rest it neither answers nor says the task cannot be done. Its other four benchmark scores sit between 60 and 89, so this one is far out of line.
The same harness and the same tasks give Qwen 3.8 27B a score of 67, so the tasks themselves are not out of reach for an open model in Terminus-2. We have not yet traced why. Until we have, we do not know how much of the 18 is the model and how much is the run. If GeoBenchX-style GIS work is what you need, run Step 5 Preview on your own tasks before relying on this score either way.
GeoBenchX is one of three agent benchmarks, and agent tasks are 4 of 6 parts of the index. That is why Step 5 Preview scores 52 on agent tasks and 64.6 on the index, below models it matches or beats on the other benchmarks.
What it costs
Cost per task is what one attempt would cost at the provider's list prices, failed attempts included.
- On GS-QA2, where the three are level, Step 5 Preview costs about a fifth more than Qwen 3.8 27B and Mistral Large 4.
- On Earth-Bench V2 it costs about a third more than Qwen 3.8 27B in the same harness, for 6 points more.
- Across all agent tasks it costs about the same as Qwen 3.8 27B in Terminus-2, and about a third more than Mistral Large 4.
Step 5 Preview is open weights, so you can also run it on your own servers. Our costs use the list prices of hosted providers; on your own hardware the cost depends on that hardware.
What this means if you are choosing a model
- Questions about places, terrain and spatial databases on an open model: Step 5 Preview is level with Qwen 3.8 27B and Mistral Large 4. Put it on your shortlist.
- GIS workflows where you give the steps: it is ahead of Qwen 3.8 27B and Mistral Large 4 on guided workflows.
- Open-ended GIS tasks like GeoBenchX: its score there is far below its others and not yet explained. Run it on your own tasks first.
Every score for every model is on the Geospatial Intelligence Index. If you want to know how Step 5 Preview does on a task your team runs today, send it to us and we will benchmark it.
