Qwen 3.8 27B (xhigh), Qwen Code: geospatial agent results

What this shows. The same attempts as the index, grouped by task format. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by kind of work. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by data type. Groups with few tasks are less certain. About these breakdowns.

What this shows. The same attempts as the index, grouped by task length. Groups with few tasks are less certain. About these breakdowns.

What this metric means. The equal average of the three text evaluations (Earth-Bench V2, GeoBenchX and GS-QA2). The Text + vision view adds Vision agents. Each evaluation score is the share of its tasks passed, 0 to 100.

Not yet ranked. An entry is ranked only when it has an attempt on every task of every index evaluation. Entries without that are listed as not yet ranked, with the tasks attempted, for example "36 of 117 Earth-Bench V2 tasks", and are left out of the ranking, the index charts and the Pareto lines.

Stripes. A striped bar marks an entry that has not attempted every task yet, so its score may still change (methodology).

One attempt per task. Scores use one attempt per task. An entry without an attempt on every task is marked with the tasks attempted, for example "177 of 488 tasks".

Not yet ranked. Entries without an attempt on every index task are not plotted; the table lists them. What is still to run is on the models page.

What cost is measuring. What one attempt at a task would cost if paid token by token at the provider's published API prices, averaged over attempts: ordinary input tokens, input tokens served from the provider's cache at the lower cached price, tokens written to the cache where the provider charges for that, and output tokens (thinking included). Cost assumes each harness uses the model's pay-per-token API, priced at public list rates; sandboxes and our own engineering time are not included. Failed attempts count too.

How to read this chart. Each point is one entry. Further left means a lower average cost per task (log scale); higher means a higher Geospatial Agent Index. Both axes fit the models shown, so the score axis does not start at zero. The shaded most attractive quadrant is the top-left quarter of the plot, split at the middle of each axis: a reading aid that moves with the axes when you change the models shown, not a fixed bar. The dotted Pareto line joins the shown models that no cheaper shown model beats on score.