Geospatial AI benchmarks

Geospatial AI benchmarks we run

As of 2 November 2026, we run 14 geospatial AI benchmarks and datasets, each under the same setup for every model. Every evaluation has its scores, tasks and licence.

• Earth-Bench V2: Earth observation data analysis: the agent computes indices, temperatures and land-cover figures from satellite products to answer each question. Source: OpenDataLab. Papers: Earth-Agent (arXiv 2509.23141) and Earth-Agent-Pro (arXiv 2609.12533); dataset LvZTTTT/Earth-Bench-V2.
• GeoBenchX: Maps, numbers and lists from country and city data, including tasks that cannot be done with the data given, where the right answer is to say so. Source: Krechetova and Kochedykov (2025), Solirinai/GeoBenchX.
• GS-QA2: Questions about places and terrain in the United States, answered from map and elevation data in a spatial database. Source: Shang et al., University of California, Riverside. Paper: GS-QA (arXiv 2605.22811); dataset Zoe/GS-QA2.
• ThinkGeo: Satellite and aerial images with questions that ask to find, count and measure objects: with code tools in Vision agents, and in one model call on Vision. Source: ThinkGeo (MBZUAI), dataset MBZUAI/ThinkGeo.
• OpenEarthAgent: Earth observation tasks on satellite images, where the agent uses code tools to find, measure and compare what the question asks. Source: OpenEarthAgent (MBZUAI), paper arXiv 2602.17665, dataset MBZUAI/OpenEarthAgent.
• GeoAgentBench: GeoAgentBench gives an agent a geospatial question and the data to answer it, and expects the result files and a map. Source: GeoX-Lab.
• GeoAnalystBench: GeoAnalystBench collects GIS analyst workflows (suitability analysis, interpolation, hot spots, network and overlay work). Source: GeoDS Lab.
• EVE question sets: Closed-book multiple-choice questions on GIS and Earth observation, one model call each, no tools. Source: ESA Φ-lab's EVE (Earth Virtual Expert) project; Atrio et al., "EVE: A Domain-Specific LLM Framework for Earth Intelligence", 2026; eve-esa question sets.
• OGD4All: OGD4All asks everyday questions about the City of Zurich's open government data: trees, buildings, transport, schools and more, from about 430 public datasets in GeoPackage and CSV form. Source: the OGD4All authors (ETH Zurich, Computational Social Science).
• GeoNatureAgent: GeoNatureAgent asks for figures from environmental rasters over Spanish provinces and regions: here, the share of land suitable for CO₂ storage, by province, with follow-up questions that compare, rank or break the figures down. Source: Diaz-Ireland, Prieto-Herraez, Garcia Peces, Velazquez and Jain.
• MultiGlobeQA: MultiGlobeQA asks spatial questions about places all over the world: what lies within, next to or nearest to what, how far apart things are, and how many there are. Source: Böckling, Nosova, Paulheim and Iana.
• GeoAgentEval: GeoAgentEval tests whether an agent can build, audit and repair spatial data in a PostGIS database, and decline honestly when a request cannot be done: flood zones, wards, routes, coordinate systems and stale records in one made-up town. Source: Harsh Dange.
• GeoChrono: GeoChrono asks about long-term change in a place seen over many years: what the land cover is, whether an area grew or shrank, when an object appeared and how things changed in order. Source: the GeoChrono authors (IntelliSensing).
• BigEarthNet.txt: BigEarthNet.txt asks about 1.2 km image patches over Europe: which land cover is present, how much, how many patches of it, what is next to what, and the country, season and climate zone. Source: Herzog et al. (BIFOLD and TU Berlin).

How we benchmark

• Every model gets the same real GIS and Earth observation tasks, data and limits.
• Each task is scored the same way for every model, on one attempt.
• Costs are at public API list prices, and every change to the method is in its version history.

The methodology