Tasks that show where models misread the physical world
We write private evaluation tasks on real satellite, radar and lidar data, each with a verifier. Labs use them to measure what public benchmarks do not cover.
Key findings
Frontier models know the Earth better than they see it: 95 of 100 on knowledge questions, 59 on reading images.
- Geospatial agent work86Best agent on GIS and Earth observation work, in a sandbox with tools.
- Vision59Best score on questions about satellite and aerial images, with no tools.
- Geospatial knowledge95Best score on closed-book questions about places, maps and coordinates.
- Scope48 modelsFrom 14 labs, on the same tasks.
On our private sample tasks, weaker models got 0 of 5 attempts right and stronger models 1 of 3, with web search and tools on. They guess the place and skip reading the picture.
The public index shows how the method reads across the major models. Your engagement is the private version: tasks that stay unpublished, built for the gap you name.
Three ways to work with us
Ready-made task sets
Tasks we have already written and measured models on.
Custom evaluations
Tasks built for the gap you name.
Partner benchmark
Built with a lab or an industry partner and published under both names on the index.
Task areas
Change in satellite imagery
What was built, cleared or flooded between two dates, and how much.
Beside DVL-Bench
Radar (SAR)
Reading Sentinel-1 scenes: slicks, water and structures that optical images miss.
Beside InSAR Regional Snow Mapping
Lidar and terrain
Earthworks, slopes and volumes from 1 m elevation models.
Beside GS-QA2
Monitoring over time
When something started, how fast it grew and when it stopped, from a series of scenes.
Beside GeoChrono
One sample task

The picture shows a port with four numbered terminals, and only one handles containers. The model must pick it from the pixels, look up its quay length and the port's container traffic in public records, and work out the boxes handled per metre of quay per day.
Weaker models got 0 of 5 attempts right and stronger models 1 of 3, with web search and tools on.
How it works
- Specify. You name the work and where your model fails.
- Measure. We run current models on private tasks with verifiers and report where each one fails.
- Improve. We write task sets and environments aimed at those failures. Evaluation tasks and any training data are kept separate.
- Re-measure. We run the evaluation again, on tasks the model has not seen.
