Geospatial agent benchmark · not-downloaded

GeoNatureAgent Benchmark

Also known as: GeoNatureAgent

Evaluates: agents with tool access

Tool orchestration for environmental analysis, including rejection, multi-turn interaction, and cross-indicator synthesis.

At a glance

93Tasks
YesAgents run real code
YesData + code public
YesChecked by recomputation
underlying indicator d…Data licence

Sources

every claim on this page traces to these records

In shortAgents run real code inside this benchmark, and answers are checked by recomputation, not by another model's opinion. Dataset and code are published, so you can verify claims yourself. General geospatial competence outside the published API, production deployment, or rights to redistribute underlying indicator datasets.

Evidence boundaries

what a score on this benchmark does and does not tell you

A good score shows

Tool orchestration for environmental analysis, including rejection, multi-turn interaction, and cross-indicator synthesis.

A good score does not show

General geospatial competence outside the published API, production deployment, or rights to redistribute underlying indicator datasets.

Arena status

not-downloaded — Public paper, repository, and dataset reviewed; no Arena import or reproduction recorded.

All recorded facts (22 fields)
Evaluates
agent+tools
Task families
environmental indicators, administrative and vector operations, charts, multilingual and multi-turn tasks
Task count
93
Difficulty
multi-turn tool orchestration
Geography
Spain and Portugal
Modality
mixed
Input types
environmental API requests and spatial data
Output types
structured tool calls, text, numeric results, and charts
Execution environment
self-hostable API with structured tool calling
Ground truth method
benchmark cases with expected tool use and outputs
Verifier method
eight mechanistic checks per case, including expected tools, required or forbidden text, numeric tolerance, chart generation, and loop limits
Deterministic verification
yes
Scoring dimensions
mechanistic pass checks per case
Aggregation formula
per-case checks; aggregate method pending verification
Trials policy
pending verification
Variance reporting
pending verification
Data licence
underlying indicator data requires review
Code licence
Apache 2.0 (repository and API)
Contamination concerns
public benchmark cases and underlying public indicators
Reproducibility status
high (public repository, dataset, scorer, API, and reproduction documentation)
First published
2026-06
Latest known update
2026-06-11

Machine-readable record

This entry is available as JSON at /api/registry#geonatureagent. See the registry endpoint.