Geospatial knowledge (closed book)
What a model knows about geospatial (GIS and Earth observation) work, with no tools and no internet: 1,685 multiple-choice questions, built on ESA's published question sets. It is the knowledge part of the Earth science group of the Geospatial Intelligence Index, and is not part of the Geospatial Agent Index. Earth science: knowledge now, reasoning coming.
Ongoing evaluation: results update as runs complete. Last updated 12 October 2026.
Score
| Model | Score | Questions scored | Not scored (provider error) | Status | Cost per question | Time per question |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 (high) | 95 | 1,685 of 1,685 | 0 | Counted | $0.0073 | 10s |
| Claude Opus 5.5 (xhigh) | 94 | 1,685 of 1,685 | 0 | Counted | $0.0084 | 10s |
| Claude Opus 5.5 (medium) | 94 | 1,685 of 1,685 | 0 | Counted | $0.012 | 14s |
| Claude Sonnet 5.5 (xhigh) | 94 | 1,685 of 1,685 | 0 | Counted | $0.0042 | 4s |
| Claude Sonnet 5.5 (high) | 94 | 1,685 of 1,685 | 0 | Counted | $0.0033 | 4s |
| Claude Sonnet 5.5 (medium) | 93 | 1,685 of 1,685 | 0 | Counted | $0.0054 | 3s |
| GPT-6.1 Sol (high) | 93 | 1,685 of 1,685 | 0 | Counted | $0.0019 | 13s |
| GPT-6.1 Sol (xhigh) | 93 | 1,685 of 1,685 | 0 | Counted | $0.0037 | 20s |
| GPT-6.1 Sol (medium) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0017 | 16s |
| DeepSeek V4 Pro (high) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0051 | 22s |
| Kimi K2.7 Code (always on) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0038 | 20s |
| Claude Haiku 5.5 (high) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0003 | 4s |
| Claude Haiku 5.5 (xhigh) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0004 | 4s |
| Kimi K2.6 (always on) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0086 | 47s |
| GLM-5.3 (max, default) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0100 | 35s |
| Claude Haiku 5.5 (medium) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0004 | 4s |
| GLM-5.3-Flash (max) | 92 | 1,685 of 1,685 | 0 | Counted | $0.0010 | 51s |
| DeepSeek V4 Flash (max) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0027 | 34s |
| GLM-5.2 (high) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0052 | 14s |
| GPT-6 Luna (xhigh) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0001 | 14s |
| DeepSeek V4 Flash (high) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0013 | 22s |
| GLM-5.3 (low) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0009 | 13s |
| Qwen 3.8 27B (xhigh) | 91 | 1,685 of 1,685 | 0 | Counted | $0.0037 | 46s |
| gpt-oss-120b (high) | 90 | 1,685 of 1,685 | 0 | Counted | $0.0011 | 19s |
| GLM-5.2 (none) | 89 | 1,685 of 1,685 | 0 | Counted | $0.0013 | 10s |
| GLM-5.2 (max) | 89 | 1,685 of 1,685 | 0 | Counted | $0.021 | 40s |
| Gemma 4 26B A4B (thinking on) | 89 | 1,685 of 1,685 | 0 | Counted | $0.0010 | 54s |
| gpt-oss-120b (low) | 88 | 1,685 of 1,685 | 0 | Counted | $0.0003 | 2s |
| GLM-4.7-Flash (thinking on) | 86 | 1,685 of 1,685 | 0 | Counted | $0.0008 | 52s |
| GLM-5.3-Flash (low) | 85 | 1,684 of 1,685 | 1 | Counted | $0.0001 | 38s |
| Mistral Small 3.1 24B (Non-reasoning) | 82 | 1,685 of 1,685 | 0 | Counted | $0.0003 | 7s |
| Claude Sonnet 4.6 (adaptive, effort high) | 98* | 840 of 1,685 | 0 | Stopped part-way | $0.014 | 13s |
| Claude Opus 4.6 (adaptive, effort high) | 98* | 946 of 1,685 | 0 | Stopped part-way; not in the Intelligence Index yet | $0.014 | 10s |
An asterisk marks a quiz that does not count yet. No quiz result: Kimi K3 (always on), Llama 3.3 70B (Non-reasoning), Gemini 3.8 Flash (low), Gemini 3.1 Pro (preview) (low), Gemini 3.1 Pro (preview) (high), Gemini 3.1 Flash-Lite (low), Gemini 3.8 Flash (high), Gemini 3.1 Flash-Lite (minimal), Claude Opus 5 (high), Claude Sonnet 5 (high), Claude Opus 4.7 (high), Claude Opus 4.8 (high), Gemini 3.1 Flash-Lite (high), Claude Haiku 4.5 (thinking 32k), gpt-oss-20b (high).
How it is run
- 1,255 questions have one correct option and 430 have one or more; a multiple-answer question is correct only when the model names exactly the correct set.
- One model call per question, at the model's index reasoning level. The model may reason, then ends with its answer; code checks the option letters. No judge.
- 1 attempt per question. A question a provider error left unanswered is asked again. A quiz counts in the index only when every question was asked and at most 1% are left unscored by provider errors; those are left out of the score.
- The score is the percentage of scored questions answered correctly. Cost is the reported tokens at the public API list price.
- Questions: the multiple-choice quiz sets of ESA Φ-lab's EVE (Earth Virtual Expert) project, eve-esa/mcqa-single-answer and eve-esa/mcqa-multiple-answers on Hugging Face, CC BY 4.0 (Atrio et al., "EVE: A Domain-Specific LLM Framework for Earth Intelligence", 2026). We dropped 7 items, each with its reason, and changed nothing else; the prompt wording around each question is ours. This site publishes scores only.
- The questions are all Earth observation questions. The quiz measures recall of published Earth observation knowledge, not the quality of the source project's own model.
- Methodology.