EVE hallucination detection benchmark leaderboard
Deciding whether an answer about Earth observation contains a hallucination, with no tools and no internet. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.
EVE hallucination detection score
Share of questions answered correctly, 0 to 100 · Higher is better
How to read this chart
Rows. Each model family once, at its best model and reasoning level, with at least 75% of the questions scored; the table lists every model and level, with how many questions each has scored.
Data table: EVE hallucination detection score
| Model | Score | Questions scored | Cost per question | Time per question (at 100 output tokens/s) |
|---|---|---|---|---|
| Claude Opus 5.5 (medium) | 88 | 65 of 2,326 | $0.012 | 5s |
| Claude Fable 5.1 (high) | 87 | 510 of 2,326 | $0.017 | 4s |
| Claude Opus 4.8 (high) | 86 | 2,316 of 2,326 | $0.010 | 7s |
| Claude Fable 5.1 (medium) | 86 | 2,326 of 2,326 | $0.019 | 3s |
| Claude Opus 4.7 (high) | 86 | 1,426 of 2,326 | $0.0087 | 3s |
| Claude Sonnet 5.5 (medium) | 86 | 2,326 of 2,326 | $0.0045 | 3s |
| Claude Sonnet 5.5 (xhigh) | 85 | 2,116 of 2,326 | $0.0052 | 5s |
| GLM-5.3-Flash (low) | 85 | 2,326 of 2,326 | $0.000094 | 3s |
| Claude Sonnet 5.5 (high) | 85 | 2,326 of 2,326 | $0.0040 | 4s |
| Claude Sonnet 5.5 (low) | 85 | 2,326 of 2,326 | $0.0044 | 3s |
| Nemotron 3.5 Super VL 120B (Reasoning) | 85 | 2,326 of 2,326 | $0.00098 | 80s |
| Claude Opus 5.5 (low) | 85 | 2,326 of 2,326 | $0.0099 | 4s |
| Step 5 Preview (Reasoning) | 85 | 2,326 of 2,326 | $0.0040 | 86s |
| MiniMax M3.1 Flash (max) | 85 | 661 of 2,326 | $0.00075 | 514s |
| Claude Sonnet 5 (high) | 85 | 1,337 of 2,326 | $0.0051 | 3s |
| Claude Haiku 5.5 (xhigh) | 85 | 2,326 of 2,326 | $0.00044 | 4s |
| Claude Opus 5.5 (xhigh) | 84 | 853 of 2,326 | $0.012 | 6s |
| Nemotron 3 Ultra (high) | 84 | 2,326 of 2,326 | $0.0024 | 28s |
| Claude Opus 5.5 (high) | 84 | 2,326 of 2,326 | $0.012 | 5s |
| Claude Haiku 5.5 (low) | 84 | 2,326 of 2,326 | $0.00038 | 3s |
| Claude Haiku 5.5 (high) | 84 | 2,326 of 2,326 | $0.00043 | 4s |
| Nemotron 3.5 Lightning 30B (Reasoning) | 82 | 2,326 of 2,326 | $0.00028 | 130s |
| Nemotron 3 Nano Omni 30B (Reasoning) | 80 | 2,326 of 2,326 | No data | 28s |
About this evaluation
Each item is a question about Earth observation and an answer; the model says whether the answer contains a hallucination, from its own knowledge, with no tools and no internet. All 2,326 items are scored.
The score is the share of items answered correctly, checked by code.
Published by ESA Φ-lab (EVE). Licence: No licence stated on the dataset card; used as open.
