Axis Spatial Spatial Intelligence Index

Search modelsMenu

EVE hallucination detection benchmark leaderboard

Deciding whether an answer about Earth observation contains a hallucination, with no tools and no internet. A standalone evaluation: its scores are shown here and on model pages, but it is not counted in any index or ranking until more models have run it.

EVE hallucination detection score

Share of questions answered correctly, 0 to 100 · Higher is better

EVE hallucination detection scoreScores 0 to 100: Claude Sonnet 5.5 (medium): 86; GLM-5.3-Flash (low): 85; Nemotron 3.5 Super VL 120B (Reasoning): 85; Step 5 Preview (Reasoning): 85; Claude Haiku 5.5 (xhigh): 85; Nemotron 3 Ultra (high): 84; Nemotron 3.5 Lightning 30B (Reasoning): 82; Nemotron 3 Nano Omni 30B (Reasoning): 800255075100Claude Sonnet 5.5 (medium)86GLM-5.3-Flash (low)85Nemotron 3.5 Super VL 120B (Reasoning)85Step 5 Preview (Reasoning)85Claude Haiku 5.5 (xhigh)85Nemotron 3 Ultra (high)84Nemotron 3.5 Lightning 30B (Reasoning)82Nemotron 3 Nano Omni 30B (Reasoning)80
How to read this chart

Rows. Each model family once, at its best model and reasoning level, with at least 75% of the questions scored; the table lists every model and level, with how many questions each has scored.

Data table: EVE hallucination detection score
EVE hallucination detection score (share of questions answered correctly, 0 to 100)
ModelScoreQuestions scoredCost per questionTime per question (at 100 output tokens/s)
Claude Opus 5.5 (medium)8865 of 2,326$0.0125s
Claude Fable 5.1 (high)87510 of 2,326$0.0174s
Claude Opus 4.8 (high)862,316 of 2,326$0.0107s
Claude Fable 5.1 (medium)862,326 of 2,326$0.0193s
Claude Opus 4.7 (high)861,426 of 2,326$0.00873s
Claude Sonnet 5.5 (medium)862,326 of 2,326$0.00453s
Claude Sonnet 5.5 (xhigh)852,116 of 2,326$0.00525s
GLM-5.3-Flash (low)852,326 of 2,326$0.0000943s
Claude Sonnet 5.5 (high)852,326 of 2,326$0.00404s
Claude Sonnet 5.5 (low)852,326 of 2,326$0.00443s
Nemotron 3.5 Super VL 120B (Reasoning)852,326 of 2,326$0.0009880s
Claude Opus 5.5 (low)852,326 of 2,326$0.00994s
Step 5 Preview (Reasoning)852,326 of 2,326$0.004086s
MiniMax M3.1 Flash (max)85661 of 2,326$0.00075514s
Claude Sonnet 5 (high)851,337 of 2,326$0.00513s
Claude Haiku 5.5 (xhigh)852,326 of 2,326$0.000444s
Claude Opus 5.5 (xhigh)84853 of 2,326$0.0126s
Nemotron 3 Ultra (high)842,326 of 2,326$0.002428s
Claude Opus 5.5 (high)842,326 of 2,326$0.0125s
Claude Haiku 5.5 (low)842,326 of 2,326$0.000383s
Claude Haiku 5.5 (high)842,326 of 2,326$0.000434s
Nemotron 3.5 Lightning 30B (Reasoning)822,326 of 2,326$0.00028130s
Nemotron 3 Nano Omni 30B (Reasoning)802,326 of 2,326No data28s

About this evaluation

Each item is a question about Earth observation and an answer; the model says whether the answer contains a hallucination, from its own knowledge, with no tools and no internet. All 2,326 items are scored.

The score is the share of items answered correctly, checked by code.

Published by ESA Φ-lab (EVE). Licence: No licence stated on the dataset card; used as open.

Related links