Skip to main content

Spatial agents \u00b7 Findings

Which vision model finds traffic signs in street images? A small, cheap one did best

We asked 12 vision models to find speed signs, warning signs and time plates in street-level images. GLM-5.3-Flash found 93% with one false alarm, at about a twentieth of the cost per call of the most expensive model.

Research

In short

  • We gave 12 vision models the same 18 street-level frames of German roads and asked them to find speed limit signs, children warning signs, the small plates that give times, and road markings. 13 frames have every sign marked by hand.
  • GLM-5.3-Flash found 93% of the marked signs with one false positive. Qwen 3.8 also found 93%, with three. Gemini 3.8 Flash found 86%.
  • Moondream 3.1 found every sign, and 22 that were not there. It is good for finding candidates for another model to check, not for the final answer.
  • The time plate was the hardest class for every model, found 16% to 83% of the time.

Signs found in street images (share of marked signs, higher is better)

  1. GLM-5.3-Flash1 false positive93
  2. Qwen 3.83 false positives93
  3. Gemini 3.8 Flash2 false positives86
  4. Gemini Robotics-ER 2 (preview)1 false positive80
  5. Llama 3.2 11B Vision40
Presence of each marked sign in 13 hand-marked frames. September 2026. Source: How we measured.

Why it matters

Checking a map against what a camera sees is a common task: is the speed limit in the map the one on the sign? A vision model can do the first step, finding the sign in the frame, without a trained detector. Which model you pick changes the cost by more than ten times and the false alarms by more than twenty.

How we measured

Each model saw each frame with two prompt wordings, at full resolution and as a smaller thumbnail. We scored whether each marked sign was found, not how tight the box was. Over the whole run the models gave about 1,480 answers. The runs were on 4 and 5 September 2026. Thirteen marked frames is a small set, so read differences of a few points as noise. When we ran Gemini 3.8 Flash twice with the same settings, its score moved from 87% to 79%.

What else we saw

  • Full resolution helps. For most models, the full frame beat the thumbnail by 5 to 15 points. Shrinking the long side to 512 pixels made the time plates disappear for GLM-5.3-Flash.
  • Cropping can hurt. Cropping to the area around the sign lost road markings, because the crop cut away the road.
  • Some models return nothing. Kimi K2.6 gave an empty answer to 13 of 72 calls and was the most expensive model in the run, about 20 times GLM-5.3-Flash per call.
  • Claude Haiku 4.5 missed time plates. It found them 16% of the time.

What this means if you are choosing a model

For finding signs in street images, start with a small model and measure it on 20 of your own frames before trying a big one. Use a high-recall model like Moondream to propose and a careful one to confirm. Keep images at full resolution until the model has found what it needs. If you want this measured on your imagery and then built into a pipeline, tell us about the work.