Skip to main content

Grounding \u00b7 Findings

Grounding DINO finds a traffic sign 92% of the time when the caption describes what it looks like

We tried six captions for one German warning sign on 66 street-level images. Describing the shape and the picture on the sign won. Naming the sign did not.

Research

In short

  • Grounding DINO finds objects from a text caption. We tried six captions for the German children warning sign on 66 street-level images from Berlin.
  • The caption "red triangle sign . children silhouette sign" found the sign in 92.4% of the images. Florence-2's best caption found it in 77.4%, and the run stopped at image 53.
  • Captions that described what the sign looks like beat captions that named it. "Achtung Kinder sign", the German name, was not the winner.

Children warning sign found (IoU at least 0.3), best caption per detector, 66 images

  1. Grounding DINOred triangle sign . children silhouette sign92.4
  2. Florence-2its best caption; stopped at image 53 of 6677.4
Share of images where the detector drew a box overlapping the sign by at least 0.3 IoU. Grounding DINO at box threshold 0.1. July 2026. Source: How we measured.

Why it matters

A grounding model connects words to a place in an image. That makes it the cheap way to find a sign, a pole or a building without training a detector: you write a caption instead of labelling a thousand boxes. But the caption is now the model. A word that makes sense to a person may not match anything the model saw in training.

How we measured

We took 66 street-level images from Berlin, each with a hand-checked box around one children warning sign (the red triangle with two figures). For each caption we ran Grounding DINO once at a low box threshold (0.1) and kept the best box. A hit is a box that overlaps the true box by at least 0.3 IoU (intersection over union). We ran Florence-2 on the same images with its own best caption. The runs were in July 2026.

The six captions were: "children warning sign", "red triangular warning sign with children", "Achtung Kinder sign", "red triangle sign . children silhouette sign", "warning sign", and "triangular traffic sign . children warning sign". Grounding DINO reads a full stop with spaces as a break between classes, so the winner asks for two things at once: the shape and the picture on it.

Small plates are harder

We did the same for the white plate under a speed sign that gives the hours. On 5 images, "white rectangular sign plate" scored highest on 3 and lowest on the image with no plate. But the scores overlapped: tight crops of real plates scored 0.40 to 0.59, loose crops of the same plates 0.22 to 0.31, and false boxes on the empty image 0.25 to 0.36. No single threshold separates them, so we treat 0.2 to 0.4 as "check", not "no". Five images is a small sample.

The same lesson with vision-language models

Later we asked 11 vision-language models to draw boxes around the same kinds of signs with three wordings: plain nouns ("speed limit 30", "children crossing", "time plate"), the German sign names, and the descriptive phrases that won with Grounding DINO. Over 1,505 answers, plain nouns gave the tightest boxes, with a mean IoU of 0.689 against 0.555 to 0.587 for the other wordings. German names did worst.

The descriptive phrase helped some models and hurt others. It raised GLM-5.3-Flash and Qwen 3.8, and cut Gemini Robotics-ER 2 (preview) from 80% to 43% of signs found. A caption that works for one model is not a general fix.

What this means if you use grounding models

Try several captions on 50 to 100 of your own images before you trust one, and keep the winner per model. Describe what the object looks like in the image, not what it is called. If you need a detector for your own objects, from street images or satellite images, tell us what it should find.