Part geometry changes what an AI confidence score means

NIST published research on 24 September 2026 about the Segment Anything Model in automated bin picking. The model can separate previously unseen objects from two-dimensional images. When it receives a point prompt, it produces three possible masks with different confidence scores, giving a robot-vision system several candidates for the boundary of a part.

The researchers report that the correct mask does not always carry the same confidence rank. For parts that appear visually decomposable into subcomponents, the correct result is more often linked to the medium-confidence mask. For shapes that appear less decomposable, the highest-confidence mask is more often correct. Simulation with gradually modified computer-aided-design models found that the transition between those behaviours can occur across a relatively narrow range of geometric changes.

Dunes Bureau interpretation: a factory should not reduce model acceptance to one average segmentation score. A useful trial should group parts by visible complexity, vary the features that matter to gripping and measure which candidate mask produces a safe, successful pick. The decision rule may need to use known part geometry rather than always select the model's top-ranked output.

That approach matters in low-volume, high-variety production, where a robot may encounter unfamiliar parts and limited training examples. A narrow change in a flange, recess or connected subcomponent could alter the model's preferred output even when the overall object still looks similar to an operator. The failure may then appear at the gripper rather than on a vision dashboard.

The study establishes sensitivity to part type in simulation; it does not prove a universal threshold or production success rate. A deployment would still need tests with the actual camera, lighting, clutter, gripper, materials and range of orientations used on the line.