An AI benchmark is not an acceptance test

NIST published its ARIA Evaluation Planning Manual on 18 September 2026. The 36-page document lays out a method for assessing an AI application in a specific context of use, beginning with the sector, use cases and one defined concept to measure. It treats evaluation as a designed study rather than a single benchmark run.

The manual separates three evidence channels. Model testing uses fixed prompts that can be repeated at scale. Red teaming asks people to try to produce an undesirable outcome and react to the system’s responses. User testing observes people attempting realistic tasks and records their experience. NIST notes that an evaluation may use all or only some of these methods, depending on its objective.

That distinction matters because each method sees a different part of performance. Fixed prompts support consistent comparison, but existing benchmarks may have been present in training data and may not represent the intended use. Human testers can explore unexpected interactions and realistic scenarios, but they are harder to recruit and scale. The manual therefore asks evaluators to document the scenarios, instructions, questionnaires, annotation rules, data structure and scoring method behind the result.

Dunes Bureau interpretation: an organisation buying an AI tool can turn this structure into an acceptance plan. Define one operational use case and one failure or quality concept; run repeatable prompts, targeted adversarial sessions and representative user tasks where each is relevant; then preserve the application, scenario and testing method attached to every result. If vendors are being compared, use the same design for each rather than combining scores produced under different conditions.

The report does not validate a particular product or claim that ARIA is the only suitable approach. NIST describes the method as relatively new, says more applications are needed to establish good practice, and notes a trade-off between realistic specificity and results that generalise broadly. The practical lesson is narrower: a model score becomes decision evidence only when the task, testers, method and limits are visible.