Artificial IntelligenceArchive edition6 min read

Creating a test set for language model outputs

Evaluate the behavior that matters to a real product rather than relying on a handful of impressive examples.

Luminous geometric nodes arranged in a layered lattice — Creating a test set for language model outputs
Original ARTware computational illustration.

A demonstration often shows a language feature working on the same examples used to develop it. Those examples have value, but they answer a limited question: can the current configuration produce the intended result under familiar conditions? A product team needs to know whether the feature will behave usefully on the range of inputs that occur in ordinary operation, and what happens outside that range. A test set makes those expectations explicit. It allows changes to be evaluated against consistent material and gives reviewers a way to explain disagreement without relying on an impression that the new responses somehow seem better.

Translate product behavior into evaluation examples

Take a prototype that assigns incoming messages to categories. A meaningful test case consists of the message, any context the application will actually provide and the expected category or judgment. That last item needs a reason behind it. If the distinction between a general question and a technical issue is unclear, an example should identify the rule that determines the preferred label. A question about instructions that a user has not read might belong to general support. A message saying instructions were followed but the system returned an error might belong to technical support. Stating the distinction helps avoid arbitrary labeling decisions.

A useful set covers the composition of everyday work. If ninety percent of actual messages are short and straightforward, the set should include those ordinary inputs. It should also contain rare but consequential cases, such as messages that clearly do not fit any offered category. Treat representation and stress testing as related but different purposes. A collection made entirely of difficult cases can suggest that a system fails frequently even when such cases seldom arise in production. A collection limited to the most common easy cases can conceal the failures that matter most when a feature is released.

Collect and label examples before testing

Begin with a manageable collection. Fifty ordinary messages, fifteen borderline ones and ten outside the supported categories could serve as an initial set for a small classifier. Those numbers are illustrative; adjust them to the task, the expected use and the cost of labeling. Use material that the team is allowed to process and retain. Remove confidential information when the precise identity of a person is irrelevant to the evaluation. Where actual messages cannot be reused, create realistic fictional inputs while keeping track of how they were constructed. Avoid designing only perfect examples that mirror the language of the category definitions.

Label the material before comparing candidate configurations. In straightforward cases, include the expected single result. In cases where different responses could be valid, record the acceptable alternatives or the conditions an acceptable answer must satisfy. Allow a case to remain disputed when the guidance does not resolve it. The goal is not to persuade every reviewer to agree immediately; disagreements reveal an aspect of the product specification that requires attention. Change a category definition if necessary and revisit the affected examples. Keep the original reasons for disagreement, which may become useful when similar situations occur after release.

Related reading: Structuring prompts for repeatable application behavior.

Decide what a passing output means

An exact string comparison is appropriate for a narrow, fixed set of labels, provided the interface's contract really requires those labels. For a generated explanation, many different phrasings may be equally useful. A rubric might check whether an answer includes specific facts, avoids unsupported assertions, follows an intended output format and covers the requested task. These are distinguishable dimensions. An answer can be exceptionally clear but omit the most important condition. It can also include the correct information in an invalid structure that the application cannot process. Define and record which of these dimensions determines whether the result can be accepted.

Specify criteria at a useful degree of detail. For an assistant that extracts meeting actions, the presence of an action description is not sufficient if it needs to identify the assigned owner. A test should distinguish an explicitly stated owner from a guessed name. A null owner may be the expected result when assignment is absent. Include a note explaining how to judge the case, so another reviewer can apply the same rule. A rubric that merely asks whether output is good encourages subjective judgments, and ratings may shift between versions even if the underlying output behaves in roughly the same way.

Layered flowing curves with fine network-like intersections
ARTware visual study. A complementary computational composition.

Keep cases suitable for development apart from final evaluation

Developers need examples to investigate failures and improve how the feature behaves. Reserve some material for that purpose. A separate locked set gives a better view of the behavior on examples that have not directly shaped the changes. If a team repeatedly studies every final test failure and changes its instructions to address exactly that wording, the final set has effectively become part of development. The resulting higher score may reflect familiarity with those particular cases rather than a general improvement. Use new representative examples for later confirmation when the development process has made the previous final cases familiar.

Keeping the collections separate does not require an elaborate evaluation platform. For a small project, versioned data files containing case identifiers, input text, expected results and tags for the categories of difficulty can provide a clear starting point. Document the procedure used to run the tests, including any settings that materially affect output. Store the result for each case rather than only a total score. The identifiers make it possible to compare a case from the previous run with its current result. Preserve the tested configurations so later maintainers can interpret changes without guessing which instructions were used.

Read the failures behind the aggregate measure

Suppose a revised classifier gets three additional ordinary messages right but now assigns several billing questions to technical support. Its overall accuracy might improve while the product becomes worse at a distinction that directly affects queues. Group results by the kinds of input and the consequences of incorrect assignments. For fixed classes, a confusion table can show which labels are being substituted for which others. For explanation tasks, counts of omitted facts and unsupported statements can provide more insight than one rating for each answer. When the collection is deliberately weighted toward rare cases, state this rather than presenting its overall score as an estimate of production frequency.

If a model can produce different results for the same input under the chosen settings, repeat representative evaluations where that variability matters. A single successful run does not establish that an important condition will always be met. Examine how regularly the format is valid, whether alternative labels remain within the accepted set and whether unacceptable claims recur. There is no need to claim that a modest repeat trial predicts every future outcome. Treat it as additional evidence about consistency under the recorded conditions, and combine it with ordinary application checks for enforceable constraints.

Maintain the set as the feature changes

The test collection should grow thoughtfully, not just accumulate every message that ever produced an error. Add an example when it represents a distinct situation, clarifies a boundary or protects against a meaningful regression. Avoid counting tiny paraphrases of the same failure as if they represented many independent problems. Revise the expected result when a product rule deliberately changes, keeping a note explaining the revision. Use the same core evaluation for routine updates so meaningful differences remain visible. A separate extension can cover recently introduced tasks without making old and new scores impossible to compare.

The value of this work becomes especially clear when a change produces a mixed result. Reviewers can discuss individual cases and refer to specific requirements. They might decide to accept minor variation in wording but demand that a structured field remain valid; they might delay release because the revision introduces a more consequential error. The evidence helps turn disagreement into a product decision. A well-maintained test collection does not eliminate judgment. It gives that judgment stable examples, explicit reasons and an established method for checking the effect of future changes.

Explore the implementation area: AI solutions.