LLM features tend to arrive at the same place. There is a prompt in a Python file, it has been edited forty times, and nobody can say whether edit thirty-eight was better than edit twelve. Someone changed the model version last week and the team is not sure whether quality moved.
That is not a prompting problem. It is a measurement problem, and it is the reason the feature cannot ship.
What “an eval” actually needs to be
The industry word makes this sound heavier than it is. At the minimum useful size, an eval is three things:
- A golden set. Thirty to a hundred real inputs with the output you would accept. Real, not synthetic — pull them from your logs, your support inbox, your existing users. Synthetic cases agree with your assumptions, which is exactly the failure you are trying to catch.
- A scorer. For extraction and classification, exact or fuzzy match. For open-ended generation, a rubric scored by a second model, calibrated once against human judgement on twenty cases so you know how far to trust it.
- A number that moves. One aggregate you can put in a pull request description, plus the per-case breakdown for when the aggregate lies.
That is a day of work. Teams postpone it for months because it feels like infrastructure rather than progress, and then spend those months arguing about prompts in review comments.
Wire it into CI on day one
The harness only pays off if it runs without anyone deciding to run it. Put it in the pipeline:
- Run the golden set on every pull request that touches a prompt, a retrieval parameter, or a model identifier.
- Fail the build on a regression past a threshold you agreed in advance, not one you set after seeing the number.
- Post the per-case diff as a comment. Reviewers should see which cases got worse, because the aggregate will happily hide a systematic failure inside an average.
Cost is the usual objection. A hundred cases against a mid-tier model is cents, and it runs in under a minute if you parallelise. If your set is expensive enough to matter, sample it on every PR and run the whole thing nightly.
The cases worth collecting
Golden sets rot toward the easy middle. Deliberately keep four categories in yours:
- Typical. The boring path, which is most of your traffic.
- Adversarial. Prompt injection, contradictory instructions, requests outside scope.
- Ugly input. Truncated documents, mixed languages, scans that OCR’d badly. This is where retrieval quietly fails.
- Known past failures. Every production bug becomes a case. This is the single highest-value habit on the list — it is a regression suite, and it is the only thing that stops the same bug shipping twice.
Retrieval gets measured separately
If the feature retrieves, evaluate retrieval on its own before you evaluate generation. Recall at k against a set of query-to-document pairs will tell you in ten minutes whether the model is hallucinating or whether it simply never received the right chunk. Those two failures look identical in the output and have nothing in common in the fix.
What changes once it exists
The arguments stop. A prompt change is either a measurable improvement or it is not, and the pull request says which. Model upgrades become a routine experiment instead of a leap of faith. And the question every stakeholder eventually asks — “is it good enough to launch?” — turns into a number with a threshold next to it, which is the only form of that question anyone can actually answer.