How to test an LLM feature before it reaches your customers
A plain guide to evals: the small set of real examples that tells you whether an AI feature works, and keeps it working every time you change something.
Hussain CFreelance AI engineer in Surat, IndiaBefore an AI feature goes in front of customers, I want to see it pass a test. Not a demo, a test: a fixed set of real examples, scored the same way every time. That set is called an eval, and it's the most useful thing you can build on an AI project.
It's also the thing most teams skip. Then the model gets upgraded, or someone tweaks the prompt, and nobody notices the answers got worse until a customer does.
What does an eval look like?
At its simplest, it's a spreadsheet. One row per example:
- The input. A real customer email, a real question, a real document.
- What good looks like. Not always the exact answer, but what a good answer must and mustn't do.
- How to score it. A rule, a reference answer, or a rubric.
Then a script runs every row through your AI feature and scores the results. You get a number, and a list of the cases that failed.
Where do the examples come from?
From the real work, never from your imagination. I ask clients for things like:
- Twenty recent support tickets, including the annoying ones.
- The questions new staff ask in their first week.
- Every case where a person said "the AI got this wrong".
Then I add the ones that should make it refuse or hand over: questions it can't answer from the sources, requests it mustn't act on, attempts to make it say something silly.
How do you score the answers?
There are four ways to score, from cheapest to most expensive. Use the cheapest one that works for each check.
| Question | Good for | Cost |
|---|---|---|
| Code rules | Format, required fields, banned words, valid JSON | Free, instant |
| Reference match | Classification, extraction, yes or no answers | Free, instant |
| AI grader | Tone, relevance, sticking to the sources | Cents per run |
| A person | Final judgement on a sample of results | Your time |
An AI grader is just another model call with a strict rubric: "Does this answer use only facts from the sources below? Answer yes or no, then explain." It's surprisingly good at this. Still, check a sample of its scores by hand now and then. Graders drift too.
How does it fit into the work?
- 01Collect30 to 50 real examples
- 02Define goodWhat each answer must do
- 03RunEvery example, every time
- 04Read failuresFix the prompt, data or tools
- 05Re-runShip only if nothing got worse
The last step is the one that matters. Before a model upgrade, a prompt change or a new data source, run the set. If the score drops or a case that used to pass now fails, you've found a bug before your customers did.
What should you measure?
It depends on the feature, but these come up on almost every project:
- Correctness. Did it get the right answer, label or field?
- Groundedness. For RAG, did it only use facts from the sources?
- Refusals. Did it say "I don't know" when it should, and only then?
- Safety. Did it stay within what it's allowed to do?
- Cost and speed. Tokens and seconds per answer. A great answer that takes forty seconds is a bad answer.
What tools do you need?
You can start with a spreadsheet and a short script. When it grows, tools like promptfoo, Langfuse, LangSmith or Braintrust store the runs, compare versions side by side and show you what changed. Pick one your team will actually open. The tool matters much less than the habit.
When is it good enough to launch?
When it passes the cases you care about most, fails safely on the rest, and you have a way to catch what you missed. That last part means logging real conversations after launch and turning every new failure into a new row in the set.
The eval set is never finished. It's a record of everything your AI has got wrong, and proof that it doesn't get those things wrong any more.
If you have an AI feature in the works and no way to know whether it's getting better or worse, I can help you set one up.