18 — Evals and Regression Testing: Scoring LLM Output with Confidence

Reading model outputs by eye feels fine for a prototype and quietly falls apart the moment a real decision depends on it. The habit that replaced eyeballing: evals are structured, repeatable tests that turn "does it work?" from a vibe into a number, and regression suites catch the silent quality drops that vendor model updates and prompt edits cause before users notice. [1] Reading outputs by eye works for a prototype. It stops working the moment I have to decide whether a prompt change made things better or worse, or whether switching models is safe — because without a number, I'm guessing, and I'm usually wrong.

The framing that finally landed is a hierarchy of evaluation methods, each with a different cost and a different kind of confidence. At the base, deterministic evals apply fixed rules — does the output contain the required keyword, does it match the expected format, is it exactly the known-correct answer. Above that, model-based evals use another LLM as a judge to score open-ended quality dimensions the rules can't capture. At the top, human evals have people review and score outputs — the ground truth other methods are validated against, expensive but irreplaceable for nuanced judgment [2]. The art is layering them: deterministic first because it's cheap and reproducible, model-based for the subjective dimensions, human for the high-stakes calls and for calibrating the automated layers.

LLM evaluations: structured tests, not vibes

An LLM evaluation is a structured test that measures how well a model or system performs against defined criteria [1]. The shift from "reading outputs" to "running evals" is the shift from informal judgment to repeatable, quantitative measurement. With an eval suite, I can compare prompt versions, model updates, and system changes on the same scale — and I can run the same suite before and after a change to know whether I improved or regressed.

The reason this matters more for LLMs than for traditional software is non-determinism. A deterministic function either passes its unit tests or doesn't; the same input always produces the same output. An LLM might produce a great answer on test run one and a mediocre one on test run two. Evals handle this by running many cases and aggregating scores, so a single weird output doesn't dominate the signal — the trend across the suite is what tells me whether the system is actually good.

Deterministic evals: the cheap, reproducible base

Deterministic evals use fixed, rule-based checks to score outputs [3]. Examples: does the response contain a required keyword? Does it match the expected JSON schema? Does it equal the known-correct answer exactly? Is it under the maximum length? These checks are fast, cheap, and fully reproducible — the same output always gets the same score.

Their strength is also their limit. Deterministic evals are perfect for the dimensions of quality that have a clear right answer — format compliance, presence of required content, length. They _cannot_ assess open-ended or subjective quality. "Is this answer helpful?" is not a question a regex can answer. So I use deterministic evals as the first layer — they catch the easy failures cheaply — and reach for higher layers when I need to judge subjective quality [3].

Model-based evals: LLM-as-a-judge

Model-based evals use a separate AI model to score the outputs of my system — the technique commonly called LLM-as-a-judge [4]. I write a prompt describing the evaluation criteria, and the judge model rates the response. This handles the subjective, open-ended quality dimensions that deterministic rules cannot — relevance, helpfulness, tone, factual groundedness — while scaling far more cheaply than human review.

The caveat I had to learn the hard way: the judge is itself an LLM, with its own biases and inconsistencies. A poorly written judge prompt produces noisy scores that don't reflect anything real. The discipline is to write judge prompts as carefully as production prompts, to validate the judge against human-labeled examples (calibration), and to treat the judge's score as a strong signal but not ground truth. LLM-as-a-judge is most credible when I've shown that, on a labeled subset, the judge agrees with humans most of the time [4].

Human evals: the ground truth

Human evals involve people directly reviewing and scoring outputs against defined criteria [5]. They are the most accurate form of evaluation for nuanced or subjective quality, and they serve as the ground truth that deterministic and model-based evals are validated against. They are also the slowest and most expensive, so I use them strategically: for high-stakes decisions, for calibrating the automated evals, and for reviewing the edge cases the automated layers flag.

The mistake to avoid is treating human evals as the only "real" eval and skipping the automated layers. That scales badly — every prompt change would need a fresh round of human review, which is why teams that rely only on human evals end up not evaluating at all. The point of the lower layers is to make human evals rare and high-leverage, reserved for the cases where they actually add unique signal.

Evaluation metrics: choosing what to measure

Evaluation metrics are the specific measurements used to score outputs [6]. Each targets a different quality dimension:

• Faithfulness / groundedness — is the answer supported by the retrieved context? (Critical for RAG.)
• Relevance — does it actually address the user's question?
• Safety / toxicity — is it free of harmful content?
• Helpfulness — is it genuinely useful to the user?

Choosing the right metrics is one of the most consequential decisions in building an eval system, because metrics determine what the system optimizes for and what failures it can detect [6]. A system optimized only for "is it fluent?" will ship fluent nonsense; a system optimized for faithfulness will refuse to answer when the context doesn't support it. The metrics I pick encode my values for the feature, and I should pick them deliberately.

Regression testing: catching silent drops

Regression testing for AI systems means verifying that a change — to a prompt, a model version, a retrieval strategy, anything — doesn't degrade performance on cases that previously worked [7]. Because LLM behavior can shift subtly with small changes, a regression suite is what lets me catch quality drops before they reach users.

The mechanism is the eval suite run before and after a change, on the same set of cases (a "golden dataset"). If the score drops, the change regressed something, and I investigate before shipping. This is the AI equivalent of the unit-test safety net — not perfect, but infinitely better than discovering the regression in production. The roadmap points to golden datasets as the core of regression testing, and the pattern is: curate a set of representative cases with known-good (or known-acceptable) outputs, run them on every change, and treat any score drop as a release blocker until understood [7].

The tooling: LangSmith, LangFuse, Helicone, Arize, DeepEval, RAGAS

The ecosystem provides purpose-built tools for each layer of this. LangSmith is LangChain's observability and evaluation platform — it captures traces and lets me run evals directly on them [8]. LangFuse is the open-source counterpart, offering tracing, prompt management, and evaluation with self-hosting or cloud options [9]. Helicone is a logging proxy that captures every request and response with zero code changes, with dashboards for cost, latency, and errors [10]. Arize AI is an ML observability platform suited to enterprise settings, with tracing, drift detection, and evaluation [11].

On the eval-framework side, DeepEval is an open-source framework that feels like pytest for LLM apps — over 50 built-in metrics covering accuracy, faithfulness, relevance, toxicity, and agent behavior, with CI/CD integration so evals run automatically on every change [12]. RAGAS is purpose-built for evaluating RAG pipelines, with reference-free metrics that don't require hand-labeled ground truth — its core metrics cover the main ways RAG fails: retrieving the wrong context, generating ungrounded answers, and producing responses that don't address the question [13]. These two are the ones I reach for most, because they encode the eval patterns (golden datasets, LLM-as-judge, faithfulness/relevance metrics) as tested libraries rather than something I have to build from scratch.

How I use this

I build the eval suite alongside the feature, not after. For any feature I intend to ship, I curate a golden dataset of representative cases — small at first, growing as I learn the failure modes — and I score them with a layered harness: deterministic checks for format and required content, an LLM-as-judge for relevance and faithfulness, and human review on a sample. I run the suite on every change and treat score drops as release blockers. I wire it into CI with DeepEval or RAGAS so it runs automatically, not when I remember. And I use the trace-and-observe tools (LangSmith, LangFuse) to connect eval failures back to the specific retrieval, prompt, or model decision that caused them. The discipline is to never ship a change to an LLM feature without running the suite — because in a non-deterministic system, the suite is the only thing that turns "I think it's better" into "I know it's not worse."

References

[1] Evidently AI, "LLM evaluation: a beginner's guide," 2024. [Online]. Available: https://www.evidentlyai.com/llm-guide/llm-evaluation

[2] Anthropic, "Demystifying evals for AI agents," 2025. [Online]. Available: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

[3] Confident AI, "How I Built Deterministic LLM Evaluation Metrics for DeepEval," 2024. [Online]. Available: https://www.confident-ai.com/blog/how-i-built-deterministic-llm-evaluation-metrics-for-deepeval

[4] Evidently AI, "LLM-as-a-judge: a complete guide to using LLMs for evaluations," 2024. [Online]. Available: https://www.evidentlyai.com/llm-guide/llm-as-a-judge

[5] SuperAnnotate, "LLM-as-a-judge vs. human evaluation: Why together is better," 2024. [Online]. Available: https://www.superannotate.com/blog/llm-as-a-judge-vs-human-evaluation

[6] Evidently AI, "LLM evaluation metrics and methods, explained simply," 2024. [Online]. Available: https://www.evidentlyai.com/llm-guide/llm-evaluation-metrics

[7] Evidently AI, "Watch the language: A tutorial on regression testing for LLMs," 2024. [Online]. Available: https://www.evidentlyai.com/blog/llm-regression-testing-tutorial

[8] LangChain, "LangSmith," 2024. [Online]. Available: https://docs.langchain.com/langsmith/home

[9] LangFuse, "Overview," 2024. [Online]. Available: https://langfuse.com/docs

[10] Helicone, "Docs," 2024. [Online]. Available: https://docs.helicone.ai/getting-started/quick-start

[11] Arize AI, "Docs," 2024. [Online]. Available: https://arize.com/docs/ax

[12] DeepEval, "Introduction," 2024. [Online]. Available: https://deepeval.com/docs/introduction

[13] RAGAS, "Documentation," 2024. [Online]. Available: https://docs.ragas.io/en/stable/