Diagrams

AI Evals Explained | How to evaluate AI Agents?

Companion diagrams to the article — an agent is only as trustworthy as the eval that scores it.

Channel: Aishwarya Srinivasan

1 · What an AI Eval Measures: From Agent Decisions to a Score

Illustrates Section 1 of the article, "What Is an AI Eval?"

Trigger · every change you make
New prompt Different model Retrieval tweak
▼re-runs the agent
The agent · a chain of decisions
1Pick a tool 2Call it 3Read the result 4Decide what's next ↻ and repeat
▼outputs and action traces
The eval Structured, repeatable scoring process
Instead of eyeballing a couple of answers, you get an actual score on your own data.
▼
The one question that matters
Is it getting better? → ship and keep measuring Did I break it? → find where, fix it, re-run the eval

2 · Benchmarks, Classic ML Evals, and AI Evals: What Each One Tests

Illustrates Section 2 of the article, "Evals vs. Benchmarks vs. Classic Machine Learning Evaluation"

Fair question: "isn't this just benchmarks?" "we've evaluated models for decades — what's the big deal?"

Benchmarks

Test the model — the same generic test applied to every model.

SWE-bench Tau-bench Terminal-Bench composite indexes
Good for picking which model to start with.

Classic ML evaluation

A multiple-choice test: output is clean, an answer key exists, grading is objective.

Clean outputs: a category, a number, yes or no.
Accuracy Precision Recall F1

AI evals

Grading an essay: no answer key, so you need a rubric and judgment.

Tests your system — your task, your data, your prompts, your users — not the model in general.
open-ended text action sequences rubric + judgment
Benchmarks → pick a starting model then evals → does it actually work for you? · LLM outputs moved grading from multiple-choice to essay

3 · Choosing Metrics by Task: From the Core Four to Task-Specific

Illustrates Sections 3.1 and 3.2 of the article, "The Metrics That Matter"

Core top-level metrics
Relevance Faithfulness Correctness Coherence
▼the right metric depends entirely on your task

Summarization

  • Faithfulness — no inventing
  • Coverage — main points
  • Conciseness — keep it tight

Classification

Clean label output — back to old-school ML.

  • Accuracy
  • Precision
  • Recall
  • F1

Translation

Meaning carries over; reads naturally.

  • BLEU
  • COMET

RAG

Answer relevance + faithfulness together catch both ways a RAG system breaks.

  • Answer relevance
  • Faithfulness

4 · The Three Families of Grading Metrics

Illustrates Section 3.4 of the article, "Three Families of Metrics"

Family 1 · Classic overlap

Old NLP workhorses: how much does the output text overlap the reference?

BLEU ROUGE METEOR
Fast, cheap, dead simple — but blind to meaning: the same idea in different words scores differently.

Family 2 · Semantic

Compare meaning using embeddings instead of matching words.

BERTScore BLEURT COMET
Handles paraphrasing far better than word overlap.

Family 3 · Model-based

An LLM does the grading for you — your LLM as a judge.

LLM as a judge G-Eval
The only family flexible enough to grade open-ended, messy outputs with no reference answer at all.
Family 1 · word overlap → Family 2 · adds meaning → Family 3 · adds judgment

5 · Golden Dataset First, Then Four Ways to Grade Outputs

Illustrates Sections 4 and 5 of the article, "Start With a Golden Dataset" and "Four Ways to Grade Agent Outputs"

Phase 1 · Build the golden dataset — before touching any metric list

1Example inputs for your task
2Paired with a great output 2band a bad output
3Cover common cases, edge cases, past failures
▼
Golden dataset
About 52 examples is plenty to start Work backwards → metrics fall out of the dataset
▼now choose a grader

Phase 2 · Four ways to grade agent outputs

Humans

Domain expert grades against the rubric. The gold standard — nothing beats it for quality.

Slow, expensive, does not scale → small careful samples, kept as source of truth.

User feedback

Thumbs up or down, edits, accepted or rejected, task finished, repeat visits.

Real ground truth — but noisy, and only available after you ship, so it cannot be your safety net.

Programmatic

Simple checks written in code: right value, valid JSON, right tool and arguments, time, cost.

Cheap and instant, run on every change — only works with clearly checkable answers.

LLM as a judge

A strong model grades outputs against your rubric — essay grading that scales to thousands.

Judge models have biases: they favor longer answers and whatever they see first.
↩ validate the LLM judge against human grades on a sample first — then it becomes your workhorse

6 · The Evaluation Loop: CI/CD for AI Agents

Illustrates Sections 6 and 7 of the article, "Evals Are a Loop" and "Tools for Running and Monitoring Evals"

Step 1 of 8

Step 01

Decide what "good" even means

Get specific about the goal before touching a single metric: what does a great answer look like for your users, on your task?

This is the step everyone is tempted to skip

Step 02

Build the golden dataset

Collect example inputs paired with great and bad outputs — common cases, annoying edge cases, and every failure you have watched blow up.

About 52 examples is enough to start

Step 03

Pick your metrics — backwards

Work backwards from the dataset: the quantified way to judge your examples is the metric. Do not start from a metric list.

Relevance Faithfulness Correctness Coherence + task-specific metrics

Step 04

Run a baseline

Run your current system across the whole dataset and get a number. That number is your starting line — without it, "it feels better" is just vibes, not data.

Eval runners: Promptfoo, RAGAS

Step 05

Go look at the failures and group them

Are they all retrieval misses? Formatting screw-ups? The same weird edge cases? The grouping tells you exactly what to fix in the agent.

Everyone rushes past this step

Step 06

Fix it

Usually a small change: tweak your prompts, your retrieval, or your tool definitions.

Step 07

Rerun against the exact same dataset

Did your numbers go up — and just as important, did you accidentally break something that used to work? Catching that is the entire reason the loop exists.

Numbers down or regressionSomething broke → return to step 5, group the failures again, fix, rerun.
Numbers up, nothing regressedThe change is safe → ship and keep watching.

Step 08

Keep going — this never stops

In production you always watch live traffic. Your provider can quietly update the models underneath you and shift behavior overnight, so continuous evaluation and monitoring run forever.

Feedback loopReal-world failures flow right back into your golden dataset — then run the whole loop again.
Monitoring toolsTracing and monitoring: LangSmith, Langfuse, Arize, Braintrust.