---
title: "Your RAG demo is lying to you"
slug: evaluating-rag
date: 2026-09-02
url: https://mohammadhameed.dev/posts/evaluating-rag
tags: ["ai", "rag", "evaluation"]
---

The first demo of a retrieval system is a magic trick. The thousandth user
query is a probability distribution. Somewhere between the two, someone
builds trust on answers that are confidently, fluently wrong.

## The three failures nobody tests

In every engagement I've run — from NuronsAI client work to my own
products — the same three failures were invisible until I built scoring to
see them.

| Failure | How the demo hides it | How the harness catches it |
| --- | --- | --- |
| Unfaithful grounding | retrieval looks great on the easy questions | faithfulness scoring on golden answers |
| Silent refusal | model answers *around* the missing context | refusal-correctness axis against "not in docs" |
| Paraphrase fragility | the test set reads like the docs | same questions, rewritten adversarially |

## The minimal stack that earns its keep

Not a platform. One file of golden pairs, a runner, and two gates that block
merge. The full shape lives on the [RAG Eval Harness project page](/projects/rag-eval-harness):

```python
assert run.faithfulness >= 0.92, "answers left the documents"
assert run.regression_count == 0, "last month was better"
```

<Callout type="insight">
The cheapest golden set you will ever build is one question from every
support ticket. By the time your product has users, they've already written
your evaluation suite — most teams just never go and collect it.
</Callout>

<Figure src="/images/posts/evaluating-rag-alt.svg" alt="Faithfulness against latency across runs" caption="The upper-left is where you ship. Everything else is a trade you chose on purpose." />

## What I stopped doing

- **Tuning prompts against a favorite question.** One question overfit is
  worse than no tuning; it teaches the model to be wrong with confidence.
- **Retrieval metrics alone.** Top-k accuracy measures the library, not the
  answer the user receives.
- **Treating eval as a phase.** The harness has to be running before the
  first bug ships, not scheduled after it.

## The bottom line

If you can't tell me the faithfulness distribution of your last release,
your demo isn't evidence — it's marketing. The good news: this problem is
small, boring, and solvable in a week. That week is always cheaper than the
support ticket where a confident wrong answer becomes a public one.

If your team would rather not spend that week, [reach out](/contact).