Proof of Tech
Menu
← FIELD NOTES
TAGGEDEVAL 18 POSTS

Posts about EVAL.

2026.09.07 EVAL

The same RLHF that made your model helpful made its confidence a liar.

A pre-alignment calibration number does not survive RLHF: the reward model prefers high-confidence-sounding answers regardless of correctness, so abstention and routing built on the model's own confidence break on the deployed model.

→
2026.09.02 EVAL

A perfect needle-in-a-haystack score tells you almost nothing about long context.

A 99% needle-in-a-haystack number is bought by literal word overlap between the question and the planted fact. Strip the overlap, or make the haystack realistic, and the same model collapses. NIAH does not predict long-context ability.

→
2026.09.02 EVAL

Your retriever's recall is not your answer quality.

Teams optimize the retriever — chase recall@k, retrieve more documents — and expect answer quality to follow. It does not. Recall and answer accuracy are decoupled, and past a small number of documents, more retrieved context routinely makes the answer worse.

→
2026.08.24 EVAL

Your LLM judge can be fooled by a colon.

An LLM judge does not just have soft biases you average out — it has content-free inputs that force a 'correct' verdict on demand. A lone colon or raw token gibberish flips judges and reward models at roughly 80-100% rates: a judge used as a reward or gate is exploitable, not noisy.

→
2026.08.24 EVAL

Sparse autoencoders make a bad alarm and a great microscope.

Sparse autoencoders are the interpretability advance of the last two years, and the instinct is to wire one in as a runtime monitor for a known concept. The benchmarks say not to: for detecting, probing, or steering a concept you can already name, SAEs lose to a logistic-regression baseline and flip under a one-token adversarial nudge. Their real strength is the opposite job — discovering concepts you did not know to look for.

→
2026.06.30 EVAL

Constrained decoding doesn't cost you accuracy — your prompt does.

The widely-cited result that structured output hurts a model's reasoning measured something real — but not what it is quoted as measuring. Most of the drop is a non-equivalent prompt and regex-based scoring, plus a decoder that renormalizes probabilities naively. With an equivalent prompt, a distribution-aware decoder, and a schema roomy enough to hold the model's reasoning, the causal effect of the format constraint on accuracy is near zero.

→
2026.06.10 EVAL

Your agent's benchmark score is not its reliability.

A benchmark score is a point estimate on a curated, static, in-distribution task set. Production reliability is a distribution under drift and stress — and the measured gap between the two widens exactly as conditions get harder.

→
2026.06.10 EVAL

Your LLM router is probably losing to the baseline.

An LLM router is supposed to send each query to the cheapest model that can handle it. Under unified evaluation, many routers — including a commercial one — fail to beat the trivial baseline of always using the single best model. The dominant failure is routing collapse to the expensive model, and the cause is structural: routers are trained to predict scalar quality scores for what is really a discrete ranking decision.

→
2026.06.08 EVAL

An agent benchmark a do-nothing agent can win.

On a popular agentic benchmark, an agent that returns nothing scores as successful, and passing the bundled unit tests need not mean the bug was fixed. Auditing the grader, not the agent, moves headline scores by tens of points — so a leaderboard rank is an artifact of its checker until proven otherwise.

→
2026.05.27 EVAL

GraphRAG or vector RAG: a decision guide.

A knowledge graph earns its build-and-maintenance cost over plain vector retrieval only for specific query classes — multi-hop reasoning and global summarization. For fact lookup it loses, and it always costs more to index. A guide by query type.

→
2026.05.25 EVAL

Hybrid search: when BM25 still beats your embeddings.

Dense embeddings win in-domain and on paraphrase. Lexical BM25 wins on rare exact terms and on corpora the embedding model never trained on. Production retrieval usually needs both — and a clear reason for the split.

→
2026.05.22 EVAL

Reranking is the cheapest RAG win you are not using.

A cross-encoder reranker often beats retrieving more documents — it scores the query and passage together, where vector search only compared two embeddings made apart. But it scales linearly with candidates and does not always earn its latency. When to add one.

→
2026.05.20 EVAL

Eval-driven development: write the eval before the feature.

An eval written after a feature ships can only ratify what the feature already does. Written first, the graded set becomes the specification — it forces the ambiguity out of the requirement before any code exists. Why the eval is the real spec.

→
2026.05.16 EVAL

Stop OCR-ing your PDFs: retrieve on the page, not the transcript.

Most document RAG OCRs a PDF, rebuilds the layout, and chunks the text — losing every table, chart, and column it touches. ColPali-class visual retrieval embeds the page image directly. When that wins, when it doesn't.

→
2026.05.16 EVAL

Faithfulness is not groundedness. And "accuracy" is not a RAG metric.

Teams say their RAG is 'accurate' and mean different things. Faithfulness is whether the answer is true; groundedness is whether every claim traces to a source. They fail differently — and a deploy depends on measuring both.

→
2026.04.29 EVAL

Your golden set is rotting.

A golden evaluation set is not a fixed asset — it decays. The world changes, the product shifts, the team overfits, and the pass rate quietly stops meaning anything. Eval data needs a maintenance protocol.

→
2026.04.22 EVAL

You don't have a RAG problem. You have a chunking problem.

Most teams blame the retriever. The retriever is fine. Your chunks don't carry their context — and no amount of reranking saves them.

→
2026.03.30 EVAL

LLM-as-judge is a model you also have to evaluate.

Teams wire an LLM into the eval harness as the judge and treat its scores as ground truth. But the judge is a model — with measurable biases, shaky calibration, and silent drift. Evaluate it before you trust it to gate a deploy.

→
NEW ENGAGEMENT · INTAKE

Tell us about it.

The more specific you are, the more useful our first reply.

SERVICE AREA
↩ ENCRYPTED IN TRANSIT
ASK THE FIELD NOTES BETA