◇ ARCHIVEPAGE 9 / 10 · OLDEST → NEWEST

The field notes archive.

2026.08.28 SECURITY

A handful of documents owns your RAG answers.

RAG corpora ingest documents from sources no one fully trusts, on the assumption that a wrong document is just noise the retriever averages out. It is not noise. A few crafted passages — sometimes one — reliably flip the answer to a targeted question.

14 min →

2026.08.29 OPINION

Your empirical privacy defense never beat the DP-SGD baseline you declined to run.

Heuristic privacy defenses report safety by beating weak attacks on average-case samples. Audit them honestly and they leak about 10x more — and lose to the provable DP-SGD baseline they never ran, even one whose formal epsilon is vacuous.

13 min →

2026.09.02 EVAL

Your retriever's recall is not your answer quality.

Teams optimize the retriever — chase recall@k, retrieve more documents — and expect answer quality to follow. It does not. Recall and answer accuracy are decoupled, and past a small number of documents, more retrieved context routinely makes the answer worse.

16 min →

2026.09.02 EVAL

A perfect needle-in-a-haystack score tells you almost nothing about long context.

A 99% needle-in-a-haystack number is bought by literal word overlap between the question and the planted fact. Strip the overlap, or make the haystack realistic, and the same model collapses. NIAH does not predict long-context ability.

13 min →

2026.09.06 OPERATIONS

You are out of inference capacity because you are out of power, not GPUs.

The dashboard counts GPUs, but the breaker and the chiller count watts — and a decode phase that draws a fifth of TDP leaves a row half-empty in power terms. That headroom is oversubscribable capacity most operators never claim.

15 min →

2026.09.07 EVAL

The same RLHF that made your model helpful made its confidence a liar.

A pre-alignment calibration number does not survive RLHF: the reward model prefers high-confidence-sounding answers regardless of correctness, so abstention and routing built on the model's own confidence break on the deployed model.

16 min →

2026.09.07 AGENTS

Your agent's reliability is an exponential, not an average.

Teams read a 92%-per-step success rate as a 92%-reliable agent. The arithmetic is multiplicative, not average: at 92% per step, a 30-step task succeeds barely one time in twelve. Reliability is an exponential in step count.

15 min →

2026.09.07 OPERATIONS

FIFO is the default scheduler in your inference server and the reason its tail latency is bad.

Continuous batching did not abolish first-come-first-serve — your inference server still admits requests in arrival order, so one long generation blocks every short request behind it and p99 explodes. Shortest-job-first is the tempting fix and it just moves the tail.

15 min →

2026.09.09 PRIVACY

You ran the model on-device for privacy and leaked the conversation through the cache.

On-device inference moves the prompt off the network so there is nothing to intercept. It does not move it off the CPU cache — and an unprivileged co-resident process reads your input and output text back from there at near-perfect fidelity.

13 min →

2026.09.11 STANDARDS

Your text watermark does not survive a paraphrase.

Output watermarking is offered as the provenance answer for AI-generated text. A cheap paraphrase removes it at near-100% — so does a translation round-trip, and so does reverse-engineering a watermark built to resist exactly that. Provenance needs signing, not a statistical signal.

15 min →

2026.09.11 PAYMENTS

Your agent's treasury is one depeg away from being unable to pay.

Settling an agent in a stablecoin imports that coin's run-and-depeg risk straight into its treasury — and the safe-looking instinct, pick the most transparent coin, can be exactly backwards when reserves are weak.

13 min →

2026.09.11 STANDARDS

Nothing checks your provenance manifest against your AI watermark.

A C2PA Content Credential and an AI watermark are two authenticity layers, and a pipeline that runs both feels like defense in depth. But each validator only checks its own layer — so an asset can carry a cryptographically valid manifest claiming a human authored it and a watermark identifying it as AI-generated, and every validator passes it. The layers interact for manifest recovery; no deployed validator cross-checks their claims.

11 min →

The field notes archive.

A handful of documents owns your RAG answers.

Your empirical privacy defense never beat the DP-SGD baseline you declined to run.

Your retriever's recall is not your answer quality.

A perfect needle-in-a-haystack score tells you almost nothing about long context.

You are out of inference capacity because you are out of power, not GPUs.

The same RLHF that made your model helpful made its confidence a liar.

Your agent's reliability is an exponential, not an average.

FIFO is the default scheduler in your inference server and the reason its tail latency is bad.

You ran the model on-device for privacy and leaked the conversation through the cache.

Your text watermark does not survive a paraphrase.

Your agent's treasury is one depeg away from being unable to pay.

Nothing checks your provenance manifest against your AI watermark.

Tell us about it.

Got it.