Menu
← FIELD NOTES
TAGGEDAGENTS 15 POSTS

Posts about AGENTS.

2026.09.07 AGENTS

Your agent's reliability is an exponential, not an average.

Teams read a 92%-per-step success rate as a 92%-reliable agent. The arithmetic is multiplicative, not average: at 92% per step, a 30-step task succeeds barely one time in twelve. Reliability is an exponential in step count.

2026.08.16 AGENTS

Your agent's benchmark number is mostly the harness you didn't disclose.

A leaderboard row reads as a property of the model, but it is jointly produced by the model and the undisclosed execution harness. Swapping the harness on a fixed model moves the score far more than the model does — often enough to reverse the ranking.

2026.08.13 AGENTS

Giving your agent more memory makes it less reliable, not more.

The default 'append every turn, summarize, re-trust' memory design is not neutral storage. Benign, un-attacked growth drifts stored knowledge off ground truth and erodes constraints — the fix is admission control and bounded distilled state, not capacity.

2026.08.13 AGENTS

Poison an agent's memory once, and it drifts forever.

Tiered agent memory does not just store facts — it consolidates and re-trusts them. So a single poisoned write is not corrected out; it is promoted, and it compounds across every later session.

2026.06.23 AGENTS

Tool-calling reliability plateaus at your tool descriptions, not your model.

An agent that calls the wrong tool too often looks like a model problem, so teams reach for a bigger model or a fine-tune. The wrong calls cluster where two tools look near-identical to the model — and rewriting or de-duplicating the ambiguous descriptions recovers double-digit accuracy a bigger model does not. The tool interface is a primary ceiling on tool-calling reliability, and the model's own choice margin is a usable pre-execution alarm.

2026.06.03 AGENTS

Shipping an agent: canaries and rollback for prompts.

A one-word change to a system prompt can move accuracy by dozens of points, and a provider's model update can regress your app overnight. A prompt or model swap is a deploy. Give it a staged rollout and a one-action rollback path.

2026.06.01 AGENTS

Cost observability for an agent fleet.

The monthly inference bill arrives as one number, and nobody can say which agent, which customer, or which tool spent it. Agent cost is too variable to estimate and has to be attributed after the fact — per run, per tool, per tenant. The layer most stacks skip.

2026.05.29 AGENTS

Human-in-the-loop checkpoints without killing throughput.

An agent that asks permission for everything trains its reviewers to rubber-stamp, and the one dangerous action slips through in the noise. Approval gates belong on consequence and on uncertainty — not on every step. Where to put them.

2026.05.18 AGENTS

Tool design for agents: the schema is the prompt.

A tool's name, description, and JSON schema are the only thing the model sees — it picks and fills tools from those words alone. Treating that surface as API plumbing instead of as prompt is why agents call the wrong tool. How to design it.

2026.05.16 AGENTS

Agent memory is a microservice, not a vector store.

"Add memory" usually means "dump everything into a vector DB." Real agent memory is tiered, has a write/forget/consolidate policy, and runs as a service the agent calls — not a library bolted into the loop.

2026.05.16 AGENTS

Your APM cannot see your agent failing.

Request traces and dashboards were built for request/response services. The ways an agent fails — a tool returning 200 with garbage, a truncated context, a looping planner — trip none of them. What agent-native observability has to capture.

2026.05.13 AGENTS

Deterministic replay: debugging agents that will not reproduce.

An agent run is non-deterministic — sampling, tool responses, and timing all vary — so a bug seen once may never recur. Deterministic replay records every non-deterministic input so the run can be replayed exactly.

2026.05.07 AGENTS

Context engineering beats prompt engineering for long-running agents.

For a long-running agent, the system prompt is a small part of the problem. The real discipline is managing the context window across the whole run as a budget — keep, drop, compact, retrieve.

2026.04.14 AGENTS

Multi-agent systems are usually one agent too many.

Splitting a task across coordinating agents adds context-handoff loss, compounding latency and cost, and a wider failure surface — overhead that usually exceeds the benefit. Start with one agent.

2026.03.08 AGENTS

Notes on agent budgets: why "let it think longer" is a bug.

An agent that hits a wall and asks for more compute is not reasoning. It is panicking. The budget is part of the spec, not a fallback.

NEW ENGAGEMENT · INTAKE

Tell us about it.

The more specific you are, the more useful our first reply.

SERVICE AREA
↩ ENCRYPTED IN TRANSIT
ASK THE FIELD NOTES BETA