Proof of Tech
Menu
← Field notes
◇ ARCHIVEPAGE 8 / 10 · OLDEST → NEWEST

The field notes archive.

2026.08.13 AGENTS

Giving your agent more memory makes it less reliable, not more.

The default 'append every turn, summarize, re-trust' memory design is not neutral storage. Benign, un-attacked growth drifts stored knowledge off ground truth and erodes constraints — the fix is admission control and bounded distilled state, not capacity.

→
2026.08.16 AGENTS

Your agent's benchmark number is mostly the harness you didn't disclose.

A leaderboard row reads as a property of the model, but it is jointly produced by the model and the undisclosed execution harness. Swapping the harness on a fixed model moves the score far more than the model does — often enough to reverse the ranking.

→
2026.08.19 PRIVACY

Your confidential-inference enclave attests the binary, not the half of the model an attacker actually wants.

Heterogeneous TEE-GPU inference shields the non-linear layers and offloads the projection and embedding layers to the GPU. Those offloaded layers are the ones a sub-$20 black-box attack already steals — and attestation never measured them anyway.

→
2026.08.19 STANDARDS

ERC-8004 gives an agent a name and a place to post reviews. It cannot tell you the agent is honest.

ERC-8004 standardizes who an agent is and what others said about it, then concedes in its own text that it cannot guarantee the agent is functional or non-malicious. The Validation Registry records a check; it does not run one. The standard ships the socket, not the plug.

→
2026.08.19 PRIVACY

Your vector database is a plaintext store.

Storing only the embeddings feels like storing a hash — a one-way projection, safe to treat as opaque. It is not: text embeddings invert back to the original text at high fidelity, and the newest attacks need no access to the model that produced them.

→
2026.08.22 PRIVACY

The epsilon on your private model is computed for an algorithm you did not run.

Your trainer shuffles the dataset each epoch; your privacy report cites the math for Poisson subsampling. They are different mechanisms with different guarantees — and the reported epsilon understates the real leakage by up to 4×, up to 10× for the common shuffling variants.

→
2026.08.24 EVAL

Sparse autoencoders make a bad alarm and a great microscope.

Sparse autoencoders are the interpretability advance of the last two years, and the instinct is to wire one in as a runtime monitor for a known concept. The benchmarks say not to: for detecting, probing, or steering a concept you can already name, SAEs lose to a logistic-regression baseline and flip under a one-token adversarial nudge. Their real strength is the opposite job — discovering concepts you did not know to look for.

→
2026.08.24 EVAL

Your LLM judge can be fooled by a colon.

An LLM judge does not just have soft biases you average out — it has content-free inputs that force a 'correct' verdict on demand. A lone colon or raw token gibberish flips judges and reward models at roughly 80-100% rates: a judge used as a reward or gate is exploitable, not noisy.

→
2026.08.24 SECURITY

Your chain-of-thought trace is not an audit log.

An agent's chain-of-thought reads like a record of how it reached an answer. The research says otherwise: CoT is frequently a post-hoc story, swayed by cues the model never mentions — useful to monitor, unsafe to trust as the audit log of what actually happened.

→
2026.08.24 PAYMENTS

Your agent's payment is a public intent, and someone reorders it before it lands.

An honest on-chain agent broadcasts a reorderable intent every time it pays — front-run and sandwiched at agent frequency, that is a continuous tax. Routing private does not close it; it trades execution leakage for centralization into a few builders.

→
2026.08.25 ZKML

Proving your model ran is a different cryptographic object than proving you trained it.

A proof of inference certifies an output came from a committed model. It says nothing about how the weights came to be. Proving the training run is a separate, far heavier object — and it is the one compliance actually asks for.

→
2026.08.27 VOICE

Your streaming transcriber is a worse transcriber, and the benchmark you bought it on was offline.

The WER on the ASR model card was measured with the whole utterance in hand. Your real-time voice agent never gets the whole utterance — it runs a different, worse operating point of the same weights, and no latency ledger records the bill.

→
NEW ENGAGEMENT · INTAKE

Tell us about it.

The more specific you are, the more useful our first reply.

SERVICE AREA
↩ ENCRYPTED IN TRANSIT
ASK THE FIELD NOTES BETA