# You ran the model on-device for privacy and leaked the conversation through the cache.

A team builds a feature the careful way. The model that reads the user's messages runs on the user's own device — a laptop, a phone, a workstation that never sends the conversation off-box. The pitch writes itself, and it is the pitch the whole category is built on: the prompt never leaves the machine, so there is nothing on the wire to intercept, no provider to subpoena, no traffic for an ISP to classify. The threat model everyone learned to fear — words read in transit — is closed by construction. Local inference is the private option because locality removes the network, and the network was the channel everyone was watching.

The network was _a_ channel. It was not the channel that matters once the model runs next to you. A co-resident, unprivileged process — a browser tab, a background app, anything sharing the CPU — can read the user's input and the model's output off the hardware cache and the timing of the decode loop, and reconstruct the conversation at near-identical fidelity to the original. The data did leave the network; it landed on a microarchitectural side channel the privacy pitch never accounted for, and that channel carries the plaintext, not a summary. This post is about why "local" is not "private," why the autoregressive design leaks both halves of the conversation through the same door, and what actually narrows the exposure.

## Local removed the network, not the observer

The premise is true as far as it goes. On-device inference is, in the words of the systems work that builds it, "driven by increasing privacy concerns and advancements of mobile-sized models" — [_Fast On-device LLM Inference with NPUs_](https://arxiv.org/abs/2407.05858) (arXiv 2407.05858) names privacy as the reason the category exists. The prompt stays on the device, no remote party sees it, and for the network adversary that is the end of the story. The error is in what "the device" is assumed to be: a single trusted boundary, safe once the data is inside. But a modern machine is not one tenant. It runs a browser with a dozen origins, background OS services, app code you forgot you installed, and on a shared box, someone else's processes — and the CPU's caches are shared across all of it. The model's memory-access patterns, while it runs, are visible: not as content (the memory is protected) but as _timing_, which cache lines get touched and when. An attacker who shares that cache need not read the model's memory; it watches the model use it. This is no reframing — the defensive literature already treats the on-device environment as adversarial. [_Amulet: Fast TEE-Shielded Inference for On-Device Model Protection_](https://arxiv.org/abs/2512.07495) (arXiv 2512.07495) describes the state of practice plainly: "the current mainstream solution for on-device model protection is storing the weights and conducting inference within Trusted Execution Environments (TEEs)." You do not bring a TEE to a place you trust. The mainstream defense exists precisely because the device's own untrusted memory and processes are assumed hostile — so "nothing to intercept" was already known to be false.

## The embedding lookup is a token oracle

The mechanism is not an exploit in the usual sense — no malformed input, no overflow, no leaked key. Every transformer-based LLM begins each token's journey by looking up its embedding vector in a table. The vocabulary is fixed and known — tens of thousands of tokens, each mapped to a distinct vector at a distinct memory location — so processing a token accesses that token's row, pulling a specific, identifiable set of cache lines into the CPU cache.

That is the whole vulnerability. An attacker who can tell _which_ embedding rows were just accessed knows _which_ tokens were just processed. The primitive that reads it is Flush+Reload: flush a line out of the shared cache, wait, then time the reload — fast means something else touched it and pulled it back (a cache hit), slow means it did not. Pointed at the embedding table, each cache hit names a token. [_Spill The Beans: Exploiting CPU Cache Side-Channels to Leak Tokens from Large Language Models_](https://arxiv.org/abs/2505.00817) (arXiv 2505.00817) states the construction exactly: "we flush and reload embedding vectors from the embedding layer, where each token corresponds to a unique embedding vector. When accessed during token generation, it results in a cache hit detectable by our attack on shared lower-level caches."

The companion result, [_I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference_](https://arxiv.org/abs/2505.06738) (arXiv 2505.06738), names the two pieces of information this yields: "adversaries can infer the token values from the cache access patterns of the token embedding operation, and deduce the token positions from the timing of autoregressive decoding phases." Token values from the cache pattern — _what_ was said. Token positions from decode timing — _where_ each token sits, which lets you reassemble values into ordered text rather than a bag of words. It reconstructs the message, not a word cloud of it.

And the attacker is not privileged and never touches the model. The same paper is explicit: "the attack framework does not directly interact with the victim's LLM and can be executed without privilege." There is no API call to rate-limit, no query to log, no anomalous access to detect — it runs its own code, watching its own cache, inferring a neighbor it never speaks to. "Nothing to intercept" assumed interception meant touching the data path; this touches nothing on the path and reads the data anyway.

## Autoregression leaks the reply, not just the prompt

There is a tempting partial comfort here — that maybe only the input leaks, the part the user typed, while the response stays private. The architecture refuses it, for a structural reason. Autoregressive generation feeds its own output back in: each emitted token becomes input for the next step, through the same embedding lookup the prompt went through. [_I Know What You Said_](https://arxiv.org/abs/2505.06738) (arXiv 2505.06738) draws the consequence directly: "in response to the input, the LLM generates an output text … each new token is sent to the token embedding operation. Therefore, the token embedding operation leaks both the victim's input and output tokens."

So the single cache channel that betrays the prompt betrays the reply, on the same hardware, with no extra attacker effort: the output traverses the exact path the input did, so there is no separate "output protection" to add. The decode loop that makes these models work — emit, feed back, emit again — walks both sides of the conversation past the attacker's cache one token at a time. Not by coincidence, it is the loop the systems literature optimizes: [_Fast On-device LLM Inference with NPUs_](https://arxiv.org/abs/2407.05858) (arXiv 2407.05858) documents the prefill/decode split that defines on-device inference, and it is the per-token decode stage — the one whose timing leaks positions — that the attack reads. Performance structure and leak structure are the same.

## The recovered text is near-identical, and the secrets fare worst

A side channel that recovered "roughly the gist" would be a footnote; these do better, and the numbers are the reason this is a post. [_I Know What You Said_](https://arxiv.org/abs/2505.06738) (arXiv 2505.06738) measures reconstruction fidelity two ways: "the restored output and input text have an average edit distance of 5.2% and 17.3% to the ground truth, respectively. Furthermore, the reconstructed texts achieve average cosine similarity scores of 98.7% (input) and 98.0% (output)." An edit distance of 5.2% on the output means the recovered reply differs from the true one by about a character in twenty; a cosine similarity near 98% means the meaning is, for any practical purpose, intact. This is not topic inference. It is transcription — demonstrated on the models people actually run locally (Llama, Falcon, Gemma), not a lab-only toy.

The independent result sharpens the part that should worry anyone who pastes a credential into a local model. [_Spill The Beans_](https://arxiv.org/abs/2505.00817) (arXiv 2505.00817) reports "an attacker can recover as much as 80%-90% of a high entropy API key with single shot monitoring. As for English text we can reach a 40% recovery rate with a single shot." Read the asymmetry. English prose recovers at a more modest 40% per shot, because language is predictable. But a high-entropy secret — an API key, a token, a password, the very thing local inference is supposed to keep off the network — recovers at 80% to 90% in one shot, because there is no redundancy smoothing it out and every recovered token is a token of the secret. The attack is most precise exactly where the data is most sensitive, the inverse of what you want from a privacy boundary. Two independent 2025 papers, on the same embedding-layer cache mechanism against different model stacks, have turned this from a single demonstration into a property of how these models run on shared hardware.

## The network channel is the one everybody hardened

It clarifies the seam to look at the channel people _did_ account for. [_Whisper Leak: a side-channel attack on Large Language Models_](https://arxiv.org/abs/2511.03675) (arXiv 2511.03675) shows that even with TLS protecting the content, "a side-channel attack that infers user prompt topics from encrypted LLM traffic by analyzing packet size and timing patterns in streaming responses … these metadata patterns leak sufficient information to enable topic classification." Notice what it recovers and what it does not. It recovers _topics_ — classified across 28 popular LLMs at often above 98% AUPRC, holding even at a 10,000:1 noise-to-target ratio, hitting 100% precision on a topic like "money laundering" while recovering 5-20% of target conversations — which subject you were discussing, inferred from the shape of encrypted packets, not the plaintext of your prompt. And it is the defended surface: the paper evaluates three mitigations (random padding, token batching, packet injection) and reports that "while each reduces attack effectiveness, none provides complete protection." Mitigations exist and are studied, because the network is the channel everyone knew to watch.

Line the two up and the trade is stark. The network channel is encrypted, mitigated, and leaks topics; the cache channel is unencrypted by the very fact of running in shared memory, unmitigated in typical local deployments, and leaks the plaintext at 98% cosine similarity. Going on-device closes the first completely and relocates the whole conversation onto the second — both more revealing and less defended. Everyone reasoned about the network adversary and defeated it by removing the network, hardening the door already being watched and walking the data out through a wall nobody was.

## What this is not, and where it stops

The argument overstated loses the reader who knows the hardware, so here are the load-bearing limits. The attack requires co-residency. The adversary must run code on the same physical machine, sharing the same cache — [_Spill The Beans_](https://arxiv.org/abs/2505.00817) (arXiv 2505.00817) frames its threat model as exactly this: "by co-locating an attack process on the same hardware as the victim model, we flush and reload embedding vectors from the embedding layer." A machine running only software you fully trust, with no untrusted co-tenant and no hostile background code, is not exposed. The danger lands in the realistic cases — a multi-tenant box, a personal device running a browser (every visited page is co-resident code) or an app of uncertain provenance, a managed endpoint where another team's process shares the CPU. "Co-resident, unprivileged process" is the default condition of most machines doing real work — but it is a precondition, and a single-tenant device with a clean software supply chain does not meet it.

Two further caveats keep the numbers honest. The rates are per shot: the 40% on English text is single-shot, the attacks improve with repetition, and the 80%-90% API-key figure is alarming precisely because high-entropy secrets do not need many shots. But "40% per shot on prose" is a degraded, probabilistic transcript, not a clean wiretap out of the box — the secrets are the worst case, not the average. And the fidelity figures come from research harnesses on specific stacks (Llama, Falcon, Gemma) under controlled conditions: strong evidence the channel is real and high-bandwidth where tested, not proof every quantization and runtime exposes the embedding table identically. But the mechanism is fundamental to how autoregressive models use an embedding table on shared CPU caches, so the burden is now on any "local equals private" claim to show its deployment closes the channel, not on the attacker to prove it open.

## What actually narrows the leak

A software fix in the inference library helps at the margin, but the control that changes the leak math is keeping the inference state out of memory the rest of the machine can observe — running the model in an enclave shielded from the OS, hypervisor, and co-resident processes. That is the TEE the on-device defense literature is built around. But [_Amulet_](https://arxiv.org/abs/2512.07495) (arXiv 2512.07495) is equally clear about the bill: the naive partitioned-TEE approach, forced by limited trusted memory, triggers constant world-switches, and "this frequent interaction between untrusted and trusted worlds dramatically increases inference latency, sometimes by orders of magnitude." Amulet's own obfuscation scheme gets that to 2.8-4.8x unprotected latency — better, still a multiple, not free. Most local deployments are exposed not from ignorance but because that fix is expensive enough that the default is to run in untrusted memory and hope the box is clean. Three things follow for anyone calling on-device inference private.

**Stop equating local with private in the threat model.** "The prompt never leaves the device" defeats the network adversary and says nothing about the co-resident one. If the device can run untrusted code, the threat model must name the cache side channel explicitly — it is not a bug a patch removes, it is a property of running the model in shared memory.

**Decide whether the box is single-tenant, and enforce it if you depend on it.** The clean escape is a device with no untrusted co-resident code. Removing co-residency is a legitimate defense, but only if the claim is real and maintained — not "a personal laptop, which also runs a browser."

**If the conversation carries secrets, the cache channel is the channel — price the enclave.** The 80%-90% single-shot key recovery is the number that should drive the decision. Inference handling credentials or high-entropy secrets on a multi-tenant or untrusted-software box needs the state shielded, which in practice means a TEE and its latency multiple. The right tool is the same one the [FHE-vs-TEE decision](/blog/fhe-vs-tee-for-ml/) turns on — confidential compute that keeps state out of observable memory — and the same trade [confidential RAG](/blog/confidential-rag/) makes for the context channel. This is mitigation with a measured cost, paid on purpose by someone who has seen both the recovery and the latency numbers.

## Reading list

- [_I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference_](https://arxiv.org/abs/2505.06738) (arXiv 2505.06738) — recovers token values from embedding-lookup cache patterns and token positions from decode timing, reconstructing input and output text at 98.7%/98.0% cosine similarity, without privilege and without touching the model.
- [_Spill The Beans: Exploiting CPU Cache Side-Channels to Leak Tokens from Large Language Models_](https://arxiv.org/abs/2505.00817) (arXiv 2505.00817) — independent confirmation that Flush+Reload on the embedding layer leaks generated tokens, recovering 80%-90% of a high-entropy API key in a single shot from a co-located process.
- [_The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems_](https://arxiv.org/abs/2409.20002) (arXiv 2409.20002) — establishes the same class of cache-and-timing channel in shared LLM infrastructure, recovering prompts token-by-token at 89.0% accuracy against real commercial services.
- [_Fast On-device LLM Inference with NPUs_](https://arxiv.org/abs/2407.05858) (arXiv 2407.05858) — the systems-side confirmation that privacy is the marketed reason for on-device inference, and that the prefill/decode split the attack exploits is intrinsic to how these models run locally.
- [_Amulet: Fast TEE-Shielded Inference for On-Device Model Protection_](https://arxiv.org/abs/2512.07495) (arXiv 2512.07495) — confirms from the defense side that the on-device environment is treated as adversarial, and that the fix that closes the channel inflates latency by orders of magnitude in the naive case.
- [_Whisper Leak: a side-channel attack on Large Language Models_](https://arxiv.org/abs/2511.03675) (arXiv 2511.03675) — the contrast case: the network channel that people did harden, which leaks only topics from encrypted traffic and still resists complete mitigation, sharpening why going on-device trades a defended topic-leak for an undefended plaintext-leak.

Going local is a real privacy win against the adversary it was designed for — the one on the wire. It is not a privacy property in itself. The conversation does not vanish when it leaves the network; it moves into the device's shared memory, where the model's own embedding lookups spell out every token, input and output alike, to any process patient enough to watch the cache. The data never left the device. That was the point, and it is also the problem.