Heterogeneous TEE-GPU inference shields the non-linear layers and offloads the projection and embedding layers to the GPU. Those offloaded layers are the ones a sub-$20 black-box attack already steals — and attestation never measured them anyway.
A team is standing up confidential inference for a proprietary model, and they have done the homework that gets a clean security review. The model runs inside a Trusted Execution Environment. The enclave produces a remote attestation, and before any client sends a token of input, it verifies that attestation against the vendor’s measurement: the right enclave image, the right code, untampered, genuine silicon. On the strength of that attestation, the model owner’s key is released and the weights are pulled in. Everyone in the room reads the same conclusion off the report — the model is protected, because the attestation says the enclave is exactly what it claims to be.
Now look at what that attestation actually measured, and what it did not. It measured the enclave’s code. It did not measure which weights are in memory, because the weights loaded from disk after the enclave booted — and on the performance-tuned designs this team will reach for next, it could not have measured half of those weights anyway, because half of them are not in the enclave. They are on the untrusted GPU. The attestation certified the binary. The part of the model an attacker is paying to steal sits where the binary measurement never reached.
This post is about a gap that two true things open when you set them side by side. First: standard TEE attestation binds the code, not the loaded model. Second: the heterogeneous TEE-GPU split that confidential-inference systems are converging on offloads exactly the projection and embedding layers that published black-box attacks already extract. The split protects the layers that were never the prize and exposes the ones that are, and the attestation you trust to say the model is safe was never speaking about the weights.
Start with the narrower of the two claims, the one most people get wrong with complete confidence. A remote attestation is a signed statement from the hardware about what was loaded into a protected environment. In a real confidential-inference stack — the kind Red Hat’s Emerging Technologies group describes — attestation “cryptographically proves the vault is authentic and untampered.” That is the gate, and the weights come through it only afterward: “only upon successful mutual attestation is the enclave authorized to pull the encrypted OCI image and the LLM provider’s private key permitted to be securely transferred to the enclave.” The attestation comes first; the weights come after. The thing measured and the thing protected are two different objects.
Tinfoil, whose whole business is confidential inference, states the consequence in one sentence: “attestation measures launch state, not runtime state. Weights are loaded from disk storage after the enclave has already booted.” So “if you rely on basic attestation, the attestation report will not include this post-boot loading from disk storage.” The report can be valid, fresh, signed by genuine hardware, and verified by a careful client — and still say nothing about which model is answering your queries.
This is not an implementation slip in one product; it is the shape of attestation itself. The canonical statement comes from outside the LLM world entirely: in Proof of Cloud (arXiv 2510.12469), the authors note that “existing CVM attestation mechanisms only certify what code is running, not where it is running.” Their gap is geography, and the silence is exploitable: “this gap enables proxy attacks in which valid attestations are combined across machines to falsely attest trusted execution.” Attestation certifies the code measurement and is mute on everything outside it. For Proof of Cloud the unmeasured fact is location. For confidential inference it is the weights.
And closing that kind of gap is never free. Proof of Cloud does not get location from the standard quote; it builds it, introducing “Data Center Execution Assurance (DCEA), a design that generates a cryptographic Proof of Cloud by binding CVM attestation to platform-level Trusted Platform Module (TPM) evidence.” Binding attestation to an additional, independent piece of evidence is the whole technique — and the analog for weights is to bind it to the loaded model, which the fact that people are building exactly that machinery confirms the base case does not do. Tinfoil’s fix is “to attest that the enclave includes both the expected hash of the data and some code that will check that hash after it’s loaded” — a Merkle-root commitment plus a dm-verity-style check, bolted on because the raw attestation does not reach the post-boot load. The on-device world built its own bolt-on: AttestLLM (arXiv 2509.06326) concedes that “existing attestation techniques are not suitable for billion-parameter Large Language Models (LLMs)” and so embeds “robust watermarking signatures onto the activation distributions of LLM building blocks,” checked in the TEE, because the code measurement alone will not tell you which model loaded — and even that protects “the hardware-level intellectual property (IP) of device vendors by ensuring that only authorized LLMs can execute,” an authorization property, not the confidentiality of the offloaded weights.
So far this is a gap in what attestation covers. It becomes a privacy problem because of where the architecture puts the weights — and three independent 2025-2026 systems put them in the same place, for the same reason: performance. Shielding an entire model inside a CPU TEE is slow — No Privacy Left Outside (arXiv 2310.07152) measures “the sharply increased latency (up to 50X)” of full in-enclave execution — so nobody ships the simple, safe design. To recover GPU speed, systems partition the model: keep a “sensitive” slice in the enclave and offload the heavy linear compute to the accelerator, since, as Towards Confidential and Efficient LLM Inference with Dual Privacy Protection (arXiv 2509.09091) puts it, “due to high inference latency in TEEs, researchers use partition-based approaches that offload linear model components to GPUs.”
Now look at which slice they keep and which they offload. SecureInfer (arXiv 2510.19979) is the split in its purest form: “security-sensitive components, including non-linear layers, projection of attention head, FNN transformations, and LoRA adapters, are executed inside an SGX enclave, while other linear operations (matrix multiplication) are performed on the GPU after encryption and are securely restored within the enclave.” The big linear matmuls — the bulk of the parameters — go to the GPU. Its stated threat is model extraction — “securing them against model extraction attacks has become a pressing concern” — and yet it names that adversary and then hands much of the model to what it itself calls “untrusted accelerators.”
The Dual Privacy system, CMIF, makes a different cut and lands on the same exposure: it “confidentially deploys the embedding layer in the client-side TEE and subsequent layers on GPU servers.” Where SecureInfer keeps the attention projections inside, CMIF keeps the input embedding inside and offloads everything after it. Two systems, two different “sensitive” slices, one identical consequence: the overwhelming majority of the weights live on the untrusted GPU. And CMIF’s threat model is not even the weights — it “optimizes the Report-Noisy-Max mechanism to protect sensitive inputs,” guarding the user’s input with differential privacy, not the offloaded parameters.
A third system confirms the convergence is no coincidence. MirageNet (arXiv 2601.13826) is built on the same split, its ConvShatter scheme storing “minimal recovery parameters securely in the TEE” while reconstructing obfuscated layer outputs during inference. Tellingly, it spends its effort obfuscating the offloaded linear weights — “injects confounding decoys and permutes channel/kernel orders” — precisely because those layers sit on the untrusted GPU and would otherwise be exposed, and justifies the architecture on speed, cutting “overhead by 16% relative to GroupCover.” When three systems independently choose the same split and one has to add decoys and permutation to defend the offloaded side, the offloaded linear weights are the asset at risk, by the designers’ own admission.
Here is where the two halves meet. The layer these systems push to the GPU is no low-value remainder — for one of the cuts it is the single most extractable layer in a transformer, and the attack needs only the public API.
Stealing Part of a Production Language Model (arXiv 2403.06634) demonstrates a black-box attack that “recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access.” No enclave compromise, no side channel — just ordinary queries. And it is cheap, proven on real production models: “for under $20 USD, our attack extracts the entire projection matrix of OpenAI’s Ada and Babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively.” It scales toward frontier endpoints: the authors “also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under $2,000 in queries to recover the entire projection matrix.”
Sit with that. CMIF keeps the embedding layer in the TEE — defensible, since that projection is exactly what Carlini et al. steal from a black box. But the broader point lands hardest on the partition premise itself, whose whole justification is that shielding part of the model downgrades the attacker’s position — No Privacy Left Outside (arXiv 2310.07152) describes how shielding on-device models “aims to downgrade (easy) white-box attacks to (harder) black-box attacks.” That downgrade is supposed to be the safety margin. But a sub-$20 black-box attack already recovers a production model’s projection layer; if black-box access is enough to take the prize, the downgrade buys no safety on the layer that matters. And the paper does not stop at the premise — across architectures, datasets, and metrics it finds “existing TSDP solutions are vulnerable to privacy-stealing attacks and are not as safe as commonly believed.” The architecture our team is about to adopt has already been measured and found leaky, against the exact attacker it names.
The argument is sharp, so be equally sharp about its limits — the version that overstates loses the reader who knows the internals.
The most important caveat is about which “projection” is which. SecureInfer keeps the attention-head output projection inside the enclave — the projection inside each transformer block. That is not the embedding/unembedding projection that produces the final logits, the layer the sub-$20 attack recovers. So “the system offloads the exact bytes the published attack steals” is precise for CMIF’s cut and the linear bulk, and a category claim, not a byte-for-byte one, for SecureInfer’s attention projections. Do not conflate the two; the people who build these systems do not.
Nor are “the projection layer is extractable from a black box” and “the offloaded weights are wide open” the same statement. The black-box attack works from the API and does not even need the offloaded weights to be visible — which, if anything, sharpens the point: those layers are exposed through the front door regardless of what runs on the GPU. An attacker positioned on the untrusted accelerator is a separate, additional threat that MirageNet’s decoys respond to directly. The two threat models stack; neither cancels the other.
And none of this says TEE confidential inference is worthless — the honest reading is the opposite. Attestation does prove the environment is genuine and untampered, which is real and load-bearing; it is the gate that makes the rest possible, and the weight-binding work from Tinfoil and AttestLLM closes the launch-versus-runtime gap when deployed. The failure is specific: the unstated assumption that a clean attestation report implies the model is protected, when it describes the binary, the weights load afterward, and on the performance split a large share never enter the measured enclave at all. The mitigations exist. The gap is in believing you do not need them.
You cannot make a useful model uninvertible from its API, and on a performance-driven split you cannot keep every parameter inside the enclave. What you can do is stop reading off the attestation a guarantee it does not give.
Bind the weights, do not infer them from the binary. Treat the attestation as covering the code and nothing else, then add the weight commitment explicitly — a hash of the model folded into the measured launch config and enforced on every read. An attestation that did not bind a weight hash told you about the enclave, not the model.
Know which layers leave the enclave, price the attack on them, and protect them where they land. A heterogeneous split is a decision about which weights sit on untrusted hardware; write down the partition. For any offloaded layer — especially an embedding or output projection — assume the black-box recovery cost is the one Carlini et al. measured, not infinity, and put the protection on the accelerator where MirageNet does: decoys, permutation, encryption restored only inside the enclave, not a line in the attestation report.
Do not let “model extraction” go unmeasured. SecureInfer and the others name model extraction as the adversary; name it in your own threat model, then test it. Run the black-box projection-recovery attack against your own endpoint and measure what it pulls, rather than assuming the enclave stopped it.
Separate legitimacy from confidentiality. AttestLLM-style attestation answers “is this an authorized model,” not “can the offloaded weights be read” — a deployment that proves the first and assumes it got the second for free has conflated two different guarantees.
The TEE-vs-FHE question of whether to trust silicon or mathematics for the parts you do shield is a separate decision, laid out in FHE vs TEE for ML; this is the prior question of whether the attestation you already trust even describes the weights. And if you are reaching for confidential inference to protect a private corpus rather than the model, that corpus has its own leaks — covered in confidential RAG — and the embeddings it produces invert back to text whether or not the model is in an enclave, the subject of your vector database is a plaintext store.
The attestation report is honest about exactly one thing: the enclave is running the code you measured. It was never a statement about which weights are in memory — they load from disk after it is signed, and on the performance split, a large share of them are not in the enclave to be measured at all. They are on the GPU, where the published attack already reaches them. Verify the attestation. Then bind the weights, because the binary is not the part anyone is trying to steal.
A membership-inference attack that separates your book from a held-out one is usually reading the calendar, not the weights. A successful attack is not a sound proof — and the null it would need to test against cannot be sampled.
17 min →
Unlearning promises a model can forget specific data without a full retrain. What it delivers is output suppression: a small benign relearning set — or simply quantizing the model — brings the 'forgotten' knowledge back.
14 min →
The EU AI Act requires AI output to be marked in a robust, reliable way. A formal impossibility theorem says strong watermarking cannot exist. Five words in the statute carry the collision.
13 min →