A clean backdoor scan on a downloaded model certifies one artifact. Quantization, distillation, merging, and fine-tuning each produce a different artifact — and a quantization-activated backdoor is engineered to be invisible in the model you scan and live in the model you serve. For the other transforms the scan misleads you either way. The only sound scan is of the post-transform bytes you actually deploy.
A team downloads a model from a hub and does the responsible thing before trusting it: a backdoor scan, a safety evaluation, the checks that catch a tampered or poisoned model. The model comes back clean. The team deploys it.
But between the model the team scanned and the model the team serves, the deployment pipeline runs a sequence of transforms. The model gets quantized, because the full-precision weights do not fit the serving GPU at the batch size the team needs. It may get distilled into a smaller, faster student. It may get merged with a task-specific fine-tune. It may get fine-tuned on the team’s own data. Each of those steps takes the weights the team scanned and produces different weights. The artifact answering production traffic is not the artifact that passed the scan.
“The model passed the backdoor scan” and “the model we serve is safe” are different claims whenever a transform sits between them — and the gap between those two claims is the subject of this post.
This post is about why a backdoor scan certifies one specific checkpoint and not the pipeline’s output, why quantization in particular can be the trigger an attacker designed the backdoor around, why distillation, merging, and fine-tuning mislead a pre-transform scan in both directions, and why the only scan that means anything is a scan of the bytes you deploy.
A backdoor scan is a measurement of a specific artifact. It examines a particular set of weights and reports whether that set of weights carries a backdoor. The report is true — about those weights.
Quantization produces a new set of weights. So does distillation, so does merging, so does fine-tuning. After any of them, the model you hold is not the model the scan examined, and the scan’s certificate does not automatically travel across the transform. A team that scans the downloaded model and then quantizes it has a clean report about a model it is not serving and no report at all about the model it is.
That would be a manageable gap if the transforms were neutral — if quantizing a clean model reliably produced a clean model and quantizing a backdoored one reliably produced a backdoored one. They are not neutral, and the reason is the part teams miss: an attacker who knows your deployment pipeline can build a backdoor that interacts with the transform on purpose. The transform stops being a thing that happens to the model after the scan and becomes a thing the attacker is counting on. The scan runs before the transform; the backdoor is designed to come alive at the transform. The ordering is the whole attack.
The cleanest, most unconditional version of this is quantization, because there the attacker does not merely survive the transform — the attacker uses it as the trigger.
Exploiting LLM Quantization (arXiv 2405.18137), a peer-reviewed result, constructs a model that is benign at full precision and malicious once quantized. The attacker fine-tunes a malicious model, then adjusts the full-precision weights so that they still round to the same quantized values — so the full-precision model behaves well and the quantized model follows the malicious behavior. The paper reports that up to 97.2% of the time, quantized versions of these models produce vulnerable code where the full-precision model produced secure code, with comparable results for over-refusal and for injecting attacker-chosen content. The paper names the supply-chain shape explicitly: host the clean full-precision model on a hub, and the millions of users who quantize it to serve it deploy the malicious version.
This is not one paper’s trick. Nearest is Not Dearest (arXiv 2405.12725), also peer-reviewed, studies “quantization-conditioned backdoors” — backdoors that are dormant on the released full-precision model and “come into effect after standard quantization” — and reports post-quantization attack success rates of 99.87% at INT8 and 96.74% at INT4, while the backdoors “bypass state-of-the-art detection methods” run on the full-precision model. Mind the Gap (arXiv 2505.23786) carries the same attack into the GGUF quantization formats that dominate local deployment, reporting a behavioral gap between the full-precision and quantized models of up to 88.7% on insecure code generation across nine GGUF data types. And Quantization Blindspots (arXiv 2512.06243) measures what the transform does to the defenses themselves: INT8 quantization drove the detection rate of all five backdoor defenses it tested from 20% to 0%, while attack success stayed above 99%.
Read those together and the quantization case is not “the backdoor might survive.” It is “the backdoor is designed so that the model you scan is genuinely clean and the model you serve is not.” A full-precision scan is not a weak control here. It is structurally blind, because the attacker put the payload precisely where the scan cannot look.
The other transforms are less absolute, and being precise about how is what keeps this post honest.
For distillation, merging, and fine-tuning, a naive backdoor often does not survive — and that is well documented. BadMerging (arXiv 2408.07362), from a top security venue, reports that existing backdoor attacks “all fail to backdoor a merged model” — under 20% attack success — because the arithmetic of model merging rescales the backdoor away. Work on backdoors and distillation finds the same thing: ordinary backdoors transfer poorly from teacher to student, with student attack success often falling into the low single digits. Left there, you might conclude these transforms launder a model safe.
They do not, because the attacker adapts. BadMerging’s contribution is an attack built specifically to survive merging — using a feature-interpolation loss — and it reaches over 90% attack success in the merged model, against which the paper reports existing defenses all fail. Pay Attention to the Triggers (arXiv 2510.18541) does the same for distillation: it constructs backdoors from composite triggers of common tokens specifically so they transfer, and reports student-model attack success far above the few-percent floor that ordinary backdoors hit. Revisiting Backdoor Attacks on LLMs (arXiv 2505.17601) does it for fine-tuning: against a fine-tuning process augmented with safe data, its attack’s success rate drops by only 1.67 percentage points where baseline attacks drop by an average of 33.67.
So for these three transforms the rule is not “the backdoor always survives.” It is “a naive backdoor is often destroyed, and an adaptively constructed one survives at high attack success.” Which means the transform’s effect on a backdoor is not predictable from the transform alone — it depends on whether the attacker designed for it.
The natural objection: if the other transforms sometimes destroy the backdoor, isn’t a pre-transform scan at least conservative — at worst over-cautious?
No, and Backdoor Vectors (arXiv 2510.08016) is the cleanest way to see why. It quantifies the merging case: an injected-trigger backdoor that scans as dangerous before merging is neutralized by merging, its attack success collapsing to near zero, while an adaptively built one survives. Put that together with the rest and a pre-transform scan can fail in two opposite directions. It can show you a backdoor — and that backdoor will not survive the transform, so you reject a model that was, as deployed, fine: a false alarm. Or it can show you nothing — and an adaptive backdoor will survive, or a quantization-conditioned one will switch on: a false clearance.
A scan that can be wrong in both directions is not a conservative scan. It is an uninformative one. Its output has no fixed relationship to the safety of the artifact you deploy, because the artifact you deploy is on the other side of a transform that can add a backdoor, remove a backdoor, or leave one in place. The only scan whose result is about the deployed model is a scan of the deployed model — the exact post-transform, post-quantization, post-merge, post-fine-tune bytes that will answer traffic. That is the conclusion, and it is stronger than “the backdoor survives,” because it holds even in the cases where the backdoor does not.
Three honest qualifications, because the version of this argument that overstates loses the reader who knows the field.
First, the attack-success percentages are attack-paper numbers. “97.2%,” “99.87%,” “over 90%” describe what an adversary who is trying to plant a surviving backdoor can achieve in a study. They are not a base rate of backdoors in the wild, and the post does not claim a randomly downloaded model is probably backdoored. The claim is about what a pre-transform scan can and cannot certify — and that holds whether the wild base rate is high or low.
Second, several of the strongest detection-evasion results are from vision models — the defense-blindspot and quantization-conditioned-backdoor work uses image classifiers and datasets like CIFAR-10 and GTSRB. The mechanism — a transform that defeats detection while preserving the payload — is general, and the LLM-native evidence is real and separate: Exploiting LLM Quantization, the GGUF attack, the distillation and fine-tuning results are all on language models. A post framed around LLMs should lean on the LLM papers and present the vision results as where the mechanism was first measured.
Third, the adaptive attacks require the attacker to anticipate the pipeline — to know, or guess, that you will quantize to INT8, or merge, or fine-tune. An attacker who guesses wrong gets a backdoor that the transform destroys. This does not soften the conclusion; it sharpens the reason for it. You cannot know which case you are in by inspecting the model you downloaded, so you scan the one you deploy.
The control is simple to state and is mostly a matter of moving the scan to the right place in the pipeline.
Scan the post-transform artifact — the exact bytes you will serve. The deployed model is the quantized, merged, distilled, fine-tuned weights, and that is the artifact a backdoor scan and a safety evaluation must run on. A scan of the upstream checkpoint is evidence about a model you are not running.
Re-scan after every transform, not only at the end. Each transform can add, remove, or change a backdoor. Scanning only the final artifact is the minimum; scanning after each stage tells you which stage introduced a problem, and is the difference between catching it and shipping it.
Treat quantization as an adversarial trigger surface, not a neutral compression step. Quantization is the one transform an attacker can use as the trigger, with the full-precision model genuinely clean. Evaluate the model at the precision and in the format you will actually serve — INT8, INT4, the specific GGUF type — because the quantization you choose for serving is also the quantization the attacker chose.
Pin and verify the provenance of the exact deployed artifact. A scan answers “is this artifact backdoored”; provenance answers “is this the artifact I think it is.” Both are needed, and provenance has to cover the post-transform bytes — the weights provenance problem does not end at the download, it ends at the bytes in memory on the serving node.
Do not accept a vendor’s full-precision scan as covering your deployment. A clean scan of a full-precision model says nothing about the INT4 model you will run, and a quantization-conditioned backdoor is invisible to exactly that scan. The vendor’s certificate is about the vendor’s artifact; yours is a different one.
Before you call a deployed model free of backdoors:
The team’s scan was real, its report was accurate, and the model it examined was clean. None of that protects production, because the team scanned a model it then took apart and rebuilt — quantized to fit the GPU, perhaps merged, perhaps fine-tuned — and handed the rebuilt model to its users. A quantization-conditioned backdoor counts on exactly that sequence: be clean for the scan, wake up for the deployment. And even the transforms an attacker has not weaponized make the upstream scan uninformative, because it can flag a backdoor that will not survive and miss one that will. There is one artifact whose safety the team actually needs to know, and it is the one in memory on the serving node. Scan that one. It is the only model the users ever meet.
Our taxonomy argued prompt injection is a vulnerability class you contain, not a bug you fix. Here is the quantitative half — against the strongest published defenses, indirect-injection attack success stays high, and for agents that can act it stays alarming.
10 min →
A quality-ascending curriculum beats random shuffling — until a decaying schedule delivers your best data exactly when the learning rate is too small to absorb it. The same coupling flips proxy-model ablations: which dataset wins depends on the schedule, not the data alone.
13 min →
An agent workflow that pauses for human approval and then resumes looks safe — a person clicked approve. But the approval decision and the authority the resumed step runs under are two different objects, and most systems conflate them by carrying the approval as an in-band signal the resumed call trusts. That is a confused-deputy bug: a forged resume, a replayed request, or an approval granted at one gate authorizes an action it was never meant to. The fix is to stop transporting approval and start deriving authority from the trusted approval record at resume time, scoped to the exact suspended step — capabilities enforced at the tool boundary, fail-closed by default.
14 min →