A diffusion language model decodes many tokens per step instead of one at a time, which reads as a promise of speed. It is not one: every open diffusion model you can deploy today is slower than an equal-size autoregressive model on the same GPU — because a parallel step is only safe on tokens that do not depend on each other, and how many of those a prompt contains is not something the model controls.
A team is choosing a model to serve, and a diffusion language model has made the shortlist on the strength of one unusually concrete pitch. Every autoregressive model — every GPT, every Llama, every Qwen — generates text one token at a time, and each token is conditioned on all the tokens before it. The sequence is a loop, and the loop is the thing that makes generation slow: a thousand-token answer is a thousand forward passes, strictly ordered, no two of them able to run at once. A diffusion language model does not have that loop. It begins with a row of masked positions and denoises them, and at each denoising step it can fill in many positions at once — parallel where the autoregressive model is sequential. The team reads that the way it is meant to be read: fewer steps, faster generation, a smaller GPU bill.
So they put a diffusion model where the autoregressive one was, hold the hardware fixed, and measure tokens per second. The diffusion model is slower. Not by a little, and not because of a misconfiguration they can chase down — several times slower, same GPU, same parameter count, after the obvious tuning.
“Decodes in parallel” and “generates faster” turn out to be different claims. The first is a description of the algorithm’s structure. The second is a wall-clock measurement, and it depends on something the structure does not guarantee. The gap between the two is the subject of this post.
This post is about why every open diffusion language model you can deploy today is slower than an equal-size autoregressive model, why the speedup diffusion can deliver is conditional on a property of the prompt rather than a property of the model, and what to measure before you scope an inference budget around one.
The pitch compresses to a single comparison. An autoregressive model needs N steps to produce N tokens; a diffusion model can do it in far fewer — T steps, T much smaller than N, because each step resolves a block of positions instead of a single one. Fewer steps, the pitch concludes, therefore faster.
The conclusion drops a factor. Generation time is not the step count — it is the step count multiplied by the cost of a step, and the two models do not have the same cost of a step. An autoregressive step extends the sequence by one token, and it does so with a key-value cache: the keys and values for every earlier token are already computed and stored, so the step’s work is one token’s worth of attention and one token’s worth of feed-forward. A diffusion step is a denoising pass over the entire sequence — every position, masked and unmasked — and the cache that makes the autoregressive step cheap is not straightforwardly available, because the set of tokens a diffusion model conditions on keeps changing as positions get filled. A diffusion model buys a smaller T and pays for it with a much larger cost per step. Whether that trade nets out to a speedup is an empirical question. It is not a structural guarantee, and the pitch presents it as one.
Then the second factor, which is worse. The “fewer steps” is itself not free. When a diffusion model fills K masked positions in one step, it draws each of those K tokens from that position’s own marginal distribution — its best guess for that slot, considered on its own. If the K tokens are independent of each other given the context, drawing them separately and drawing them jointly are the same thing, and the parallel step is exact. If the K tokens depend on each other — and tokens in fluent text overwhelmingly do — then the separate draws sample from the product of the marginals, which is not the joint distribution, and the step produces a combination the model would never have generated token by token. So a diffusion model that wants to keep its quality cannot fill many positions per step on the parts of the output that carry meaning. It either unmasks a few positions per step — and then T is not small, and the headline advantage is gone — or it unmasks many and accepts that the joint is wrong. The fewer-steps promise and the quality promise pull against each other, and the tension is not an implementation detail to be engineered away. It is the algorithm.
The reason this is a post and not a hunch is that the comparison has been run carefully — hardware held fixed — by people who went looking for the diffusion model’s advantage and reported what they actually found.
How Efficient Are Diffusion Language Models? (arXiv 2510.18480) is a critical examination of how diffusion-model efficiency gets evaluated, and its controlled comparison is blunt. Benchmarking LLaDA-8B-Instruct, a diffusion model, against LLaMA-3.1-8B-Instruct, an autoregressive model of the same scale, on one A800 GPU: at batch size 1, the autoregressive model has 13.7 times the inference throughput of the diffusion model. The paper also tested the acceleration techniques the diffusion ecosystem offers — dual-cache, parallel decoding — and found that they “mainly offer gains at small batch sizes, with their benefits diminishing upon scaling.” A block-diffusion variant’s batch-throughput advantage over the autoregressive baseline appears only above batch size 2 and is gone again by batch size 4, and the diffusion models ran out of memory at batch sizes 16 and 64 where the autoregressive model did not. The one regime in which a diffusion model is competitive is a single request at a time — the regime a serving system exists to avoid.
Accelerating Diffusion LLMs via Adaptive Parallel Decoding (arXiv 2506.00413) gives the cleanest single-hardware numbers. On one A5000, generating GSM8K answers: an autoregressive Qwen2.5-7B runs at 38.6 tokens per second; the diffusion model Dream-7B at 10.1; LLaDA-8B at 9.33. The diffusion models land within a point or two of the autoregressive model on accuracy and at roughly a quarter of its speed. This is not a paper hostile to diffusion models — its whole contribution is a method to make them faster — and it still reports the equal-size autoregressive model as the throughput baseline to beat, because on the same GPU it is.
Two results, two labs, two hardware setups, one direction. The diffusion model you can download today and put behind an API is slower than the autoregressive model you would otherwise have served.
The diffusion model is not slow because parallel decoding fails to work. It is slow because parallel decoding only works on a subset of the tokens, and which subset is decided by the prompt.
ParallelBench (arXiv 2510.04767) makes the dependency precise. It builds a 17-task benchmark and runs diffusion models — LLaDA, Dream, DiffuCoder, Mercury — against autoregressive baselines, and it finds that under parallel decoding the diffusion models “suffer severe quality degradation in real-world scenarios” that are trivial for an autoregressive model. The degradation is not random. It tracks one quantity the paper makes formal: the conditional total correlation of the output tokens given the input — a measure of how much the output tokens depend on one another. Where that quantity is zero, where the output tokens really are independent given the prompt, parallel decoding is exact and genuinely free. Where it is positive, parallel decoding degrades, and the more the tokens depend on each other, the worse it gets. The paper shows the failure cleanly on list-manipulation tasks, where accuracy converges toward zero as the list grows whenever more than one token is resolved per step, and it shows that single-step generation under temperature sampling has an accuracy ceiling of 1/e — roughly 37 percent — no matter how good the model is, because one step cannot represent the dependencies.
That is the conditionality. A diffusion model’s speedup is not a property it has; it is a property of the text it is asked to write. A prompt whose answer is a list of independent items has a lot of parallelism available. A prompt whose answer is a chain of reasoning, where each step depends on the last, has almost none. The model cannot change which one it was handed.
This is the part that matters for anyone evaluating diffusion models, because it tells you that the techniques are not going to surprise you. Every quality-preserving acceleration method for diffusion LLMs works the same way: it identifies which tokens are safe to decode together — near-independent given the context — and parallelizes only those.
Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models (arXiv 2604.02560) is explicit about it. Its method, DEMASK, puts a small dependency predictor — about 26 million parameters — on top of a frozen 7B diffusion backbone, estimates the pairwise dependency between masked positions, and greedily unmasks only the positions whose cumulative dependency stays under a bound. On Dream-7B it reports a 1.7x to 2.2x speedup while matching or beating the one-token-per-step baseline on accuracy, and it proves a bound on the total-variation distance between its parallel sampling and the true joint distribution. The speedup is exactly the size of the independence the predictor can certify.
Adaptive Parallel Decoding does the same thing by a different route: it multiplies the diffusion model’s marginal probabilities by the joint probability under a small auxiliary autoregressive model, and uses that to decide, per step, how many tokens it can safely emit in parallel. The revealing number is how that count varies. APD emits, on average, 7.62 tokens per step on GSM8K, 6.45 on MATH, and 4.35 on the harder GPQA. The parallel width is not a configuration value the operator sets. It is a reading of the prompt — fewer independent tokens in a hard reasoning problem, so fewer tokens per step, so less speedup. The harder the task, the more the diffusion model is forced back toward decoding one token at a time, which is to say back toward being an autoregressive model with a heavier per-step cost.
So the lever is not the model and not the decoder. The lever is how much token-independence the prompt distribution actually contains — and that is a fact about the workload, knowable only by measuring the workload.
Four honest qualifications, because the version of this argument that overstates loses the reader who knows the field.
First, the claim is “every open diffusion model you can deploy today,” and the word “today” is load-bearing. There is active research converting autoregressive models into block-diffusion models and reporting throughput at or above autoregressive baselines; the gap is a fact about released open weights in mid-2026, not a law of nature. A diffusion model that beats its autoregressive peer on wall-clock is a plausible near-future result. It is not a downloadable one now.
Second, parallel-decoding degradation can be mitigated by chain-of-thought — letting the model write out its reasoning so the hard dependencies are spread over more tokens — but ParallelBench measures the cost: roughly 8x more output tokens. That is a quality-recovery technique, not a speedup, and counting it as one double-counts.
Third, the 13.7x figure and the roughly-4x figure come from different hardware, batch sizes, and shot counts. They agree on direction and they do not compose into a single multiplier; quote each with its setup, and do not average them.
Fourth, “near-independent” is, in practice, mostly estimated rather than proven. DEMASK certifies a bound; APD relies on an auxiliary model’s probability estimate, which can be wrong. The speedup methods are sound engineering, but a team should know that the independence check between them ranges from a proof to a heuristic.
And one thing the post is not claiming: diffusion language models have genuine advantages that are not speed — native infilling, bidirectional conditioning, finer output control. A team may want a diffusion model for those. It should simply not choose one expecting it to be faster, because on today’s open weights it is not.
You cannot read a diffusion model’s speed off its architecture, because the architecture’s parallelism is conditional on the workload. You can measure it.
Measure tokens per second on your own hardware, against the autoregressive model you would otherwise ship. The comparison that matters is same-GPU, same-parameter-count, same prompt distribution, after the diffusion model’s own acceleration libraries are turned on. Treat the parallel-decoding pitch as a hypothesis your benchmark tests, the way a serving-time quantization decision is a measurement and not an assumption.
Measure at the batch size you will actually serve. The diffusion model’s least-bad regime is batch size 1. A serving system runs many concurrent requests, and that is exactly where the published comparisons show the autoregressive model pulling further ahead and the diffusion model running out of memory. A batch-1 benchmark is a benchmark of the case you will not run.
Profile your prompt distribution for token-independence. The speedup, where it exists, is the size of the independence in the output. If your workload is structured, list-like, form-filling text, there is parallelism to capture. If it is reasoning chains, code with tight data dependencies, or anything where each token leans on the last, there is little, and the diffusion model will be decoding close to one token per step regardless of the method.
Test quality at the parallelism you would actually run. A diffusion model evaluated at one token per step is being evaluated as a slow autoregressive model. Evaluate it at the tokens-per-step setting that produces the speed you are quoting — that is the configuration whose quality you are actually buying, and ParallelBench is the shape of what goes wrong if you do not.
Watch the converted-model frontier, and write down the result that would change the decision. The signal is a downloadable open-weight diffusion or block-diffusion model with a published, reproduced wall-clock win over an equal-size autoregressive model at production batch sizes. Decide now what that result looks like, so you re-open the decision when it lands rather than on the strength of a press cycle.
Before you scope an inference budget around a diffusion language model:
The parallel decoding was real. The loop that makes an autoregressive model slow genuinely is gone, and watching a diffusion model resolve a block of text at once is a striking thing. But the team’s speed was never in the loop’s absence — it was in how many tokens of their particular prompts are actually independent of one another, and that number is small for the reasoning and code most production workloads are made of. A diffusion model on those workloads spends its steps the way an autoregressive model spends them, one meaningful token at a time, and pays more for each step. Measure the independence in your own traffic. If it is there, the diffusion model can spend it. If it is not, the architecture cannot manufacture it, and the faster model is the one you were already running.
Our taxonomy argued prompt injection is a vulnerability class you contain, not a bug you fix. Here is the quantitative half — against the strongest published defenses, indirect-injection attack success stays high, and for agents that can act it stays alarming.
10 min →
A quality-ascending curriculum beats random shuffling — until a decaying schedule delivers your best data exactly when the learning rate is too small to absorb it. The same coupling flips proxy-model ablations: which dataset wins depends on the schedule, not the data alone.
13 min →
An agent workflow that pauses for human approval and then resumes looks safe — a person clicked approve. But the approval decision and the authority the resumed step runs under are two different objects, and most systems conflate them by carrying the approval as an in-band signal the resumed call trusts. That is a confused-deputy bug: a forged resume, a replayed request, or an approval granted at one gate authorizes an action it was never meant to. The fix is to stop transporting approval and start deriving authority from the trusted approval record at resume time, scoped to the exact suspended step — capabilities enforced at the tool boundary, fail-closed by default.
14 min →