# The epsilon on your private model is computed for an algorithm you did not run.

A team trains a model under differential privacy, does everything the textbook asks, and reports the result with a clean conscience. They clip per-example gradients, add calibrated Gaussian noise, run a numerically tight privacy accountant over every step, and publish a number: this model satisfies (ε, δ)-DP at ε = 8. The number is correct arithmetic. The accountant did not make a mistake. The clipping norm and the noise multiplier are exactly what the report says they are. And the ε on the model is still wrong — not loose, not conservative, wrong in the direction that matters — because the accountant priced one algorithm and the training loop ran a different one.

The difference is buried in a single line of the data loader. The privacy analysis of DP-SGD assumes each batch is drawn by _Poisson subsampling_: every example is included independently with some fixed probability, so batch size is random and a given record may appear zero, one, or several times across an epoch. That is the sampler the tight composition theorems are derived for, and it is the sampler almost nobody runs. Real training loops _shuffle_ the dataset once per epoch and slice it into fixed-size batches, because shuffling is what every framework's data loader does by default and Poisson sampling is awkward to implement at scale. So the standard pipeline shuffles, then reports the privacy parameters as if it had Poisson-subsampled. The accountant is exact — about a mechanism that was not used.

This is not a rare misconfiguration a careful team avoids; it is the default path. The numerically tight accountants exist for Poisson subsampling; shuffling, which is what people implement, has no comparably tight analysis — so the field substitutes the number it can compute for the number it cannot, and ships. The two samplers do not give the same privacy, the gap has been proven, and the gap has been measured. The privacy budget you are citing is computed for a sampler your trainer is not running.

This post is about why shuffling and Poisson subsampling are different mechanisms with different guarantees, how large the gap is when you audit a real model, and why a separate line of work shows that even the "loose" part of the bound was hiding real leakage that better attacks recover. It is not an argument that DP-SGD is broken. It is an argument that the number on the model has to describe the algorithm that produced the model.

## Shuffling and Poisson subsampling are not the same mechanism

The whole problem rides on a fact that is easy to wave past: the privacy of DP-SGD comes entirely from the randomness of which examples land in each batch, and the two samplers randomize differently.

The reason the field cares about subsampling at all is _amplification_. Adding Gaussian noise to a gradient gives a base level of privacy; sampling a small random subset for each step amplifies it, because an adversary trying to detect one record now also has to get past the uncertainty of whether that record was even sampled into the batch they observe. The more cleanly that uncertainty is characterized, the more amplification you can prove. Poisson subsampling has exactly that clean characterization — each record in or out independently, with a known probability — which is why, as the work on how private DP-SGD implementations really are ([_How Private are DP-SGD Implementations?_](https://arxiv.org/abs/2403.17673) (arXiv 2403.17673)) puts it, "Poisson subsampling-based DP-SGD is challenging to scalably implement, but has a well-understood privacy analysis, with multiple open-source numerically tight privacy accountants available."

Shuffling produces a different distribution over batches. Permute the dataset and cut it into fixed-size chunks and every record appears exactly once per epoch, batch sizes are deterministic, and the inclusion events are not independent — knowing record A is in batch 3 tells you something about where record B can be. That is a different random process feeding the noise mechanism, and a different process gives a different amplification, which means a different ε. The same paper names the practice and why it is a problem: "This has led to a common practice of using shuffling-based DP-SGD in practice, but using the privacy analysis for the corresponding Poisson subsampling version." The analysis is sound. The mechanism it analyzes is not the mechanism in the loop.

So this is not a bug in anyone's accountant, and that is what makes it slippery. Each piece is individually correct — the Poisson amplification math is correct, the shuffled data loader is correct, the noise calibration is correct — and the composite is wrong, because a correct analysis of mechanism A was attached to an execution of mechanism B. Nothing in the pipeline is going to throw an error. The number just quietly describes a model you did not train.

## The gap is proven, and it grows over a full training run

The natural first defense is to hope the two samplers are close enough that the substitution is harmless — that shuffling and Poisson subsampling land near the same ε and the mismatch is a rounding concern. The lower bounds say otherwise.

[_How Private are DP-SGD Implementations?_](https://arxiv.org/abs/2403.17673) (arXiv 2403.17673) studies the mechanism underneath DP-SGD — Adaptive Batch Linear Queries, which DP-SGD is a post-processing of — and demonstrates "a substantial gap between the privacy guarantees of the Adaptive Batch Linear Queries (ABLQ) mechanism under different types of batch sampling: (i) Shuffling, and (ii) Poisson subsampling." Substantial, not marginal. The paper's conclusion plants a caution flag in the standard workflow: the gap "advises caution in reporting privacy parameters for DP-SGD." The two numbers are not interchangeable, and the analysis proves it rather than asserting it.

There is a sharper worry the same group addresses next. The first result was for a single epoch, and real models train for many; if the per-epoch gap merely persists across a full schedule, the multi-epoch number — the one that governs a shipped model — is where the damage concentrates. The follow-on work on scalable DP-SGD ([_Scalable DP-SGD: Shuffling vs. Poisson Subsampling_](https://arxiv.org/abs/2411.04205) (arXiv 2411.04205)) extends the lower bounds to exactly that setting: "new lower bounds on the privacy guarantee of the multi-epoch Adaptive Batch Linear Queries (ABLQ) mechanism with shuffled batch sampling, demonstrating substantial gaps when compared to Poisson subsampling; prior analysis was limited to a single epoch." It does not hedge the implication: the result "brings into serious question the common practice of implementing shuffling-based DP-SGD, but reporting privacy parameters as if Poisson subsampling was used." That is the standard practice named in plain language by the people who proved the bound. The gap is not a curiosity at the edge of parameter space; it sits on the default training path, and it does not wash out over the length of a real run.

## Auditing a shuffled model shows up to 4× — and the common variants are worse

A lower bound on an abstract mechanism is one kind of evidence; a number measured against a trained model is the one an operator should care about, and the auditing work supplies it. [_To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling_](https://arxiv.org/abs/2411.10614) (arXiv 2411.10614) takes the practice head-on, starting from the same observation — "computing tight theoretical DP guarantees under shuffling remains an open problem. As a result, models trained with shuffling are often evaluated as if Poisson subsampling were used, which might result in incorrect privacy guarantees" — and then measuring how incorrect. The headline: models trained this way "have considerably overestimated their privacy guarantees (by up to 4 times)." Up to four times. A model reported at ε = 8 can be leaking at a rate honest accounting would price several times higher — and the over-optimism is in the reported number, against the model's own behavior under audit.

The detail that should change how you read your own stack is that the shuffle is not one thing. Common variations in how a loop shuffles — when it reshuffles, how it handles epoch boundaries — are not privacy-equivalent. The same paper studies "two common variations of the shuffling procedure that result in even further privacy leakage (up to 10 times)." Ten times. The specific shuffle your library implements is a privacy parameter, an undocumented one, and some of the most common choices are the worst. This is the opposite of a corner case: it is the mainstream behavior of mainstream tooling, and the variants that leak the most are not exotic.

It would be convenient if the gap were largest only in some artificial regime far from where real models live, leaving production training accidentally safe. The audit closes that escape too: "the gap between the theoretical Poisson DP guarantees and the actual privacy leakage from shuffling is not uniform across all parameter settings and threat models" — the discrepancy concentrates rather than averaging away, and not in a region you can assume your training avoided. The reported ε on a shuffled model is not a conservative bound you can trust to be on the safe side. It is an underestimate, sometimes by a factor of ten, exactly where it matters.

## The "looseness" that the folklore dismissed was real leakage

There is a second, independent way the reported ε on a DP-SGD model has been comfortably misread, worth separating from the sampler mismatch because it has a different cause. For years the standard reassurance about a large theoretical ε was that empirical audits came back far below it — train at ε = 10, audit, get an empirical ε of two or three, and conclude the bound is loose and the real privacy is much better than the number says. That reassurance was measuring the weakness of the attacks, not the strength of the privacy.

The work on nearly tight black-box auditing ([_Nearly Tight Black-Box Auditing of Differentially Private Machine Learning_](https://arxiv.org/abs/2405.14106) (arXiv 2405.14106)) shows it cleanly. Instead of benign canaries it crafts "worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters" — an adversarial starting point the formal guarantee already promises to hold against, so it is fair game. With that, and without any white-box access, the audit reaches near the theoretical ceiling: "for models trained on MNIST and CIFAR-10 at theoretical ε=10.0, our auditing procedure yields empirical estimates of ε_emp = 7.21 and 6.95, respectively, on a 1,000-record sample and ε_emp = 6.48 and 4.96 on the full datasets." An empirical 6.48 against a theoretical 10.0 is not a loose bound with a comfortable margin. It was nearly tight all along, the slack hiding in the auditor's imagination.

The threat model is what makes this load-bearing. Earlier tight audits needed a white-box adversary — "previous audits were only (relatively) tight in stronger white-box models, where the adversary can access the model's inner parameters and insert arbitrary gradients" — and the folklore leaned on that: the bound is tight only if you let the attacker reach inside and inject gradients, which no real adversary can do. This result removes the crutch; the tightness holds in the black-box model, where the adversary only trains on a poisoned dataset and observes the released model. And the authors note the procedure "can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations" — so the audit doubles as a correctness test for the shipped library, the bridge from the abstract gap to your code.

The hardest setting confirms the pattern. When the adversary sees only the final released model — the most realistic deployment posture — auditing historically came back loosest. The work on adversarial-sample-based auditing ([_Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios_](https://arxiv.org/abs/2412.01756) (arXiv 2412.01756)) tightens even that "by crafting worst-case adversarial samples through loss-based input-space auditing," against a baseline where final-model auditing "often results in empirical lower bounds that are significantly looser than theoretical privacy guarantees." Crafted inputs recover signal standard canaries miss: "with a theoretical privacy budget of ε = 10.0, our method achieves empirical lower bounds of 4.914, compared to the baseline of 4.385 for MNIST." The gain is modest and the gap to the ceiling is still real here — but it moves the same direction as every result above: as the attacks improve, the empirical number climbs toward the theoretical one.

## The honest limits: what this does and does not say

A post that has spent five sections undermining reported epsilons has to be exact about the claim it is making, because two larger claims are tempting and both are wrong.

The first over-reading is "DP-SGD is broken, so the guarantee is worthless." That is not what any of this shows. The formal guarantee of DP-SGD is a theorem, and the theorem is true for the algorithm it describes. The sampler-mismatch results do not refute the math — they show it was applied to the wrong sampler, an implementation-and-reporting failure, not a failure of differential privacy. The fix is not to abandon DP but to make the number describe the run: either run Poisson subsampling, the mechanism the tight accountant prices, or account for the shuffling you actually do. It is a guarantee-gap between code and report, and it closes by aligning the two. Whether DP fine-tuning is worth its accuracy cost at all is a separate question — the subject of [differential privacy for fine-tuning](/blog/differential-privacy-fine-tuning/) — and that cost-benefit call takes the reported ε as given. This post is upstream of it: before you decide whether the price is worth paying, the number on the price tag has to be real.

The second over-reading is "the audits prove your model leaks 4× more, full stop." Mind the quantifiers. The 4× and 10× figures are _up to_ — the worst observed over the settings audited — and the auditing paper is explicit that the gap "is not uniform across all parameter settings and threat models." Your configuration might sit near the low end or the high end; the point is you cannot tell from the reported Poisson ε alone, because that number is blind to which sampler and which shuffling variant you ran. The black-box numbers — 6.48, 4.96, 4.914 — are likewise empirical _lower_ bounds against specific datasets at theoretical ε = 10.0, evidence that the slack is smaller than assumed, not a constant to paste onto your model. The defensible claim is narrower than either slogan and harder to dismiss than both: the reported ε on a standard shuffled DP-SGD model is computed for a mechanism the trainer did not run, the resulting error is optimistic, its magnitude has been measured as large in realistic regimes, and the historical comfort of "loose audits" was largely an artifact of weak attacks. The number is not a safe over-estimate. That is the whole claim, and it is enough.

And there is genuine progress to credit, the counter-evidence to any fatalism. The 2026 systematization of DP auditing ([_The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing_](https://arxiv.org/abs/2506.16666) (arXiv 2506.16666)) sets out what a trustworthy audit must achieve — "three cross-contextual desiderata that DP audits should target -- namely, efficiency, end-to-end-ness, and tightness" — and in doing so explains why the field's numbers have been systematically off rather than occasionally mistaken: the framework highlights "key details overlooked by prior work." That the gaps are now named, measured, and organized into desiderata is why this is a fixable reporting problem, not a permanent indictment.

## Reporting an epsilon that describes the run

You cannot trust a Poisson ε attached to a shuffled run. You can make the reported number describe the algorithm that produced the model, and verify it with an audit rather than asserting it.

**State the sampler next to the epsilon.** The highest-value change is to record which batch sampler the loop actually used — Poisson subsampling or shuffling, and if shuffling, which variant and reshuffling policy. An ε with no named sampler is an incomplete claim, because the same ε means different things under the two mechanisms. Make the sampler a required field in the privacy report, not a detail left in the data loader.

**If you report a Poisson epsilon, run Poisson subsampling.** The tight, audited accountants are for Poisson subsampling; that is the mechanism whose number you are quoting. If the report says Poisson, the loop has to subsample independently per record — accept the throughput cost of doing so rather than shuffling for convenience and borrowing the Poisson number. The mechanism and the analysis have to be the same mechanism.

**If you shuffle, account for shuffling — and treat the Poisson number as an underestimate until you do.** Where shuffling is non-negotiable for scale, the reported ε must come from an analysis of shuffling, not a substitution. Until such a number exists for your exact procedure, the Poisson ε you have is known to be optimistic, by up to 4× in the audited cases and up to 10× for some common shuffling variants — so it is a lower bound on your spend, not a guarantee, and should be labeled as such.

**Audit the model with worst-case inputs, not benign canaries.** A low empirical ε from a weak attack is not evidence of strong privacy; it is evidence of a weak attack. Audit with worst-case crafted inputs — worst-case initial parameters, adversarial input-space canaries — the methods that reach near-tight empirical ε in the black-box model. A benign-canary audit that comes back comfortably below your theoretical ε has told you almost nothing, because better-crafted inputs recover leakage it missed.

**Audit the artifact you ship, in the threat model you ship it in.** If you release only the final model, audit the final-model setting; if you expose checkpoints, audit those. The empirical ε depends on what the adversary can see, and validating a privacy claim against an easier threat model than the one you deploy under is validating a model you are not shipping.

**Treat a failed audit as a bug, not noise.** Tight auditing detects DP violations and bugs in real libraries. An empirical ε that exceeds — or sits implausibly close to — your theoretical ε is a signal to inspect the clipping, the noise calibration, and the sampler for a defect, the same way you would treat any assertion that fired in production. The audit is a correctness test for the privacy code; a red audit means something is wrong in the code.

## The checklist

Before you publish an (ε, δ) on a model trained with DP-SGD:

- [ ] The batch sampler — Poisson subsampling or shuffling, and which shuffling variant — is recorded next to the reported ε, not left implicit in the data loader.
- [ ] If a Poisson ε is reported, the training loop actually performs Poisson subsampling, not shuffling.
- [ ] If the loop shuffles, the reported ε comes from an analysis of shuffling; absent that, the Poisson number is labeled an optimistic estimate, not a guarantee.
- [ ] The model has been audited with worst-case crafted inputs (worst-case initial parameters or adversarial input-space canaries), not only benign canaries.
- [ ] The audit was run in the threat model the model is deployed under — final-model-only if only the final model is released.
- [ ] An empirical ε that approaches or exceeds the theoretical ε is treated as a probable implementation bug and investigated, not dismissed.

## Reading list

- [_How Private are DP-SGD Implementations?_](https://arxiv.org/abs/2403.17673) (arXiv 2403.17673) — proves a substantial gap between the shuffling and Poisson-subsampling guarantees of the mechanism underlying DP-SGD, and names the practice of reporting the Poisson analysis for a shuffled implementation.
- [_Scalable DP-SGD: Shuffling vs. Poisson Subsampling_](https://arxiv.org/abs/2411.04205) (arXiv 2411.04205) — extends the lower bound to the realistic multi-epoch setting and states that reporting Poisson parameters for shuffled training is in serious question.
- [_To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling_](https://arxiv.org/abs/2411.10614) (arXiv 2411.10614) — audits real shuffled models and finds privacy guarantees overestimated by up to 4×, with two common shuffling variants leaking up to 10×.
- [_Nearly Tight Black-Box Auditing of Differentially Private Machine Learning_](https://arxiv.org/abs/2405.14106) (arXiv 2405.14106) — reaches empirical ε of 6.48 against a theoretical 10.0 with no white-box access, showing the audit "looseness" was an artifact of weak attacks.
- [_Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios_](https://arxiv.org/abs/2412.01756) (arXiv 2412.01756) — tightens auditing in the hardest final-model-only setting with crafted adversarial inputs, beating standard canaries.
- [_The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing_](https://arxiv.org/abs/2506.16666) (arXiv 2506.16666) — systematizes DP auditing into three desiderata (efficiency, end-to-end-ness, tightness) and flags key details prior work overlooked.

The accountant did its arithmetic perfectly; it just did the arithmetic for Poisson subsampling while your data loader shuffled, and those are different mechanisms with different privacy. The lower bounds prove the gap, the audits measure it at up to 4× and up to 10× for the common variants, and a separate line of work shows the slack you trusted was real leakage that better attacks recover. None of this means DP-SGD is broken — it means the number has to describe the run. Report the sampler you used, account for the mechanism you ran, and audit the model you ship. An epsilon computed for an algorithm you did not run is not a conservative guarantee. It is fiction with a decimal point.