Differential privacy budget accounting is the calculation that turns a DP-SGD run into a reported (ε, δ) pair. The accountant takes four inputs: the noise multiplier σ, the sampling rate q, the number of steps T and a target δ. It tracks the privacy loss of every noisy gradient step and converts the running total into ε. A reported ε therefore describes the whole training run, and it means little without the accountant and settings that produced it.
What ε and δ measure in this setting
This article uses the add-or-remove-one-record neighbouring relation. The TensorFlow Privacy tutorial reads ε as bounding how much the output distribution can change when one record is added or removed, and δ as the probability that this guarantee fails. The tutorial advises setting δ below the inverse of the dataset size. That is a rule of thumb, not a requirement.
ε is an upper bound on a formal guarantee. It is not a measured leakage rate. A large ε does not by itself show poor practical privacy, and a small ε does not by itself show low real-world leakage.
Inputs the accountant needs
σ is the noise multiplier, the ratio of the Gaussian noise standard deviation to the clipping norm. q is the probability that each example enters a batch. T is the number of steps and δ is the target failure probability. In this article's reading, the clipping norm does not appear as a separate privacy parameter; it enters through the normalisation of σ. That convention should be checked against the Opacus and TensorFlow Privacy documentation before any figure is reproduced.
Why the budget does not simply add per step
Each noisy release is computed from the same underlying data, so privacy loss accumulates across steps. Classical composition bounds are loose over many steps. The moments accountant, introduced by Abadi et al. (2016), tracks the log moment generating function of the privacy-loss random variable. That quantity composes additively across steps and is converted to (ε, δ) at the end. We did not retrieve the 2016 paper, so this description comes from a 2025 survey that cites it.
How subsampling changes the per-step cost
DP-SGD draws a random batch at each step, so the cost of a step depends on q. According to the research summary of Mironov, Talwar and Zhang (arXiv 1908.10530), the cost of a sampled step scales quadratically rather than linearly in q for small q. We have not checked the exact form against that paper. The amplification result assumes a particular sampling scheme, so the scheme used in code has to match the scheme the accountant assumes. Wang, Balle and Kasiviswanathan (AISTATS 2019) showed that the moments technique extends to subsampled RDP mechanisms.
Accountants compared: moments, RDP, PRV and GDP
Moments accountant
This is the Abadi et al. (2016) approach described above. It is still in use, but the research we reviewed does not establish its accuracy against the alternatives on the same run.
Rényi DP and the sampled Gaussian mechanism
A mechanism satisfies (α, ε)-RDP if the Rényi divergence of order α between output distributions on neighbouring datasets is at most ε. Mironov, Talwar and Zhang analyse this for the sampled Gaussian mechanism. The Opacus migration guide states that RDP is the default and recommended accountant. A 2026 preprint states that the moments accountant is subsumed by RDP. That is a single recent source we did not verify directly, so treat the wider framing as medium confidence. RDP is not shown here to be tighter in every setting.
PRV and GDP accountants
A third-party Opacus API mirror lists rdp, prv and gdp as accountant options. We did not check this against the official Opacus site. We found no same-run comparison of RDP, PRV and GDP, so we do not rank them.
From an RDP curve to a reported (ε, δ)
The standard conversion is that (α, ε_α)-RDP implies (ε_α + log(1/δ)/(α−1), δ)-DP. The accountant chooses the order α that gives the smallest ε, so the reported figure depends on the α grid searched. The 2025 preprint that gives this formula is secondary for it. Tighter conversions exist and are used in some tools, but we did not retrieve their formulas from a primary source. Opacus's get_epsilon returns ε together with the optimal order α.
Reading and reporting a result
The TensorFlow Privacy tutorial reports ε = 0.563 at δ = 1e-5 with optimal RDP order 18 for one DP-SGD configuration. It separately reports ε = 1.18 for the trained model. The research summary says these come from different settings, and we have not confirmed the configuration behind either number. Do not merge them into one result. A report should state:
- the accountant type;
- δ;
- the α grid or optimal order;
- σ, q and T;
- the sampling method;
- whether the figure covers one training run or several.
Where accounting goes wrong in practice
- Per-round resets. In multi-round or federated training, resetting the accountant each round makes the reported ε cover one round rather than the full run. Our support is a community forum answer, not a verified error report. It is consistent with composition.
- Sampling mismatch. The Opacus API mirror exposes poisson_sampling. One 2021 review (PMC8564206) argues that random shuffling for batching can underestimate the loss the moments accountant computes. This is contested, comes from one review, and is not settled. Verify it against primary work before relying on it.
- Sweeps and retraining. Each run spends budget unless it is accounted for. This is the author's reading from composition, not a cited finding.
Open questions and sourcing limits
We did not retrieve the primary texts of Abadi et al. (2016) or the original RDP paper. The tight RDP conversion came from secondary sources. The Opacus accountant list came from a mirror. We checked no published large-scale model ε values with verified accounting. Typical-ε ranges on consumer sites are unsourced and excluded. The evidence reflects sources gathered as of October 2026.