Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD

Key points

• We study DP-SGD with random allocation, where each epoch uses every record once, at a randomly chosen step.1
• For this setting we give a one-line formula that bounds the accuracy of every membership inference attack (MIA) on the trained model.1
• With $M$ steps per epoch, $E$ epochs and noise multiplier $σ$, and with membership and non-membership equally likely a priori, the attack accuracy is at most $\frac12+\frac14\sqrt{(1+(e^{1/σ^2}-1)/M)^E-1}$.1
• In training, the resulting $σ$ outperforms the formula and matches a published accountant in test accuracy.1

Sources (1)

• [1]Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 08:23 AM

  DP-SGD protects training data by adding Gaussian noise to clipped gradients.

  We study DP-SGD with random allocation, where each epoch uses every record once, at a randomly chosen step.

Extractive summary: sentences quoted from the sources.