Adversarial Robustness · Membership-Inference and Model-Inversion Mitigation
Test against membership inference
Verify that membership-inference attack simulations demonstrate that attack accuracy does not exceed random guessing on evaluated data.
OWASP AISVS v1.01, requirement 11.2.5
In plain English
A membership inference attack tries to tell whether a given record was in the training data. Simulate the attack against your own model. It should do no better than guessing. If it does better, the model is leaking information about the people in its training set. This is a Level 3 control.
What good looks like
- Membership inference attacks are simulated on the trained model before release.
- Attack accuracy is compared with random guessing, including at low false-positive rates.
- A model that fails is not released until fixed.
- The test covers the most exposed records, such as outliers and rare cases.
How teams usually do it
- Open privacy-testing tools that run the standard attacks: the Adversarial Robustness Toolbox, TensorFlow Privacy's attack library, or ML Privacy Meter.
- A strong modern attack such as LiRA, the likelihood ratio attack, in addition to the simple loss-threshold test.
- Fixes when a model fails: differential privacy, more regularisation, less memorisation through deduplication.
Suggested evidence
- Attack simulation results, with the method used.
- The release decision that referenced them.
Common mistakes
- Reporting average accuracy only. An attack can be near chance on average and still identify a few people with confidence.
- Testing only on typical records the model generalises well on.
- Running one weak attack and declaring the model safe.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.