Implementation guide
OWASP AISVS v1.01
Adversarial Robustness · Model Alignment, Safety, and Robustness Testing and Training
Test against known adversarial attacks
Verify that models are evaluated against known adversarial attack techniques relevant to their modality.
OWASP AISVS v1.01, requirement 11.1.3
In plain English
Test the model against the attack techniques known for its type of input. For text models that means jailbreak and injection techniques. For image and audio models it means adversarial perturbations: small changes a person would not notice that push the model to a wrong answer.
What good looks like
- The test set covers published attack techniques for each input type the model accepts.
- It is refreshed on a schedule as new techniques are published.
- Results are recorded per model version.
- Tests run against the deployed configuration, including the system prompt and filters.
How teams usually do it
- Open red-teaming tools: garak and PyRIT for language models; the Adversarial Robustness Toolbox, Foolbox or TextAttack for classifiers and vision models.
- Public benchmarks such as HarmBench or JailbreakBench for comparable numbers.
- Periodic manual red-team exercises, which find what automated tools miss.
Suggested evidence
- The test set, its sources and when it was last updated.
- Results per model version.
- Findings from the latest manual exercise.
Common mistakes
- Testing with a handful of famous jailbreaks the model was already trained against.
- Testing text attacks only on a model that also accepts images or audio.
- Using MITRE ATLAS and the OWASP Top 10 for LLM Applications as reading material and never turning them into test cases.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.