Adversarial Robustness · Model Alignment, Safety, and Robustness Testing and Training
Model is safety-trained
Verify that the model has undergone alignment and safety training or fine-tuning to prevent the model from generating disallowed content categories.
OWASP AISVS v1.01, requirement 11.1.1
In plain English
The model you use should have been trained or fine-tuned to refuse the kinds of content you do not allow. If it comes from a provider, find out what safety training it has had and compare that with your own list. Filters around the model help, but a model with no safety training leaves them carrying the whole load.
What good looks like
- Your disallowed content categories are written down.
- You hold evidence that the model has had safety training covering those categories.
- You have tested the model's refusals in each category yourself.
- Gaps are recorded and covered by other controls, such as input and output filtering.
How teams usually do it
- For a provider's model, read the model card, system card or usage policy and map it to your categories.
- For your own models, safety fine-tuning with refusal examples in each category, and a check that later fine-tuning has not undone it.
- Choosing instruction-tuned, safety-trained versions over raw base models or "uncensored" community variants for anything users can reach.
Suggested evidence
- The category list.
- The model's safety documentation, or your training records.
- Test results showing refusal rates by category.
Common mistakes
- Putting a base model with no safety training into a customer-facing product.
- Assuming a provider's general safety training covers categories particular to your domain.
- Fine-tuning on your own data and not checking that the refusals survived.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.