Implementation guide
OWASP AISVS v1.01
Adversarial Robustness · Model Runtime Anomaly Detection
Anomaly-check outside input
Verify that inputs from external or untrusted sources pass through anomaly detection before model inference.
OWASP AISVS v1.01, requirement 11.4.1
In plain English
Run input from outside your organisation through anomaly detection before the model processes it. Known-attack filters catch what they have seen before. Anomaly detection flags input that simply does not look like normal use, which is how new attacks first show up.
What good looks like
- External and untrusted inputs are scored for anomalies before inference.
- Scoring covers every route in, including documents, retrieved content and tool results.
- Scores are logged with the request.
- The definition of "normal" is built from your own traffic and refreshed.
How teams usually do it
- Simple statistical checks first: length, language, character mix, entropy, and repetition.
- Embedding-based outlier scoring against a baseline of normal inputs.
- For language models, perplexity checks, which catch the machine-generated gibberish suffixes used in some automated jailbreaks.
Suggested evidence
- Detector configuration and baseline.
- Score logs.
- Tests with known unusual inputs.
Common mistakes
- Scoring the chat interface only.
- A baseline built once and never refreshed, so every new feature looks anomalous.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.