Implementation guide
OWASP AISVS v1.01
Adversarial Robustness · Model Runtime Anomaly Detection
Act on anomalous input
Verify that inputs flagged as anomalous trigger gating actions.
OWASP AISVS v1.01, requirement 11.4.2
In plain English
When an input is flagged as anomalous, something has to happen: block it, ask for confirmation, limit what the model may do with it, or send it for review. A flag that changes nothing is a log entry.
What good looks like
- A defined action exists for each anomaly level.
- The actions run automatically.
- Outcomes are logged.
- False-positive rates are tracked, and thresholds tuned.
How teams usually do it
- Rules in the input pipeline keyed on the anomaly score.
- Graduated responses: answer without tools or retrieval for mid-range scores, and block or queue for review at high scores.
- For agents, dropping to read-only mode when the input looks anomalous.
Suggested evidence
- The rules.
- Logs showing actions taken.
- False-positive measurements.
Common mistakes
- Flagging anomalies and processing them as normal.
- A single block-or-allow threshold that is either too noisy to keep on or too loose to matter.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.