Implementation guide
OWASP AISVS v1.01
Monitoring, Logging & Anomaly Detection · Model, Data, and Performance Drift Detection
Tell unexplained shifts from normal drift
Verify that unexplained behavioral shifts are distinguished from gradual, expected operational drift.
OWASP AISVS v1.01, requirement 12.3.4
In plain English
Behaviour drifts gradually for ordinary reasons, such as new users and new topics. A sudden or unexplained shift is different. It may mean tampering, a poisoned data source or a silent model change by your provider. Your monitoring should tell the two apart. This is a Level 3 control.
What good looks like
- Expected drift is described, with bounds on its normal rate.
- Sudden shifts, or shifts with no known cause, trigger an investigation.
- Known causes (releases, prompt changes, data refreshes, provider updates) are recorded so they can be matched against shifts.
- Investigations and conclusions are written up.
How teams usually do it
- Change-point detection on behaviour metrics such as refusal rate, response length, topic mix and tool-call patterns.
- A fixed set of canary prompts run on a schedule, whose answers should stay stable.
- A change log that monitoring can overlay on the metrics.
Suggested evidence
- Detection rules.
- The change log.
- An example of a shift that was investigated, and the finding.
Common mistakes
- One threshold for all change, so a sudden shift looks like ordinary drift.
- No record of your own changes, so every shift is unexplained.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.