Monitoring, Logging & Anomaly Detection · Model, Data, and Performance Drift Detection
Detect drift in inputs
Verify that data drift detection monitors input distribution changes that may impact model performance, using statistically validated methods matched to the input data type (e.g., KS test or PSI for tabular numeric features, embedding-distance metrics for text or image).
OWASP AISVS v1.01, requirement 12.3.1
In plain English
Watch whether the inputs reaching your model are changing in ways that could affect how it behaves. Models are tested on one mix of inputs. When the real mix shifts, performance and safety results stop applying. Use statistical tests that suit the type of data.
What good looks like
- Input distributions are compared with a baseline on a schedule.
- The method fits the data: tests for numeric features differ from those for text or images.
- Significant drift raises an alert that someone reviews.
- The baseline is updated deliberately after legitimate changes.
How teams usually do it
- The Kolmogorov-Smirnov test or population stability index (PSI) for numeric features, as the requirement suggests.
- Embedding-distance measures for text and images: compare the embeddings of recent inputs with the baseline set.
- Monitoring libraries such as Evidently, NannyML or Alibi Detect, or your platform's model monitor.
Suggested evidence
- Monitoring configuration and the methods chosen.
- Drift reports.
- A record of a drift alert and what was done.
Common mistakes
- Setting a baseline once and never updating it.
- Updating the baseline automatically, so slow drift is never seen.
- Applying a numeric test to text by way of crude proxies such as prompt length.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.