Implementation guide
OWASP AISVS v1.01
Monitoring, Logging & Anomaly Detection · Model, Data, and Performance Drift Detection
Detect hallucinations
Verify that hallucination detection monitors identify and flag model outputs that contain factually incorrect, inconsistent, or fabricated information.
OWASP AISVS v1.01, requirement 12.3.2
In plain English
Monitor answers in production for factual errors, contradictions and made-up content, and flag them. Testing before release shows how the model behaved on your test set. Production shows how it behaves on what people really ask.
What good looks like
- A detection method runs on production answers, or on a defined sample.
- Flagged answers are logged and reviewed.
- The method has been checked against examples with known answers.
- User reports of wrong answers feed the same process.
How teams usually do it
- Groundedness checks against the retrieved sources for retrieval-based systems, using tools such as Ragas, TruLens or a cloud groundedness detector.
- A second model that checks claims, or consistency checks across several generations.
- A "report a problem" control for users, with reports reviewed.
Suggested evidence
- Detector configuration.
- Flagged samples and review notes.
- The detector's measured accuracy.
Common mistakes
- Measuring hallucination before release and never afterwards.
- A detector nobody has validated.
- Collecting flags that nobody reviews.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.