Implementation guide
OWASP AISVS v1.01
Model Behavior, Output Control & Safety Assurance · Hallucination Detection & Mitigation
Fall back when confidence is low
Verify that the application automatically blocks answers or switches to a fallback message if the confidence score drops below a defined threshold.
OWASP AISVS v1.01, requirement 7.2.2
In plain English
When the confidence score for an answer falls below a set level, block the answer or replace it with a fallback message, automatically. This is what makes the scoring in 7.2.1 useful.
What good looks like
- A threshold is defined for each use case, based on measured error rates.
- Answers below it are replaced with a fallback, such as "I could not find a reliable answer", or routed to a person.
- The fallback is honest. It does not present a guess as an answer.
- Fallback rates are monitored, and a sharp change is investigated.
How teams usually do it
- A post-processing step that compares the score with the threshold before the response is released.
- Stricter thresholds for topics where a wrong answer costs more.
- A hand-off route to a human or a search results page for questions the system declines.
Suggested evidence
- Threshold settings and how they were chosen.
- Fallback statistics over time.
- A test showing a low-confidence answer is replaced.
Common mistakes
- A threshold set so low it never fires.
- Showing the weak answer anyway with a small disclaimer underneath.
- Streaming the answer to the user before the score is known.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.