Implementation guide
OWASP AISVS v1.01
Model Behavior, Output Control & Safety Assurance · Hallucination Detection & Mitigation
Estimate confidence in answers
Verify that the system assesses the reliability of generated answers using a confidence estimation method.
OWASP AISVS v1.01, requirement 7.2.1
In plain English
Language models state wrong answers with the same fluency as right ones. Put a method in place that estimates how reliable each answer is, so the system can treat shaky answers differently from solid ones.
What good looks like
- A confidence method is chosen for each use case and written down.
- A score is stored with each response.
- The method has been compared with real error rates on a labelled sample, so you know what a given score means.
- The comparison is repeated when the model or the retrieval setup changes.
How teams usually do it
- For retrieval-based answers, a groundedness check: does each statement in the answer follow from the retrieved passages? Evaluation tools such as Ragas, TruLens and cloud groundedness detectors do this.
- Self-consistency: generate several answers and measure how much they agree. Wide disagreement is a good sign of a guess.
- Token probabilities (logprobs), where the API exposes them, for short or classification-style answers.
Suggested evidence
- The chosen method and why it suits the use case.
- Calibration results: scores against measured accuracy.
- Sample responses with their stored scores.
Common mistakes
- Asking the model how confident it is and trusting the reply. Self-reported confidence is poorly calibrated.
- Choosing a method and never checking it against real errors.
- Using one method for every use case.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.