Model Lifecycle Management & Change Control · Model Validation & Testing
Re-test models after quantisation
Verify that models subjected to post-training quantization are re-evaluated against the same safety and alignment test suite on the compressed artifact before deployment.
OWASP AISVS v1.01, requirement 3.2.2
In plain English
Quantisation shrinks a model by storing its numbers at lower precision. It can weaken refusals and other safety behaviour even when task scores barely move. If you quantise after training, run the full safety suite again on the compressed file you will actually deploy.
What good looks like
- Every quantised variant goes through the same safety and alignment suite as the original.
- Results are compared side by side with the original model's.
- A drop beyond an agreed tolerance blocks deployment.
- Each quantised file is registered as its own artifact with its own results.
How teams usually do it
- Treat each quantised build (for example each GGUF or AWQ variant at each bit width) as a separate model in the evaluation pipeline.
- Record both sets of results in the model registry against the artifact hashes.
- Apply the same rule to quantised models downloaded from third parties, which were often produced by someone other than the original publisher.
Suggested evidence
- Test results for the original and quantised models, side by side.
- The tolerance you set and the decision record for the deployed variant.
Common mistakes
- Assuming the quantised model behaves the same because it scores the same on the main task.
- Testing one quantisation level and shipping another.
- Evaluating the original and deploying a community quantisation nobody on the team has tested.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.