Infrastructure, Configuration & Deployment Security · AI Workload Sandboxing & Validation
Protect weights during inference
Verify that confidential inference services protect model weights during runtime through isolated execution environments.
OWASP AISVS v1.01, requirement 4.1.4
In plain English
Protect model weights while they are in use, so that even someone with administrator access to the host cannot read them out of memory. This needs hardware support, known as confidential computing. It is a Level 3 control, mainly for weights that are highly valuable or run on infrastructure you do not own.
What good looks like
- Inference runs in a hardware-isolated environment with encrypted memory.
- Weights are stored encrypted and are decrypted only inside that environment.
- The decryption key is released only after attestation succeeds (see 4.1.3).
- If a GPU is involved, the protection extends to GPU memory and the link between CPU and GPU.
How teams usually do it
- Confidential virtual machines from your cloud provider, with confidential GPU support where inference runs on accelerators (for example NVIDIA confidential computing on recent data-centre GPUs).
- A key management service that ties key release to the attested environment.
- For smaller models, CPU-only inference inside an enclave or confidential VM.
Suggested evidence
- An architecture diagram showing where weights are decrypted.
- Instance and attestation configuration.
- Key policy showing release is conditional on attestation.
Common mistakes
- Encrypting weights at rest and then loading them into ordinary memory on a shared host.
- Protecting the CPU side while the weights sit unencrypted in GPU memory.
- Leaving debug access or memory dumps enabled in the protected environment.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.