AI-Assisted Secure Coding (Appendix C) · AI Tool Qualification & Threat Modeling
Evaluate each tool end to end
Verify that the evaluation of each tool covers the local components (static and dynamic analysis), the SaaS endpoints (TLS, AuthN/AuthZ, logging, data residency), and the vendor's model supply chain (training-data provenance, fine-tune history, RAG sources). Each of these is reviewed and the review is written down.
OWASP AISVS v1.01, requirement AC.2.2
In plain English
Evaluate each tool end to end. There are three parts: what runs on your machines, the vendor's online service, and the supply chain behind the vendor's model. Review each part and write the review down.
What good looks like
- Local components (IDE extensions, command-line tools, agents) are analysed with static and dynamic analysis.
- The SaaS endpoints are checked for TLS, authentication and authorisation, logging and data residency.
- The vendor's model supply chain is reviewed: where the training data came from, the fine-tuning history and any retrieval sources.
- Each review is written up and kept.
How teams usually do it
- A vendor security questionnaire and a read of their trust documentation (SOC 2 or ISO 27001 reports, data processing terms, retention and training-use settings).
- Your own testing of local components: what they send, where, and what permissions they hold.
- Recording what the vendor could not or would not answer about model provenance, as a stated risk.
Suggested evidence
- Evaluation records per tool covering all three parts.
- Network captures or analysis notes for the local components.
Common mistakes
- Relying on the vendor's marketing security page.
- Reviewing the service and never looking at the extension that runs with the developer's full permissions.
- Leaving the model supply chain section blank and unremarked.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.