AI-Assisted Secure Coding (Appendix C) · AI Code-Review & Assistant Bot Hardening
Keep attack-testing review bots
Verify that AI review and assistant bots are subject to continuous adversarial testing: indirect-prompt-injection corpora are replayed against the bot through simulated PRs, issues, and comments. Detection effectiveness is tracked over time, and a regression blocks the model or prompt update that caused it.
OWASP AISVS v1.01, requirement AC.11.8
In plain English
Keep attack-testing your review bots. Replay a library of known indirect injection attacks through simulated PRs, issues and comments, track how many the bot resists over time, and block any model or prompt update that makes things worse. This is a Level 3 control.
What good looks like
- Injection corpora are replayed against the bot on a schedule and before each update.
- Detection and resistance rates are tracked over time.
- A regression blocks the model or prompt update that caused it.
- The corpus is extended with new techniques and real attempts seen in your repositories.
How teams usually do it
- A scheduled job that opens test PRs, issues and comments in a sandbox organisation and scores the bot's behaviour.
- Canary secrets and canary actions in the sandbox, so a successful attack is unmistakable.
- Wiring the results into the release check for bot configuration (AC.6.4).
Suggested evidence
- Detection trends over time.
- An update that was blocked by a regression.
- The corpus and its change history.
Common mistakes
- Testing once at adoption.
- A corpus that never changes.
- Tracking results on a dashboard that does not block anything.
Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.