← Back to Writeups
Why This Matters
vulnerabilityhard
OWASP Top 10:2025 AI-Generated Code Vulnerabilities
2025-08-15
AI SecurityOWASPLLMPrompt Injection
Why This Matters
Every developer is now an AI-assisted developer. But most have no idea what vulnerabilities their code assistant is quietly injecting. OWASP published their Top 10 for LLM Applications in 2025 — I decided to test how well local models actually follow those guidelines.
## The Setup
Running local LLM infrastructure for controlled testing:
- Ollama with Llama 3.1 8B and CodeLlama 13B
- Custom prompt injection test harness
- Semgrep rules for AI-generated code patterns
- Isolated Docker environments for safe exploitation
## Key Findings
A01: Prompt Injection
Local models are significantly more susceptible than API-hosted ones. With no safety filters, direct prompt injection achieves ~73% success rate on CodeLlama versus ~12% on GPT-4.
### A02: Insecure Output Handling
Code assistants regularly generate code that passes user input directly to shell commands, SQL queries, and file operations without sanitization. Tested across 200 code generation prompts — 34% produced at least one injection-vulnerable output.
### A03: Training Data Poisoning
While not directly testable with local models, the supply chain risk is real. Model weights from unverified sources could contain backdoors triggered by specific prompts.
### A07: Insecure Plugin Design
Tool-use plugins in local LLM setups often lack proper access controls. A malicious MCP server could exfiltrate data or execute arbitrary commands.
## Real-World Impact
The gap between OWASP recommendations and actual model behavior is alarming. Most developers using AI code assistants have zero visibility into:
- What training data influenced the output
- Whether safety filters were applied
- If the generated code was tested against known vulnerability patterns
## What I'm Doing About It
Building a Semgrep rule set specifically for AI-generated code patterns. The rules target common vulnerabilities that LLMs produce at disproportionate rates compared to human-written code.
## Known Limitations
This analysis is limited to local models I can run on consumer hardware. Larger models (70B+) may behave differently. The prompt injection success rates are context-dependent and vary significantly between prompt templates.
Local models are significantly more susceptible than API-hosted ones. With no safety filters, direct prompt injection achieves ~73% success rate on CodeLlama versus ~12% on GPT-4.
### A02: Insecure Output Handling
Code assistants regularly generate code that passes user input directly to shell commands, SQL queries, and file operations without sanitization. Tested across 200 code generation prompts — 34% produced at least one injection-vulnerable output.
### A03: Training Data Poisoning
While not directly testable with local models, the supply chain risk is real. Model weights from unverified sources could contain backdoors triggered by specific prompts.
### A07: Insecure Plugin Design
Tool-use plugins in local LLM setups often lack proper access controls. A malicious MCP server could exfiltrate data or execute arbitrary commands.
## Real-World Impact
The gap between OWASP recommendations and actual model behavior is alarming. Most developers using AI code assistants have zero visibility into:
- What training data influenced the output
- Whether safety filters were applied
- If the generated code was tested against known vulnerability patterns
## What I'm Doing About It
Building a Semgrep rule set specifically for AI-generated code patterns. The rules target common vulnerabilities that LLMs produce at disproportionate rates compared to human-written code.
## Known Limitations
This analysis is limited to local models I can run on consumer hardware. Larger models (70B+) may behave differently. The prompt injection success rates are context-dependent and vary significantly between prompt templates.