Source note
Context-Based Adversarial Attacks on AI Code Generators: Vulnerability Analysis and Implications
Code Generation SecurityAdversarial AttacksPrompt InjectionSoftware Foundation ModelsSecure Code Generation
Summary
The paper shows that comments, documentation, variable names, and examples can push AI code generators to emit insecure code at inference time. It tests four models in 2,800 trials and reports a two-layer detector for risky context and generated code.
Problem
- The target problem is context-based manipulation of AI code generators: an attacker can place insecure guidance in normal-looking comments, docs, or examples, then let developers' tools turn that context into vulnerable code.
- This matters for software supply chains because the attacker does not need model access, training data access, or deployment access. A poisoned library docstring or reference example can affect downstream code generation.
- The paper studies five vulnerability classes: SQL injection (CWE-89), XSS (CWE-79), hardcoded credentials (CWE-798), path traversal (CWE-22), and insecure cryptography (CWE-327).
Approach
- The study runs 2,800 controlled generations across CodeT5+, CodeLlama-7B, GPT-3.5-Turbo, and GPT-4.
- It compares neutral prompts with three adversarial prompt types: direct insecure instructions, subtle semantic cues such as "quick prototype," and vulnerable reference examples.
- It measures vulnerability generation rate, attack success rate, and cross-model transfer rate.
- It checks generated code with Bandit, Semgrep, ESLint, regex and AST pattern matching, plus manual review on a 15% stratified sample with Cohen's kappa of 0.87.
- The defense system combines prompt checks before generation with code checks after generation, including linguistic scoring, semantic contradiction checks, perplexity, position monitoring, static analysis, taint analysis, and differential comparison.
Results
- Adversarial context increased mean vulnerability generation from 3.5% to 37.4%, a 10.7x rise across 2,800 trials. The paper reports chi-square = 847.3, p < 0.001, and a 95% CI of 35.1% to 39.7% for adversarial VGR.
- GPT-3.5-Turbo had 0.0% baseline VGR and 68.7% adversarial VGR. GPT-4 had 0.0% baseline VGR and 40.0% adversarial VGR. CodeT5+ rose from 2.0% to 22.7%, and CodeLlama rose from 12.0% to 18.3%.
- Direct instruction attacks had the highest mean attack success rate at 55.0%, including 100% on GPT-3.5-Turbo and 80% on GPT-4. Example-based attacks reached 31.4% mean ASR, and semantic priming reached 17.5%.
- Context placed 10 to 50 tokens before the target function had 62.1% ASR. Distant context placed 300 or more tokens away had 22.7% ASR. Attention weights for pre-function tokens were 2.3x higher than for distant tokens.
- SQL injection had the highest adversarial VGR at 36.6%. Hardcoded credentials reached 30.1%, XSS 23.0%, path traversal 19.9%, and insecure cryptography 17.0%.
- Cross-model transfer rates ranged from 0.58 to 1.00. Open-source to open-source transfer reached 0.95 to 1.00, and GPT-3.5-Turbo to GPT-4 transfer reached 0.90. The combined defense system reported 89.1% detection, 0.3% false positive rate, 99.7% precision, F1 = 0.942, and 520 ms average latency on a held-out set of 560 examples.