Recoleta
GitHub
Source note

VulKey: Automated Vulnerability Repair Guided by Domain-Specific Repair Patterns

Authors
Jia Li, Zhuangbin Chen, Yuxin Su, Michael R. Lyu
Source
arXiv
Published
Collection
Software Intelligence
Automated Vulnerability RepairCode IntelligenceLLM Code GenerationSoftware SecurityProgram Repair
Open arXivMarkdown source

Summary

VulKey is an LLM-based automated vulnerability repair system that guides patch generation with selected repair patterns instead of raw CWE labels or full example patches. It reports 31.5% repair accuracy on PrimeVul, 7.6 percentage points above the best baseline in the excerpt.

Problem

  • Automated vulnerability repair matters because public vulnerability volume is high: NVD lists 40,009 disclosed vulnerabilities in 2024, up 38.83% year over year.
  • Existing LLM repair methods use CWE knowledge weakly. Adding a CWE prefix reduced Exact Match from 23.9 to 23.6 on StarCoderBase-15B and from 23.2 to 22.5 on Qwen2.5-Coder-32B.
  • Full few-shot repair examples add project-specific noise. On 435 PrimeVul instances, random examples dropped EM to 11.9 and 12.9, while BM25 retrieval reached 17.9 and 16.8, below base fine-tuning.

Approach

  • VulKey encodes each repair pattern with three fields: CWE type, syntactic action, and semantic key element.
  • The syntactic action names the edit shape, such as inserting a range check or method call. The semantic key element names the security-specific code cue, such as mutex_lock, get_net, or a boundary condition.
  • It extracts these patterns from historical vulnerability-fix diffs covering 3,789 real-world vulnerabilities.
  • A CodeT5p-based matcher takes vulnerable code and CWE context, then predicts top-k action and key-element patterns.
  • A fine-tuned code generation model receives the vulnerable function plus the selected patterns and generates the repaired code.

Results

  • On PrimeVul for C/C++, VulKey reports 31.5% repair accuracy.
  • The 31.5% score is 7.6 percentage points above the best baseline, a StarCoder model fine-tuned on bug-fix and vulnerability-fix data.
  • The paper claims VulKey beats VulRepair by 8.5x, VulMaster by 3.6x, and GPT-5 by 3x on PrimeVul.
  • On the Java Vul4J benchmark, VulKey repairs 18 cases, compared with 9 for VulMaster and 4 for VulRepair.
  • In the paper's preliminary study, concrete examples hurt performance: StarCoderBase-15B EM fell from 23.9 to 11.9 with random examples and to 17.9 with BM25 retrieval.

Link