Recoleta
GitHub
Source note

EvoRepair: Enhancing Vulnerability Repair Agents Through Experience-Based Self-Evolution

Authors
Haichuan Hu, Guoqing Xie, Quanjun Zhang, Jiawei Liu, Shengcheng Yu, Chunrong Fang, et al.
Source
arXiv
Published
Collection
Software Intelligence
Automated Vulnerability RepairCode AgentsLLM Self EvolutionSoftware SecurityExperience Retrieval
Open arXivMarkdown source

Summary

EvoRepair is an LLM agent method for automated vulnerability repair that learns reusable repair experience from past repair attempts and feeds it into later repairs. It reports higher repair success than 12 AVR baselines on PATCHEVAL and SEC-bench using GPT-5-mini.

Problem

  • LLM repair agents often treat each CVE as a fresh task, so they repeat failed edits and miss fixes seen in related CWE/CVE cases.
  • This matters because reported CVEs reached 38,942 in 2024, and manual or isolated repair does not scale to that volume.
  • Existing retrieval methods use static examples or patches, while AVR needs repair rules that change after each success or failure.

Approach

  • EvoRepair runs a cyclic learn-and-repair loop: retrieve relevant experience, attempt a patch in Docker, summarize the trajectory, score the new experience, and update the experience bank.
  • Each experience entry records vulnerability analysis, repair strategy, trajectory analysis, reusable rules, and follow-up notes.
  • Retrieval selects top-M similar experiences by CVE/CWE, reranks them with similarity plus an experience score, and injects top-K entries into the agent memory.
  • The base agent uses ReAct-style shell interaction, minimal Bash tools, defined repair skills, test feedback, and early stopping based on turn-level yield rate.
  • New experiences are scored by an LLM judge on quality and generality, with repeated scoring and comparison to human ratings; lower-scoring updates for the same vulnerability can be discarded.

Results

  • On PATCHEVAL, EvoRepair reaches 93.47% with GPT-5-mini and reports gains of 39.56% over LoopRepair and 70.86% over IntentFix.
  • On SEC-bench, EvoRepair reaches 87.00% and reports gains of 33.50% over LoopRepair and 50.50% over IntentFix.
  • Across PATCHEVAL and SEC-bench, EvoRepair reports 90.46% overall performance.
  • Across both benchmarks, EvoRepair exceeds Live-SWE-Agent by 6.98% overall.
  • Transfer experiments on VUL4J are reported to work across datasets, programming languages, and model backbones; the excerpt does not provide the VUL4J metric values.

Link