Recoleta
GitHub
Source note

ContraFix: Agentic Vulnerability Repair via Differential Runtime Evidence and Skill Reuse

Authors
Simiao Liu, Fang Liu, Li Zhang, Yang Liu, Yinghao Zhu
Source
arXiv
Published
Collection
Software Intelligence
Automated Vulnerability RepairCode IntelligenceLLM AgentsMulti Agent Software EngineeringRuntime Analysis
Open arXivMarkdown source

Summary

ContraFix is a multi-agent system for automated vulnerability repair that uses paired crashing and safe executions to infer the condition a patch must enforce. It also stores successful repair specifications and input-mutation strategies for reuse on later vulnerabilities.

Problem

  • LLM repair agents often patch the symptom shown by a crash report instead of the root cause, because one failing run shows where execution failed but not which nearby variable or state transition caused it.
  • This matters for real repositories: wrong repair direction can produce code that compiles while leaving the vulnerability unfixed or moving the crash elsewhere.
  • Existing agents usually discard useful evidence after each case, so similar vulnerabilities require fresh mutation, probing, and diagnosis.

Approach

  • The Mutator creates Proof-of-Concept variants close to the original input, then separates them into crashing and non-crashing groups.
  • The Analyzer inserts runtime probes near the fault site, runs both groups, and compares recorded state values to find which variables separate unsafe execution from safe execution.
  • The Analyzer writes a repair specification with a source location and a safety condition, such as extending a VM stack before regs[] access when argc reaches a boundary.
  • The Patcher turns the specification into source edits, then accepts a patch only if it compiles and passes re-execution on the original PoC and crashing variants.
  • A two-track skill store records verified repair specifications and successful mutation strategies; retrieval uses same-repository matches, same-vulnerability-type matches, then embedding similarity over sanitizer reports.

Results

  • On SEC-Bench, which has 200 C/C++ CVE instances, ContraFix with GPT-5-mini resolves 84.0% of tasks.
  • On PatchEval, using 225 Go, Python, and JavaScript instances after 5 Docker failures were excluded, it resolves 73.8% of tasks.
  • The paper claims best reported performance on both SEC-Bench and PatchEval while using less than one-third of the per-task cost of the strongest comparable baseline.
  • On SEC-Bench with the same GPT-5-mini backbone, ContraFix beats AgentMem by 34 percentage points.
  • Ablations report a +27.0 percentage-point gain from contrastive runtime analysis and a +9.0 percentage-point gain from skill accumulation.
  • ContraFix resolves 25 SEC-Bench instances that no existing compared method fixes.

Link