ATTAIN: Automated Exploit Failure Analysis through Trace-Driven Diff Analysis

Summary

ATTAIN uses exploit execution traces and version diffs to decide which library versions are affected by a CVE when exploits fail across releases.

Exploit-based checks can miss affected versions when API, build, or runtime changes stop an exploit from running even though the vulnerability remains present.
Commit-based methods such as V-SZZ and LLM4SZZ can choose the wrong vulnerability-introducing commit and then label long version chains incorrectly.
Accurate affected-version ranges matter because downstream projects rely on them to decide whether a dependency version exposes them to a known vulnerability.

ATTAIN runs a public exploit across historical Java library versions with Maven and records method-entry traces using an ASM-based Java agent.
It compares a version where the exploit triggers the vulnerability with a nearby target version where the exploit fails, crashes, or diverges, then records the first trace mismatch, unique methods, dependency changes, and execution errors.
An LLM-guided diff search inspects version changes and collects concrete diff hunks that explain the trace divergence, with fallback and repair evidence when direct hunks are missing.
A judgment step classifies each case as absent, breakage, or neutral, assigns a presence score, aggregates evidence across exploits, and backfills labels along the version chain.

The evaluation covers 224 CVEs, 25,943 library versions, and 128 Java libraries.
ATTAIN reports an F1-score of 93.24% for affected-version identification.
The reported F1 improvement is 116.28% over V-SZZ and 33.30% over LLM4SZZ.
Version-pair preprocessing produces 845 boundary version pairs; 459 unique pairs are selected for LLM-based analysis.
The paper claims ATTAIN matches or beats existing methods on frequent CWE types, including cases where exploit runs fail for non-vulnerability reasons, but the excerpt gives no per-CWE metric values.
The paper says bounded iterations and short trace-guided prompts keep token use low, but the excerpt gives no token-count result.