Source note
Taint-Style Vulnerability Detection and Confirmation for Node.js Packages Using LLM Agent Reasoning
Summary
LLMVD.js is an LLM-agent pipeline for finding and confirming taint-style vulnerabilities in Node.js packages without relying on dedicated static or dynamic taint engines. On the benchmarks in the excerpt, it confirms far more vulnerabilities than prior Node.js analysis tools and also finds validated issues in newly released npm packages.
Problem
- Node.js packages are a major part of software supply chains, and many contain taint-style flaws such as command injection, code injection, path traversal, and prototype pollution.
- Existing program-analysis tools have trouble with JavaScript dynamics, weak type information, native Node.js behavior, brittle parsing and instrumentation stacks, and SMT solving on strings and regex-heavy constraints.
- Detection alone is not enough; a useful system must also confirm exploitability with working proof-of-concept inputs so teams can separate real bugs from false positives.
Approach
- The paper builds LLMVD.js, a multi-stage ReAct-style agent that reads package code, searches for suspicious flows, proposes candidate vulnerabilities, and processes each candidate through later stages.
- The pipeline has four main stages: Finder proposes possible bugs, Judge filters candidates by likely reachability and exploitability, Constraints Inferencer extracts the conditions an exploit must satisfy, and Exploiter generates and runs proof-of-concept code.
- Instead of deriving taint paths with a dedicated analysis engine, the system uses LLM reasoning over raw source code plus lightweight tools such as code search, file inspection, and package execution.
- Confirmation uses class-specific execution oracles based on concrete side effects: sentinel files for path traversal, marker functions for code injection, object-property probes for prototype pollution, and cleaned sentinel artifacts for command injection.
- The design aims for high recall early, then higher precision through execution-backed validation.
Results
- On public benchmark packages, LLMVD.js confirms 84% of vulnerabilities, compared with less than 22% for prior program-analysis tools named in the excerpt.
- The paper states that LLMVD.js also outperforms a prior LLM plus program-analysis hybrid approach (PoCGen) while needing no vulnerability annotations and no prior vulnerability reports.
- On 260 recently released packages without ground-truth vulnerability labels, traditional tools produced validated exploits for at most 2 packages, while LLMVD.js produced validated exploits for 36 packages.
- The paper reports 36 previously undocumented vulnerabilities in recently released Node.js packages and says the authors disclosed them to maintainers, with 3 maintainer acknowledgments received.
- The excerpt does not provide a full metric table by vulnerability type, dataset split, or exact baseline names for every number beyond the comparisons above.