---
source: arxiv
url: http://arxiv.org/abs/2604.08417v1
published_at: '2026-04-09T16:17:58'
authors:
- Kevin Lira
- Baldoino Fonseca
- "Davy Ba\xEDa"
- "M\xE1rcio Ribeiro"
- "Wesley K. G. Assun\xE7\xE3o"
topics:
- vulnerability-detection
- llm-evaluation
- interprocedural-analysis
- software-security
- multi-language-code
relevance_score: 0.91
run_id: materialize-outputs
language_code: en
---

# Vulnerability Detection with Interprocedural Context in Multiple Languages: Assessing Effectiveness and Cost of Modern LLMs

## Summary
This paper tests whether giving LLMs caller or callee code helps them find vulnerabilities that depend on cross-function behavior. Across 509 ReposVul vulnerabilities in C, C++, and Python, extra interprocedural context usually did not help and often hurt accuracy while increasing cost.

## Problem
- Prior LLM vulnerability studies often inspect one function at a time, which misses bugs that only appear when data or control flow crosses function boundaries.
- Security teams also need results across multiple languages, plus usable explanations and reasonable inference cost for deployment in real tools.
- The paper asks whether adding caller or callee context improves detection, how results change across C/C++/Python, and what cost and explanation quality each model delivers.

## Approach
- The authors evaluate four models: Claude Haiku 4.5, Gemini 3 Flash, GPT-4.1 Mini, and GPT-5 Mini.
- They build an evaluation set of 509 interprocedural vulnerabilities from ReposVul, using AST analysis to identify caller-callee relations.
- Each target function is tested under three prompt settings: code only (CO), code plus callees (CC), and code plus callers (CK).
- They measure accuracy and F1, run paired McNemar tests with Holm-Bonferroni correction, estimate API cost from token counts, and manually rate 1,004 explanations for correctness and comprehensiveness on a 0-2 scale.

## Results
- Overall, code-only prompts were best for every model. Gemini 3 Flash scored **0.9853 accuracy / 0.9926 F1** in CO, Claude Haiku 4.5 scored **0.9756 / 0.9877**, GPT-5 Mini scored **0.9072 / 0.9514**, and GPT-4.1 Mini scored **0.7561 / 0.8611**.
- Adding context often reduced performance. GPT-4.1 Mini fell from **0.7561 accuracy in CO** to **0.5401 in CC** and **0.6923 in CK**. GPT-5 Mini fell from **0.9072 in CO** to **0.8348 in CC** and **0.7742 in CK**.
- Statistical tests show significant context-related drops for some models: GPT-4.1 Mini CO vs CC had **p=0.0000, Φ=0.404**, GPT-4.1 Mini CC vs CK had **p=0.0000, Φ=-0.436**, and GPT-5 Mini CO vs CK had **p=0.0003, Φ=0.259** after correction. Claude Haiku 4.5 showed no significant differences across context settings.
- The abstract reports a language-specific cost result for C: Gemini 3 Flash achieved **F1 ≥ 0.978** at an estimated **$0.50-$0.58 per configuration**, giving the best cost-effectiveness trade-off for C vulnerabilities.
- Adding interprocedural context nearly doubled token use and inference cost without matching accuracy gains, according to the abstract.
- For explanation quality, Claude Haiku 4.5 correctly identified and explained the vulnerability in **93.6%** of evaluated cases. The abstract also states that the GPT models had **zero-score rates 5× higher** than Haiku 4.5 and Gemini 3 Flash for correctness and comprehensiveness.

## Link
- [http://arxiv.org/abs/2604.08417v1](http://arxiv.org/abs/2604.08417v1)