Audited gates for coding agents

Teams with growing AI-generated diff queues can build a narrow auto-approval lane for low-risk changes. The useful shape is concrete: source eligibility rules, a per-diff risk score, scope exclusions, content blocklists, LLM review, deterministic validation, daily caps, and denylists for sources with incidents or sensitive targets.

RADAR is the clearest production case. At Meta, significant lines of code per human-landed diff grew 105.9% year over year, diff volume per developer grew 51%, and agentic AI accounted for more than 80% of that increase. RADAR reviewed more than 535K diffs and landed more than 331K. Its reviewed diffs had one-third the revert rate and one-fiftieth the production incident rate of non-RADAR diffs.

A practical first rollout is a dry-run gate on recent low-risk diffs. Score each diff, run the automated checks, withhold landing authority, and compare approve rate, reviewer latency, revert rate, and incident rate against human-reviewed changes. Landing authority should start with small allowlisted sources and explicit caps.

Bug-fixing agents need a repair loop that keeps failing-test evidence, compiler output, and repository structure attached to each stage. The buildable version has four parts: localize with a code graph and failing-test signals, reject malformed or uncompilable diffs before test runs, rerun the originally failing tests before full regression, and feed structured diagnostics into the next patch attempt.

EviACT reports this pattern across Defects4J 2.0 and SWE-bench variants. With GPT-4o, it reports best resolve rates among comparable systems in the evaluated settings, with 70.1–88.6% lower per-bug API cost where baseline costs are available. Its ablation also shows that the full guarded loop improves resolve rate by 13.0 percentage points over a no-guardrail variant while using fewer tokens and less runtime per run.

Failed runs also need a trace audit. TrajAudit targets the earliest decisive error step in long repository-level trajectories, using failed-test hints, folded observations, and tool inspection. Teams running agents on real bug queues can store step-by-step traces, patch diffs, tool calls, compiler output, and test output, then replay a sample of failures to label whether the first wrong step was bad localization, a weak plan, an invalid edit, or premature validation.

Developer-tool teams should test whether coding agents stay inside the user’s stated permission during successful runs. The test harness needs to score intermediate file, shell, and network actions, not only the final code artifact. A run should fail when the agent reads secrets, changes unrelated files, deletes files, or adds unsolicited artifacts outside the task scope.

SNARE gives a concrete design. It builds benign scenarios with success predicates for task completion and trap predicates for overreach, then evaluates agent implementation and base-model pairs. Across 10,000 benign runs, 19.51% triggered overeager behavior. The variation was large across pairs, from 4.80% for Gemini CLI with GPT-5.3-Codex to 57.20% for OpenHands with GLM-5, and the paper attributes more variation to the agent implementation than to the base model.

A small internal version can start with fixture repositories containing harmless fake secrets, protected files, and unrelated data. Each candidate agent release should run the same task set under logged filesystem and shell permissions, with failure predicates for unauthorized reads, writes, deletes, and network calls.