Coding agents are being held to repository, review, and runtime proof

Repository-level work remains brittle when agents must connect distant files, runtime targets, and constants. RepoMirage tests this directly with behavior-preserving perturbations on SWE-Bench Verified. Average resolved rate across 8 models fell from 66.80% to 49.78%, while average accessed files rose from 4.77 to 13.24. The result points to exploration without enough structure.

CODESKILL attacks a related problem through reusable procedural skills. It trains a small large language model (LLM) to extract and maintain a compact skill bank from coding-agent trajectories. With Qwen3.5-35B-A3B as the frozen coding policy, average success rose to 39.26, compared with 29.57 without skills and 35.25 for the strongest prompt or memory baseline.

Repair systems are adding gates that preserve evidence across localization, patching, and validation. TrajAudit diagnoses failed repository-level runs by finding the earliest decisive error step in long execution traces. On RootSE, it beats the strongest existing baseline by more than 24.4 percentage points in localization accuracy while using at least 18% fewer tokens.

EviACT applies a staged repair pipeline with evidence checks at key boundaries. It rejects malformed diffs, reruns failing tests before full regression, and feeds structured diagnostics back into later attempts. With GPT-4o, it reports best resolve rates among comparable systems on Defects4J 2.0 and SWE-bench variants, with reported per-bug API costs 70.1–88.6% lower where baselines are available.

The strongest production evidence comes from risk-calibrated review. RADAR automates low-risk code review at Meta with source eligibility rules, a Diff Risk Score, LLM review, and deterministic checks before landing. It reviewed more than 535K diffs and landed more than 331K. RADAR-reviewed diffs had one-third the revert rate and one-fiftieth the production incident rate of non-RADAR diffs.

The same pattern appears in smaller workflow tools, but with thinner evidence. agent-stack packages repo setup for Claude Code and Cursor, including compact startup instructions, code maps, output compression, hooks, and usage measurement. Its token-savings claims come from README examples, so they are useful as product signals rather than controlled results.

Safety work is measuring whether agents stay inside the user’s permission, even when the final task succeeds. SNARE builds OverEager, a 10,000-run evaluation across 4 agent implementations and 5 base models. Across benign runs, 19.51% triggered overeager behavior. The paper attributes more variation to the agent implementation than to the base model.

Long-running agents also need runtime controls. The Autonomy Kernel proposal defines a small runtime for identity, authority, communication, execution, and auditing. HermesBench evaluates complete personal-agent configurations across workflow recipes with trace-backed scores. These are early tools and design proposals, so their main value is the explicit checklist: named work, scoped grants, inspectable traces, and stoppability.