Coding agents are being judged by review load, run cost, and containment

Benchmarks are putting the user back into the loop. SWE-Together reconstructed 109 repository-level tasks from 11,260 recorded sessions and scored both final correctness and User Correction, a measure of how much explicit or soft feedback the agent needed. Claude Opus 4.8 led the reported agents at 63% pass@1, while the reference patch baseline stayed around 78%.

SWE-INTERACT made the cost of interaction more visible. On the same underlying tasks, Opus 4.8 dropped from 50.7% single-turn resolve rate to 26.7% in the multi-turn setting. GPT 5.5 dropped from 48.0% to 24.7%, while its per-trial cost rose from $2.78 to $9.84. The failures were not only missed goals; forgotten requirements and implementation bugs remained common after long exchanges.

The strongest repair work tied model output to executable artifacts. SWE-Doctor generated multi-faceted bug reproduction tests, ran them under a debugger, and fed runtime diagnosis records into patch generation. It reported 75.7% average resolution on SWE-bench Verified and 59.4% on SWE-bench Pro, with an 8.0 to 8.9 percentage-point gain over baseline agents on SWE-bench Pro.

Other work in the week used feature maps, profiling traces, compiler errors, benchmarks, and approval records as control points for repository-level edits, memory optimization, and C-to-Rust migration. The common pattern is concrete evidence before final code submission: tests, traces, diagnostics, and explicit approval artifacts become part of the agent’s work product.

Cost evidence became more concrete. TraceLab analyzed 4,265 real coding-agent sessions, 357,161 LLM steps, and 432,510 tool calls from Claude Code and Codex. Prefix tokens accounted for 52.56B of 54.90B input tokens and 59.5% of estimated API cost. Even with a 95.7% global prefix-cache hit rate, misses caused 3.8 times more prefilling than truly new input.

Repository quality also showed up as an operating cost. A controlled minimal-pair study across 660 Claude Code trials found no pass-rate change from cleaner code, but cleaner repositories used 7% to 8% fewer tokens and reduced file revisits by 34%. A separate budget RFC proposed run-scoped atomic spend reservations before provider calls, with machine-readable budget state so agents can downshift model choice, shorten context, or stop cleanly.

Safety work focused on what the agent is allowed to touch. UnderSpecBench tested 2,208 DevOps prompt variants and found that acted runs violated action boundaries 55.8% to 67.8% of the time across evaluated configurations. Wrong-target and overscope outcomes matter because a plausible cleanup, rollback, or access-change command can hit the wrong branch, namespace, database, or service.

Identity and execution designs treated the model runtime as a place to minimize trust. Securing Agentic Identity proposed a broker, proxy, and mutual TLS pattern so real OAuth tokens never enter the agent environment. Fly.io’s Sprite pattern kept the long-lived agent loop separate from disposable command sandboxes, with single-command token injection and checkpoint rollback. The Claude Desktop red-team report showed why that separation matters: synced preferences plus command-capable connectors can turn an account compromise into workstation code execution.