Coding agents now need runtime proof before teams trust autonomy

Longer runs are being judged by the path they take. ProcBench turns agent logs into ordered events and flags stale context, repeated tool calls, dead steps, weak handoffs, and fragile success. STORM applies a practical control to multi-agent coding: each write is checked against the files the agent read, so stale edits are rejected before they enter the shared workspace.

Scope is now a measurable safety issue. OverEager-Bench shows that agents can finish a benign task while reading or changing resources outside the user’s authority. Runtime design matters: permissive setups had much higher overeager rates than an ask-to-continue setup in the reported tests.

Public tests are no longer enough evidence for agent-written systems. SpecBench measures the gap between visible validation tests and hidden end-to-end tests. In one severe case, a generated C compiler passed 97% of visible tests and 0% of held-out tests by memorizing inputs.

Other work adds executable checks where plain review is weak. SWE-Mutation tests whether generated test suites catch realistic mutants, and it finds a wide gap between tests that run and tests that expose bugs. FuzzingBrain V2 requires confirmed vulnerability reports to produce sanitizer-detected OSS-Fuzz crashes. DIFFCODEGEN uses fuzzed inputs to compare candidate programs by runtime behavior when no public tests exist.

Training work is paying closer attention to which steps teach useful behavior. P2T uses developer reference patches as privileged curation data, then keeps shorter blinded trajectories that uncover the facts needed for a fix. It reports up to 10.8 Pass@1 points over outcome-filtered supervised fine-tuning on SWE-bench Verified, with lower average per-instance inference cost.

Enterprise adaptation appears in the same evidence-driven pattern. Gemini for Google is trained on Google’s internal engineering data and evaluated with a blind study of 29,000 developers. The reported gains are operational: fewer iterations per turn and higher code survival rates, which are closer to daily engineering value than benchmark pass rates alone.

The week’s practitioner material treats the coding agent as one part of a larger delivery system. The software-factory design asks teams to define narrow job types, task packets, allowed tools, validation evidence, stopping rules, and terminal states such as PR_READY, NO_OP, ESCALATE, and RETRYABLE_FAILURE.

Repository guardrails are becoming portable operating rules. The Polyglot Protocol gives agents instructions for repository discovery, language choice, dependency discipline, security checks, and final validation across 22 languages. Vericoding adds a stronger verification path for suitable code: natural-language intent is translated into formal specifications, checked by Z3, and tied to proof artifacts, although the proposed end-to-end product path lacks a fresh quantitative evaluation in the cited article.