Code agents need executable proof, bounded work loops, and auditable traces

Benchmarks treated coding as a full work cycle. SWE-Cycle asks agents to set up repositories, change code, and write verification tests across 489 GitHub issue instances. Phoenix-bench adds hardware-engineering repositories and executable EDA checks, where agents transfer poorly because domain toolchains and project structure matter. SaaSBench and WebGameBench push the same standard into delivered applications: enterprise SaaS systems and playable browser games are judged by runtime behavior, configuration, and cross-component integration.

Several papers used execution results as the source of supervision or task pressure. DuST trains from execution-labeled candidate code produced during test-time scaling. FrontierSmith turns closed coding tasks into open-ended optimization problems, and DIO-Agent uses execution errors during code discovery. Orchard adds reusable sandbox infrastructure so agents can run tasks across software engineering, browser use, and assistant settings. The practical point is consistent: scoring, errors, and sandbox state are becoming part of the data pipeline.

The week also treated agent scores as objects that need inspection. BenchJack audits benchmarks for reward-hacking paths before normal runs and found exploits across 10 popular benchmarks. Rollout Cards asks agent papers to publish rollout records, views, reporting rules, and omitted fields so scores can be rechecked. Model Context Protocol (MCP) deployments raise the same control issue for enterprise tools: access, permissions, and third-party skills need review paths before agents act in production settings.

Security repair work focused on runtime evidence and reusable repair knowledge. ContraFix compares crashing and safe executions to infer what a patch must enforce, then stores successful repair specifications and input-mutation strategies. MemRepair adds persistent memory for repository-level vulnerability repair and reports gains on SEC-Bench, PatchEval, and Multi-SWE-bench C++. These systems treat past fixes and concrete executions as inputs to the next repair attempt, which makes evaluation depend on trace quality as much as patch text.