Coding-agent research is being judged by runnable proof, repo realism, and harness quality

Evaluation kept tightening around executable proof. Daily trend documents across the week repeatedly favor systems scored by full runs, tool traces, state changes, and live workflow survival. The item-level evidence adds the same message at repository scale: RealBench tests code generation with real repositories, UML design inputs, and human-verified tests, and the best average Pass@1 is still 19.39%. The week reads as a demand for runnable artifacts, not polished code text.

Repository-scale coding still has a hard ceiling. RealBench reports that performance falls from above 40% Pass@1 on repositories under 500 LOC to below 15% above 2000 LOC. RAT shows why environment setup matters so much in this regime: its automated configuration pipeline raises executable setup success across Python, Java, Rust, and JS/TS, but it also carries real cost in tokens and time. The practical story this week is simple: getting code to run is still a major part of the task.

Agent quality depends heavily on the harness around the model. The week’s trend coverage points to scaffolding, documentation, and control surfaces as outcome-critical variables. HARBOR makes that claim concrete: in its manual tuning study, a modest flag set improved Terminal-Bench 2 results from 15/89 to 17/89, while later stacks with extra self-evaluation and reflection features fell to 13/89 and 12/89. Better agent behavior here comes from disciplined configuration work, not just choosing a stronger model.