Full operating loops define agentic coding evaluation

Asuka-Bench evaluates web-app agents under vague initial requests and multi-round user feedback. The benchmark hides the full product requirements, tests rendered browser behavior, and sends direct failure feedback into later rounds. The reported spread is large: after three rounds, weighted task pass rate ranges from 51.8% to 90.1% across 13 model-runtime configurations.

ADK Arena treats Agent Development Kits (ADKs) as measurable engineering choices. One coding agent builds benchmark agents for 51 Python kits inside isolated Docker environments. Generation succeeds in 57% of runs, while cost varies 5.6x across kits. The best single-benchmark agents reach 80% task resolution, and the median kit reaches 32%.

Two papers make the agent harness a target for measured repair. Retrospective Harness Optimization selects hard past tasks, reruns them, asks the agent to diagnose its own rollouts, and chooses a harness update through self-preference. On SWE-Bench Pro, it raises held-out pass rate from 0.59 to 0.78 without external grading.

HarnessFix gives the repair process finer grounding. It converts failed traces and harness code into step-level records, links failures to execution, tooling, context, lifecycle, observability, verification, or governance layers, and applies scoped patches. Across SWE-Bench Verified, Terminal-Bench 2.0 Verified, GAIA, and AppWorld, the paper reports 15.2% to 50.0% held-out performance gains over initial harnesses.

Memory work is being judged by downstream task effect, not by whether a note looks useful. MemOp accepts a memory only when it leaves all measured metrics unchanged or better and improves at least one. It reports up to 5.25 percentage-point single-episode success gains and at least 9.79% compute-cost reduction.

CL-Bench adds a caution: dedicated memory systems do not automatically beat full-context in-context learning. Across six domains, full-context in-context learning with Claude Sonnet 4.6 leads the aggregate results, while Mem0 and ACE trail it on normalized reward and gain.

Repository adaptation is another route. Code2LoRA generates repository-specific low-rank adapters from codebase embeddings, with an evolution mode updated by commit diffs. On RepoPeftBench, the static version reaches 63.8% cross-repository exact match, and the evolution version reaches 60.3% cross-repository exact match on evolving repositories.