Coding agents earn trust through context, traces, and executable checks

Several results treat the agent interface as part of the capability being measured. Context-Augmented Code Generation reports that adding Brief, a product-context retrieval system, raises decision compliance for Claude Code on an 8-task benchmark from 46% to 95%. The result is useful, but the paper also notes a workflow confound: Brief changes available context and adds specs, acceptance criteria, and mid-build guidance.

SWE-Edit makes a narrower systems claim. It splits file viewing and patch writing into separate subagents, so the main agent keeps a cleaner reasoning context. On SWE-bench Verified, it raises resolved rate from 69.9% to 72.0%, cuts total inference cost by 17.9%, and improves edit success from 93.4% to 96.9%. The common lesson is practical: agent quality depends on what the model can see, what it is asked to write, and how edits are applied.

The strongest evaluations this week ask agents to leave checkable artifacts. AutoMat tests reproduction of 85 computational materials-science claims. The best tested agent reaches a 54.1% success rate, while paper-only reproduction has near-zero success across systems. The weak point is reconstructing underspecified procedures and judging whether outputs support the scientific claim.

LiveFMBench applies the same discipline to formal specifications for C programs. It uses 630 ACSL-annotated programs and filters outputs that change the program or assertion. After that faithfulness check, measured accuracy drops by about 20%. Claw-Eval-Live extends trace-based grading to workflow agents by recording tool traces, service audit logs, command traces, files, tests, and service state. Its leading model passes 66.7% of 105 tasks, so the benchmark still exposes many failed real workflows.

Security and testing papers give the model a constrained job, then verify the output against a concrete signal. FeedbackLLM feeds uncovered line and branch data back into later prompts for C and Python test generation. Its reported gains are large on several PALS/RERS programs, although the excerpt also shows weaker cases and no aggregate mean coverage.

QASecClaw keeps Semgrep as the high-recall scanner, then asks a coding-focused large language model to judge each finding with source context. On OWASP Benchmark v1.2, false positives fall from 560 to 64, with recall down by 3.1%. VulKey takes a similar bounded approach for repair: it predicts repair patterns using CWE type, syntactic action, and security-specific key elements, then generates a patch. On PrimeVul, it reports 31.5% repair accuracy, 7.6 percentage points above the best baseline in the excerpt.

The week also treats orchestration and serving as measurable parts of agent performance. SAGA schedules an entire agent workflow on GPU clusters, rather than treating each model call as independent. The reason is concrete: agent tasks can make 10 to 100 chained calls with tool gaps, and request-level schedulers regenerate cache and inflate end-to-end latency.

On 64 A100 GPUs, SAGA reduces task completion time by 1.73x on SWE-bench and 1.55x on WebArena versus vLLM with automatic prefix caching. The tradeoff is explicit: peak throughput is about 30% lower than throughput-optimized batching. That kind of accounting matches the week’s broader standard. Agent work is judged by complete task time, stored traces, reproducible artifacts, and failure modes that users can inspect.