Coding Agent Feedback Loops

Teams running coding agents should add a repair queue for failed trajectories. The queue should store tool calls, observations, file changes, final submissions, and the harness code version that produced each run. A reviewer or repair agent can then assign each failure to a concrete harness area such as tool schema, context assembly, lifecycle control, logging, verification, sandboxing, or policy checks.

HarnessFix gives a useful pattern: convert raw traces into step-level records, link each bad outcome to a harness layer, generate a scoped patch, and validate that the patch reduces the target flaw without broad regressions. RHO adds a lighter deployment loop for teams without labeled validation sets: pick hard and varied past failures, rerun them, let the agent compare its own rollouts, and accept the harness update preferred over the baseline. A small first test could use 20 to 30 recent failed coding-agent jobs, with a held-out slice of similar issues used only after candidate patches are chosen.

Web teams evaluating coding agents should test agents through a deployed UI, not only through source diffs or unit tests. A practical setup is a hidden product-requirements file, an intentionally incomplete user request, browser tests for visible behavior, and two or three feedback rounds where failures are converted into plain-language user feedback.

Asuka-Bench shows why this is worth building for web-app agents. It starts with underspecified requests, hides the full clarified requirements, tests rendered browser behavior, and sends direct failure feedback into later rounds. Across 13 model-runtime configurations, three-round weighted Task Pass Rate spans 51.8% to 90.1%, so the loop separates configurations that may look similar in one-shot code generation. The first internal version can be small: 10 common UI tasks, Playwright checks arranged by prerequisite behavior, and a score split between first-pass success and recovery after feedback.

Repository memory should have an admission test before it is added to a coding agent’s context. A useful gate compares the same task with and without a candidate memory and keeps the memory only when measured outcomes stay unchanged or improve, with at least one gain on success, localization, resolve efficiency, or related task metrics.

MemOp reports this kind of utility test for software-engineering agents, with accepted memories trained from trajectory-based rejection sampling and evaluated on SWE-Bench Verified. The reported gains include up to 5.25 percentage points in single-episode success rate and at least 9.79% lower compute cost. CL-Bench adds a check on adoption: full-context in-context learning with Claude Sonnet 4.6 beats several dedicated memory systems on aggregate normalized reward and gain. A team can start by measuring its best long-context baseline against a memory-gated variant on repeated tasks from the same repository before saving agent-written notes for general reuse.