Coding Agent Control Gates

Teams using Claude Code, OpenHands, Aider, or similar tools should add an explicit merge gate for agent-authored code. The gate should record the agent name, model version, prompt or task summary, tool calls, edited files, tests run, and any network or secret-touching behavior. Pull requests with new outbound calls, credential handling, build-script changes, dependency changes, or generated security-sensitive code should require a reviewer to inspect the trace before merge.

The production pressure is clear: Anthropic says more than four-fifths of the code it published in May was written by Claude, up from low single digits before Claude Code launched. A separate sabotage study found that 94% of developers failed to detect and remove data-exfiltration sabotage in a five-hour coding task without a monitor. Even with a monitor that flagged 16 of 19 sabotage attempts, malicious code still remained in 9 of those 16 sessions.

A cheap internal test is to replay 20 recent AI-assisted pull requests through the gate and measure how many require extra review, how often traces are missing, and whether reviewers can explain the riskiest tool calls without asking the author. The first useful version can be a CI check plus a pull-request template; it does not need a new development environment.

Engineering teams with large repositories should test whether a coding agent can find the right code before judging its patch quality. The practical test is simple: give the agent an issue and ask for ranked file-and-line ranges under a fixed line budget, then score whether those ranges include the code a human or known-good fix used. This separates repository exploration from patch generation and exposes cases where an agent writes plausible changes after reading the wrong files.

SWE-Explore shows why this step matters. It covers 848 issues across 203 repositories and finds that upstream exploration metrics track downstream repair results closely: Context Efficiency has Pearson r=0.950 against resolve rate, and first useful hit has r=0.928. TeleSWEBench shows the same pain in telecom code, where file localization drops hard on difficult srsRAN 5G tasks and the strongest evaluated tools reach up to 25% ship-ready changes.

The first adoption check can use 30 closed issues from the team’s own repository. Ask each candidate agent to return five line ranges before it is allowed to edit. If the agent cannot locate the relevant code with a small context budget, widening autonomy for that repository will mainly create review load.

Teams training or fine-tuning coding agents should keep full solver traces as training assets: searches, file reads, edits, shell commands, test failures, final diffs, and reviewer outcomes. Those traces can feed two practical loops. One loop distills repeated failure patterns into skills or task templates. The other loop updates diagnostics when training branches fail through reward leakage, zero-variance rollouts, behavior collapse, or misleading pass rates.

Socratic-SWE uses repository-solving traces to create targeted repair tasks with execution-based validation and reaches 50.40% on SWE-bench Verified after three iterations under a 36k-task training budget. EvoTrainer reads metrics, rollouts, configs, logs, and code diffs to decide whether to keep, prune, revert, or merge training branches, and reports SWE-9B at 38.16 Avg@8 BC% versus 33.77 for a human-engineered RL setup.

A small version is buildable inside an existing evaluation harness. Store every failed and successful agent run in a queryable format, tag failure types after review, and generate a weekly set of execution-checked repair tasks from the most common misses. The key adoption test is whether the generated tasks catch regressions or improve pass rate on a held-out set without increasing flaky tests.