Research idea

Coding-agent controls tied to verification failures and harness changes

Week · 2026-W29 · Software Intelligence

Repository exploration can be deferred until verification identifies a concrete knowledge gap, reducing unnecessary context while preserving a path to deeper repair. Separately, harness upgrades need paired security regression tests because changing the interaction layer can reverse whether the same model blocks or executes an unsafe action.

2 ideas

Verification-triggered repository question answering for coding-agent repair

Coding-agent teams should trigger targeted repository question answering when a minimal repair fails verification, rather than performing the same broad exploration before every edit. E3 shows that starting with the smallest viable execution path and expanding after failed verification can reduce redundant work, while ACQUIRE shows that explicit questions about repository mechanisms, dependencies, and contracts can recover otherwise failed repairs. DiffTestGen supplies a concrete escalation signal: tests aimed through public entry points at changed functions can reveal unexplained old-versus-new behavior or stalled changed-line coverage.

A practical repair loop would estimate scope, make the smallest plausible patch, and run change-directed tests. A failure would be translated into repository questions for read-only agents; their file- and function-cited answers would then inform the next patch. The cheapest useful evaluation is an ablation on the same issue set comparing always-on question answering, no question answering, and verification-triggered question answering, measuring resolved issues, tokens, and repair rounds.

Harness-swap security tests for coding-agent releases

Coding-agent release engineers should treat a harness upgrade as a security-relevant behavior change even when the model is unchanged. AgentCompass found that harness choice changed benchmark scores and trajectory failures. In package-install experiments, swapping only the harness changed one untrusted-registry attack from 10/10 caught to 9/30 caught, while another variant moved in the opposite direction. Model-level safety claims therefore cannot substitute for testing the shipped model–harness combination.

Before releasing a harness or permission-policy change, run the current and candidate versions with the same model on fixed dependency-install attacks and benign controls, then compare whether unsafe commands are blocked before execution and inspect the recorded trajectories. The release receipt should bind the result to the harness version, tool policy, command set, and source state; Proof-or-Stop demonstrates the underlying mechanism for rejecting stale, tampered, or reconfigured evidence, while explicitly not proving semantic correctness.