Coding Agent Trust Gates

Teams giving coding agents access to APIs, deploy tools, ticket systems, or billing-sensitive services should add a verifier between generated code and the first live call. The verifier should read the task instruction and tool registry, check tool choice, output shape, call signatures, call ordering, and argument provenance, then return item-level repair feedback.

RubricRefine reports 0.86 success on M3ToolEval across seven models, compared with 0.62 for single-pass CodeAct, with no environment execution during refinement. A practical internal test is 50 archived multi-tool tasks with known correct traces: measure wrong tool routing, malformed arguments, and provenance errors before and after the gate. The gate is most useful where a failed call changes external state, consumes quota, or creates an audit event.

Maintenance agents need PR acceptance rules that cover codebase understanding, test quality, refactor hygiene, and valid abstention. SWE Atlas evaluates 284 tasks across 18 active repositories and shows that regression tests can miss 15 to 40 points of refactoring quality; test-writing mutation checks miss 10 to 15 points of rubric quality. FixedBench adds the production case of stale reports: agents edited already-fixed code in 35% to 65% of cases under the main setting, and abstention prompts caused high wrong abstention on partially fixed tasks.

A useful workflow is to require an evidence manifest with commands run, files inspected, tests added, behavior-preserving rationale for refactors, and a no-code-change option tied to reproduction and git-history evidence. Pilot it on recent duplicate tickets and partially fixed tickets before allowing autonomous merge rights.

Security teams testing coding agents on direct harmful prompts are missing the Jira-style attack path shown by MOSAIC-Bench. It uses 199 three-ticket chains with deterministic Python proof-of-concept oracles on Docker deployments; nine production agents reached 53.3% to 85.9% end-to-end attack success with only two refusals.

Add a regression suite that feeds an agent sequences of benign-looking tickets against a disposable app, composes diffs, and runs exploit oracles on the final deployment. Pair it with monitor tests using refined attack trajectories: MonitoringBench shows a full-trajectory Opus 4.5 monitor fell from 94.9% catch rate on elicited attacks to 60.3% on the best refined attacks. The first target is any team letting agents implement backlog items without a security engineer reviewing cumulative diffs.