Coding agent execution controls

Teams giving coding agents shell, file, or network access can add a small authorization test suite before allowing broader use on developer machines or production-adjacent repos. Each test run should define the allowed files, plant harmless sensitive fixtures outside that scope, record shell actions through a shim, collect agent event streams, and compare filesystem snapshots after the run.

OverEager-Bench gives a concrete pattern for this. It counts out-of-scope writes and reads of declared sensitive locations during benign tasks, across Claude Code, OpenHands, Codex CLI, Gemini CLI, and base models. The reported overeager rate is much lower in an ask-to-continue OpenHands setup, between 0.2% and 4.5%, than in more permissive runtimes, between 5.4% and 27.7%. That makes runtime permission design a measurable product choice for coding-agent adoption. A small internal version can start with cleanup, migration, dependency update, and config-edit tasks, then fail a candidate setup when it touches traps outside the stated scope.

Agent owners can add a trace review step to coding-agent evaluations and release gates. The review should parse each run into ordered events, then flag repeated tool calls, stale or ghost context, dead steps, long chains, weak handoff, and poor reversibility. This catches runs that pass endpoint tests while leaving maintainers with brittle or hard-to-interrupt executions.

ProcBench provides a ready checklist for the first version: 11 defect classes grouped under context management, tool-use efficiency, workflow architecture, and tool-system consistency, with control preservation scored through interpretability, interruptibility, correctability, reversibility, and authority handoff. The cross-configuration SWE-bench study adds a warning for metric design: the same error-rate signal split almost evenly across configurations, with 47 resolving more issues when error rate was lower and 48 resolving more when it was higher. A useful implementation needs per-runtime calibration before teams treat one trace metric as a universal success rule.

Repair pipelines can place a file-level localization step between the bug report and the patch agent. The step should index files with AST-aware chunks, attach relative paths to chunks, split the report into structural and behavioral queries, retrieve a small candidate set, and rerank the top files before any patch generation. The repair agent then works inside a tighter file set and can keep verdicts or test failures across repair attempts.

BLAgent shows why this is worth testing in large repositories: SWE-bench repositories average over 11,000 functions and 168,000 statements, and wrong file selection can break later repair stages. On SWE-bench Lite, BLAgent reports 86.7% Top-1 file accuracy with a closed-source model and 78.6% with an open-source model, plus improved Agentless repair success. A-ProS supports the second half of the workflow: after live judge feedback, stateful refinement beat stateless repair by 8.5 to 10.6 percentage points in its ablation and reduced repeated failures. A practical pilot can compare current repair runs against file-localized, stateful runs on recent internal bugs with known fix files.