--- kind: ideas granularity: day period_start: '2026-06-11T00:00:00' period_end: '2026-06-12T00:00:00' run_id: fe29e522-4572-4dae-9d5a-d4bb56e61559 status: succeeded topics: - coding agents - runtime enforcement - agent evaluation - software engineering - AI security - CAD automation tags: - recoleta/ideas - topic/coding-agents - topic/runtime-enforcement - topic/agent-evaluation - topic/software-engineering - topic/ai-security - topic/cad-automation language_code: en pass_output_id: 249 pass_kind: trend_ideas upstream_pass_output_id: 248 upstream_pass_kind: trend_synthesis --- # Agent autonomy checkpoints ## Summary Coding-agent teams now have concrete places to add control: executable checks for repository instructions, rejection gates before agent pull requests reach reviewers, and sandboxed programmatic action channels for CAD work. The common pressure is review waste and unsafe autonomy in workflows where agents can edit repositories, open pull requests, or drive professional Windows software. ## Runtime verifier hooks for repository agent instructions Teams using coding agents should turn recurring repository instructions into executable pre-completion checks. A practical version is a small rule file with three fields per rule: when it applies, what behavior is required, and how the agent proves it passed. The hook can run before the agent declares a task done, checking items such as test commands, forbidden file edits, cleanup of temporary files, migration rules, or required review notes. Trace gives the clearest pattern: it extracts user corrections, rewrites them as atomic rules, and blocks completion until a verifier passes. The reported drop in ClawArena preference violations, from 100.0% to 37.6% in distribution and to 2.0% out of distribution, is a useful target for teams that already keep agent instructions as Markdown. The repository-level study adds a warning about quality: instruction files had mixed effects across 148 projects, and projects with large merge-rate gains had longer, more structured files than projects with large drops. A cheap pilot is to pick five instructions that reviewers repeat most often, attach one verifier to each, and compare violation rate and reviewer comments for agent pull requests over two weeks. ### Sources - [Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents](../Inbox/2026-06-11--getting-better-at-working-with-you-compiling-user-corrections-into-runtime-enforcement-for-coding-agents.md): Trace converts user corrections into atomic runtime rules with verifiers and reports large reductions in preference violations. - [Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests](../Inbox/2026-06-11--toward-instructions-as-code-understanding-the-impact-of-instruction-files-on-agentic-pull-requests.md): The instruction-file study finds mixed merge-rate effects and links better outcomes to longer, more structured files. ## Pre-review rejection gates for agent bug-fix pull requests Repositories that accept agent-generated bug fixes need a gate before human review. The gate should check whether the issue is still active, whether another pull request supersedes the fix, whether CI passes, and whether the agent states the chosen repair approach with the files and tests it touched. Pull requests that fail the gate can return to the agent with a specific rejection reason before a maintainer spends review time. The AIDev rejection study shows why this belongs before review: 46.41% of bug-fix pull requests from Copilot, Devin, Cursor, and Claude were rejected. The leading visible causes included inactivity, superseded fixes, incorrect implementations, wrong approaches, and CI failures. A repository can start with a GitHub Action that labels likely rejection causes, blocks review-request assignment until CI and issue-status checks pass, and asks the agent to revise against a short taxonomy. The success measure is simple: fewer reviewer comments on pull requests that later close unmerged, plus no increase in false blocks on merged fixes. ### Sources - [Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset](../Inbox/2026-06-11--understanding-the-rejection-of-fixes-generated-by-agentic-pull-requests-insights-from-the-aidev-dataset.md): The study quantifies rejected agent bug-fix pull requests and identifies rejection reasons useful for a pre-review gate. - [Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset](../Inbox/2026-06-11--understanding-the-rejection-of-fixes-generated-by-agentic-pull-requests-insights-from-the-aidev-dataset.md): The authors point to approach hints, constraints, CI validation, and task prioritization as ways to reduce wasted review and agent resources. ## Sandboxed COM scripts for CAD agent tasks CAD teams can test agents on bounded SolidWorks or AutoCAD tasks by asking the agent to generate Python COM scripts in a disposable Windows environment. The first useful scope is repetitive geometry setup, drawing cleanup, export, and cross-application file preparation where GUI agents often lose state across many clicks. The script should run under a restricted account, with an allow-list of COM objects and methods, artifact comparison after execution, and logs that an engineer can inspect. ComAct reports that COM control gives agents semantic access to professional Windows software and avoids long chains of fragile GUI actions. Its ComCADBench covers 1,000 tasks across SolidWorks, Inventor, and AutoCAD, with GUI-based agents reported near zero in many settings and COM-based execution far higher. The security boundary matters because the same agent stack often depends on gateways and tool brokers. Obsidian Security’s LiteLLM report shows a CVSS 9.9 chain where a low-privilege user could reach admin routes and execute code on the gateway server; teams should patch LiteLLM to v1.83.14-stable or later, keep CAD automation credentials separate, and prevent the model gateway from gaining direct desktop or filesystem reach. ### Sources - [ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm](../Inbox/2026-06-11--comact-reframing-professional-software-manipulation-via-com-as-action-paradigm.md): ComAct uses COM as the action space for CAD agents and reports large gains over GUI-based agents on ComCADBench. - [ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm](../Inbox/2026-06-11--comact-reframing-professional-software-manipulation-via-com-as-action-paradigm.md): The paper explains COM as semantic programmatic access across Windows professional software including SolidWorks. - [Breaking LiteLLM: From Low-Privilege User to Admin and RCE](../Inbox/2026-06-11--breaking-litellm-from-low-privilege-user-to-admin-and-rce.md): Obsidian Security reports a CVSS 9.9 LiteLLM chain reaching admin access and server-side code execution. - [Breaking LiteLLM: From Low-Privilege User to Admin and RCE](../Inbox/2026-06-11--breaking-litellm-from-low-privilege-user-to-admin-and-rce.md): The LiteLLM report describes the gateway as a point that can tamper with agent responses and steer downstream tools.