Patch Quality Gates

Repository repair agents can now be packaged around AST-level read and edit tools, with review focused on named code entities and patch size. CodeStruct reports that this interface lifts SWE-Bench Verified Pass@1 by 1.2 to 5.0 points for frontier models, and GPT-5-nano jumps from 19.6 to 40.4 as empty-patch failures fall from 46.6% to 7.2%. PRepair points to the same operational change on the training side: reward small correct edits, because pass rate alone hides over-editing. On HumanEvalFix, Qwen2.5-Coder-7B moves from 47.44 to 81.62 on fix_1@1 with only a 1.37-point pass@1 gain. For teams shipping internal code agents, the build is concrete: expose functions, classes, and methods as the editable unit; reject syntax-breaking edits at the tool layer; log edit distance per successful patch; and review agents on minimal accepted diffs, not only test success. The first users are repository maintenance teams where bad patches fail in formatting, touch too much unrelated code, or create long review cycles.

Issue-resolution evaluation needs a design-constraint gate alongside test pass rate. SWE-Shield shows why: agents score 70.25% to 75.95% pass rate on the verified split, while design satisfaction is only 32.64% to 50.20%, and the paper finds little statistical relation between functional correctness and design compliance. This supports a practical workflow change for teams already running SWE-Bench-style evaluation or agent-assisted bug fixing. Mine design rules from merged pull requests and review threads, link them to issue types, and run a verifier that checks architecture choices, error handling, API consistency, and maintainability conditions before a patch is marked acceptable. The near-term product is not a new benchmark brand. It is an internal acceptance check that catches patches which pass tests but still conflict with repository conventions. Teams with active code review queues would care first, because that is where these failures become visible and expensive.

AI-generated code needs a post-generation security review step with exploitability checks on high-risk code paths before merge. Broken by Default reports vulnerabilities in 55.8% of 3,500 generated programs and 1,055 findings formally proven exploitable with Z3 witnesses. Integer arithmetic reaches 87% vulnerability rate and memory allocation 67%. The same paper also shows a useful asymmetry for workflow design: models catch their own vulnerable outputs 78.7% of the time in review mode, while secure prompting only cuts the mean vulnerability rate by 4 points on the tested subset. AutoPT evidence points in the same direction for offensive security agents: many systems hallucinate, 8 of 13 frameworks produce hallucinated flags, and only 16.67% of chained-vulnerability samples complete the full exploit chain. A practical build here is a merge gate for agent-written code that routes memory management, arithmetic, auth, input handling, and crypto changes through model review plus formal or symbolic checks where available, with human escalation on any proved witness or uncertain result.