Agent Workflow Checkpoints

Teams running coding agents on bug fixes can add a repair gate that preserves failing-test evidence across localization, patching, and validation. The gate should reject unappliable diffs, syntax errors, and build failures before test validation, then rerun the originally failing tests before a full regression suite. Failed attempts should return structured diagnostics to the agent and store the earliest decisive error step for review.

EviACT gives a concrete version of this workflow: it maps RED failing-test evidence to suspect code spans, filters invalid patches with a compile gate, and runs target tests before regression. With GPT-4o, it reports 25.0% resolve rate on Defects4J 2.0, 40.4% on SWE-bench Verified, and 70.1–88.6% lower per-bug API cost where baseline costs are available. TrajAudit adds the audit trail: it diagnoses failed repository-level agent runs by predicting the first step that sent the run off course, with more than 24.4 percentage points better localization accuracy on RootSE while using at least 18% fewer tokens than baselines.

A low-cost rollout is a wrapper around the existing agent runner and CI system. Run it on recent failed agent patches, compare accepted fixes, build-fail rate, full-regression invocations, and tokens per accepted patch, then decide whether to make the gates mandatory for agent-authored pull requests.

Teams asking agents to write formal specifications should test the specification as a program artifact before trusting a verifier result. The practical build is a harness that turns requirements into executable checks over valid inputs, invalid inputs, correct outputs, and incorrect outputs, then runs hidden and adversarial cases against the generated spec.

Verus-SpecGym shows why this check matters. A verifier can prove code against the wrong specification, so the spec has to match user intent. The benchmark gives agents Codeforces-derived tasks, Verus skeletons, examples, documentation, and tool access, then evaluates generated specs with executable Rust checks. The best reported model solves 77.8% of 581 tasks, while LLM-as-a-judge evaluation misses 26% of failures caught by the executable evaluator.

The adoption path is narrow and useful: start with small pure functions or service-level invariants where product owners can write examples and edge cases. Treat a generated spec as ready for proof work only after it accepts valid behavior and rejects invalid behavior under concrete tests.

Teams with multi-agent workflows can add a test suite that covers the declared coordination structure, including reachable agents, allowed tool calls, restricted tool attempts, and delegation edges. The build is straightforward: extract a manifest from agent entry points, turn it into a graph, generate witness scenarios for each obligation, and judge runtime traces for the expected event or violation.

The workflow-testing paper demonstrates this on OpenAI Agents SDK-style systems. Across 10 SDK-derived workflows with 49 reachable agents, 47 tools, and 403 structural obligations, generated scenarios witnessed 54 of 75 allowed-tool obligations and 36 of 48 delegation obligations within the stated refinement budget. Restricted-tool probing found 23 violations among 248 restricted-tool obligations.

This fits teams that already review tool permissions manually or rely on end-to-end task success. A first check can run in CI after workflow edits and fail only on restricted-tool violations; coverage thresholds for allowed calls and delegation paths can start as reported metrics until the scenarios stabilize.