Coding-agent research is getting harder to game and easier to verify
This period is strongest on coding agents that face real state, real failure modes, and real execution consequences. SWE-STEPS and ABTest make evaluation more concrete.
Topic summary
Benchmarks
Repository Agent Safety Gates
Coding-agent evaluation is moving into real repository state, real user failure traces, and real extension security checks.
Software-agent research is getting more executable, replayable, and production-bound
Today’s research is strongest where software work can be checked by execution. The main emphasis is stricter evaluation for coding agents, plus better test generation for code and APIs.
Operational Agent Validation
Production-facing software-agent work is getting concrete in three places: private replay benchmarks for coding agents, deterministic testing for tool-call failure and recovery, and requirement-driven API test…