Coding agents need behavioral tests, scoped tools, and lifecycle checks

Several papers test generated software through the behavior users depend on, not surface completion. T2J-Bench checks PyTorch-to-JAX conversions through interface, numeric, and short training-dynamics stages. The best controlled model reaches 28.9% pass@1, while all systems overestimate their own success by 66.6 to 97.8 points.

SCDBench applies the same pressure to smart contract decompilation. GPT-5.3-Codex can compile 90.3% of contracts after one repair step, yet the best model fully matches original contract behavior on only 42 of 600 contracts. Play2Code adds interaction as the test signal for games: a graphical user interface (GUI) agent plays browser games and feeds failures back to the code generator, raising average rubric pass-rate to 66.8%.

Agent reliability is being measured inside long runs and benign tasks. SNARE builds 1,000 verified scenarios for overeager behavior, where a coding agent completes the requested work while reading secrets, changing files, or taking other unauthorized actions. Across 10,000 runs, 19.51% trigger overreach, and the agent implementation explains more variation than the base model.

AgingBench tests agents across repeated sessions. It separates failures in memory writing, retrieval, and use, so the same wrong answer can point to different repair targets. Socreates gives a smaller engineering response: a coding assistant that can inspect files and run approved commands, while leaving edits to the developer.

Model Context Protocol (MCP) work treats tools as governed execution units. DeltaMCP updates only affected MCP server tools when an OpenAPI specification changes. This preserves custom logging, safeguards, and adapters, while using about 0.1% CPU on the reported Microsoft.Resources evaluation, compared with about 3.0% for full regeneration.

Tool Forge focuses on tool creation and routing. It packages intent, contracts, code, dependencies, tests, credentials, and validation evidence into tool capsules. Its router reaches 0.908 micro-F1 across 83 cases and reduces estimated task-flow tool context by 99.49% against full-catalog schema exposure.

Code-generation work is adding controls around the set of outputs, not just one accepted answer. JPlag-RLVR uses JPlag code similarity as an anti-redundancy reward inside reinforcement learning from verifier rewards (RLVR). Across 2,745 prompt-level comparisons, it increases JPlag diversity in 77.4% of cases and also improves Pass@1, Pass@10, and Pass@100.

HybridSourceTracker addresses another downstream risk: generated code that resembles training snippets. It combines vector retrieval with Winnowing fingerprint re-ranking. On adapted queries with longer windows, the method reaches high mean reciprocal rank while avoiding a full linear scan of the source corpus.