Coding agents are being judged by interfaces, token cost, and repository evidence

Agent execution is becoming a costed subsystem. Terminus-4B trains a 4B model to run terminal commands for a coding agent, then return compact summaries to the main agent. The reported Serilog example cuts main-agent tokens from 2.46M to 740k, while the subagent handles nine internal commands and reports build, test, and failure details in about 200 tokens.

TSCG attacks the same deployment pressure at the tool-catalog level. It compiles JSON tool schemas into structured text before the model sees them. The paper reports about 19,000 benchmark calls across 12 models, with large gains for small models at 20 to 50 tools and substantial token savings on production-style schemas. The practical claim is narrow and useful: agents can call tools more reliably when their interface is written for model consumption.

Repository-level repair papers center on what evidence the agent can inspect before it edits code. ARISE adds statement-level definition-use edges to a repository graph, then exposes slicing and context tools to a SWE-agent setup. On SWE-bench Lite, it reports Function Recall@1 gains of 17.0 points, Line Recall@1 gains of 15.0 points, and 22.0% Pass@1, fixing 66 of 300 issues.

Structured spec-driven engineering (SSDE) makes a related point for generation. It gives LLMs Gherkin scenarios, domain models, generated API signatures, and templates while building Python MVC backends. The pilot shows that generated signatures often help more than raw domain models: across all tested LLMs, signatures improve average test pass rate by 7.82 percentage points and reduce variance. The failure analysis is also useful, since 49.0% of errors are non-existent API calls and 20.2% are type mismatches.

ARIADNE treats competitive programming as a search problem under a fixed budget. It uses Monte Carlo Tree Search (MCTS) with a shared blackboard that stores constraints, candidate strategies, generated tests, counterexamples, diagnostics, and repair notes. With GPT-4o, it reports Pass@1 of 41.30 on APPS, 46.67 on CodeContests, 27.27 on CodeContests+, and 20.91 on LiveCodeBench, with gains over the listed CodeSim baseline.

This line of work values execution feedback as reusable state. The important detail is the feedback format: ARIADNE combines scalar rewards with structured notes, then sends the search budget toward branches that have better tests and repairs. That design makes hidden edge cases and failed attempts part of the next coding decision.

One paper widens the evaluation target to maintainability. The AI-generated smells study compares LLM outputs on 90 CodeContest problems with human submissions, then audits 20 MetaGPT projects across increasing requirement stages. It reports smell patterns such as Long Method, Too Many Branches, Potential Improper API Usage, Unstable Dependency, and God Class structures.

The evidence is less numeric than the repair and tool-calling papers, but the warning is concrete. Passing code can still accumulate maintenance debt, and larger generated systems need static checks alongside test pass rates. SSDE’s failure analysis supports the same engineering need: many repository-generation errors are API and type mistakes that static analysis can catch before runtime tests.