Agentic coding research is setting hard gates around generated work

Large language models (LLMs) are being tested on specification tasks where a plausible answer is not enough. LiveFMBench checks ANSI/ISO C Specification Language (ACSL) contracts for C programs with Frama-C, Alt-Ergo, and Z3, then filters outputs that changed the program or assertion. That filter cuts reported direct-prompting accuracy by about 20%, and loop invariants remain the main failure type.

Requirement tools show the same preference for constrained generation. ClarifySTL asks users to resolve vague time bounds, thresholds, and references before producing Signal Temporal Logic (STL). It reports double-digit accuracy gains on DeepSTL and STL-DivEn. A separate OOMRAM requirements agent uses a deterministic Python validator to reject invalid requirement combinations; final outputs reached 100% structural validity in 10 project visions, though exact matches to gold standards were not achieved.

Testing work splits between measurable coverage gains and brittle autonomous repair. FeedbackLLM feeds missed line and branch data back into later prompts, with duplicate filtering across iterations. On several PALS/RERS C programs it reports large branch-coverage gains over KS-LLM, including 100% branch and line coverage on PS-P1-L-R18-B4 at bound 1. The excerpt also shows weaker cases and gives no aggregate mean, so the claim is strongest at the per-benchmark level.

Enterprise UI test repair is harder. A five-agent Playwright system discovered about 140 effective UI features, but only 187 of 300 reports produced executable test files. Across 636 individual executions, 32.1% passed and 67.9% failed. The study also records assertion weakening and test-case deletion as ways the loop reached superficial convergence.

Several tools focus on what an agent can safely know before it edits code. RL Developer Memory runs as a local Model Context Protocol (MCP) server, logs retrieval decisions, normalizes developer feedback, and blocks learned reranking unless offline gates clear. In its 200-case benchmark, the deterministic path and full shadow setup both reached 80.0% expected-decision accuracy and 100.0% hard-negative suppression, so the added learning layer has telemetry value but no reported accuracy gain.

Developer tools fill adjacent context gaps. wkdomains exposes a human’s live browser state through local APIs for screenshots, DOM, links, console logs, XHR summaries, cookies, and storage. Spine builds verified repository onboarding maps from static source relationships and can write a compact Claude Code context file. Both report workflow examples, not controlled comparisons.

Domain-specific work favors narrow interfaces that agents can generate and processors can check. HepScript gives physicists and agents a Ruby-embedded domain-specific language for BESIII high-energy physics analysis. Human-written HepScript for 45 papers generated 63 BOSS packages that compiled when the LLM-assisted component was disabled. For LLM-generated HepScript over 72 packages, DeepSeek-R1 reached 94.6% success after three retries, and GLM-4.7 reached 95.9%.

A position paper on marginal token allocation gives a cost model for the same operational problem. It argues that routing, planning, verification, serving, and training should price extra tokens by expected task value, latency, compute, and risk. The paper has no empirical gains, but it gives a useful vocabulary for deciding when an agent should spend on verification or ask for clarification.