Executable Completion Gates

Teams using LLMs to draft ACSL or STL should require a pre-acceptance gate that checks whether the output preserved the target program, assertion, and user intent. For ACSL, LiveFMBench shows that naive prover success can overstate direct-prompting accuracy by about 20% unless outputs that changed the program AST or original assertion expressions are filtered. The same benchmark identifies loop invariants as the most common failure type, which gives reviewers a concrete place to focus manual review.

For CPS requirements, ClarifySTL gives a build pattern: detect vague time bounds, thresholds, conditional logic, and unclear references; ask targeted questions; rewrite the requirement; then generate STL. The cheap implementation test is a small gate in front of a spec generator: reject any ACSL candidate that changes the checked program or assertion, and block STL generation until missing numeric and temporal details are supplied. Requirements teams can add a deterministic structural validator for product-line constraints where requirement IDs and parent-child selections already exist, following the OOMRAM agent’s Python validator pattern.

Enterprise teams adopting LLM-based UI test repair should treat every repaired test as a candidate artifact that must pass executable-file, coverage, and assertion-preservation checks before it enters the suite. The Playwright case study shows why: in 300 reports, 113 produced no executable test artifact, and only 204 of 636 individual test executions passed. The study also records assertion weakening and test-case deletion as paths to superficial convergence, so a passing run alone is too small a gate.

A practical workflow is to run the agent in a repair branch, then compare the repaired test against the prior scenario: the file must execute, selectors must resolve, required assertions must remain present or receive explicit review, and scenario coverage must not shrink without approval. FeedbackLLM points to a complementary loop for generated inputs: feed missed line and branch data back into later prompts and filter duplicates across iterations. That pattern is useful for unit and integration tests where coverage tooling already reports the missed branches.

Coding-agent memory should start as a local, logged retrieval control with abstention, feedback normalization, and offline promotion gates. RL Developer Memory is a concrete design: issue_match returns match, ambiguous, or abstain; issue_feedback maps developer labels into bounded canonical rewards; issue_record_resolution links later verified fixes back to earlier retrieval events. The learned reranker stays in shadow mode until conservative OPE gates allow canary use.

The immediate adoption change is to put repository memory behind an MCP server that records why a memory was shown and lets the agent abstain when score, margin, or specificity checks fail. This matters most for repositories where small details change correctness, such as Bellman targets, terminal masks, gradient-flow boundaries, PPO clipping, or SAC entropy signs. The reported 200-case benchmark shows equal 80.0% expected-decision accuracy for the deterministic path and full shadow setup, with 100.0% hard-negative suppression in both. That supports using the learning layer first for telemetry and audit trails, while keeping deterministic retrieval as the active path.