Coding agents need state, measured harnesses, and action gates

Claw-SWE-Bench makes the harness itself part of the score. With the same GLM 5.1 model, OpenClaw rises from 19.1% Pass@1 with a minimal adapter to 73.4% with the full adapter. The paper also reports that harness choice changes Pass@1 by up to 27.4 percentage points under fixed models, and its Lite-80 set tracks the full 350-instance benchmark at about 23% of the run cost.

Several companion items treat agent operation as a maintained runtime. Loop-Harness schedules Claude Code runs in isolated git worktrees, then requires a second Claude session to print VERDICT: PASS before shipping. Agents All the Way Down favors command-line interface (CLI) agents for product integration and cites a matched task where Model Context Protocol (MCP) used about 35× more tokens than CLI. SkillJuror adds a smaller but useful result: reorganizing the same skill content with Progressive Disclosure improved pass rate from 42.0% to 46.1% across 410 matched trials.

PROJECTMEM targets a common failure mode in coding sessions: the agent forgets failed fixes and repeats context reconstruction. It stores issues, attempts, fixes, decisions, and notes in an append-only plain-text log, then builds deterministic summaries for the agent. Its precheck_file(path) gate warns before edits to paths tied to failed attempts, open issues, or high churn. The paper estimates 5,000 to 20,000 tokens per session for rebuilding project context, though it does not report a controlled task-success benchmark.

Exploration Structure in LLM Agents attacks the file-localization step before patching. On an Ansible slice from SWE-bench Pro, it assigns domain agents to repository regions such as CLI, module utilities, plugins, and docs. The result favors subsystem-parallel search for small models. The same study also shows that bounded input matters: one large source file drops from 29,895 tokens to 719 tokens under bounded context, and a large docs file drops from 14,366 to 121 tokens.

Production-agent safety is being specified at the action layer. The five-plane reference architecture checks intent, delegation chains, capability sets, and session state before actions reach enterprise infrastructure. It also proposes stop-anywhere mediation with pause, escalate, narrow, modify, defer, and rollback, plus tamper-evident audit records. The paper’s evidence is mostly structural, with microsecond policy-core timings and no live full-system benchmark.

Two practical variants appear in the same period. agent-gate makes completion fail closed unless all required evidence fields are true, including deterministic checks, independent review, no secrets, human approval for irreversible acts, and an honest receipt. Bootstrapped Monitoring adds a model oversight protocol: a stronger untrusted monitor reviews actions, while a weaker trusted model audits the monitor’s raw chain-of-thought (CoT). The gains depend on access to raw CoT; summarized reasoning sharply weakens detection in the reported tests.

Code generation has its own safety failure path. CodeSpear shows that grammar-constrained decoding (GCD), a technique used to force syntactically valid code, can remove natural-language refusals from the output space. On local models such as Qwen2.5-Coder-7B, the attack reaches an 81.82% average success rate, and across tested models it beats representative jailbreak baselines by more than 30 percentage points on average. CodeShield answers by training harmless code outputs for cases where grammar constraints force code.

Multisage addresses a different code-specific weakness: translation outputs that compile while changing program behavior. It extracts control flow, types, and API signals from the source, generates several semantic views, then filters them with execution validation, code mutations, and cross-view checks before prompting the translator. On HumanEval-X, it reports success-rate gains up to 2.22× over vanilla prompting, with the largest relative gains on smaller models.