---
kind: trend
trend_doc_id: 575
granularity: day
period_start: '2026-04-20T00:00:00'
period_end: '2026-04-21T00:00:00'
topics:
- coding-agents
- execution-verification
- interactive-code-generation
- test-structure
- developer-tools
run_id: materialize-outputs
aliases:
- recoleta-trend-575
tags:
- recoleta/trend
- topic/coding-agents
- topic/execution-verification
- topic/interactive-code-generation
- topic/test-structure
- topic/developer-tools
language_code: en
pass_output_id: 98
pass_kind: trend_synthesis
---

# Coding papers are adding real execution checks and better handles on long-horizon work

## Overview
April 20’s coding research is strongest where papers add harder contact with real execution. SolidCoder, OpenGame, and the OpenROAD verifier all reduce trust in surface-correct outputs by checking behavior in sandboxes, browsers, or domain graphs. A second thread looks smaller but useful: test placement and editing history both affect whether developers and models can keep work verifiable.

## Findings

### Execution checks are moving earlier and getting more concrete
Execution-backed coding is the clearest signal in this period. SolidCoder replaces model self-checks with sandbox runs and property-based assertions, then keeps every failing test it finds during repair. The gains are concrete: on GPT-4o, pass@1 reaches 77.0% on CodeContests, up from 72.7% for CodeSIM, and 26.7% on APPS versus 23.3%. The strongest ablation drop comes from edge-case planning before code generation, which cuts CodeContests performance from 77.0% to 53.3% when removed. The cost is higher inference work, so this line of work looks strong where correctness matters more than token budget.

#### Sources
- [SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution](../Inbox/2026-04-20--solidcoder-bridging-the-mental-reality-gap-in-llm-code-generation-through-concrete-execution.md): Summary with method and benchmark deltas for execution-backed coding

### Evaluation is expanding to interactive and domain-specific execution
Some papers now treat code generation as a runtime systems problem, not just a prompt-following task. OpenGame targets full playable 2D web games, where failure comes from scene wiring, assets, and cross-file state. Its benchmark uses headless browser execution and scores Build Health, Visual Usability, and Intent Alignment; with Claude Sonnet 4.6 it reaches 72.4, 67.2, and 65.1, beating Cursor with the same model by about six points on each metric. In EDA, the OpenROAD verifier paper adds a structural dependency graph before execution. On single-step tasks it reaches 82.5% pass rate, above 76.0% for tool-in-loop debugging, while using 1.00 tool call per task instead of 1.77. On multi-step tasks, reported pass rate rises from 30.0% to 70.0%, then 84.0% with trajectory-level reflection.

#### Sources
- [OpenGame: Open Agentic Coding for Games](../Inbox/2026-04-20--opengame-open-agentic-coding-for-games.md): Summary with OpenGame method and benchmark results
- [Structural Verification for Reliable EDA Code Generation without Tool-in-the-Loop Debugging](../Inbox/2026-04-20--structural-verification-for-reliable-eda-code-generation-without-tool-in-the-loop-debugging.md): Summary with structural verification and OpenROAD efficiency gains

### Prompt-adjacent code structure affects what models keep and pass
Test format itself is showing up as a real model control surface. The doctest study compares inline Python tests with separated Rust test blocks across 830+ generated files and 12 models. Inline tests are much more likely to survive generation: with prompt-provided doctests, preservation is 100% for all models except Claude 3.5 Haiku, and correctness stays in the 92–99% range. Rust results are less stable. Some Claude models preserve and pass every test, while several Opus versions drop to 0% preservation by stripping all test blocks. The paper also reports that temperature 0 still leaves determinism gaps, with 0% for Mistral Medium and 30–64% for Claude Opus 4.6 in cited settings. That makes prompt and file structure part of the reliability story, not just model choice.

#### Sources
- [Co-Located Tests, Better AI Code: How Test Syntax Structure Affects Foundation Model Code Generation](../Inbox/2026-04-20--co-located-tests-better-ai-code-how-test-syntax-structure-affects-foundation-model-code-generation.md): Summary with preservation, correctness, and determinism findings

### Workflow control and provenance remain active, with lighter empirical support
The interface layer is also getting attention, though the evidence here is more qualitative. EvoGraph records AI prompts and code edits as a branching graph inside VS Code, so developers can revisit, compare, and merge alternative paths. In a study with 20 participants, the paper reports better support for exploration, prompt management, and tracking AI-generated changes, plus lower reported cognitive load than a baseline interface. A separate theory paper argues that multi-agent coding should be studied at repository scale, where interaction effects can degrade the codebase even when local tasks look correct. That claim is not backed by new experiments in this period, but it fits the day’s broader focus on traceability, coordination, and system-level checks around agent work.

#### Sources
- [Choose Your Own Adventure: Non-Linear AI-Assisted Programming with EvoGraph](../Inbox/2026-04-20--choose-your-own-adventure-non-linear-ai-assisted-programming-with-evograph.md): Summary with user study scope and reported workflow effects
- [More Is Different: Toward a Theory of Emergence in AI-Native Software Ecosystems](../Inbox/2026-04-20--more-is-different-toward-a-theory-of-emergence-in-ai-native-software-ecosystems.md): Summary with theory claim and cited system-level failure evidence