---
source: arxiv
url: http://arxiv.org/abs/2604.19742v1
published_at: '2026-04-21T17:59:16'
authors:
- Zhiyuan Peng
- Wei Tao
- Xin Yin
- Chenhao Ying
- Yuan Luo
- Yiwen Guo
topics:
- gui-code-generation
- multi-agent-systems
- code-evaluation
- repository-aware-generation
- interactive-testing
relevance_score: 0.91
run_id: materialize-outputs
language_code: en
---

# PlayCoder: Making LLM-Generated GUI Code Playable

## Summary
PlayCoder targets a gap in code generation for GUI apps, where code can compile and even pass tests but still fail during real user interaction. The paper adds a benchmark, a behavioral metric, and a multi-agent repair loop built around automated GUI playtesting.

## Problem
- Existing code benchmarks mostly judge functions with unit tests, which misses GUI failures tied to event handling, state changes, timing, and multi-step interaction.
- GUI applications can look correct under compile and test checks while still breaking core logic during use, such as a Flappy Bird clone that lets the bird pass through pipes.
- This matters because interactive apps and games need end-to-end behavioral correctness, and current repository-aware code agents do not test that reliably.

## Approach
- The authors build **PlayEval**, a repository-aware benchmark with 43 GUI applications across Python, TypeScript, and JavaScript, covering 6 categories and 188,432 lines of code.
- They introduce **Play@k**, a stricter metric than compile or unit-test success: at least one of k generated candidates must be playable end-to-end without logic errors after passing tests.
- They create **PlayTester**, an agent that drives GUIs through task-oriented playthroughs, uses visual feedback and interaction traces, and checks for behavioral violations that unit tests miss.
- They propose **PlayCoder**, a multi-agent system with a coding agent (PlayDeveloper), a testing agent (PlayTester), and a repair agent (PlayRefiner) that iterates through test-and-fix cycles using repository context plus GUI feedback.

## Results
- On PlayEval, the paper says 10 state-of-the-art LLMs have **near-zero Play@3** in many settings despite high compilation rates, showing a large gap between executable code and correct interactive behavior.
- In the preliminary benchmark results, **Claude-Sonnet-4** drops from **18.6% Exec@3** to **9.9% Play@3**, and **GPT-5** drops from **17.5% Exec@3** to **6.9% Play@3** on Python tasks.
- With **GPT-5-mini**, PlayCoder reaches **26.8% Exec@3** and **9.8% Play@3**, compared with the best baseline **DeepCode** at **17.9% Exec@3** and **6.4% Play@3**.
- With **Claude-Sonnet-4**, PlayCoder reaches **36.8% Exec@3** and **20.3% Play@3**.
- The paper also claims gains of up to **20.2 percentage points on Exec@3** and **11.0 percentage points on Play@3** over baselines.
- The benchmark itself contains **43 applications**, **637 files**, **4,497 functions**, **595 classes**, and **2,104 test cases** across **6 categories** and **3 languages**.

## Link
- [http://arxiv.org/abs/2604.19742v1](http://arxiv.org/abs/2604.19742v1)