---
source: hn
url: https://www.normaltech.ai/p/did-googles-ai-agents-really-build
published_at: '2026-05-22T22:50:27'
authors:
- randomwalker
topics:
- coding-agents
- multi-agent-software-engineering
- open-world-evaluation
- software-engineering-ai
- agent-benchmarks
relevance_score: 0.86
run_id: materialize-outputs
language_code: en
---

# Did Google's AI agents build an operating system for $916?

## Summary
The article argues that Google’s operating-system-by-agents claim is not strong evidence without the prompt, code, logs, retry count, and copying checks. It treats the demo as a useful case for independent evaluation of long-running coding agents.

## Problem
- Google claimed that agents built an operating system for about $900 from a single prompt, but the public writeup leaves out details needed to judge the claim.
- The case matters because agent vendors now use long-running software tasks as proof of capability, while standard benchmarks cannot test this kind of work well.
- Missing methodology makes it hard to tell whether the result came from model capability, heavy prompt engineering, task-specific scaffolding, retries, or copied public code.

## Approach
- The authors audit Google’s public blog post rather than rerunning the experiment.
- They check whether the writeup defines human intervention, reports retries and dry runs, releases artifacts, and tests for copied or memorized code.
- They separate the model from the scaffold around it: specialized roles, subagent delegation, tool access, stuck-agent restarts, and an anti-cheating component.
- They argue that open-world evaluations need stronger norms: public artifacts, clearer intervention logs, cost reporting, and independent review.

## Results
- Google reported a final cost of $916.92 in API fees and a total budget of 2.6B tokens.
- Google described the task as starting from a single prompt, but the prompt later became many thousands of lines; the article says the number of prompt-writing attempts is not reported.
- Google said a few dozen subagents worked together, with specialized roles and delegation through the Antigravity 2.0 setup.
- Google said the final run needed no additional human guidance or corrections, but the article says the writeup gives 0 clear counts for manual restarts, approvals, escalations, dry runs, or retries.
- Google released 0 of the key artifacts needed for independent review: the long prompt, the generated source code, and the agent logs.
- The article reports no benchmark baseline, no similarity analysis, and no log analysis to test whether the agents copied or memorized existing toy operating-system code.

## Link
- [https://www.normaltech.ai/p/did-googles-ai-agents-really-build](https://www.normaltech.ai/p/did-googles-ai-agents-really-build)