---
kind: trend
trend_doc_id: 1870
granularity: day
period_start: '2026-07-12T00:00:00'
period_end: '2026-07-13T00:00:00'
topics:
- coding agents
- model routing
- security review
- agent memory
- local AI
run_id: materialize-outputs
aliases:
- recoleta-trend-1870
tags:
- recoleta/trend
- topic/coding-agents
- topic/model-routing
- topic/security-review
- topic/agent-memory
- topic/local-ai
language_code: en
pass_output_id: 320
pass_kind: trend_synthesis
---

# Agent products are becoming operational systems, but their evidence remains thin

## Overview
Agent products are being designed as controlled operational systems. OneDev anchors coding work in issues, pull requests, and continuous integration (CI); Avriz gates learned model routing through shadow tests and traffic caps; Mango keeps memory and permissions on the user’s device. Measured evidence is uneven. The strongest benchmark covers only ten synthetic pull requests, while most product claims lack comparative evaluation.

## Findings

### Supervised coding workflows
OneDev places an agent inside the existing software trail: the issue holds requirements, an isolated workspace contains execution, and the linked pull request records review and CI results. This design makes agent work inspectable and lets failed builds trigger revisions. It has no measured evaluation.

Fast code generation also increases the human review burden. A three-month first-person account reports prototypes completed in hours, alongside sustained architecture decisions, mental fatigue, and difficulty reconstructing why generated code took a particular form. Together, these reports make review capacity and decision pacing practical constraints on coding-agent throughput.

#### Sources
- [OneDev AI: Coding Agents as Teammates in Issues, Pull Requests, and CI](../Inbox/2026-07-12--onedev-ai-coding-agents-as-teammates-in-issues-pull-requests-and-ci.md): Summarizes OneDev's issue-to-PR workflow, controlled workspaces, CI feedback, and lack of quantitative evaluation.
- [The cost of AI-assisted development: cognitive fatigue](../Inbox/2026-07-12--the-cost-of-ai-assisted-development-cognitive-fatigue.md): Reports faster prototyping, architecture-level fatigue, review blind spots, and the absence of controlled evidence.

### Cost-aware model selection
Model choice is becoming a workload-specific operating decision. Dam Secure tested ten models on ten synthetic pull requests with planted access-control flaws, repeating each model five times. GPT-5.6 Sol reported 100% recall, 0.91 F1, and $0.70 per pull request. Grok 4.5 reached 0.77 F1 at $0.20, while Gemini 3.1 Flash Lite reached 0.75 F1 at about $0.04. The small, private corpus limits broader conclusions, especially for full-code scans.

Avriz addresses the same cost problem during live coding-agent use. Its contextual bandit selects among five model tiers using 11 content-free features and billing-derived rewards. Learned routing remains in shadow mode until coverage gates pass, then reaches only a capped traffic share under a model-tier ceiling. The report gives implementation detail, though it does not report aggregate savings or quality gains.

#### Sources
- [Grok 4.5 and GPT5.6 beat Anthropic for finding security vulnerabilities in PRs](../Inbox/2026-07-12--grok-4-5-and-gpt5-6-beat-anthropic-for-finding-security-vulnerabilities-in-prs.md): Provides the benchmark design, workload scope, recall, F1, and cost results.
- [We taught our platform to learn its own pricing decisions](../Inbox/2026-07-12--we-taught-our-platform-to-learn-its-own-pricing-decisions.md): Details Avriz's features, five-tier learner, delayed rewards, deployment gates, and missing aggregate outcomes.

### Agent memory and user control
Persistent context is being treated as data with explicit ownership rules. Mango proposes local execution, plain-text memory, replaceable models, and deterministic client-side permissions for actions across logged-in services. xysq targets teams with isolated vaults that collect context from Slack, Drive, Notion, and other tools; access is consent-gated, encrypted, exportable, and deletable.

These designs specify where memory lives and who can authorize its use. Their evidence is architectural and promotional. Mango has no controlled security audit, and xysq provides no deployment metrics, retrieval evaluation, or independent verification of its privacy claims.

#### Sources
- [A Technology for Free Will](../Inbox/2026-07-12--a-technology-for-free-will.md): Summarizes Mango's local execution, portable memory, model choice, client-side safety, examples, and evaluation gaps.
- [Show HN: Collaborative context-sharing memory platform for agents and teams](../Inbox/2026-07-12--show-hn-collaborative-context-sharing-memory-platform-for-agents-and-teams.md): Describes xysq's isolated team vaults, consent controls, encryption, export and deletion claims, and lack of metrics.