bench.yair-tech.com
← yair-tech.com

Run 001 · agent web bench

One prompt. Five coding agents. Stopwatch and receipts.

Every agent got the identical brief in an empty directory — headless, one shot, no retries unless noted. The prompt is published in full below, and so is the bill. No scoring rubric, no vibes column: wall clock, tokens, cost, and the artifact itself. Judge the results with your own eyes.

9 Jul 2026 · sequential on one idle Linux workstation · 25-minute cap per agent

Newer: Run 002 — Claude Code conducting Grok 4.5 subagents, on tape, with the bill →

Results

Sorted by wall clock. Tokens as reported by each harness; agents on flat-rate plans have zero marginal cost — the metered column shows what the same run bills on Anthropic's API for Fable, the only metered agent in the lineup.

AgentWall clockTokens in / outOutputMarginal costLive result
Grok 4.5
grok CLI · SuperGrok plan
 2:33
 not reported1 71.3 KB $0 · flat-rate open →
Kimi K2.7
Claude Code harness · Kimi Code plan
 3:07
 178k / 12k (+231k cached) 32.9 KB $0 · flat-rate open →
Claude Fable 5
Claude Code · Anthropic, metered
 4:48
 20k / 23k (+187k cache read, +68k write) 38.3 KB $2.89 measured open →
Antigravity
agy CLI · Gemini 3.1 Pro (High)
 6:42
 not reported1 129.5 KB $0 · flat-rate open →
GLM-5.2
Claude Code harness · z.ai Coding Plan
 9:50
 178k / 33k (+288k cached) 67.7 KB $0 · flat-rate open →

1 The grok and agy harnesses don't report token usage in headless mode. Disclosure: Grok 4.5 and Antigravity were initially invoked with wrong harness flags on my side (no tool auto-approval / wrong flag shape) and produced nothing; the published numbers are each agent's first correctly-configured one-shot run. GLM, Kimi and Fable were one-shot on the first attempt.

Head to head

Pick any two builds and drag the divider — same brief, same viewport, pixel against pixel.

vs
Left build Right build Grok 4.5 GLM-5.2

Compare pattern adapted from motion-primitives' Image Comparison, found via 21st.dev.

Full disclosure

The prompt, verbatim

Secret prompts can't be checked. This one is public — same text, every agent, character for character.

You are in an empty directory. Build the complete official website of "Meridian Subsea", a fictional company that operates a fleet of ships repairing undersea internet cables.

Requirements:
- Exactly one self-contained file: ./index.html (inline CSS and JS, no external resources, no fonts/CDNs, no build step, no frameworks).
- It must look and feel like the real website of a serious maritime-infrastructure company, not a template: hero, the fleet (invent 3 ships with names and specs), a live "cable health" status board with invented but plausible data, an incident-response timeline, careers, and contact.
- Design quality matters and will be judged: typography, color, spacing, hierarchy. Dark or light is your call.
- Plain scrolling page, working internal nav links, no console errors.

When you are done, ./index.html is the only deliverable. Do not ask questions; make all decisions yourself.

Method

  1. Fresh empty working directory per agent; no repo context, no skills, no MCP servers.
  2. Each agent runs its own vendor harness headless with permissions auto-approved, 25-minute cap, sequentially on the same idle machine.
  3. Wall clock is the full harness run, measured outside the process. Tokens and metered cost come from the harness's own JSON result where the harness reports them.
  4. The deliverable is one self-contained index.html. No post-editing: what the agent wrote is what's served and screenshotted (1280×800, Chromium).
  5. Failures are published, not re-rolled quietly. Any retry is footnoted with the reason.