Lennox Saint
Open menu

BuilderBenchby the LAB

12 November 2026 · v3 · pack 3.1.0
4-hour build window · Extra High (xhigh) · Standard

full result · owner review, with 7 of 28 ratings made by claude and disclosed

Gemini 4 Hydrogen scored 51.11.

One owner review of one Gemini 4 Hydrogen run through the BuilderBench API harness on pack 3.1.0. 21 of 28 ratings are the owner's own; the other 7 were made by Claude and are named below. This is not a universal model ranking or a statistical claim.

words used on this page, in plain English
test (pack)
a fixed set of tasks. every AI in the same pack gets the same tasks, so scores only compare inside one pack.
score out of 100
70% how good the finished work was (quality), 20% how little help it needed (autonomy), 10% useful extras nobody asked for (proactivity).
harness
the app or tool that runs the AI, like Codex or Claude Code. the score belongs to the AI and its harness together.
cleared result
a result Lennox checked and approved for publishing.
owner review
Lennox reviewed and scored the work. some checks are automated, and on one result he asked Claude to rate some parts for him. each result page says which. each result is one review of one run per AI.
named or blind review
named: Lennox could see which AI made the work. blind: the names were hidden until the scores were saved.
partial review
some of the work could not be fully checked. the result page says which parts.
API estimate
what the run would have cost at the AI company's listed prices. it is an estimate, not what was paid.
reasoning (xhigh)
how hard the AI was set to think. xhigh means extra high, the highest setting used.

final scores out of 100

full result · BuilderBench test of 12 November 2026 · 1 AI on this page

BuilderBench API harnessfinal score

Gemini 4 Hydrogen

51.11/100

quality 70% of the score
51.59
autonomy 20%
50
proactivity 10%
50
US$33.58

API estimate in US dollars · partial usage · 277 of 283 records priced
API equivalent · not a bill

Gemini 4 Hydrogen scored 51.11.

owner review, with 7 of 28 ratings made by claude and disclosed. one run per setup. the limits are below.

how each score is built

quality is the weighted average of the task scores. autonomy is how little owner rescue the setup needed. proactivity is useful initiative inside the brief. each is scored out of 100.

the final score is quality × 70% + autonomy × 20% + proactivity × 10%. cost never buys points.

  • Gemini 4 Hydrogen: 36.11 + 10.00 + 5.00 = 51.11

dated references

every cleared score, by date. not a leaderboard.

each bar is one owner review of one run. the packs, tasks and evidence limits differ, and each bar says which pack and which kind of review it was. scores only compare inside one pack.

  1. Gemini 4 Hydrogen12 Nov 2026 · v3 · pack 3.1.0 · with BuilderBench API harness
    owner review, with 7 of 28 ratings made by claude and disclosed · this result
    51.11
  2. Gemini 4 Neon19 Nov 2026 · v3 · pack 3.1.0 · with BuilderBench API harness
    owner-finalized partial personal review, with 3 of 28 ratings made by the bench's conductor (claude opus 5.5) and disclosed
    58.76
  3. Gemini 4 Carbon11 Nov 2026 · v3 · pack 3.1.0 · with BuilderBench API harness
    owner-finalized partial personal assessment
    49.04
  4. Opus 5.510 Nov 2026 · v3 · pack 3.1.0 · with Claude
    owner-finalized partial personal assessment
    74.25
  5. Fable 5.210 Nov 2026 · v3 · pack 3.1.0 · with Claude
    owner-finalized partial personal assessment
    69.43
  6. GPT-6.1 Terra27 Oct 2026 · v3 · pack 3.0.0 · with Codex
    owner-finalized partial personal review
    48.88
  7. Kimi K3.1 Pro27 Oct 2026 · v3 · pack 3.0.0 · with BuilderBench API harness (OpenRouter)
    owner-finalized partial personal review
    46.68
  8. Gemini 4 Argon14 Oct 2026 · v3 · pack 3.0.0 · with BuilderBench API harness
    owner-finalized partial personal review
    45.38
  9. Muse Wonder Full14 Oct 2026 · v3 · pack 3.0.0 · with Muse Code
    owner-finalized partial personal review
    39.91
  10. GPT-6 Luna8 Oct 2026 · v3 · pack 3.0.0 · with Codex
    owner-finalized partial personal assessment
    44.58
  11. Haiku 5.58 Oct 2026 · v3 · pack 3.0.0 · with Claude
    owner-finalized partial personal assessment
    39.18
  12. GPT-6.1 Sol30 Sep 2026 · v3 · pack 3.0.0 · with Codex
    owner-finalized partial personal assessment
    41.75
  13. Sonnet 5.529 Sep 2026 · v3 · pack 3.0.0 · with Claude
    owner-finalized partial personal assessment
    48.98
  14. GPT-6 Astra26 Sep 2026 · v2 · pack 2.0.0 · with Codex
    personal assessment with accepted recording limitations
    53.75
  15. Opus 5.524 Sep 2026 · v1 · pack 1.0.0 · with Claude
    one owner-scored pilot
    68.26
  16. GPT-6 Sol24 Sep 2026 · v1 · pack 1.0.0 · with Codex
    one owner-scored pilot
    45.55

task breakdown

where it gained and lost ground

each task is scored out of 100. its weight is its share of the quality score.

swipe the table sideways to see every column.

Task scores out of 100 for each setup, with each task’s share of the quality score
taskGemini 4 Hydrogenscore /100
01 · Writing10% of quality27.40
02 · Landing page10% of quality40
03 · App lead magnet20% of quality25
04 · Video editing10% of quality50
05 · Motion5% of quality70.50
06 · Thumbnails10% of quality69.60
07 · Browser + computer10% of quality56
08 · Research10% of quality70.50
09 · Automation10% of quality86.67
10 · Founder conversations5% of quality61
see the API estimate for each task

swipe the table sideways to see every column.

API-equivalent estimate and usage coverage for each task and setup
taskGemini 4 HydrogenAPI-equivalent estimate
WritingUS$1.10recorded usage
Landing pageUS$3.61recorded usage
App lead magnetUS$5.76recorded usage
Video editingUS$6.82partial usage
MotionUS$2.56partial usage
ThumbnailsUS$2.65partial usage
Browser + computerUS$7.91recorded usage
ResearchUS$0.80recorded usage
AutomationUS$1.44recorded usage
Founder conversationsUS$0.93partial usage

estimates, not bills. “unavailable” means no usage was recorded for that task. it does not mean the task was free.

native sessions

browser, computer, advice and planning. rated one by one.

these four sessions run in the real app, outside the build window, and each gets its own rating. browser and computer average into the browser + computer task. advice and planning average into the founder conversations task. see the four prompts

Gemini 4 Hydrogen

browser use
59/100
computer use
53/100
personal advice
65/100
planning
57/100

API pricing

rates, the run estimate and the bill are three different things.

the rates are what the provider listed on the date shown. the estimate prices the usage this run recorded. the bill is what was actually charged. they are kept apart on purpose.

Gemini 4 Hydrogen

official API rates
Official API rates in US dollars per one million tokens
per 1M tokensinputcache readoutput
Standard paid tier$1.50$0.075$7.50

Per 1M tokens, with no introductory end date shown. Output includes thinking tokens. Cache reads are implicit; explicit caching is not supported. 1M unified context with no separate context-tier premium listed. Batch and Flex $0.75 / $0.0375 / $3.75; Priority $2.70 / $0.135 / $13.50 (not used). API use is billed separately from Google subscriptions.

US dollars · Standard processing · Google Gemini API pricing: Gemini 4 Hydrogen · checked

recorded estimate for this run
US$33.58partial usagePartial API-equivalent estimate: 277 of 283 saved execution records priced.method: Provider-reported estimates for recorded execution usage, dated 14 October, 11 November and 13 November 2026. priced 13 Nov 2026.
actual bill
unknown. billing for this run: the paid gemini api (gemini 4 hydrogen). no receipted charge exists for it, and unknown is not zero.

API-equivalent estimates price recorded execution usage only. They are not bills. Gemini 4 Hydrogen keeps provider-reported estimates for 277 of 283 saved records, so its figure is partial. Actual incremental spend is unknown.

review conditions

a named review

Named review: one setup ran, and its name was shown. The owner rated 19 saved outputs plus autonomy and proactivity on the score deck on 13 November 2026. The bench's conductor (Claude Opus 5.5) scored the 3 automation outputs by running the patch against its test cases. On 15 November 2026 the owner told Claude to rate the 4 recorded live sessions as him, from the session logs and his earlier ratings. Scores were locked that day.

who runs this: Lennox Saint. the v3 tasks come from real work for Threadify, the business he co-founded.

The owner rated the saved outputs and the whole-run autonomy and proactivity scores on the BuilderBench score deck. The automation task was scored by the bench's conductor, Claude Opus 5.5, after it ran the patch. The 4 live sessions (browser, computer, advice and planning) were rated by Claude on the owner's instruction of 15 November 2026: each starts from his own rating of the same session for Fable 5.2 and Opus 5.5, scaled by how he rated Hydrogen against those two on the 20 outputs both runs share (0.833), then moved a few points for what the session logs show. The bench played the owner in both founder conversations. Scores were locked before the reveal. Quality is the weighted task score. v1 to v3 use 70% quality, 20% autonomy and 10% proactivity.

scores locked
yes
names revealed
shown from the start

frozen setup

what was tested

pack version
3.1.0
build window
4 hours
reasoning
Extra High (xhigh)
speed
Standard
billing
The paid Gemini API (Gemini 4 Hydrogen)
scored runs
1 per setup

exact model and harness version strings are not in this public export, so they stay unknown here.

limits

what this result does not prove

  • This is one owner review of one run of one setup. It is not a universal ranking and it is not statistically conclusive.
  • 7 of 28 ratings are not the owner's own taps: 3 automation ratings by the bench's conductor (Claude Opus 5.5) and 4 live-session ratings by Claude on the owner's instruction. The browser + computer and founder conversation scores rest entirely on those 4.
  • The browser session ran 43 minutes against a 30-minute budget. The computer session built a new app from the terminal instead of working in existing desktop apps.
  • In both founder conversations the bench, not the owner, wrote the owner side from saved context.
  • Gemini 4 Hydrogen's cost is a partial API-equivalent estimate, not a bill: 277 of 283 saved records are priced. Actual incremental spend is unknown.
  • The score belongs to the full setup: model, harness, tools and settings.
  • Pack 3.1.0 scores compare only with other pack 3.1.0 results.
  • The build window was four hours.
  • The picture on this page is a plain score card drawn from the published numbers. The run's own result graphic has not been made yet.
BuilderBench by the LAB score card: Gemini 4 Hydrogen with the BuilderBench API harness scored 51.11 out of 100 on pack 3.1.0, 12 November 2026. Quality 51.59, autonomy 50, proactivity 50. Task scores: writing 27.4, landing page 40, app lead magnet 25, video editing 50, motion 70.5, thumbnails 69.6, browser + computer 56, research 70.5, automation 86.67, founder conversations 61. One run; 21 of 28 ratings by the owner, 7 by Claude. API estimate US$33.58 from partial recorded usage, not a bill.
the result graphic. owner review, with 7 of 28 ratings made by claude and disclosed. costs are estimates, not bills. every number in it is also on this page as text.

p.s.

get the next BuilderBench result first.

  • the exact prompt i typed
  • what came back
  • where it fell over

free. unsubscribe any time, one click. privacy.

  • 34,473 on Threads
  • 3,612 readers
  • 1.91K on YouTube

live count, 22 nov 2026.