Lennox Saint
Open menu

BuilderBench / methodology

judge the work. keep the receipt.

BuilderBench compares complete AI setups on the same practical work. The score belongs to the model, harness, tools and settings used in that dated run.

what changed, and when

current method · v3 · pack 3.0.0

latest cleared result: 19 November 2026

13 setups across 2 packs. Pack 3.0.0: Sonnet 5.5 with Claude (29 September), GPT-6.1 Sol with Codex (30 September), GPT-6 Luna with Codex (8 October), Haiku 5.5 with Claude (8 October), Gemini 4 Argon with BuilderBench API harness (14 October), Muse Wonder Full with Muse Code (14 October), GPT-6.1 Terra with Codex (27 October), Kimi K3.1 Pro with BuilderBench API harness (OpenRouter) (27 October). Pack 3.1.0: Opus 5.5 with Claude (10 November), Fable 5.2 with Claude (10 November), Gemini 4 Carbon with BuilderBench API harness (11 November), Gemini 4 Hydrogen with BuilderBench API harness (12 November), Gemini 4 Neon with BuilderBench API harness (19 November). Scores compare only inside one pack. One owner review of one run each. Not a universal model ranking.

open the latest result

how a run works now

five steps, in this order

  1. 01

    freeze the test

    Each setup gets the same pack, task weights, clock and revision rules. Task versions are frozen when the run is created, and the pack version stays attached to the result.

  2. 02

    run the build window

    The default build window is 4 hours. The four native sessions (browser, computer, advice, planning) happen in the real app and are timed separately. They are never counted as part of the build window.

  3. 03

    review the work

    Named review is the default: the model name is shown. A blind comparison is something the owner asks for when two or more models run together. Each published result says which one it was, and where the blind was imperfect.

  4. 04

    lock, then reveal

    Every required score is saved and locked. In a blind comparison the names are revealed only after the lock. Scores are never edited after that. A mistake gets a dated correction.

  5. 05

    publish the limits

    The result names the setup, the review conditions, each model's API rates, the recorded cost estimate, missing proof and sample size. A partial assessment is labelled partial everywhere the score appears.

the score

new runs use 70/20/10

70%

quality

the weighted average of the ten task scores

20%

autonomy

how little owner rescue was needed

10%

proactivity

useful initiative within the brief

Each part is scored out of 100. The final score is quality × 70% + autonomy × 20% + proactivity × 10%. Every result page shows that sum for each setup. v1, v2 and v3 all use these weights.

a later pack · from v4

90% quality · 5% autonomy · 5% proactivity

These weights are announced for the next major pack only. No v4 run has started. Runs on pack 3.0.0 and all earlier results keep 70/20/10, and old scores are never recalculated.

who scores what

taste is scored by a person. code is checked.

the owner scores taste

Lennox rates the work a founder would judge by eye: writing, pages, video, motion, thumbnails, research and the native sessions, plus autonomy and proactivity.

task 9 is conductor-scored

Technical repair is checked against tests and scored by the conductor, with the checks and reasons recorded. The owner can review or adjust those scores before locking.

blank stays blank

Every other owner rating stays empty until the owner supplies it. An unrated item is not a zero, and nothing is filled in on the owner's behalf. If the owner delegates a specific rating for one run, that result says so.

lead magnets: look and feel

For new runs, the lead-magnet app gets one owner rating for look and feel: layout, type, colour, spacing, hierarchy and polish on desktop and mobile. Its decision notes, tests and run instructions are supporting evidence, checked separately. Runs created before 8 October 2026 keep the four-part rubric they were frozen with.

help on every criterion

Each rating has a plain label and its own explanation of what to open, what to judge and what is outside that rating. The help works on hover, keyboard focus and touch.

old runs keep their rules

A run keeps the task versions, rubric and weights it was created with. Ratings are never migrated or copied forward. Results from different packs or rubrics are dated references, not one ranked list.

v3 task pack

ten task families. 100 points.

4-hour build window · native sessions timed separately

  1. 01

    writing

    10

    useful founder writing from the frozen Threadify source pack

  2. 02

    improved Threadify landing page

    10

    an improvement on the current public page, built as a benchmark artifact

  3. 03

    app lead magnet

    20

    the setup chooses and builds the most useful Threadify-adjacent interactive lead magnet

  4. 04

    video

    10

    a finished edit from the same supplied source media

  5. 05

    motion

    5

    clear motion that helps explain the idea

  6. 06

    thumbnails

    10

    finished title and thumbnail packaging with a visible selection

  7. 07

    browser + computer

    10

    two separately timed native sessions: an open browser-use demonstration, and a computer-use demonstration without a browser

  8. 08

    research

    10

    sourced research that ends in a useful decision

  9. 09

    automation

    10

    a technical repair with checks and a handoff, scored by the conductor with recorded evidence

  10. 10

    conversation

    5

    two real founder conversations in the native app: personal advice and YouTube-topic planning

the landing page must improve the current public Threadify page, but it stays a benchmark preview and never replaces production. the app is separate: the setup chooses the strongest Threadify-adjacent interactive lead magnet and builds the working product.

from pack 3.1.0 · planned, not run yet

the tasks are asked the way Lennox asks

The v3 briefs read like a spec. Lennox doesn’t talk to Codex or Claude like that. From pack 3.1.0, every brief, revision and curveball uses the patterns in 1,334 messages he typed himself across his local Codex and Claude Code sessions.

  • outcome first

    Each task opens with what done looks like to him, not a procedure.

  • exact counts

    Three pairs, five pieces, 4-7 scenes. Never "several options".

  • "make sure" carries the checks

    Quality bars read the way he writes them: make sure it works at 390px, click every button.

  • one blunt limit line

    Draft only, send nothing. Don't invent my face. Stop and tell me if it asks for a password.

  • curveballs as add-ons

    Stress variants arrive as "also", "by the way" or "one more thing", the way his do mid-task.

  • feedback names one fix

    Revisions say what to fix, what to keep and what to check again.

Only the wording changed. The work, files, checks, weights and rules are the same as 3.0.0. Only patterns were kept: no message he wrote is stored in the pack or shown here. 3.0.0 scores do not change, and 3.1 scores can’t be compared with them.

from pack v4 · planned, not run yet

the writing tests use the real templates

From pack v4, the writing tasks use the same templates Lennox uses for his real work every week. A benchmark should test the real job, not a task made up in a vacuum.

  • x article

    Follows the X article skill Lennox uses to write his own X articles.

  • threads post

    Follows his listicle plug template: a claim with proof, a short list, then one reason to click.

  • threads thread

    Follows the long-form thread template Threadify uses to write his long threads.

  • youtube script

    Follows the shape of his Threadify proof loops videos: proof, a promise, a live demo, the honest result, then one next step.

v3 scores do not change. v4 is a new test, so its scores can’t be compared with v3. The templates stay private: this page only says what they are.

native sessions

four sessions in the real app. four separate ratings.

Two sessions show what the setup can do with a browser and with the computer. Two are real founder conversations: personal advice and YouTube-topic planning. The same prompt goes to every setup. Each session is timed on its own and gets its own rating out of 100.

browser use
Choose and complete a task that shows off your browser-use capabilities.
computer use
Choose and complete a task that shows off your computer-use capabilities, without using a browser.
personal advice
Based on everything you know about me, what is the single highest-leverage thing I should do right now, and how can you help me do it?
planning
Help me choose my next YouTube topic. Start in plan mode. Question me until you can narrow the options to one grounded, creative topic I should make next, then show me the strongest angle and why it earns the slot.

open demonstrations

the setup picks the task

The browser and computer prompts are open on purpose. The setup chooses a task that shows its own ability, and the computer session may not use a browser. These two prompts were corrected on 8 October 2026 and replace the earlier fixture tasks.

blind comparisons

rated by name, without breaking the blind

In a native app the owner can see which model it is. So native ratings are collected by model name in a separate form that never shows which blind candidate that model is. The blind scorecard hides those ratings and any total built from them until the reveal.

what stays private

the answers are archived, not published

The exact founder conversations are saved privately with their source, model and date. They hold personal context, so they are not on this site. Scoring forms and controls are private too: public results are read-only.

API pricing

every model gets a price panel

official rates

Each model's official input, cache-read and output rates are read from the provider's own page before a result is shown, with the source link and the date it was checked.

the run estimate

The recorded usage for that run, priced as an API equivalent. It states its method and how much usage it covers. It is shown apart from the rates.

the actual bill

What was really charged. Most runs use existing subscriptions, so this is usually unknown. Unknown is shown as unknown, never as zero.

If a provider has not published a rate, or the rate could not be checked, the panel says exactly that. The model is never left out, and another model’s rate is never used in its place. A published result keeps the prices it was checked with. Any newer price is added as its own dated scenario. Cost never buys points.

review conditions, run by run

not every test was blind. here is each one.

Each line below is that result’s own record of how it was reviewed.

How each published BuilderBench result was reviewed
runsetupsreviewin its own words
19 Nov 2026pack 3.1.0Gemini 4 Neon with BuilderBench API harnessnamed reviewowner-finalized partial personal review, with 3 of 28 ratings made by the bench's conductor (claude opus 5.5) and disclosedNamed review: one setup ran, and its name was shown. The owner rated 25 items on the BuilderBench score deck, including autonomy and proactivity. The bench's conductor (Claude Opus 5.5) rated the automation outputs (repair patch 85, test matrix 80 and receipt 70) after running its own checks on the work. The deck also showed advisory AI judge scores; they are not counted, and only the owner's own ratings were imported. Scores were locked before the reveal.
12 Nov 2026pack 3.1.0Gemini 4 Hydrogen with BuilderBench API harnessnamed reviewowner review, with 7 of 28 ratings made by claude and disclosedNamed review: one setup ran, and its name was shown. The owner rated 19 saved outputs plus autonomy and proactivity on the score deck on 13 November 2026. The bench's conductor (Claude Opus 5.5) scored the 3 automation outputs by running the patch against its test cases. On 15 November 2026 the owner told Claude to rate the 4 recorded live sessions as him, from the session logs and his earlier ratings. Scores were locked that day.
11 Nov 2026pack 3.1.0Gemini 4 Carbon with BuilderBench API harnessnamed reviewowner-finalized partial personal assessmentNamed review: one setup ran, and its name was shown. The owner locked the scores in the review deck on 12 November 2026 as a partial personal assessment. Unfinished tasks were listed on the lock screen and stay disclosed.
10 Nov 2026pack 3.1.0Opus 5.5 with Claude, Fable 5.2 with Clauderequested blind comparisonowner-finalized partial personal assessmentRated blind as Candidate A and Candidate B on the BuilderBench score deck; names were revealed after the scores locked. Model names were masked in transcripts and videos, but one planning transcript mentions a model's release date and list price.
27 Oct 2026pack 3.0.0GPT-6.1 Terra with Codex, Kimi K3.1 Pro with BuilderBench API harness (OpenRouter)requested blind comparisonowner-finalized partial personal reviewRated as Candidate A and Candidate B on the BuilderBench score deck; names were revealed after score lock.
14 Oct 2026pack 3.0.0Gemini 4 Argon with BuilderBench API harness, Muse Wonder Full with Muse Coderequested blind comparisonowner-finalized partial personal reviewRequested as a blind comparison. The main build jobs were rated as Candidate A and Candidate B, and the names were revealed after score lock. It was not fully blind: operator logs leaked identity clues, autonomy and proactivity were rated by name, and the native sessions were rated by model name in a separate form.
8 Oct 2026pack 3.0.0GPT-6 Luna with Codex, Haiku 5.5 with Clauderequested blind comparisonowner-finalized partial personal assessmentRequested as a blind comparison. The build work was rated as Candidate A and Candidate B, and the names were revealed after score lock. It was not perfectly blind: earlier operator diagnostic output included cost clues. The four native sessions were rated by model name in a separate form that never shows which candidate is which.
30 Sep 2026pack 3.0.0GPT-6.1 Sol with Codexsee its noteowner-finalized partial personal assessmentThe owner rated the saved outputs of each of the ten v3 tasks, plus autonomy and proactivity, in a named single-model review. The saved scores were then finalized as a partial personal assessment and locked. Quality is the weighted task score. v1 to v3 use 70% quality, 20% autonomy and 10% proactivity.
29 Sep 2026pack 3.0.0Sonnet 5.5 with Claudesee its noteowner-finalized partial personal assessmentThe owner rated the saved outputs of each of the ten v3 tasks, plus autonomy and proactivity, in a named single-model review. The saved scores were then finalized as a partial personal assessment and locked. Quality is the weighted task score. v1 to v3 use 70% quality, 20% autonomy and 10% proactivity.
26 Sep 2026pack 2.0.0GPT-6 Astra with Codexsee its notepersonal assessment with accepted recording limitationsThe owner rated the saved outputs of each of the nine v2 tasks, plus autonomy and proactivity, then locked the scores. An accepted evidence limitation covers the browser and computer task. Quality is the weighted task score. v1 to v3 use 70% quality, 20% autonomy and 10% proactivity.
24 Sep 2026pack 1.0.0Opus 5.5 with Claude, GPT-6 Sol with Codexsee its noteone owner-scored pilotThe owner scored quality, autonomy and proactivity after a blind review. Quality is the weighted score across eight task families. The score lock happened before identities were revealed.

method changelog

every method change, dated

  1. · v3.1

    v3.1 plan: the tasks are asked the way Lennox asks

    From pack 3.1.0, every task brief, revision and curveball is worded the way Lennox actually prompts Codex and Claude: the outcome first, exact counts, "make sure" for the checks, one blunt line for the limits, and curveballs that arrive as "also" or "by the way". The style comes from 1,334 messages he typed himself across his local Codex and Claude Code sessions. Only patterns were kept, never the messages. The work, files, checks, weights and rules are the same as 3.0.0. A model that only does well on spec-shaped prompts should not win a benchmark about his real work. 3.0.0 scores do not change, and 3.1 scores can't be compared with them. No 3.1 run has started.

  2. · v4

    v4 plan: the writing tests use the real templates

    From pack v4, the X article, Threads post and Threads thread follow the same templates Lennox uses for real: his X article skill, his listicle plug template and Threadify's long-form thread template. The YouTube script follows the shape of his Threadify proof loops videos. A benchmark should test the real job, not a task made up in a vacuum. The templates stay private. v3 scores do not change, and v4 scores can't be compared with v3. No v4 run has started.

  3. · v3

    how runs work now: named by default, separate native ratings, API prices on every result

    new runs keep ten task families, 70% quality, 20% autonomy and 10% proactivity, a four-hour build window and separately timed native sessions. named review is the default. a blind comparison is requested when two or more models run together. task 9 is conductor-scored with evidence. other owner ratings stay blank until supplied. future lead magnets get one owner rating for look and feel. the browser and computer prompts are now open demonstrations, and the computer one excludes browsers. each rating has its own help text. every result shows each model's official API rates, apart from the run estimate and the actual bill. older runs keep their frozen task versions and rubrics.

  4. · v4

    future versions: 90% quality, 5% autonomy, 5% proactivity

    quality will carry 90% of the overall score in future benchmark versions, with autonomy and proactivity at 5% each. this starts with the next major pack, v4. the current v3 run and all earlier results keep their original 70/20/10 weights. no v4 scored run has started. scores from different packs remain separate historical references.

  5. · v3

    clearer review, saved answers and visible cost coverage

    single-model reviews show the model name. blind review is reserved for comparisons of two or more models. the conductor reviews task 9 and records its evidence and scores. founder answers and generated documents are archived for later review. future review screens render Markdown tables and reject predominantly black recordings. the private cost view shows API-equivalent estimates with the number of saved records priced. estimates are not bills, and missing usage stays missing. the current v3 review is still provisional; there is no cleared public v3 result.

  6. · v3

    v3 method: ten task families on real Threadify work

    Threadify work, ten task families, a working lead-magnet app, and repeatable owner-supplied computer tasks. Calibration is pending, so there is no v3 score yet.

dated cohorts

new work or weights means a new version

versiondatedpublic statewhat changed
v3pack 3.1.019 November 2026latest cleared result: 19 November 202613 setups across 2 packs. Pack 3.0.0: Sonnet 5.5 with Claude (29 September), GPT-6.1 Sol with Codex (30 September), GPT-6 Luna with Codex (8 October), Haiku 5.5 with Claude (8 October), Gemini 4 Argon with BuilderBench API harness (14 October), Muse Wonder Full with Muse Code (14 October), GPT-6.1 Terra with Codex (27 October), Kimi K3.1 Pro with BuilderBench API harness (OpenRouter) (27 October). Pack 3.1.0: Opus 5.5 with Claude (10 November), Fable 5.2 with Claude (10 November), Gemini 4 Carbon with BuilderBench API harness (11 November), Gemini 4 Hydrogen with BuilderBench API harness (12 November), Gemini 4 Neon with BuilderBench API harness (19 November). Scores compare only inside one pack. One owner review of one run each. Not a universal model ranking.
v2pack 2.0.026 September 2026latest cleared result: 26 September 2026One owner-scored personal assessment of one GPT-6 Astra run with Codex on the v2 pack. This is not a universal model ranking or a statistical claim.
v1pack 1.0.024 September 2026latest cleared result: 24 September 20262 setups on pack 1.0.0: Opus 5.5 with Claude (24 September), GPT-6 Sol with Codex (24 September). Each score is one owner review of one run, with its own limits. Not a universal model ranking.

how to read a result

a scorecard is a dated experiment

One owner-scored run can show what happened in that test. It cannot prove a universal model order. Compare task scores, review conditions, setup details, cost coverage and limits before you copy the bold headline number. Results from different packs do not share a leaderboard.

see the public results