i tested if the old GPT is worth paying double

Lenny's AI Builders - LAB 0024

GPT-5.6 Sol vs GPT-6.1 Sol on three of my real jobs, a blind judge, and one prompt to run the same test on yours.

devs are paying double for the old GPT on the API - and swearing it's better than the new one.

so i put both on three of my real jobs, twice each, with a blind judge from a different AI. then i turned the whole test into one prompt you can hand your own agent.

why it matters

GPT-5.6 Sol costs $4 in / $20 out per million tokens. GPT-6.1 Sol is $2 / $10. same company, half the price - and some devs are switching back to the old one anyway.

benchmarks won't settle it. one leaderboard favours the new model, another favours the old one. the only score that matters is the one on your work.

what you get

• one prompt you can copy. it makes your agent run the same job on two models in clean copies of your project, time them, run your tests and hide the names.
• a blind judge. a different AI scores A and B out of 100 without knowing which model wrote which.
• one table. tests passed, judge score, minutes and API cost - then a plain call on which model to use for that kind of job.

the real example: my three jobs

my agent ran it for me on 17 October 2026 in the Codex CLI, medium effort, two runs per job. the judge was a fresh Claude Opus session that only saw A and B.

1. a broken test in Threadify - 6.1 Sol won both runs (80 vs 35, 74 vs 66).
2. a video pipeline bug - 5.6 Sol won both runs (88 vs 80, 81 vs 78).
3. my landing page copy - one each (80 vs 58 to 6.1 Sol, then 84 vs 52 to 5.6 Sol).

wins: 3-3. but the tie hid the rest:

• average score: 6.1 Sol 74.0, 5.6 Sol 68.7
• total time: 6.1 Sol 13.1 min, 5.6 Sol 24.8 min
• API cost: 6.1 Sol $1.19, 5.6 Sol $6.79

so is the old one worth paying double? not on my jobs. it did win my video bug twice - so test the kind of work you actually do.

try it

1. pick one real job from your week - a bug, a failing test, a page of copy.
2. open Claude Code or Codex inside that project.
3. paste the prompt below and drop your job into the first line.
4. let it run. it works in clean copies, so your real folder stays untouched.
5. read the table - and if it says it's too close to call, believe it.

copy this prompt

paste this into Claude Code or Codex inside your project:

I want to know which AI model is actually better for MY work - not on benchmarks. Run a blind bake-off.

The job: [paste one real task from your week - a bug, a failing test, a page of copy]
Model A: gpt-5.6-sol    Model B: gpt-6.1-sol    (swap in any two models your CLI can run)
Prices per million tokens (OpenAI pricing page, 17 Oct 2026): gpt-5.6-sol $4 in / $0.40 cached / $20 out; gpt-6.1-sol $2 in / $0.10 cached / $10 out.

1. Make two clean copies of this project in a temp folder. Never touch my real folder.
2. Run the job once in each copy with the same prompt and the same effort setting:
   codex exec -m [model] -c model_reasoning_effort=medium --json "[the job]"
   Time each run and save its token usage.
3. Run my tests in both copies and note what passed.
4. Hide the model names. Label the two results A and B in a random order. Ask a different AI to judge them
   (for example: claude -p) - give it only the job, the two diffs and the test results, and ask for a 0-100 score
   and one sentence why for each.
5. Show me one table: tests passed, judge score, minutes, and cost at today's API prices. Then tell me which
   model to use for this kind of job - and say plainly if it's too close to call.
Drafts only: don't push, merge or publish anything.

you'll know it worked when

you get one table with all four numbers for both models - tests, judge score, minutes and cost - and a plain call on which model to use for that kind of job.

a bad result: the agent edits your real folder, the judge can see the model names, or the same model grades its own work.

limits and dates

• three jobs, two runs each, my own repos. a test, not a benchmark. run on 17 October 2026 in Perth.
• one judge family: Claude Opus judged every pair blind. the landing page job was re-judged with the product facts.
• API-equivalent cost: worked out from token use at OpenAI's list prices on 17 October 2026. OpenAI labels the 5.6 Sol price promotional through at least 21 November 2026.
• drafts only: nothing was pushed, merged or published. my agent ran the test for me.

the rest of this LAB

• GPT-6 Pro runs agents in parallel. OpenAI's release notes say multiple agents can now work on different parts of a task in parallel, and you can follow each one. i watched three spin up on one question. Pro users only; OpenAI didn't publish limits. my take: chat usage is separate from your Codex usage, so if you're on Pro, go crazy with it.
• Claude writes about 90% of Claude Code. Anthropic's write-up says merged pull requests more than doubled from June to September while reverts fell by more than 40%. low-risk changes need no human once checks pass; risky ones need one or two people. Anthropic's own numbers on its own code.
• X pays creators to post video. an invite-only pilot called Video Kickstart Bonus: post 15 videos on at least 15 different days in a 30-day window. one screenshot shows $3,000, Meet Kevin's shows $5,000. invite-only; enrollment closes 30 October 2026. my take: check your email for an invite.
• more Codex usage. Tibo says Codex & Work got changes that stretch your usage further - shared context across subagents, longer caching, better routing. he gave no number.
• Claude Code answers sooner. large requests now use gzip compression, even with telemetry off. no speed numbers published.
• move your AI memory into Grok Bot. a community bot brings what Claude or ChatGPT knows about you into Grok Bot, and you keep only what you choose. a community bot built by a third-party user, not a built-in feature.

sources

• Irushi on GPT-5.6 Sol vs GPT-6.1 Sol
• OpenAI API pricing
• ChatGPT release notes
• OpenAI: GPT-6 Pro in the Chat tab
• How the Claude Code team does code review
• Thorsten Ball: why are you still using pull requests?
• Video Kickstart Bonus announcement
• Tibo on Codex usage
• Claude Code changelog
• ClaudeDevs: responding sooner
• Grok Bot: Memory Transfer

next step

watch LAB 0024 on YouTube - the practical starts at 11:55.

ran the bake-off on your own job? tell me under the video which model won.

want the next one? get the next field notes. optional - these notes stay free to read.