Back to the lab

Build a test set before you switch models.

Open the sample report, then change the models or tasks you want to test.

Runs on worked examples — how this works

Change the models · Change the task
Set up your model switch

Which model are you moving from?

Choose a reference and two or three candidates. Older references are not retirement announcements; check the provider’s current availability.

Candidate models Choose 2–3

What does good work look like?

Use three to five repeatable tasks. Keep the same inputs for every model. Don’t paste personal, customer or confidential information.

Preview runs do not use the daily allowance.

One model answering one task counts as one Model Switch run. The current model is included. This set uses 9 runs when live. A Compare pair counts as one run.

Your test set. Your judgement.

Task 1 · Summarise this email

GPT-5.6 Luna

Current model · Authored sample · not this model’s output

The workshop has moved to Thursday at 10am. The draft agenda is still due Monday, and Priya needs to confirm the room.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 22
Your checks · GPT-5.6 Luna, task 1

Claude Sonnet 5

Authored sample · not this model’s output

Thursday workshop, 10am. Two follow-ups: finish the agenda for Monday; get Priya’s room confirmation.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 14
Your checks · Claude Sonnet 5, task 1

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

Gemini 3.7 Flash

Authored sample · not this model’s output

Next steps 1. Prepare the draft agenda by Monday. 2. Ask Priya to confirm the room. 3. Attend the rescheduled workshop on Thursday at 10am.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 25
Your checks · Gemini 3.7 Flash, task 1

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

Task 2 · Extract the details from this invoice

GPT-5.6 Luna

Current model · Authored sample · not this model’s output

Supplier: Birch Office Supplies Invoice: INV-204 Issued: 20 August 2026 Total: £480 Currency: GBP PO: PO-28 Missing: due date and bank details. This is extraction, not payment approval.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 28
Your checks · GPT-5.6 Luna, task 2

Claude Sonnet 5

Authored sample · not this model’s output

Invoice INV-204 | Birch Office Supplies | 20 Aug 2026 | GBP 480 | PO-28. Exceptions: no due date; no bank details. Route for verification only, not payment.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 28
Your checks · Claude Sonnet 5, task 2

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

Gemini 3.7 Flash

Authored sample · not this model’s output

I found six supplied fields: Birch Office Supplies; INV-204; 20 August 2026; £480; GBP; PO-28. Two required checks remain — due date and bank details — before any payment process continues.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
41 / 31
Your checks · Gemini 3.7 Flash, task 2

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

Task 3 · Draft a reply to this customer

GPT-5.6 Luna

Current model · Authored sample · not this model’s output

I’m sorry your order has not arrived, especially with the gift needed this weekend. Could you share your order reference so the team can check its status and available options? I can’t confirm a delivery date from this message alone.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
50 / 40
Your checks · GPT-5.6 Luna, task 3

Claude Sonnet 5

Authored sample · not this model’s output

Thanks for letting us know — I understand this is time-sensitive because it is a gift. What is your order reference? We need that before checking the delivery status, and I don’t want to promise an outcome without it.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
50 / 39
Your checks · Claude Sonnet 5, task 3

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

Gemini 3.7 Flash

Authored sample · not this model’s output

Sorry about the delay. To investigate, could you provide your order reference? Once the team can identify the order, it can review the tracking and explain the available next steps; no delivery date can be confirmed yet.

Cost of this answer
$0 · sample
Time taken
Not measured
Words in / out
50 / 37
Your checks · Gemini 3.7 Flash, task 3

Cost and speed differences are available after a successful live run. Samples are not benchmarks.

How cost, time and word counts work

Live cost is an estimate in US dollars from the provider’s real input/output usage, cached usage where reported, and the listed rates checked 2026-08-31. It is not an invoice; taxes, currency conversion and provider-account discounts are excluded. Time covers the full server-side model request. Word counts split visible text on spaces; billing uses tokens, which are different.

Samples have no measured latency or model usage. Their $0 means no inference was purchased, not that the selected model is free. Model Switch deltas appear only for two successful live answers.

Sharing is optional. Links contain model and preset selections only—not custom prompts, responses, grades or keys. Anyone with the link and preview access can load the selections. Downloads do contain your prompts and outputs; review them before sharing.

How this was built

Build diary awaiting Arun’s notes. Empty fields below are intentional; no build story or time estimate has been invented.

Tools used
OpenAI Codex, Next.js 16, TypeScript, Impeccable
Models used to build
Arun to add
Rough hours
Not recorded
What failed
A same-origin hostname mismatch surfaced in the production smoke test and was fixed. Live providers remain intentionally disabled.

Actual build prompts

Not as v2, build it over this
Here you go, build it

Use a repeatable test, not a confident guess.

One run does not establish reliability. Keep the inputs and checks consistent, include edge cases and repeat the comparison before changing a production workflow.

Check the work around the model

Open the agent-readiness audit