Claude Haiku 5.5 vs Haiku 4.5

Scored higher by 5 points or more: Haiku 5.5 on building web pages, 98 to 63; Haiku 5.5 on code, 100 to 76; Haiku 5.5 on planning and architecture, 91 to 77; Haiku 5.5 on everyday, 100 to 71.

Haiku 5.5’s median max effort answer took 1 min 56 s and cost 1.5¢ at API prices, and Haiku 4.5’s median answer took 6.8 s and cost 0.5¢. A median Haiku 5.5 answer cost 3.3 times as much.

Of the 24 prompts, Haiku 5.5 passed more checks on 16, Haiku 4.5 on none, and the two passed the same number on 8.

Where they differed

Prompts where the two passed a different number of checks, each at its highest setting, widest gap first.

• F05 SQL: which customers left and came back

  Haiku 5.5 7 of 7, Haiku 4.5 1 of 7.

  Passed by Haiku 5.5, missed by Haiku 4.5: It gets the example in the question right; Periods that touch or overlap are not counted as gaps; Several gaps per customer, same-name customers, sort order; A period inside a longer one does not hide or invent a gap; Open-ended (NULL end) periods never produce a gap after them; Days without access is exact across month, leap and year ends

• I01 Check a market receipt adds up

  Haiku 5.5 5 of 5, Haiku 4.5 1 of 5.

  Passed by Haiku 5.5, missed by Haiku 4.5: Says the tomatoes line is wrong and should be 2.95; Gives the right total, 27.01; Says the overcharge is 1.00; Does not wrongly call any other line a mistake

• F03 Some subscriptions renew a day early

  Haiku 5.5 7 of 7, Haiku 4.5 2 of 7.

  Passed by Haiku 5.5, missed by Haiku 4.5: Uses the customer's local date (Auckland, Kolkata, LA, London); Explains both causes: the UTC date and drifting month to month; Signups on the 29th to 31st keep their day after short months; Leap and non-leap Februaries are handled, including 2100; Signups just before or after a clock change get the right local date

• B04 Minesweeper in one HTML file

  Haiku 5.5 9 of 9, Haiku 4.5 3 of 9.

  Passed by Haiku 5.5, missed by Haiku 4.5: Numbers are right and an empty square opens its neighbours; Right click flags and unflags, the counter follows, a flagged square stays shut; Hitting a mine says Game over, shows all 10 mines as 💣 and stops the board; Opening every safe square shows You win! (and not before); New game resets the board, counter and message; Fits a 375 px phone with squares of at least 32 px, no sideways scroll

• B02 Build a dashboard from a picture of its design

  Haiku 5.5 7 of 7, Haiku 4.5 3 of 7.

  Passed by Haiku 5.5, missed by Haiku 4.5: Sidebar, active item, accent blue and the three status pills match the picture; Chart bars run Mon to Sun with heights in proportion to their values; On a phone: no sideways scroll, cards stacked, sidebar hidden until a menu button opens it; Side by side with the picture, a designer would accept it as a faithful build

• K02 Answer questions from a bread maker manual

  Haiku 5.5 6 of 6, Haiku 4.5 3 of 6.

  Passed by Haiku 5.5, missed by Haiku 4.5: Question 1: gets 4 hours 15 minutes (the footnote's extra 15 minutes included); Question 2: notices the delay needed is over the 13-hour maximum; Question 5: says the manual doesn't say how long keep-warm lasts (no made-up number)

• P02 Filling a week of volunteer shifts

  Haiku 5.5 6 of 6, Haiku 4.5 3 of 6.

  Passed by Haiku 5.5, missed by Haiku 4.5: Nobody is on a shift they can't do; Everyone works 3 or 4 shifts, and Osei exactly 2; Wren and Tomasz are never together, and Lindiwe is always with Priya

• B01 A landing page for a small invoicing app

  Haiku 5.5 9 of 9, Haiku 4.5 6 of 9.

  Passed by Haiku 5.5, missed by Haiku 4.5: One h1, no skipped heading levels, text 16 px or larger, enough contrast; Yearly shows $90, $190 and $490, and Monthly brings back $9, $19, $49; Each FAQ question opens and closes from the keyboard

• A02 Should we switch to microservices?

  Haiku 5.5 8 of 8, Haiku 4.5 6 of 8.

  Passed by Haiku 5.5, missed by Haiku 4.5: Suggests a middle path: clear modules inside the monolith; Tone is fair to the cofounder and safe to forward

• A01 Plan a booking app for a dog groomer

  Haiku 5.5 8 of 9, Haiku 4.5 6 of 9.

  Passed by Haiku 5.5, missed by Haiku 4.5: Takes the deposit through a hosted checkout and never stores card numbers; Applies the 24-hour refund rule from the appointment start time

• W02 Turn a messy thread into one clear update

  Haiku 5.5 6 of 6, Haiku 4.5 5 of 6.

  Passed by Haiku 5.5, missed by Haiku 4.5: Says each person pays £60, and gives no wrong share

• R02 Did cycling in town really double?

  Haiku 5.5 6 of 6, Haiku 4.5 5 of 6.

  Passed by Haiku 5.5, missed by Haiku 4.5: Notices the 1,890 total does not match the rows

• S02 A story of exactly 100 words

  Haiku 5.5 6 of 6, Haiku 4.5 5 of 6.

  Passed by Haiku 5.5, missed by Haiku 4.5: Exactly 100 words

• B05 Fix three layout bugs on a garden page

  Haiku 5.5 7 of 7, Haiku 4.5 6 of 7.

  Passed by Haiku 5.5, missed by Haiku 4.5: On a 375 px phone the page no longer scrolls sideways

• F04 Review a pull request before merging

  Haiku 5.5 7 of 7, Haiku 4.5 6 of 7.

  Passed by Haiku 5.5, missed by Haiku 4.5: Puts the invoice access hole first, ahead of the other issues

• A03 Rename a column with no downtime

  Haiku 5.5 8 of 9, Haiku 4.5 7 of 9.

  Passed by Haiku 5.5, missed by Haiku 4.5: Keeps returning username to old mobile apps until they are gone; Gives a way back at every step

  Passed by Haiku 4.5, missed by Haiku 5.5: Drops the old column last, after a waiting period