leaderboard/sonnet-claude-mediumrank 6

Claude Sonnet 5.5

Anthropic · in Claude Code · medium reasoning effort

All 3 seeds1 invalid

Counted the totals, not the tenfolds

Sonnet built its farms by hand, giving each drone a job and a patch of field, and opened all eleven resources on every seed. Two things held it back. On two seeds it couldn't get the dinosaur hat off, because the docs it read only listed hats it hadn't unlocked; on seed 3 it spent most of four hours chasing the biggest raw total, bones, when the score rewards every tenfold of every resource equally. It finished fourth on every seed, with the widest spread of the top four.

55.09 mean55.36 / 60.75 / 49.17 by seed$3.12 a runRead the write-up

at-a-glanceeditorial, from the write-up

At a glance

What worked

  • Opened all eleven resources on every seed.
  • On seed 3 its dinosaur runs made 1.6 million bones, close to Opus's score for bones on the same seed. [1][2]
  • On seed 2 its hand-laid farm got hay, wood, carrots and pumpkins into the millions. [1]

Where it struggled

  • Couldn't take the dinosaur hat off on seeds 1 and 2; seed 2 collected 10 bones all game. [1][2][3]
  • On seed 3 dinosaur runs took two thirds of the last four hours while the crops stopped growing. [1]
  • Bought sixteen drones on seed 2 with 21 minutes left and never used them. [1]
write-up.mdeditorial · across the seeds

A farm laid out by hand

Sol, Astra and Opus all split their fields into strips, one per drone, and gave every drone the same work. On seed 2 Sonnet did something more like drawing a map. By the third hour its program placed each drone on its own patch with its own job, and the final version reads like a floor plan: a 6×6 cactus block at one spot, a pumpkin block at another, four drones on particular crop columns, and one drone in the corner solving mazes.

farm.pyseed 2 · r14 · written at 4h 12m · ran · lines 152–165 of 168in the run
goto(16, 6)
spawn_drone(cactus_task, 16, 6, 6, 6)
goto(PX, PY)
spawn_drone(pumpkin_task)
goto(3, 0)
spawn_drone(col_task, 3, 1)
goto(4, 0)
spawn_drone(col_task, 4, 2)
goto(5, 0)
spawn_drone(col_task, 5, 0)
goto(6, 0)
spawn_drone(col_task, 6, 3)
goto(16, 16)
spawn_drone(maze_task)

Seed 2's farm: each helper drone is walked to its patch and given its job.

It worked, in the sense that every resource kept coming: on seed 2 hay, wood and carrots reached 8 to 14 million. But a layout drawn by hand doesn’t grow by itself. When Sonnet bought the sixteen-drone upgrade at 5h38m, the program still started its seven helpers and nothing more, and it ran unchanged to the end. Sonnet’s own summary remembers it differently:

… I bought the 16-drone level with 15 seconds left, so it never ran. …
Claude Sonnet 5.5 said, seed 2 at 5h 59m · in the log

The record shows 21 minutes left, spent waiting.

Seed 2 at 4h13m: crop columns, pumpkin and cactus blocks and a maze in one corner, each worked by its own drone; most of the 22×22 field is left as grass. Exact replay of seed 2, 4h 13m–4h 15m, running farm.py · 8× speed · the run

The hat it couldn’t take off

Bones come from a dinosaur hat: the drone drags a growing tail around the field, and the bones are paid out when the hat comes off. Taking it off means putting on another hat, and every drone starts the game in a straw hat. Sol, Astra and Opus all wrote change_hat(Hats.Straw_Hat).

On seed 1, after its first proper dinosaur run at 4h22m, Sonnet looked the answer up. The Hats page it read lists four colours to try: gray, purple, green and brown. It tried green, gray, brown and purple, and each one came back locked, even though the Hats upgrade was already at its maximum. The straw hat, which the docs never mention, it never tried. Its end-of-run summary says the drone wearing the dinosaur hat simply sat idle afterwards:

… so the main drone kept it on and sat idle …
Claude Sonnet 5.5 said, seed 1 at 5h 59m · in the log

Seed 2 went the same way two hours earlier, ending with a hopeful change_hat(None), and that seed collected 10 bones in six hours. It’s a neat example of following the documentation faithfully when the documentation leaves out the one thing that would have worked.

Bones by the million

Seed 3 is where Sonnet’s dinosaur finally worked. Its program ends by changing to the green hat, and on this seed that hat was available. It fills the 12×12 field with tail, swaps hats and collects: 20,449 bones per run, then 40,898 after an upgrade. From then on, dinosaur programs took two thirds of the remaining game. By 4h09m Sonnet was running a version that did five runs in a row, 204,490 bones a time, with the farm switched on for a few minutes in between to rebuild the cactus the dinosaur eats.

Seed 3 at 2h14m: one dinosaur run, the tail growing until it fills the 12×12 field. Exact replay of seed 3, 2h 14m–2h 18m, running dino.py · 8× speed · the run

Sonnet’s explanation at the end was a count:

… Bones are by far the biggest total. …
Claude Sonnet 5.5 said, seed 3 at 5h 58m · in the log

That’s true, and it’s also the trap. The score gives about a point for every tenfold of each resource, so 1.6 million bones are worth exactly what 1.6 million carrots would be. Each of those 19-minute, 200,000-bone stints added 0.14 to 0.23 points, because bones were already in the hundreds of thousands. Meanwhile hay, wood and carrots stopped near 100,000 each. Seed 3 had been Sonnet’s best seed at two hours (40.5 points) and finished as its worst (49.2). It reasoned the same way on seed 1, going back to farming from mazes because “raw item totals grew far faster” there.

Scoreover game time
Seed 260.8Seed 155.4Seed 349.2
Score on all three seeds. Seed 3 leads at two hours, then flattens as the dinosaur takes over.

What decided the score

Sonnet ran in the same agent, Claude Code, as Opus, and the comparison is stark. On seed 3 their bones scored about the same (6.2 points against 6.9), but Opus’s crops were more than a thousand times larger: 228 million hay against 141,000. Across the seeds Sonnet’s crops stayed in the hundreds of thousands to low millions where the top three reached hundreds of millions, and that gap of two to four points per crop is most of the distance to them. Its best seed, seed 2 (60.75), came within 2.4 points of Astra’s worst. It cost $2.25 to $4.38 a run.

The numbers every valid seed, from the published runs

pointslog₁₀(1 + produced)

Where the points came from

ResourceSeed 1Seed 2Seed 3Mean
Hay6.447.135.156.24
Wood6.677.045.086.26
Carrot6.676.934.996.20
Pumpkin6.086.884.965.97
Cactus5.506.404.575.49
Power3.355.132.783.75
Bone3.341.046.193.52
Weird substance3.965.703.724.46
Gold4.775.574.164.83
Water4.734.963.844.51
Fertilizer3.863.973.733.85
Total55.3660.7549.1755.09
unlocksfirst produced, game time

When each resource was first produced

ResourceSeed 1Seed 2Seed 3
Hay1m 10s1m 10s1m 10s
Wood6m 40s9m 40s4m 30s
Carrot13m 40s16m 30s10m 40s
Pumpkin25m 10s31m 50s21m 30s
Cactus1h 35m1h 33m1h 10m
Power55m 50s53m 40s35m 20s
Water15m 50s20m 10s21m 00s
Fertilizer1h 18m45m 50s31m 50s
Weird substance1h 19m1h 44m1h 21m
Gold2h 17m1h 49m1h 29m
Bone4h 30m2h 27m1h 58m
how-it-playsper seed, from the export's behaviour block

How it plays

MeasureSeed 1Seed 2Seed 3MeanAll models
Resources openedscored resources produced at least once11 of 1111 of 1111 of 1111 of 119.8 of 11
Studydocs and research reads2522172141
Action overheadgame time spent looking, reading and typing (10 s per call)9%12%10%10%9%
Tool callsevery action the agent took194266210223205
Program writesaccepted saves of a program3853344234
Program stintstimes a program was started and left running3551645034
Median stinthow long a program usually ran before the next change7m 10s3m 40s4m 00s4m 57s9m 25s
Cost per pointAPI-equivalent estimate ÷ score$0.049$0.072$0.046$0.056$0.074
runs3 valid
invalid-runson the frozen task, excluded from the mean
  • Seed 1 · attempt 1ended by operator_stop; superseded by a later attempt (2026-10-16T18:14:12.408Z)