Oriel — AI & technology worth noticing, every day.

LLMs Play Zachtronics' TIS-100

Web page · https://tis-100.wincon.dev/

First seen by Oriel on .

AI summary

TIS-100 Bench tested 16 LLMs on 55 programming puzzles, finding 133 verified Pareto-frontier points across 18 puzzles and three outright cycle records.

  • Opus 5.5 and GPT-6 Astra solved all 55 puzzles and produced new frontier points on 11, with statistically indistinguishable record proximity.
  • Each model received one timed attempt per puzzle, using the real game and diagnostics; headline results required fresh-game replay and 100,000 passing simulator tests.
  • Opus 5.5 completed Undecimal Twin RounderSPEC44149538 in 1,918 cycles, 29% faster than the frozen community record, while using one more node and 28 more instructions.
  • Providing exact targets, detailed diagnostics, randomized edge-case testing, and up to 60 minutes improved mean record proximity for every model across six puzzles.
  • Community players quickly improved selected model programs, including reducing Opus 5.5’s Undecimal Twin RounderSPEC44149538 result from 1,918 to 1,744 cycles the next day.

#llm evaluation #program synthesis #optimization #tis-100

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 91, latest observation ; +86 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 22, latest observation ; +22 over observed period since

Aliases

SourceExternal IDURL
Hacker News50448340https://tis-100.wincon.dev/ Read discussion