LLMs Play Zachtronics' TIS-100
Web page · https://tis-100.wincon.dev/
First seen by Oriel on .
AI summary
TIS-100 Bench tested 16 LLMs on 55 programming puzzles, finding 133 verified Pareto-frontier points across 18 puzzles and three outright cycle records.
- Opus 5.5 and GPT-6 Astra solved all 55 puzzles and produced new frontier points on 11, with statistically indistinguishable record proximity.
- Each model received one timed attempt per puzzle, using the real game and diagnostics; headline results required fresh-game replay and 100,000 passing simulator tests.
- Opus 5.5 completed Undecimal Twin RounderSPEC44149538 in 1,918 cycles, 29% faster than the frozen community record, while using one more node and 28 more instructions.
- Providing exact targets, detailed diagnostics, randomized edge-case testing, and up to 60 minutes improved mean record proximity for every model across six puzzles.
- Community players quickly improved selected model programs, including reducing Opus 5.5’s Undecimal Twin RounderSPEC44149538 result from 1,918 to 1,744 cycles the next day.
#llm evaluation #program synthesis #optimization #tis-100
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 91, latest observation ; +86 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 22, latest observation ; +22 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hacker News | 50448340 | https://tis-100.wincon.dev/ Read discussion |