ToyBlocks: A controlled world for robot learning and evaluation
Web page · https://huggingface.co/blog/allenai/toyblocks
First seen by Oriel on .
AI summary
The page introduces FoundationWorlds and ToyBlocks, a controlled simulation for generating robot-learning data, evaluating reasoning and dexterity, and investigating generalization, planning, and tool-using agents.
- ToyBlocks models a robot arm manipulating blocks with letters and pictures, supporting stacking, sorting, searching, pushing, flipping, and rotating tasks under randomized physical conditions.
- Reinforcement learning and motion planning generate short-horizon skills, while a task-and-motion planner sequences them into demonstrations; parallel execution reaches up to 24 times real-time performance on one GPU.
- The framework separates high-level reasoning from low-level control, enabling researchers to isolate planning and execution failures using reference components and metrics including speed, force, and task progress.
- Initial VLA experiments found poor spatial generalization despite large-scale pretraining, testing interpolation and extrapolation across workspace positions, stack heights, letters, colors, and block counts.
- Claude Opus 5.5 solved or correctly rejected 63% of search-and-spell tasks, while agent-generated skill descriptions raised success from 24% to 63%, versus 36% for human-written descriptions.
#robot learning #simulation #vision-language-action models #agentic systems
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Blog | https://huggingface.co/blog/allenai/toyblocks | https://huggingface.co/blog/allenai/toyblocks Read discussion |