Ai2’s block-stacking robot planner scores 97.5% on easy tasks, 14.5% on hard unseen ones

The Allen Institute for AI (Ai2) has published Molmo2-4B-tb on Hugging Face, a vision-language model of about 4.5 billion parameters that plans block-stacking steps for a robot. In simulation it reaches 97.5% success on easy tasks, but only 14.5% on hard ones it has not seen.

A planner that picks the next move from one camera image

According to the model card, Molmo2-4B-tb is the high-level planner for ToyBlocks, an Ai2 project built around a robot arm that stacks blocks on a table. The model takes one 640×360 front-camera image and a text goal. It writes a short plan, then names an action and the block or spot to act on, given as a Molmo2 point. The actions are pick, place, flip, rotate and done. A separate low-level model, such as a motion planner or a learned policy, carries out each step. The card says the planner runs at about 6 Hz.

It is fine-tuned from Molmo2-4B, which the card lists as built on Qwen3-4B-Instruct-2507, with a SigLIP 2 vision encoder. Training ran for 6,000 steps at batch size 256 on simulated three-block stacking episodes. The card says 25% of the training data was image-pointing material (PixMo-Points, PixMo-Count and CoSyn-Point) to keep the model’s general pointing ability.

The weights are a single model.pt file in float32 plus a config.yaml, in the native Molmo2 format rather than Transformers. Ai2 lists about 20 GB of GPU memory in bfloat16. The page gives one node of eight H100 GPUs for about eight hours as the training cost, and estimates about 61 kg of CO2e per run.

Who can use it, and what the license limits

The model is released under Apache 2.0 for research and educational use, under Ai2’s Responsible Use Guidelines. The card adds that the base model’s training datasets are limited to academic, non-commercial research use, so commercial users should read that caveat before building on it. Ai2 also says the intended scope is moving blocks on a table and that other uses are out of scope. It did not adapt any safety benchmarks, judging them not relevant to these tasks.

Ai2 released the low-level counterpart the same day: ManiFlow-tb, a 322.5M-parameter action model trained on simulated Franka arm data, also under Apache 2.0. The pair can be run together using the ToyBlocks code on GitHub.

Numbers in context: Success drops fast as tasks get harder

The model card reports closed-loop success rates in simulation, with 200 episodes per set. With a motion planner as the low-level executor, the planner scores 97.5% on easy tasks it has seen and 81.0% on unseen ones. On medium tasks that falls to 53.5% and 36.5%, and on hard tasks to 31.5% and 14.5%.

Paired with ManiFlow as the executor, the numbers are lower: 55.5% (seen) and 44.5% (unseen) on easy, 35.5% and 12.5% on medium, and 17.0% and 10.0% on hard. On a real robot with ManiFlow underneath, the model succeeded in 6 of 10 trials, according to the card.

Two outside details from the ToyBlocks repository and the ManiFlow-tb card help read these figures. First, ToyBlocks is built on NVIDIA’s IsaacSim and IsaacLab and writes goals in PDDL, which lets it score partial progress without custom metric code, according to its GitHub page. Second, ManiFlow-tb was evaluated on 128 held-out episodes with an oracle high-level policy and scored between 54% and 59% across four runs. That puts the low-level policy, not the planner, as the weaker link: even with perfect planning, it completes a bit over half of the episodes. The gap between the motion-planner and ManiFlow columns above points the same way.

What the release does not settle

The results come mostly from simulation with three blocks, and the real-robot figure rests on only 10 trials, which is too few to generalize from. The card cites a draft technical report, “ToyBlocks: A Minimalistic FoundationWorld for Robot Foundation Models” (2026), so the numbers have not been independently verified. The repository describes the goal as a controlled world where a policy’s results can be traced back to its training data, which suggests the release is meant as a research testbed rather than a product. Ai2 says nothing about wider robotics use, and the card limits the model to table-top block moves.

For researchers, the value is an open, inspectable high-level and low-level pair with training and evaluation code. For everyone else, the results are a reminder of how far vision-language planners still have to go on unseen, multi-step manipulation. For other open-weight releases this week, see our coverage of ByteDance’s sparse autoencoders for Qwen3.