Ai2’s ManiFlow-tb robot arm model finishes about 57% of block tasks in simulation

The Allen Institute for AI (Ai2) has published ManiFlow-tb on Hugging Face, a 322.5M-parameter model that controls a simulated robot arm. In Ai2’s own closed-loop tests, it completed 54% to 59% of three-block tabletop tasks.

The model is the low-level half of ToyBlocks, Ai2’s block-stacking robot project. It turns a high-level instruction into the arm’s movements. Weights and training code are released under the Apache License 2.0.

A 322.5M action model with a heatmap target

ManiFlow-tb was trained on simulated Franka arm data. It handles pick-and-place and in-place rotation of three blocks on a table. According to the model card, it uses a ManiFlowMultiModal design with a DiTX action head of 12 layers and 8 heads, with a width of 768. The weights are stored in bf16 safetensors.

It reads front and gripper camera images at 224 x 224, a 9-dimensional robot state, the task name, and a target heatmap token supplied by a high-level policy. It outputs a chunk of 25 executed actions out of a 50-step prediction horizon. Each action is 9-dimensional and absolute. The last dimension is a play mode: 1 for play, 0 for pause, -1 for skill done. The card says that value must be fed back to the model when it is 1 or 0.

Inference uses flow matching with consistency training and 12 integration steps. Training ran for 200,000 steps on 8 GPUs with a per-device batch of 128, using three ToyBlocks-generated LeRobot datasets. The published weights are the non-EMA weights of the final step.

Four evaluation runs that did not agree

Ai2 evaluated the model on 128 held-out episodes, each starting from the initial state of a recorded episode. An oracle supplied the high-level plan, so the numbers isolate the action model. Success means all goal conditions hold, the blocks have settled, and the gripper is open.

Four runs gave 73, 75, 76 and 69 successes out of 128. Collisions ended between 30 and 38 episodes per run. Ai2 attributes the spread to simulation and batched inference that are not bitwise deterministic. Two runs agree on only 82 to 97 of the 128 episodes, so single episodes flip between success and failure.

The time limit matters as much as the model. Tests used 3x the longest expert demonstration, which is 194 seconds, set through --expert-timeout-scale 3.0. With the default 30 seconds, only 26 to 28 episodes succeed. Ai2 says that measures the limit, not the model.

To run it, download the repository with hf download allenai/maniflow-tb --local-dir maniflow-tb and point ToyBlocks’ scripts/launch_eval.py --policy maniflow at checkpoints/steps_200000_model.safetensors. The repository also holds config.yaml and stats.json, and ToyBlocks expects that layout. Ai2 also publishes the training code and the data.

The weak link next to a classical planner

The model card does not show how ManiFlow-tb compares with other approaches. The nearest comparison comes from Ai2’s companion release, the Molmo2-4B-tb planner, which we covered on Wednesday. There, a classical motion planner reached 97.5% on easy seen tasks, while the same planner driving ManiFlow reached 55.5%. On hard unseen tasks the figures were 14.5% for the motion planner and 10.0% for ManiFlow. On a real robot with ManiFlow underneath, the planner succeeded in 6 of 10 trials.

Those numbers come from a different test set than the 128-episode evaluation here, so they are not directly comparable. They point the same way: the learned low-level policy is where tasks fail, and the language model that plans is not. The ToyBlocks repository, which runs on IsaacSim and IsaacLab, says the aim is to let researchers trace a trained policy’s outcomes back to its training data. That makes an open, imperfect baseline more useful than a polished one.

What the release does and does not establish

Anyone with access to the ToyBlocks setup and GPUs can reproduce the evaluation, which is the main value for researchers. The weights are permissively licensed. Results are Ai2’s own and have not been independently checked. Everything is in simulation with a Franka arm and three blocks, so the model is not a general-purpose manipulation policy. The page also gives no real-robot numbers for the model alone, and no hardware requirements for inference.