Research
RoboColosseum's Opening Results: No AI Policy Completed a Unitree G1 Humanoid Task
RoboColosseum, a new open initiative for testing AI models on physical robots, launched on 4 November 2026, and its initial published results show how far humanoids still have to go: none of the evaluated policies completed any of the five tasks set for the Unitree G1 humanoid, according to the project's paper.
The project announced itself on X as "an open robot eval initiative" backed by Hugging Face, BitRobot Network and the Singapore Embodied AI Centre, offering access to more than 100 robots, more than 140 in-the-wild scenes, open rollout data and "10,000 FREE eval hrs". The paper's authors come from NUS, the University of Washington, Stanford, UNC, Ai2, Oxford, Hugging Face, BitRobot and the Singapore Embodied AI Centre.
The paper tested five vision-language-action (VLA) policies and three frontier AI agents across four robot bodies: the SO-101 arm, a Franka arm, a Bimanual-YAM setup and the Unitree G1 humanoid. On the G1, no policy finished any of the five tasks, and average task progress ranged from 13.3% to 23.2%. In other words, the systems made partial progress but did not complete the jobs. The result concerns the policies tested, not the limits of the G1 hardware itself.
Unitree said on X it was "great to see Unitree G1s being used as the primary embodiment" in the humanoid benchmark category.
The evaluations ran from 17 September to 28 October 2026 on a fleet of more than 100 robots in the US, UK, Singapore, Malaysia and China. The Open Track gathered 5,699 blinded pairwise comparisons at 124 sites, and a separate Fine-tune Track covered 20 tasks with 100 demonstrations each.
The authors report that frontier AI agents rival specialised VLA policies, but that all approaches struggle on open-world tasks. After fine-tuning, GR00T N1.7 led on the SO-101, while pi0.5 led on the Franka and Bimanual-YAM. Agreement between rankings on different robot bodies was low, with a Kendall's W of 0.24, so a model that does well on one robot may not do well on another.
Ai2 said on X that its MolmoAct 2 placed second, behind GPT-6 Astra, in the Franka arm tests without extra training for those tasks.
Hugging Face, a founding partner, said that "every rollout, failures included" is open on its Hub. The project has pledged the 10,000 free evaluation hours to academic labs.
For humanoid makers, the G1 numbers set a measurable starting point rather than a sign of readiness: getting from partial progress to finished tasks remains the open problem.