Physical Intelligence's company blog on robot foundation models, including the π0 and π0.5 vision-language-action models and their research results.
HN commenters link it as an example of vision-language-action models for robotics, citing the π0 and π0.5 posts and a coffee-making demo.
> But factory robots haven't propelled Kuka, Fanuc, ABB, UR, Staubli and peers to anything like the levels of success nvidia is already at. A market big enough to accommodate several profitable companies with market caps in the tens of billions might not drive much growth for a company with a trillion-dollar market cap.
That's because the past year of robotics advancements (e.g. https://www.physicalintelligence.company/blog/pi0, https://arxiv.org/abs/2412.13196) has been driven by advances in machine learning and multimodal foundation models. There has been very little change in the actual electronics and mechanical engineering of robotics. So it's no surprise that the traditional hardware leaders like Kuka and ABB are not seeing massive gains so far. I suspect they might get the Tesla treatment soon when the Chinese competitors like unitree start muscling into the humanoid robotics space.
Robotics advancements are now AI driven and software defined. It turned out that adding a camera and tying a big foundation model to a traditional robot is all you need. Wall-E is now experiencing the ImageNet moment.
It’s the fundamentals that underly Stable Diffusion, Dalle, and various other SOTA image generation models, video, and audio generation models. They’ve also started taking off in the field of robotics control [1]. These models are trained to incrementally nudge samples of pure noise onto the distributions of their training data. Because they’re trained on noised versions of the training set, the models are able to better explore, navigate, and make use of the regions near the true data distribution in the denoising process. One of the biggest issues with GANs is a thing called “mode collapse” [2].
[1] https://www.physicalintelligence.company/blog/pi0
[2] https://en.wikipedia.org/wiki/Mode_collapse
> We’ve long since proven that machines are better for back-breaking labor.
This is contrary to my perception of current trends (and to a perhaps generous reading of Moravec's Paradox). Dexterous manipulation is still quite hard which is why a task like laundry folding is so impressive despite being relatively simple for many humans: https://www.physicalintelligence.company/blog/pi0
Perhaps we disagree on what "back breaking" entails but I'd much rather spend all day coding than folding laundry!
regardless of what is "really" happening under the hood, if this is model has any influence on the real world through robotics, this could lead to an actual fatality. Physical Intelligence has a robot with arms that can interact with the world. If a robot like this, or a more advanced robot like a car, ends up on the same thought pattern that this model did, it could make actions that can hurt people.
- https://www.physicalintelligence.company/blog/pi0?blog
We are certainly getting close! In 2010, watching PR2 fold some unseen towels is similar to watching paint dry [1], but we can now enjoy robots attain lazy student-level laundry folding in real-time, as demonstrated by π₀[2].
[1] https://www.youtube.com/watch?v=gy5g33S0Gzo
[2] https://www.physicalintelligence.company/blog/pi0
That's less a GenAI problem, more robotics. And perhaps it's here, literally doing your laundry:
https://x.com/physical_int/status/1852041726279794788
https://www.physicalintelligence.company/blog/pi0
https://www.entrepreneur.com/business-news/jeff-bezos-backed-physical-intelligence-robot-can-do-chores/482348