Generalist is built on research in training robots on real-world data. Source: Generalist
Summer is about creating memories. And for robots, that translates into generating lots of data. It’s also a time to catch up on reading.
My beach material last June was the research paper on “Universal Manipulation Interface: In-the-Wild Robot Teaching Without in-the-Wild Robots“(a.k.a. UMI). The academic findings quickly became the foundation for a $2 billion unicorn robotics startup, Generalist.
Essentially, teams at Toyota Research Institute (TRI),Columbia University, andStanford Universitycreated a data-collection platform that uses puppet-likeend effectorsoperated by people, with GoPro cameras capturing real-world tasks such as washing dishes and picking up objects.
The resulting demonstrations become training data for robot foundation models, enablingcollaborative robotsto learn and take over these tasks much more quickly.
UMI’s paper demonstration image, illustrating human data capture to full autonomy on co-bots. Source:https://umi-gripper.github.io/
This past June atAutomate, Generalist demonstrated live how its models could quickly create policies for different cobot systems. On one side of the hall, the company demonstrated how it could make Universal Robots (UR) arms fold and build cardboard boxes, while across the McCormick Center, it usedFlexivarms to repair robot vacuums.
According to thecompany‘s X post boasting about the display at the show: “What resonated most wasn’t just that the models could do the tasks — it was that they had the intelligence to recover in real time when things went wrong, and this got people thinking differently about automation.”
Then, almost a month later, theAIunicorn posted a blog missive titled “Towards Machines with a Thousand Hands.” To better understand the meaning of Generalist’s newest accomplishment of using different arms and grippers, including spatulas, across a plethora of use cases, I interviewed Samantha Castellanos, thecompany‘s founding mechanical engineer.
I wanted to know the process beyond research, especially how it translated into industrial implementations to accelerate adoption and expand usage into new applications.
The mechanical engineer was referring to the already crowded space that has cumulatively raised more than $4 billion, including standouts likeSkild AI($2 billion),Physical Intelligence($1 billion),Field AI($300 million), andRLWRLD($41 million).
“Everything we do is to make the model better,” she continued. “All of the data collection, all the types of tool collection, different types of interacting with the world. That is all incredibly important. In addition to all the efficiency improvements that we’ve been making, we’re just trying to build that real general intelligence by giving the model everything it needs to know about how the world works, and that I think is what’s going to set us apart.”