Deepak Pathak (left) and Abhinav Gupta (right), former Carnegie Mellon University professors, founded Skild AI in 2023 to build a “general purpose brain” for AI
Skild AI
Robots, whether bipedal humanoids handling basic factory tasks or four-legged military “robot dogs” destined for urban combat, need brains. Historically, these have been highly specialized and purpose-built. But a Pittsburgh-based robotics startup claims to have created a single off-the-shelf intelligence that can be plugged into various robots to enable basic functions.
Founded in May 2023 by Abhinav Gupta and Deepak Pathak, two former Carnegie Mellon University professors, Skild AI has created a basic model for what it describes as a “general-purpose brain” that can be embedded in a variety of robots , enabling them to do things like climb steep slopes, walk over objects that get in her way, and identify and retrieve items.
The company announced Tuesday that it has raised $300 million at a $1.5 billion valuation in a Series A funding round led by Lightspeed Ventures, Softbank, Coatue and Amazon founder Jeff Bezos with participation from CRV, Felicis Ventures, Menlo Ventures , Amazon and General Catalyst. others.
Raviraj Jain, Lightspeed partner who also led the company’s July 2023 seed round said Forbes he was extremely impressed with the Skild AI models when he first saw them being pressure tested last April. The robots they used were able to perform tasks in environments they had never seen before and were not designed for demonstrations. “Robots back then were able to climb stairs, and I think it’s really crazy how well they were able to do that because it’s a very complex stability problem,” he said.
More impressive, yet: robots using Skild’s AI models also demonstrated “emergent skills”—completely new skills that weren’t learned. These are often simple, such as retrieving an object that slips from your hand or rotating an object. But they demonstrate the model’s ability to perform unforeseen tasks, a tendency that occurs in advanced artificial systems such as large language models.
Skild has achieved this by training its model on a massive database of text, images and video – one it claims is 1,000 times larger than those used by its rivals. To create this massive database, the co-founders, both former AI researchers at Meta, blended a mix of data collection techniques they’ve developed and tested over years of research.
One way was to hire human contractors to operate the robots remotely and collect data on those actions. Another was to have the robot perform random tasks, record the results, and learn by trial and error. The AI model was also trained on millions of public videos.
As a doctoral student at UC Berkeley, Pathak developed a way to instill “artificial curiosity” in robots by rewarding the system for producing results that come when it cannot predict the results of its actions. “The more uncertain the agent is about Predicting the effect of its actions, the more curious it becomes to explore,” he explained. The technique prompted the AI to navigate more scenarios and collect more data.
His research on curiosity-driven learning was published in 2017 and has been cited more than 4,000 times, he said. Pathak also devised a way for robots to use information written from large language patterns such as GPT (how to open a can of milk, for example) and convert it into actions.
“In 2022 we found a way to bring these things together into a single coherent system,” Pathak said.. “The notion of learning from videos, learning from curiosity, learning from real data, but combined with learning from simulation.”
Skild AI’s basic model, trained with 1,000 times more data than its rivals, can be fed into a variety of robots, enabling them to do things like climb steep slopes, walk over objects that get in the way her and the identification and receipt of items.
Skild AI
Skild AI faces stiff competition from a string of robotics companies that have emerged with billions of dollars in venture funding thanks to the AI boom. Industry behemoth OpenAI recently revived its robotics team to supply models to robotics companies. Forbes reported for the first time. Then there are outfits like the humanoid robotics company Figure AI, run by billionaire CEO Brett Adcock, and Covariant, an OpenAI spin-off that is building ChatGPT for bots and has raised over $200 million to do so.
Co-founder Gupta claims that Skild AI’s access to large amounts of data separates it from others in the space, but declined to reveal exactly how much data its model was trained on.
Ken Goldberg, a professor of robotics and automation at UC Berkeley agrees that data is key to scaling robotics, but robots require a specific type of data that is not widely available online. Plus, using the data gathered from the simulation doesn’t always translate to the real world.
“The whole idea that robotics is excited about right now is that we can do something analogous to big language models and big vision language models, both of which have internet-scale data accessible where you have billions examples,” he said. It’s not a straightforward task for robotics, but Skild AI aims to address the issue by combining all data-gathering techniques with more information derived from simulations.
Pathak and Gupta envision a future for their company that is similar to OpenAI, where different use cases and products can be built on top of Skild’s core model by fine-tuning it. “This is exactly how we intend to disrupt the robotics industry,” Gupta said, adding that eventually they want to achieve general artificial intelligence (a hypothetical AI system that can rival or surpass human capabilities) for robots, but one that people can interact with in the physical world.
“A GPT-3 moment is coming to the world of robotics,” said Stephanie Zhan, a partner at Sequoia Capital and an existing investor in Skild AI. “It will spark a monumental shift that brings advances similar to what we’ve seen in the world of digital intelligence, in the physical world.”
MORE FROM FORBES