How Nvidia AI Robot Trained 42 Years In 32 Hours And Did THIS | Google DeepMind AlphaCode

Key Takeaways

Nvidia an American chipmaker just Advanced robotics 42 years in the span of 32 minutes and the secret technique it used was detailed in a 90s skid show called Dragon Ball Z one of the biggest stumbling blocks in robotics was designing something as effective as a human hand human hands evolved over millions of years to be able to grasp manipulate objects and craft fine tools trying to replicate the same ability in robots has proven futile until very recently before some used claws or grippers but those had limited applications a hand with fingers has more joints that must move in very specific and coordinated ways to be useful training a robot to have generalized fine motor skills with unlimited degrees of motion has been extremely difficult one approach that has been very effective for training AI to complete certain tasks has been called Deep reinforcement learning or deep RL for short remember this this will be important later reinforcement learning is a technique where an AI learns from trial and error and is rewarded for the successful completion of the assigned tasks the AI learns by taking actions in its environment and receiving Rewards or punishments for those actions over time the AI agent learns which actions leads to the most reward and therefore the best outcomes the AI evolves its abilities similar to how organic life evolved its abilities over time in fact machine learning uses terms like Generations species and genome to describe how different AI models improve over time machine learning can be thought of being what evolution and survival the fittest was for organic life but applied to artificial intelligence just like organic evolution takes millions or billions of years machine learning can take millions or billions of virtual simulations to learn from this is right now the biggest limitation for training robots using reinforcement learning they are expensive to build and can take many Real World hours to train the amount of money and time it takes has limited how fast robotics can advance however machine learning in Virtual Worlds has been accelerating very very fast neural networks or neural Nets is an approach to teaching computers that is Loosely modeled after the human brain Alpha zero the Google chess AI was tasked to play games against itself and improve its abilities over the course of nine hours the chess version of the program played 44 million games against itself after two hours it began performing better than human players after four it was beating the best chest engine in the world openai created a simple game where one AI competes with another in a game of Hide and Seek in this environment AI agents play a team-based hide and seek game hiders The Blue Team are tasked with avoiding line of sight from The Seekers the red team and Seekers are tasked with keeping vision of the hiders there are objects scattered throughout the environment that hiders and Seekers can grab and lock into place as well as randomly generated immovable rooms and walls that agents must learn to navigate before the game begins hiders are given a preparation phase where Seekers are immobilized to give hiders a chance to run away or change their environment it's important to understand that the AI is not given hints such as incentives for moving around objects they start learning at zero their first run through might be almost random Behavior as the eye learns even the most basic of behavior like moving around as the AI runs through tens of millions of games some glimmers of intelligent Behavior can be seen The Seekers learn to chase the hiders the hiders learn to run from The Seekers between 20 million and 80 million games The Seekers learn to construct shelters to hide it around 100 million games Seekers learn to use ramps by 200 million games hiders learn to lock the ramps to prevent Seekers from using them bite 300 million games Seekers learn a skill that we can call box surfing that helps them get over the forge that the hiders have built by 40 million games heidis learned to disable everything that The Seekers can use to their advantage as the numbers of games scale into the billions the AI begins to test the limits of the physics engine and even find certain exploits that the developers did not intend these AI agents are also tested for their ability to do other unrelated tasks that test their memory ability to solve problems and predict their environment as you can see running these simulation billions of times creates a sort of Mastery in these digital AI agents unfortunately running 8 billion scenarios or using a billion robots to run a scenario and the real world is not realistic energy is expensive robotic arms can cost hundreds of thousands of dollars and are subject to wear and tear and breaking from overuse the amount of money time and resources needed to train real robots made of real atoms is astonishingly expensive and this is where we come back to Nvidia and the complexity of reproducing a human hand say hello to Nvidia ISAC robot simulator which enables robots to be trained inside a simulated universe that can run more than 10 000 times faster than the real world and yet a base the same laws of physics if you are familiar with the kids cartoon Dragon Ball Z which start it in the 90s they had a similar idea the hyperbolic Time Chamber is a mysterious time compression chamber where one year is just a day on the outside the characters used this chamber to train themselves at an accelerated pace and now Nvidia has a similar idea for training a new Army of robot using the Nvidia ISAC gym a reinforcement learning simulation for robots Nvidia was able to train a human-like robot hand to handle and rotate a cube in one hand the neural network brain learned to do this entirely in the simulation before being transplanted to control a robot in the real world similar work has only been shown once before by researchers at openai their work required a far more sophisticated and expensive robot hand a cube tricked out with precise motion control sensors and a super Computing cluster of hundreds of computers to train the hardware used by Nvidia Project was chosen to be as simple and inexpensive as possible training a good policy takes about 32 hours on the system equivalent to 42 years of a single robots experience in the real world [Music] foreign [Music]