AI Robot's ChatGPT moment at 2024?

Key Takeaways

2023 has been the year of large larage model where we see multiple different breakthroughs at a weekly basis it really created paradigm shift that AI can also do creative Innovative work rather than just repetitive boring tasks and with all those developments as we be able to see the same level of takeoff in robots and in body a the numerous smartest people in the field are betting big on robotics for 2024 Jim fan who is a senior AI researcher from Nidia predicting we are three years away from the GPT moment for physical AI agents as in 2023 their huge amount of foundation and platforms has been built to enable robotic development from multimodel larger model like GPT 4V that can work with normal robot arms and we start seeing algorithm that can Bridge gap between system one and system 2 syncing for robotic Ai and also insane amount of progress on on robot hardware and Brad at Cook who is the CEO of leading robotic company called Figure also post a tweet a couple days ago where he said last 90 days I'm watching robots perform complex task entirely on neuronet that he didn't think was feasible until the end of decade the beginning of 2022 he believed that we have reliable humanoid Hardware way before we can have reliable AI they can control the robot to complete household tasks but now that thinking has really changed he feels strongly that we likely will have reliable Ai and hardware at the same time and 2024 will be the year of embodi AI on the other hand I'm sure you're all seen this impressive demo from mobile Aloha where they built a impressive mobile robot Hardware with less than 30k which can already complete numerous household tasks like washing clothes cooking egg and shrimp although most of those tasks are actually Clon by human rather than done autom by the robot but we also saw some more autom example from leading AI robot company like figure where robot can actually use coffee machine to make coffee autonomously with voice control as well as Tesla robot where showcase this impressive hand coordination movement so this start got me thinking what has been the key breakthrough in the robotics AI past few months and how far we actually are from Deployable human robots I did a bunch of different research and would love to share this findings with you so let's get it we can break down the robotic AI into two buckets one is physical intelligence including how the robot can do the hand coordination to pick up items or ability to balance and working to certain destination and the other is cognitive intelligence how can robots reasoning and making decisions based on the environment and tasks and both aspects got loads of breakthrough in the past few years especially in terms of generalization which is how can we have general purpose robots that can do multiple different tasks instead of just specialized in one specific task CU at the moment robots are great Specialists but poor generalist typically you have to train a model for each task and environment even just changing one variable like the background or environment can require start from scratch training but as we know the real world environment is always Dynamic and continuous changing and it's incredible and inefficient for us to pre-program all the different edge cases and this paradigm shift from specialist to journalist has been majorly driven by the dment of Transformers and large L model let's take cognitive intelligence as example there were plenty of developments in AI agents that showcase powerful decision making and planning ability for example the research paper called Voyer where they build an embod agent in the digital world Minecraft the agent can continuously explore the world learn new skills and accomplish task at even superhuman pace and all those cognitive ability developed through those digital AI agents can later be transferable as we develop physical AI agents and on the other side we already see company like Boston Dynamic start equipping their robot dog spot with large language model for better cognitive ability and deliver new experience for the end users may I have the pleasure of knowing your names I'm Matt and that's v a pleasure to meet you Matt and vacher shall we commence our journey the charging stations where spot robots rest and recharge is our first point of Interest follow me Gentlemen let's proceed to the Rock P similarly we see lots of breakthrough in terms of robotic control which is how can the robot learns to complete multiple different physical tasks back 2018 the state of art of robotic learning is called QT opt which method that use reinforced learning to learn how to do a specific task well it was quite efficient compared with the previous method but still with this method we B to train new model for a different task which is not efficient but couple years later April 2021 a research paper called Mt op were introduced which is paradigm shift from single to multitask learning so mtop represents multitask robotic reinforced learning Mt Ops Revolution this by enabling single robot to learn an excute multiple task by leveraging Shar learning principles this means instead of learning from scratch for every single new tasks the robot can apply Knowledge from one task to another much like how a chef using skills from making pastry to also Bak bread and this shared learning not only making the training more data and time efficient it also result in robot that can adapt to variety of tasks in Dynamic environment so it was huge step forward but still quite Limited in many ways and real breakthrough happening at December 2022 with where Google introduced rt1 so rt1 adopt Transformer architecture it took a nicing both inputs and outputs which basically transforming the camera image the task instruction it is was given and also the Mortal commands into a language that robot AI can't understand this represents a huge leap in the robotic learning proping us towards highly generalized robotic intelligence because now the robots understanding of the word and its task become deeply integrated with language meing how human prec reive and interact with their environment this allows robots to not just perform tasks that they were directly trained on but to generalize an SQ Tas that they were never seen before it's like giving a robot the ability to read a recipe book and cook meal is never cooked before and a result is that this type of general purpose approach I'll perform all the previous methods in both specialized tasks as well as unseen tasks and this is very similar to the phenomenon that we saw from large language model that a general purpose large language model that train on a diverse and vast data sets can outperform specialized AI model on task that they were specifically train on in 6 months later on July 2023 building on rt1 rt2 was introduced rt2 combine a visual language model that is pre-trained on extensive web scale internet data like videos and image with rt1 which original trained on specifically robotic data and this give artitude a Nuance understanding of visual cues and natural language far beyond is original robotic training data set and with rt2 robots now have cognitive ability to interpret complex commands and perform sematic reasonings they can identify different objects and even use some of the objects as tools to complete task with uning objects Dynamic backgrounds and even Dynamic environments and again at the heart of rt2 is the transformation of actions into tokenized strength which align robotic control with model's linguistic capabilities this means actions are described in a language that robot can understand enabling to adapt actions as easily as we form sentence and the result is that the robot can not only Excel a task that it's been originally trained on but also generalized to new unseen environment with remarkable accuracy r2's real world application showcase a successful rate leap from rt1 32% to 62% making a significant step towards truly intelligent adaptable robotic AI but not only that just 2 months after rt2 was introduced at October 2023 Google announced a massive robotic training data set called open X embodiment data set it is collaboration across 20 different institution to get data from 22 different robot embodiments demonstrating more than 500 skills and and 150,000 task across more than 1 million episodes and as contrast rt1 was trained on only 700 tasks so this is a massive training data set and more importantly it is C from 22 different robot embodiments and then Google introduced RTX where between rt2 and rt1 model with this massive new diverse data set and by just having more diverse and bigger data sets outperform rt2 by 300% in emerging skill evaluation it is crazy that those level of improvement happen just within two months time and most importantly it is following the scaling law which means with bigger more diverse data set you get those Improvement in term of performance for free without any fundamental change this showcasing how important training data is for robotic AI progress and just as I'm making this video Google just released a few new announcements for robot AI one of them is called Auto RT which is a method that can potentially generate huge amount of training data from real word it basically use visual language model and large language model to Ure different robots to continuously complete different tasks and use them as shared training data it is still pretty early but can be huge if it does generate huge amount of training data last but not list even though development of Transformers and large larage model made huge progress on both cognitive intelligence as well as midlevel physical intelligence it often fell short on real low-level dextrous skills like hand manipulation with Q4 2023 there also research advancement that showcase potential for large Lang model like gbd4 to design reward function to train robot lowlevel textural skills at superhuman level it basically used large larage model like GPD 4 to generate reward function to train robot in reinforced learning and reward function is a core part of reinforced learning where it will tell the agent and robot whether they're doing things right or wrong it's pretty similar to how you train a dog to give it a little tra when it does something right so they get GPD 4 to initially design a reward function for certain actions and then they will simulate this robot actions for thousand times and send better results in real time to lar L model as feedback so that it can iterate the reward function and the result is stunning it is able to outperform expert human engineer on designing those reward function 83% of their Benchmark with average Improvement of 52% as this is the first time feel like we are finally breaking morx Paradox it is concept introduced 30 Years Ago by H marvik who was a leading robotic scientist his finding is basically it is comparatively easy to make computers achieve addult level performance on intelligent tasks or playe Checkers but much more difficult or almost impossible to give them a skills of one year old when it comes to perception and mobility in short it basic means problems that feels very easy to human are actually very hard for computers like planning recognizing someone walking or grapping something the problem that feels really hard to human are actually easy for computers like math and Logics and the theory behind this Paradox is that easy problem like walking running hand manipulation it is something that has been deved by human hundreds of thousands years ago they almost became part of intuition and very very hard to translate bad to computers but hard problems like math and Logics are skills that has been recently developed by human and fairly straightforward to translate to computer but with the recent development finally it feels like we are on the path to get away from this Paradox that has been cursed robotic industry for past 30 years so those are key breakthrough from the past few months and with this pace of accelerated development it let me feel really optimistic that robot can possibly have the charb moment in the next 12 to 24 months and many leading robotic company like figure Tesla ectronic or have plan to start deploying robots in Real World scenario like manufacturing or Logistics and once these robots get deployed in real world scenario we will start collecting huge amount of training data they can again speed up the learning curve of robots so it is truly very exciting time so when do you think will be the chat GPT moment for robots am I way too optimistic or too conservative please comment below and let me know I hope you enjoy this video I continue posting interesting purchase and findings in AI so please subscribe if you want to keep updated thank you and I see you next time