Let's Hack my Tesla with Javascript LIVE

Key Takeaways

[Music] i think it's working i think we're live um hello world it's siraj okay if we are live that's great okay we're live great um hold on a second i want to show you this demo and uh what you are seeing is essentially a demo of me using javascript to hack into my new tesla and this in this live stream where i'm going to show you exactly how i did that i'm going to show you how i use javascript to create a web app that can retrieve all of this data from my tesla and then we can even modify some data and what specifically what we can modify is we can modify the climate and we're going to use reinforcement learning from scratch just pure javascript this is so simple guys i want to show you how simple it is to hack into this computer car essentially it's a computer with wheels and in this live stream i'm going to show you exactly how i did that all the code the code is going to be in the video description by the way i'm just going to rewrite that code but we're going to talk about reinforcement learning in detail and three people from this live stream are each going to win 175 dollars in bitcoin by the end of the stream if they answer all three of my questions i'm going to pick three of the people who answered all three questions correctly all right so i hope you're ready for this now what are we going to talk about today we're going to talk about autopilot we're going to talk about autonomy in this tesla vehicle what does autonomy look like well autonomy in terms of tesla looks very different from other vehicles it's something that you have to get used to right like i wasn't really even even though i've been into deep learning for a while it kind of was jarring to try autopilot for the first time um in a few minutes i'm going to show you a montage of me trying out all those autopilot features it's it's a three-minute montage but the tesla has a bunch of cameras on it it's got two cameras on either side it's got a camera on top it's got a camera behind it it's got a radar so it can detect proximity to other vehicles um and it's basically a computer all of these sensors are are connected as input output devices to an onboard computer and what else is connected to this computer everything every component in the car is driven by software from the ground up from the air conditioning to the powertrain to the battery everything is driven by software which is why we love this car right everything is driven by software as it should be now that's the first part of tesla all of the components what's the next part well there's one component in particular that i really like and that component is called the full self-driving computer the fsd and what you're seeing here is a diagram of the fsd and of course you're going to see some things that you'll find in all sorts of computers you'll see a cpu for you know single processes and you know general really basic process management and then you'll see a gpu and the gpu is for parallel processing now what do we use gpus for why would we need a parallel processor we need it for deep learning right you know this i hope you know this by now we need it for deep learning and now why does the tesla use deep learning it uses deep learning to see where it's at to interpret the world around it and to make the best actions to optimize its objective which is to keep you alive and get you from point a to point b and what you're seeing in this diagram is an npu you see two of these npus and an npu stands for a neural processing unit it's a specific type of processor used specifically for the matrix operations that neural networks require now why did they built a specific processor for this why didn't they just use a gpu like nvidia or a cpu or even a tpu like google uses well because tesla has a very specialized ai it does one thing really well and that is to get you from point a to point b and that means that a lot of the matrix operations that are going to be happening are going to be the same matrix operations in a row over and over and over again in terms of convolutional networks what do those matrix operations look like they look like add multiply right so there's like the same matrix operation convolution pooling right the same operations that are just repeated over and over and over and over again given some input image or a frame of images in the in the form of a video right so that's why we have an npu because neural network processing is constantly happening on the tesla it is constantly using a neural network to perform inference and tell you where you are and it needs to do this really fast because lives are at stake right your life is at stake you cannot mess this up so the closer to the metal you make this processing happen the faster it's going to be so it's all about speed and efficiency so you know we could do all that neural processing on a cpu or a gpu or any level any number of levels of virtual levels above the metal but the closer and closer we go to the metal in terms of running these matrix math operations the faster it's going to be which is why there's an npu make sense let's keep going so that's the npu and you know overall this system is essentially a multi-threaded operating system it's an operating system guys like i said it's a computer with its own operating system and what do operating systems have they have process management right it is looking to see which processes are running which processes should be killed which processes are interacting and it's trying to prevent any kind of thread mismanagement like what did we talk about in our rust live stream we talked about the importance of thread safety we talked about the importance of memory safety to make sure that threads know not to access the same memory location in order to prevent something like a deadlock which is a system crash we don't want that so how good is tesla's multi-threaded architecture it's pretty good it's not perfect and i will show you how it's not perfect one example that i found was i ran an easter egg on the car which was to play a simple song uh it was called ho ho ho mode like santa mode and then i try to turn it off it would not turn off you know so that was not gracefully killing that process like i wanted to i tried to play another song and i didn't know which song to play so again thread safety and memory safety very important is tesla using rust for this no they're using c plus plus how do i know this i was just googling it they're using a bunch of c plus plus of course they are andre carpathi is the lead of tesla ai who wrote the original greatest ai blog post of all time the unreasonable effectiveness of recurrent neural networks go read that if you never have yet that's the idea behind the tesla computer so now what are we going to do i have my first question for you before we get started with node but before i ask you this first question i have to show you what autopilot looks like so are you ready are you ready to see some autopilot i have this three-minute montage that i made of me trying out all the different autopilot features what i want you to do is to watch this montage and pay attention because right after it's done i'm going to ask you the first question of this stream and that first question is going to have to do with autopilot so i hope you're listening here we go enjoy autopilot montage in three two one here we go so this is basically the greatest car of all time like without a dowel i just love this car i am in this car just sometimes just sitting in here just chilling you know you can even watch netflix or something on in this car and uh it's awesome let's take a look at the autopilot so if we go to autopilot you can see that there's auto steer so that's for city streets uh navigate on autopilot is for highway on-ramp and highway off-ramp automatically you can customize navigate on autopilot as you see i have it on mad max children do not try this at home mad max for speed based lane changes uh i guess i'm sort of an auto speed demon but you'll see lane change notifications here it requires uh lanes change confirmations i put yes to require so if you put no this is like all right so it's just going to automatically change lanes and it's not even going to notify you it's just going to do it are you sure you want to do that you crazy person and we're gonna put yes we put yes because we just want the full autonomy actually that's a little too dangerous so we're gonna put yes required um and so that's how that goes that's autopilot you have summon as well summon is amazing and it just comes up to you lane departure avoidance is awesome i just avoids the car departing lanes and yeah that's oh and then there's this easter egg of course romance mode you ready for this hold on [Music] yeah ho ho it's christmas autopilot engaged autopilot engaged i am not even driving right now my hands are not on the wheel right now autopilot is engaged nothing nothing my feet are not even on the wheel you're my feet we're coming up to a stop and my feet are as you see up here they're not even here we are coming up to a stop nothing is on the wheel it's totally autonomous it's going up this bridge it's going up at an incline and their car is coming at us it's detecting everything with a convolutional network it knows what the lanes where the lanes are it's it knows what cars are behind us what cars are in front of us what cars are to the side the distance between cars it's constantly computing these things with its amazing processors onboard computing units it's amazing i love this car yeah yeah all right so so so so so so what do we have here i hope you liked that little montage now i've got the first question for you related to autopilot so i hope you're ready for the first question let's administer this question live the first question let me pull it up here for you okay we're gonna go live in a second um q a and begin accepting answers now okay here we go with the first question which of these is not a component of tesla's autopilot system is it a the neural processing unit is it b a convolutional neural network is it c multi-threaded programming or is it d k means clustering now all of these are related to machine learning but one of them is not a component of tesla's autopilot system if you're paying attention to the montage you know exactly which what the answer is and it looks like everybody knows most people have an exact idea about what the answer is and uh so you know while you guys answer that let me answer some questions here bernab desai has a question what os does it run on great question pranav i it's got to be unix-based it's definitely unix-based for sure because you can there you can run shell commands so easily on this i think it's a custom version of unix related title says i'm very grateful because it has allowed me to fund my project that i'm working on i think that he might be referring to the prize money uh from last week i think he won like 500 bucks see that's what i like to see guys i i want to see you guys gain opportunities and employment and you know all sorts of things from these games that we're playing and um that's exac absolutely what's going to happen with code royale when it's released but i just can't wait for that to be released we're going to do it even before that we're going to give out cash prizes and play games even before code royale has finished developing because you know someone once told me that if you have a job that you want to do just start doing it before you even have it just like if you have a job that you envision yourself doing just do it before you even have the job and that's what these past eight nine weeks have been let's now see what the answer is it seems like almost every single person knows exactly what the answer is all right so i'm gonna give you guys five more seconds five four three two one all right let's see what the answer was guys finish and reveal the answer was d k means clustering i hope you got that one right that was the easy one for you guys um the next ones are going to be harder so make sure to uh keep that in mind all right now we're going to keep going so the next thing we're going to do is we're going to actually build this thing in node so who's ready to build some uh hacks with tesla in node.js i'm super excited for this all right so here we go with node.js we're going to create a custom web app that will log into this tesla and we're going to run reinforcement learning on this thing because i want to show you exactly how that works why it's so important and what all the utility is of reinforcement learning in this context i want to show you a real use case of reinforcement learning all right because it's a real technology it's not just you know for show so what do i have in terms of a demo to show you here in terms of a demo i have an ubuntu browser that will show you what the output is so before i actually build it i need to show you what the output is going to be and in this case the output is going to be the climate state of our vehicle let me just transition that all right so what you're seeing is the ubuntu terminal that shows the output of of my web app so let's let me just show you so after a bunch of tries after a bunch of errors and such we logged in successfully so what you're saying here is me running that code that's in the video description that you know i hacked together so what you're gonna see here is let me go up go go where was it it was just working so i got to show you that stack trace all the way up here we go so what are you seeing here what the car does is it first logs in what the web app does i should say the web app first logs into the card just using oauth 2. just simple username password just like a server like i said guys this is a computer once you've logged into your car it's going to notify you what its vehicle identification number is and its status whether it's online or offline then you can make any number of api calls to see what its current charge level is after you do that you can make any number of api calls to see what the climate is is it on or off that's a boolean value then you can make an api call to see what the interior temperature is and the exterior temperature so both and then once you have pulled all that data from the vehicle then you can run reinforcement learning and that's what you're seeing here is that it's like yo plus i'm printing out some random value from the training loop of a deep q learning agent over and over again so i've taken both of these two values the interior and the exterior climate values i've created a custom reinforcement learning environment in javascript and i've told this agent to optimize this environment such that the interior of the car is exactly the same temperature as the exterior now your first question might be why would you do that why wouldn't you just manually set the interior to be equal to the exterior and that's because we could do that but this is more about a simple simple the world's most simple reinforcement learning problem to show you how you could use this in your day-to-day life like i said it's not perfect it does work it's not perfect though like you know tesla has a built-in climate system that's probably using ai but we are essentially going to recreate that in javascript ourselves now how are we going to even do that like yes we can you know use node.js that's fine and dandy but you know it's it's about more than just node.js right it's also about the data right any kind of ai starts with the data so in this case what does the data look like for tesla like what does the api look like so what i'm going to do is i'm going to show you the tesla api now the tesla api is never officially acknowledged by the company believe it or not some dude made this this this website right here tesla does not own this this website it's all developers so um so what are we gonna do this is the tesla api every single tesla vehicle has a set of api endpoints you can log into your tesla using the remote tesla server so it's kind of like a server client architecture right overall it's a distributed system it's not decentralized there is a master server that we could think of and all cars are pulling data from that server so if we can log into the server we can log into our cars by the vehicle identification number and once we're in the car then we can see all of the api endpoints and you might be thinking wait why hasn't tesla ever acknowledged their api i mean if you look at all of the endpoints like let's say let's see state and set it settings for energy sites what this is going to do is it's going to show us parameters and links for all of the tesla api endpoints this is going to give us live status site data so if we go to owner dash api dot tesla motors dot com slash api slash one slash energy site slash the site id slash live status it's gonna give us um what some status is going to be or historical data you know it gives us all this data from the car just insane amounts of data um which is crazy diagnostics log data i mean there's so much here that we could pull from the car um like i said it's just a server it's a computer it's our car has an api and we can do anything we want with that api so how are we going to access this api well i found this amazing library called tesla js and tesla js was the unofficial is the unofficial um library to interact with the tesla with the tesla and it basically wraps that api into a bunch of javascript functions these are all asynchronous callbacks and we're going to build that right now but you can see a very very simple example of it right here so in this 10 line example you require the library you literally just set your username and password this is how dead simple this is guys like you might be you know i thought this would be a lot harder you know like we gotta hack into the firmware we gotta you know you know i was excited to do some serious hacking and like gain uh you know admin access over the system but nope you can just literally log in with your username and password and then you've got this callback right so this is an asynchronous callback it's a design pattern used especially in node.js to retrieve data when making a call asynchronously which means um you know not sequentially just simultaneously using that username and password to retrieve an authentication token just like any other oauth 2 app and then it's going to give us a notice like hey your login was successful you logged in successfully good job so let's try that what do you see let's just try this guy says it's super easy so let's do it let's just copy and paste this 10 line snippet make our own node.js web app and let's see if it's really that easy to log into our tesla so let me now open up ubuntu and we'll do that so in ubuntu we'll go all the way down and we'll open up we'll open up um well i already have it open up actually it's called main.js so i'm going to point us to main.js in the code editor and then we're going to start coding in javascript who's ready to code in javascript javascript is live i love javascript the language of the web all right we're gonna start coding some javascript and then um i'm gonna ask you question two so be prepared for question two it's about to start so like we saw before what did this guy do he just imported his library his library was called um what was it called it was called um let me make this super big because javascript is meant to be read really big we have var tesla js equals equals require not esquire require tesla js that's it we imported the library simple right um what else do we need well we need an http library that's going to help us access the web right http what other um library do we need we need that's it actually let's just start with that simple stuff okay so we'll start with that and then once we have that simple simple library we will create our basic node app so node has this design pattern where we have this this is you know the universal function and node of creating a server and a server takes a request and it's going to give us back a response and after it's done that the response is going to return something what's the response going to look like well this response is going to look like an html html that it's going to that that's the start of it it's going to start off as html and the content type is going to be text just plain old text you know how it is and after that now we can log in with that let's paste that snippet in boom let me make this smaller so you guys can see everything okay is everything in there everything's in the screen yes okay look at that so simple oh my god way too simple now what is the email i'll just put xxx whatever for now i'll switch this later and then we'll get to question two once this runs we will get to question two because this is an example of connecting to the car and once we've connected to the car we are going to do something to the car what else are we gonna do once we're in the car let's not only connect to the car let's um let's also once we've connected to the car we're going to use that token right we're going to use the token that login was successful to run another callback so this is called this is called callback hell this is one of the things that happens with node.js it's nothing new there's a solution to callback hell callback hell is when your code looks really fat it's just a bunch of asynchronous callbacks that are nested one inside the other really easy to do for like a hacky prototype you don't want to do that for anything in production the way to prevent that is to use what's called promises in javascript promises help prevent callback hell and make your code more vertical so it's more readable you don't want unreadable code right but that we don't care about that right now we're just going to try to hack this together so we have our token we've logged in successfully now we'll use that token to uh get the vehicle right so we're going to use that token under options using off token as a parameter to get the vehicle vin and the state of the vehicle and we will see if this works now so let's save that and open up ubuntu and we will see if that compiles and runs in ubuntu so that file was called main.js so nodemain.js it's like no you we didn't expect that token on line seven right so on line seven i wanna make sure that that related title is asking why am i not using express yeah i could use express but um because because we are going to i'll tell you why because we're going to import this reinforcement learning library that requires um a state an import statement which is es6 syntax and express is one way to use that babble is the other way we're gonna use babel instead of express we'll get to that um tesla js http what am i missing here okay content type plane 200 what am i oh gotcha um okay i think i got it um let me make sure that this is visible while i fix this code all right all right all right all right all right i really want to fix this code have you guys see what i'm doing here um what was it it was like response dot write head 200 content type text plain close okay that should work so now if we compile that you can see it's compiling um nodemain.js good but we need our my username and password which i'm not going to type in live so some random person hacks into my car and makes it go all over the place don't do that um i will find you like liam neeson you know that i love that meme you know anyway so um that was an easy part let's get to question two we did something really really really basic here which brings us to question two let's bring up question two which is related to node.js so what is question two question two i pull my face up and pull the question up as well um question two let me pull up question two for us question two is okay so what's the difference between machine learning and deep learning i've had plenty of videos on both of these topics but at this point i kind of expect you to know the difference is it a machine learning is a subset of deep learning is it b deep learning is a subset of machine learning is it c deep learning is not related to machine learning or is it d deep learning is the same as machine learning those are your options and while you answer those i'm going to answer your questions got some good amount of questions here here we go reclaimer117 hey john117 from halo i hope that's a reference my favorite game of all time which architecture are these processors are they arm based mobile chipsets or x86 they're arm based um well some of them are the cpu is armbase i'm not sure what the npu is evas asks how do i manage time between school very hard and coding so it's been a while since i've been in school 10 years and i don't miss it at all basically i have my own school that i've created which is my life and i want the world to understand that it's possible to learn how to out in the open without some administrator or bureaucracy telling you how you should how you can and cannot learn learning is not just restricted to the university environment or a school environment or any environment that other people tell you learning can happen while you're dancing learning can happen in a tesla while you're driving learning can happen anywhere while you're having fun during this stream so how do you manage that you need to follow your curiosity that's what i found like if you follow your curiosity it'll feel like you'll be in your flow state and like eight hours will pass during the day and you'll be like wait i thought it was daylight and you just coded like a bunch and that's what happens when you truly are curious about what you're coding and you can't force that you know you can't force that so you really want to align what you're learning with what you're interested in what you're curious about um e1lg asks why is batch normalization so popular in computer vision because it's all about speed right batch normalization helps uh you know batch processing first of all is one technique to make training faster for these for these neural networks and batch normalization essentially well you're normalizing the data in batches so that it's all going to be on the same level playing field that it's all going to be the same scalar values all on the same scale and if you want more accurate predictions you want to normalize your data that's like one of the you know key steps of data science so that's why batch normalization is popular um victor campos says is tesla js javascript package safe can i put the password there without a leak risk is the code open and secure um so i've been using it for a few days nobody's hacked into my car um nobody would dare no i'm just kidding um is it safe and secure it's oauth 2. it's the same protocol you would use for some um what popular site is using oauth 2 right now i think facebook just switched to like something else but almost every major site uses oauth2 so the answer is yes as long as you keep your password secure don't upload it to your github you know make sure that your password is not in your code before you upload it um and then you'll be just fine all right so let's see what the answer was every single person knew the answer to this question except for one brave person who thought they knew that it was different let's see what the answer is the answer is that deep learning is a subset of machine learning you guys are making me proud by knowing the answer that question good job you get a sound effect congratulations for getting question two correct and we're gonna move on to question three but we got some code to do some reinforcement learning so let's move this out of the way and get to the code so back to the code so we just built this simple tesla js web app it connects to our car it gets that climate data um it gets the vehicle identification number now we have that climate data we want to do some reinforcement learning with it right how do we think about this problem right how do we think about a reinforcement learning problem we have an agent and an agent is going to perform an action in a simulated environment and it's going to transition from one state to the next state depending on what its action is when it performs its action it's going to move on to the next state via what's called a transition probability matrix that defines the likelihood that it's going to move from one state to the next state and one when it moves from one state to another it's going to get a reward that reward will tell it whether or not it's successfully completing its objective so let me just before we start coding let me just 30 seconds explain exactly the architecture of this reinforcement learning agent that i've set up we connected the car we get all that data we're retrieving just a bunch of data as a nicely formatted json file in that json we care about two values interior climate and exterior climate of the car we want those two values to be the exact same values we could just manually switch the interior to the exterior but that's no fun we want to continually optimize the interior until it's the same as the exterior because that will save us some energy because we want to keep the car on as opposed to off and make sure that it's not too hot or too clogged but it's cold but exactly that temperature in order to optimize that we're going to take both of these values we're going to feed it to a reinforcement learning environment so we'll create a custom environment we'll create a neural network and that neural network will will give it an objective the objective of the neural network is to minimize the difference between the interior and the exterior it will optimize that using calculus back propagation and in this training loop this is all happening in a simulated environment this is all happening on my laptop right so we're just getting the data from the car as like a data stream every you know x number of intervals let's say every five seconds we're just getting that climate data and then on the computer we are optimizing this using a deep q learning agent to make sure that any input and output data the agent knows how to act the action in this case is plus one temperature or minus one temperature and the reward it's gonna get is only if that difference is smaller than before the last training loop does that make sense so we want to continually minimize the difference between the interior and the exterior temperature using deep q learning which means we'll take a neural network we'll try to learn the q matrix which is the which is a matrix that tells us the value of every action that ever every state that we could take given an action um and if you want to learn more about deep q learning literally i promise you the best video on youtube for this i'm very confident about this is my video just search deep q suraj there's this awesome video on it you'll learn everything in nine minutes but that's the idea so let's get to the code and then we'll get to question three so um question three is not yet here's the code here's the code here's the code so there's our editor and now we're going to look at the code so where is the code at the code is here okay so what are we going to do with this code now so now what we're going to do this code is we're going to add some reinforcement learning to this code so let's add some reinforcement learning tools what are we going to do there's so many libraries that we could use for reinforcement learning which one are we going to pick if you go to npm which is the node package manager all i did was i searched reinforcement learning and then i picked one reinforced.js was the most popular uh reinforcement library for javascript and the reason i love npm is because you don't have to do anything with like shell scripts and all these like little gluey things to get your dependencies to work it's all packaged that's the whole idea behind a package manager it nicely packages everything you need into one command that you can run on your uh terminal and then that's it so assume we've run that and now we're going to require that so this is what i you know related title asks why i wasn't using express this is why so import let me make this bigger import dqn solver dqn opt and dqn environment these are the three things that we need from the reinforced js library we have the environment the agent and then the um the options the parameters of that agent that we're going to define programmatically so this is from the reinforced js library now we can't just use an import statement normally with node that's why we need we need babble which we're going to install in a second but that's the idea that's how we import our reinforcement learning agent now assume we have our reinforcement learning agent installed once we have that we're going to create a custom function called reinforcement learning we'll put everything in here we'll put everything in this function reinforcement learning we'll put the environment creation in here we'll put the whole world in this function we got the whole world in this function we got the whole wide world in this function here we go function reinforcement learning you ready you ready you ready so we'll start with the width of the environment we'll start with the height of the environment um these are really arbitrary values but we'll just have four of these you know we could have a million but we're just gonna like randomly start with this like this is the beauty of machine learning we'll have to learn what the optimal parameters should be but we're gonna start off with some simple ones and now we can create our dqn solver this is the agent and the dqn solver is going to use an environment and a set of options now what is the environment instead of options we haven't defined those things we'll have to define both of them so let's do both we'll invite we'll define our environment we'll say var env equals new dqn these are pre-loaded functions now we can create all sorts of environments um you guys have some amazing questions i'm i can't wait to answer your questions in a second um man i love you guys i love this don't you guys love this i love doing this every week it's so much fun um with height number of states number of actions that's it number says number of actions that's our environment and we have our environment and now we're going to create this um option object which is just new dqn opt and that's it and then guys this is so simple like seriously i was so surprised at how simple it was to do this and i say that because i want you to do something similar i know not everybody has a tesla but you know you know i like to think of this as our car yes it's my car but i think of it as our community's new car and what i mean when i say that is that if you guys have any suggestions as to any kind of hack that i should do a video with the tesla um anything that you want me to do with it let me know and you know we're gonna take that into account and i'd gladly do a video on that to make you guys happy and do something cool all right so that those are our options we have a var option for our dqm options now what is the one more thing remaining we need the environment which oh no we already have that so that's end that's opt and then we have to um did we use okay we used all those things so that's ops so like you know option we could have any number of options here all i'll do is one option but you know you can see the original code that i wrote there's like a million options but in this case we'll just have one that's it okay so that's a solver and we're not done yet i'm gonna i have like 10 more lines to code and then i'm going to ask you the last question we have our environment we have our options let's build this training loop so in javascript we can create this learning loop manually as this function and inside of the training loop we will tell the solver to decide what the next best state is so this is using a neural network we didn't have to write out the network it's internally using one this is already built into the solver object that we instantiated earlier and uh this was that yo part where i was like console.log yo um plus the action that it decided to take and you might be thinking well wait where's state constant state is going to be an array um of size let's say 20 states it's going to collect all the states over time these are just single values you know whatever that value is and it's going to optimize that so we have yo plus action um and so this is a single um all it's doing is it's just taking an action over and over and over and over and over again depending on the state and what we want to do is we want to make every action smarter now how do we make every action smarter well what we want to do is to this reinforcement learning we want to pass into very like var um interior climate and exterior climate let's call it interior and exterior lowercase because we don't care interior exterior so these are both climate values and um var last action for reward now here is like my hacky hacky hacky code right here here we go i'm almost like kind of embarrassed to even share this but guys literally did this in like a day and like not a day a few days i don't i'm not gonna here we go so where does this come in we're gonna say if the action that was taken is if there's an actual difference between those two values then set the reward equal to the interior temperature now what why are we setting the reward equal to the interior temperature shouldn't the reward just be plus one or minus one here's the here's the truth guys i wanted a way to inject those temperatures into this like very very much pre-built library reinforced js and i didn't find an easy way to inject it into the environment so i'm just injecting it in the form of a reward so yes this really is a reinforcement learning agent that really is using real climate data from the car but is it using it in the optimal way that it should in that these climate values are injected into the state directly no but could it in like a few days absolutely um but that's the idea this is still like an example of a markov decision process of states actions rewards over time being changed as we uh you know move forward with this and i have one more line and then we're gonna get to the the last one the last line i have to write is dqn solver dot learn reward and that takes into the account the reward and it modifies the neural network weight values behind the hood um to be better and better and better so that um that reward is going to be that interior temperature is going to be much closer and closer and closer to the exterior temperature over time and we need to call that function as well so we're going to call that reinforcement learning function inside of this loop so inside of callback hell inside of callback hell we'll call the function using um inter and exterior oh we also need to get that so i need to add another callback hell and then well we'll be good so where's that climate callback hell climate retrieve climate data where is it where is it where is it hold on hold on i'm gonna get you guys this climate start give me a second i've got this climate js and then we're done samples climate start and then there it is here we go three two one boom call back hell okay so what just happened i just pasted in the climate start it uses the same option it tells us what the climate is and then we'll give that the climate climate start yeah it's going to return the climate and uh actually that's not the one i wanted that was like a different callback hell i want here's the one i want yes perfect here we go boom climate fan status inside temperature is going to be this so that's what i'll go say climate start inside temperature and outside temperature there we go watch this give me a million errors in a second but that's the basic idea the code works if you download this code you run mpm install all five of those dependencies this code will run even if you don't have a tesla um it will run you just gotta like create a tesla account and i wonder if it'll let you log in without a vehicle identification number it will run that's for sure um so that's that let me see what's next i have a third question for you let me pull up the third question question three time here we go i hope you're ready for question three i hope you were listening we're about to do question three here we go question three is i hope this one's hard for you guys which process is used to frame almost every reinforcement learning problem is it a no.js is it b policy gradients is it c markov decision process or is it d process management and uh you know okay good this one seems to be a little harder for people that's good i'm gonna make this harder guys you guys are getting too smart for me i like that this is good i want you guys to be really good at these questions you know um let me answer some questions and and then we're gonna announce the winner okay in three minutes let me answer every single question here so we've got some five really great questions soundkit asks what's the next thing after machine learning gets saturated soundkit i give you my word that machine learning is not going to get saturated it is going to continue to improve and every time there's a blocker maybe that there's not going to be an algorithmic advance there will be some computing advance in terms of processing there will be some data in advance some new data set is available there will be some new educational advance there's new people educating people on a new topic it's not going to get saturated this is the future it's an exponential technologies we want to focus on the exponential technologies of the future so we can exponentially bring forward a future that is more equitable and more beneficial for everybody in the world right fully automated future will you do a video on image segmentation um that could be a good tesla self-driving car video i would absolutely consider that michael asks why do i use swift for tensorflow or swift for machine learning because there are plenty of developers who talk about ios related technologies i did help develop the first ios app for meetup.com a few years ago um in new york city with michael gray but um we used objective-c back then swift is way too easy and also um you know what somebody said that i should do like um a video on like deep learning on the m1 on the neural processor for apple's new computer that i will absolutely absolutely do and in swift as well arnob asks i'm currently trying to make a simple convnet good and i'm trying to use my amd radeon pro gpu to enhance the speeds but it's not working any solutions solution number one sell your amd get the cheapest cheapest nvidia gpu you can find install cuda and it will all be better if you can install cuda install the docker image for it everything is search nvidia docker gpu download install you'll be running in no time three more questions unrugged asks hi siraj i'm 16 years old good and my paper is being published in scopus congratulations on deep learning thank you for teaching a bunch it wasn't a question all right you're welcome thank you anurag great to see that uh tiago rajan asks siraj can you share your thoughts about dqn dqn great initial algorithm it's what set deepmind apart from the competition it's what made deepmind get bought out by google for what it was what was it 500 million a few years ago great generalizable reinforcement learning algorithm it's already been surpassed surpassed by alpha fold zero um in every way so um not that great anymore still a great initial learning algorithm why does everyone think they call me ken says why does everyone think we'll get level 5 autonomy with cars because we will we will get level 5 autonomy with cars go ask george hotz go ask who i interviewed go search george hot siraj for that interview it's a super old interview go ask a bunch of other podcasters um it's gonna happen and go ask elon musk go ask tesla like this is their goal like this tesla will get literally smarter in like overnight with the software update you can get hardware updates from over the air software you can increase your acceleration from a software update your torque all that stuff incredible um thank you evas for the correction one more question and then we're done is it possible kd asks is it possible to extract the tesla autopilot neural network parameters after login and some hacking that is what my initial thought was um kd i haven't been able to do that easily you're definitely not going to be able to do that from any tesla api that's for sure that's going to require some kernel hacking i would absolutely be down to do that in a future video um and that would be harder and that would make the questions harder so that's a great idea let's see what the answer was it seems like everybody knew the answer to this question the answer to which process is used to frame almost every reinforcement learning problem is see a markov decision process i hope you got that one correctly because now it's time to find out who the winner was and now the winners the three winners we're gonna announce all three winners right now so here we go the winners are i'm now gonna look up the winners i'm gonna let you know who the winners are right now and then we are going to um i'm going to sing a freestyle wrap for the winners so just got to pull up that live challenge all right i just got to find who just answered all that all right end session view the results okay here we go i'm gonna view the results now let's see who got this okay let's see we have the winners are pranav this i winner number one pranav desai congratulations pranav winner number two ishan dubey so pranav dasai and ishan dubey winner number three let me just write their names that pranav desai ishan dubey and winner number three is nuns g-n-u-n-s like that g nuns got all three of them correctly g n u ns congratulations pranav desai ishan dubey and nuns all three of you win 175 dollars in bitcoin in the next hour email me hello at siraj raval with your bitcoin address and a screenshot of your win if you don't have a bitcoin address that's fine i'll just let me know but email me within the next hour and i'll get that money to you within 24 hours congratulations guys so proud of you for winning that if you didn't win congrats you know good job playing i am now going to sing a freestyle for our winners um and it's a song plus a freestyle um and then we're gonna talk about any closing things that we need to say all right so let's get that song started where's that music at where's the beat where's it be where's it be transition here we go music start playing [Music] yo [Music] one more time [Music] i got my tesla got my autopilot going left right rack and down up sided i'm excited trying to wrap and time it i'm trying to time it every single beat line it line it up with every single line the line of best fit is called regression are you dumb don't you understand every algorithm is used to find the secrets within the data it's like a logarithm it's like a exponential rocket ship to the future [Music] to plot my line [Music] congratulations that's what i like to see guys each of you win all three of you are my heroes thank you for playing this week guys it's been so much fun being here with you guys i will be here the same time next week you know what it is if you haven't signed up for code royale go watch that trailer sign up for the wait list code royale is the world's first educational esport this is going to be the world's most valuable esport as well i am now getting more confident every week with trying to communicate what it is that i've been working on and what i want to give to this community i refuse to lose to apathy and i refuse to lose the war to games that are not upskilling people we have to retain the attention of the population of earth by upgrading and innovating in education and turning machine learning into a game a social multiplayer game we're gonna create legends we're gonna glorify these legends these legends are gonna win cash prizes and they're gonna have entire stadiums with fanboys and fan girls cheering after them i have a vision i'm not stopping until my last breath i will see you here next week three winners 175 dollars each i love you guys last thing to say before this live stream shout out to zilliqa silica helps sponsor this video they're the reason that i'm able to give out 500 worth of prizes this live stream silica is a blockchain run by milan chukri arnav these are great guys they use sharding it's a great blockchain they're sponsoring this video they're sponsoring the next video as well um uh but that's how that goes shout out to zilliqa thank you silicon for sponsoring this video zelikazel gazelle gazillica if you haven't seen them go check them out at the bottom of the video description they've got some great links thank you guys for being here i love you and peace wherever you are in the world good night good morning you