Open Interpreter Tutorial - AI Controls Computer Using GPT Vision - Self Operating Computer

in today's video I'm going to be showing you how you can control your computer with just your voice using two Cool Tools super whisper and open interpreter so let's dive right into the video so this is open interpreter you can visit their website by visiting open interpreter. comom open interpreter allows llms to run code on your computer so they can complete tasks so I'm going to help you guys get this set up today all you have to do is come over here and click on GitHub and we'll scroll down and then you're just going to run this command you need to have python installed so you're going to go over to your terminal you're going to paste that command in and I already have it installed but you're just going to run it and you'll see everything install and then what you're going to do is you're going to run interpreter and then you see a startup and then you can provide the open AI API key or you can use LM studio and you can use the server on there if you want to run a local model with it so to run the model locally you're just going to scroll down and you'll be able to see right here and you can just put dh- local and you'll be able to run whatever model you're currently running in LM Studio I also want to make a note that you need to make sure the context window is set to 3,000 and the max tokens is set to 1,000 and you can also use open interpreter inside of python as well and you can call it just like this so we're going to use in the terminal and we're going to have it run some task for us if you go into the docs and then you go to usage and then under terminal you can see Vision if you're using a vision model you'll be able to use this right here even if you're using LM studio and this is what we're going to be doing today we're going to be using vision and we're also going to be testing out without Vision to do the same task and see if it does it better with or without it they also have OS mode which allows open interpreter to control the operating system and it's also going to be recording the screen and using the vision model from GPT 4 let's run interpreter OS okay now we can talk to it and ask it to do what we want it to so the first thing we're going to ask it to is to play hip-hop music for us okay so we're going to ask it to play hiop music music using Chrome okay nice doing the spotlight search okay let me pull the terminal back up so that way you can see it oh it clicked on the wrong area probably because it probably screenshotted it when the terminal was underneath okay so it recognize that we're on Gad okay so it's finding an alternative approach okay so it's getting information see if we can get it now nope not yet there it goes wow okay okay wow wow all right nice that was pretty cool that was pretty cool took it a minute took it a minute like I could have did it way faster than that but but that was pretty cool though so let's see if it'll play some music now I'll bring back the terminal so we can see what it's doing oh okay so it says the task is done all right so now I can choose a video to play would you like me to execute playing so it's going to execute playing the little dark s I think it needed to double click it because I was on the terminal so I'm going to let it just do its thing for a minute we'll see what happens oh okay so it thought it clicked on it but it didn't so I'm going to tell it to play the little dark song okay I'm just going to leave this on here so that way we can see if it works so that way there's nothing interfering with it perfect okay that worked perfectly let's go back over and see what it says all right okay now you're just going to run interpreter without the D- OS so to bypass the approval you're going to run interpreter Dy or else you're going to have to approve everything that it runs so what we're going to do is we're going to restart that and we're going to add that argument at the end now we'll run play hip-hop music on Chrome and we'll see how it goes Al so as you can see it's already running completely different it wants to install selenium then open Chrome that way so it's going a completely different rout but we'll see what happens okay let's see if it it continues running what it's supposed to do yeah I most definitely prefer running it with the OS way better then what it's doing right now but we'll let it do its thing and we'll see what happens oh okay all right well let's see if we can find a different approach I meant like without selenium okay let's see if we can try this now okay so now we're going to test the speech mode and what you need to do is download Super whisper and this is going to allow you to do everything handsfree so this only works if you have a Mac so you're just going to click on download then after you have it downloaded I'm going to show you what to do next you just need to open the app after you download it and then you'll click on this and you'll be able to click on the settings and it'll take you to this page and once you're in here you can turn on the real time if you want to these are experiments and then you'll have the AI models which you have downloaded the pro one when you installed the software um and then they have the other ones as well you can download if you'd like so we're going to go into the configuration and we're going to change the simulate key presses this is turned off by default you need to turn this on and then the restore clipboard after paste you need to turn this on as well cuz that's going to be turned off and these are the only things we need to edit in the settings so what we're going to do now is we're going to use super whisper and we're going to use OS I'm going to ask to change the appearence from the default setting to light mode so all you have to do is click on that and it'll start recording change my appearance to light mode please recorded the first part of what I said and then it said change my appearance to light mode so we'll just remove that part but you guys get the point of it if you're not talking in a video it'll work for you pretty well so that way you can have a hands-free experience using it you just have to make sure your cursor is over the terminal so that way it can click on it properly nice okay so we didn't change it to light mode so we're going to use super whisper again we're going to ask it to do it change my appearance to light mode okay there we go it worked nice change my appearance back to dark mode please nice so this is just a few examples of what you guys can do with it play around with it let me know in the comments what you guys end up doing what it does for you and how well it works all right yall I appreciate you guys for watching the video If you enjoyed the video just drop a like comment and subscribe and I started my Discord community so if you guys want to join all the information is below in the description so I hope to see you guys in there