Claude's New Computer Use Changes Everything! (2024)

Key Takeaways

Claude just released its latest beta feature called computer use for its 3.5 Sonet model here's everything you need to know about the latest feature from Claude first things first this new feature will allow the Claude Sonet model to interact directly with computer tools and control a desktop environment now let me explain this a bit more clearly what I mean by interacting with computer tools is that the claw model can now click buttons open applications and even move the cursor around this allows the cloud model to behave the exact same way as a human using a computer now the only caveat here is that this new feature should only be run inside a safe sandbox environment such as Docker rather than your actual computer so what this means is that Docker can emulate a desktop for you so that you can try out the new computer use features you can think of this as giving Claude a virtual computer to use and watching the clae model interact directly within this virtual computer within docket container Cloud will have access to most standard apps such as a web browser and even Excel and within this environment Claude will be able to handle basic computer functions such as Mouse movements and even keyboard inputs Claude can also work with and edit text files directly as well as run command line operations so an example you can try today within this desktop environment is to have the cloud model scrape names from a company and then pasting in the contact information of these names into a formatted Excel spreadsheet now let's go over security BL shouldn't ever be given access to sensitive information like passwords or personal data especially in its current form since the computer use feature is still in beta and is currently being refined as well as improved upon so in a nutshell you should only be using this feature for non-critical tasks now one important note here if you do decide to try this feature out is to be very very careful about usage limits now because it's in beta usage of this new feature is quite expensive now from testing this out I found that 1 minute of use costs around 30 so if you're just trying to see how well this new feature works you can easily spend hundreds if you're not careful enough over time though the cost should go down significantly as anthropic refines the model capabilities so what exactly is claud's computer use good for I found that it works best with simple and clearly defined tasks you can't just give a very complex prompt to Claude and expect it to answer in one sitting I've even seen it make occasional mistakes while clicking or scrolling things that work for me were simple action tasks such as inputting mock data to a spreadsheet now for the final question should you try this beta out for yourself if you aren't too Technical and you're not trying to stress test the feature just yet I would personally wait there are plenty of issues currently that will either get fixed or optimized by the time of its official release and in the meantime it may be just helpful to think about how to best leverage this latest computer use feature and if you do decide to use this new feature always make sure to provide clear instructions and try out simple tasks to start such as opening up apps or basic file operations and even basic text edits this is just so you know the limitations about how this feature Works before you burn up millions of credits now let's go over how to actually use this new feature for starters you will need an anthropic API key a basic understanding of how API calls work as well as a development environment that can run containers you can only access this new feature through anthropics API meaning that you won't be able to use it in the regular Cloud chat interface so to get started you first want to make sure that you have an anthropic developer account now this is separate from a Cloud Pro account if you do have one so just keep that in mind after you're inside and logged in Just click on get API keys and from here you can create your own personal API key now one thing to note is that you do have to add billing before your API keys are active after you've added billing just go ahead and click create key and then for the name of our key we'll just call this computer use and our workspace will be our default workspace great next you'll want to install Docker for either your Windows or Mac device so for me here I'm on a Mac so I'll click download Docker desktop and then I'll select Mac Intel chip now inside Docker what we want to do is first open up this terminal and enable it so we'll click enable Docker terminal next what you'll want to do is run a command within Docker so I'll paste over my command here and I'll provide this in the description of the video as well all you'll want to do is just replace this API key with your own personal API key then now we can click enter and we're now able to open up the Local Host URL so let's just click this into our browser and we're now able to use the cloud computer use demo within a virtual environment so inside here we can just type in any command such as populate in Excel spreadsheet from 1 to 10 and we can see the agent is now running and Claude will now automatically perform these actions without you needing to lift the single finger this is a revolutionary Next Step within Ai and although it's only in beta the Claude Sonet model can now do some pretty incredible things with its new computer use feature the limitations here can be endless from creating entire files in your code base grinding your favorite video game for experience data entry or company research tasks and even a automated email marketing if you enjoyed this video and want to learn more about AI tools in the future make sure to like comment and subscribe for more information on all the latest AI developments especially within the coding space and if you're looking for a program that will help you build applications completely with AI with full stack developers available to help you and explain code for you when AI gets stuck make sure to check out our website at no code pathways