Claude Code + Playwright Automates Literally Anything

Key Takeaways

So, when you give a tool as powerful as Claude code, access to literally control a browser, if you think about it, you can actually automate anything. Whether that's stress testing an app, downloading reports, or even playing a game like you guys saw in my other video. So, today I walk through three use cases that you can use when you connect Playwright CLI to Claude code to control your browser. The first one is having it actually QA a web app and finding bugs and then fixing them. The second one is searching for dentists and finding their contact information. And then the third one, a question that everyone's asking, is can browser use actually sessions where you need to be logged in? So, I show off what I'm able to do in my school account. So, I don't want to waste any time, let's get straight into the video. All right, so I am in a fresh Claude code project as you can see. I'm in a folder called browser automation demo, but there's nothing in here. Just so you guys can follow along with exactly what I'm doing. So, the first thing to do is install Playwright CLI so that we can actually do some browser automation. So, I'm just going to go into plan mode and I'm going to say, "Hey Claude code, I want to use Playwright CLI in order to do some browser automation stuff, whether that means testing out web apps or getting screenshots of things, whatever the case, I just need you to figure out how you can install this for me and then go ahead and build a plan and let's do it." So, I know I'm telling you guys to use Playwright CLI, but this is literally what I did when I decided I wanted to try out some browser automations, except for I said, "Hey, I need you to research the different tools and the different pros and cons and help me figure out what I should do." I played around with Chrome dev tools for a little bit, but now I'm using Playwright CLI. And one reason for that, while we're letting this load, is that if I do {slash} context in here, you can see that the Chrome dev tools MCP takes up so many tokens because there's so many different tools and each tool has a description. So, that's how I was like, "Hmm, maybe I don't want to use the MCP server. Let me just go ahead and try this Playwright CLI." And it works really well. All right, so the plan is done. We have an empty project. We're going to initialize the project, install Playwright, create a demo script, test things out. I'm going to go ahead and accept this plan and now I'll check in with you guys when that's done. All right, so you can see that it is done. It's been installed and then it also tried it out already. So, it ran a script to open up this page and then screenshot it and that got saved right here. So, as you can see, it was able to do that. It was able to take a screenshot, which means that everything is working. So, now that we have this initial stuff set up, I'm just going to go ahead and do a {slash} init to basically just initialize the environment and give us a claw.md file. All right, so there are tons of great use cases when it comes to using a browser automation. Whether that is looking through Amazon or applying to a ton of positions or, you know, downloading reports from websites that don't have an API. And what's awesome about this is it will build out these scripts that run the browser automation, right? And when you pair those custom scripts with a skill, it gets really powerful because then that process of opening up the browser and doing certain things becomes consistent and repeatable. But one thing that I think is really cool about browser automations is the ability to basically test things and automate QA. So, an example I wanted to walk through with you guys is I'm going to use Claude code in here to build us a quick web app that's going to be like a multi-page form and I don't want to test it myself. I want to have it test it and then it has to find the bugs, find suggestions and then fix itself. So, it's going to be us building a web app or a form and it's going to be like completely hands-off. That's the goal at least. So, let's hop into plan mode and let's get started. I need you to build me a form submission website. Now, I want this to basically be one page per question. So, as soon as I open up the page, it should ask me for my first name and then there should be a button that prompts me to hit enter. When I hit enter, I go to the next page and then I have my last name and then I have my phone number and then I have my business and I have maybe, let's just say eight questions about me and kind of getting me on-boarded into this, you know, fake business or whatever. I just want to do this to see and test the functionality of what you're able to build me here. Okay, so that is phase one. Phase one, we're going to get an initial version of the website and then before I even open it up and test it, we're going to go into plan mode again and see if just Claude code with the browser automation can test it, iterate, test it, iterate. Okay, here is our plan. Multi-page onboarding form. The user wants a polished multi-page onboarding form. We have the architecture, there's 12 total questions, which look good to me. We've got some files it's going to make, some implementation details here and all of this looks good to me. So, we're going to go back into the main session and accept those changes. Okay, so that just finished up. We have the files have been created. We've got a server that this is running on. It's going to be on a local host. We also have 12 one per page questions and what I noticed it doing is it was taking screenshots and I didn't even tell it to. So, if I open up screenshots, you can see we now have these pictures. So, let me just look at these. We've got number one, which is what is your first name. Number two, last name. Best phone number to reach you at. There's even, if you can see, there's like one of those scroll bars at the top where it's like showing, you know, how many questions in they are. So, for a one-shot prompt and the fact that it's already flipping through and taking screenshots, I'm pretty impressed. But we have to see how it's able to actually test it out. So, now I'm going to go back into plan mode and I'm going to say, "Yep, absolutely. Spin up a server so that you can actually run this. And then what I want you to do is use your browser use and test it out. So, fill in the fields, click through and if there's any bugs or if there's anything wrong with the functionality of the site, make note so that you can go ahead and fix the actual site itself." And I also want you to do this in a headed browser so that I can watch what's going on. So, I'm going to shoot that off. It's going to make a plan, but that is one important distinction. By default, you could say, "Hey, I always want you to run Playwright in a headed browser." But there is headed or headless, meaning it could be running this headless where it wouldn't show us on screen, but it is still running on its own tab and clicking through things. Okay, here is the QA pass, the headed browser testing. I'm not going to read this. I think that this is pretty clear of what we want. I'm going to go ahead and accept it. So, what happens is for every kind of like bot that we want to run, it has to write its own script. So, I believe that this is this one that it's writing, which is the test onboarding.js and this is basically what instructs the bot on what to do. And then when we wanted to turn this into a skill, which is basically like QAing our website, we would say, "Okay, when you want to QA, then you run the bot and then you take the feedback from the bot and then you make changes and then you run the bot again." And so, that's how we can turn kind of like all of these different pieces into an actual process. Okay, also that wasn't even the script. The script that it's writing for this is QA-test. So, now you see what I mean. Okay, so the window just popped up. I'm going to move it into view and we should be able to watch it fill in. It's did it did Nathan. Okay, it went backwards. Now it's on the second one, Nathan Harrison. Filling in a phone number. It's continuing to click through, Harrison Tech. We've got a email. We've got founder and CEO. It shows a company size with from a drop down. Okay. Technology, referral. Primary goal. Website, continue and anything else. Okay, it looks like it's having a little bit of trouble here now. It keeps kind of glitching out. But once again, I'm not going to stop it. I'm going to let it figure out what it needs to do. Okay, so it just shut down and now it said, "Okay, found some issues. Let me review the screenshots to understand what happened." So, what you guys didn't notice is that in there, it took more screenshots. So, we can see now we have QA and we have all of these screenshots that it took during that test. So, it says, "Okay, the UI looks polished." That's great. It did look really good. But the first bug is that enter on text area didn't advance to review. It stayed on the notes page. The second bug is that the review page never loaded. The test found zero review items and the edit button was intercepted by a stale page overlay. And now it was able to use that feedback to fix the actual bugs in the site itself. And this is just really cool because if you've ever built software or websites or apps or whatever, there are so many bugs and you don't always find them until it's maybe too late or maybe a customer found it. So, the fact that you can automate QA by having multiple different headed browsers or even headless browsers spin up and you can say, "Hey, you test for X, you test for Y, you test for Z." and just have them running and fixing and running and fixing is a complete game-changer. And now, without me even asking, it says, "Okay, I'm going to start the server and I'm going to rerun the test." So, we're basically getting it stuck in this loop of testing, validating, testing, validating and, you know, it's really, really cool. So, I think it's doing another quick test just to see um if it truly is good, but Okay, nice. So, second test went all the way through and now it should be shutting down and telling us what it learned. As you can see, it just shut down. So, now you can see it has finally passed the tests and it goes ahead and ends the process because now the server is good. So, like I said, the next step from there would be to turn that into a skill. But what else can we do with browser automations because they're definitely not perfect, right? So, let's say we wanted to do something on the web. Let's say we wanted it to just be able to go to Google, search for, I don't know, let's just say dentists and maybe capture like some phone numbers. All right, here is our plan. Build a Playwright script that automates Google search to find dentist offices in California. We will basically launch the browser, do the Google search, collect the links, visit each site and print a summary. So, that looks good to me. We're going to go ahead and shoot this off and hopefully it's going to be decent on the first try. I'm going to tell you guys right now, it's probably not. But what's going to happen is we're going to have it learn. So, it's going to open up the browser, it's going to fail and we're going to say, "Okay, keep learning, keep updating the script and don't stop until you're done." I literally stepped away to go get water and it actually I came back and I saw a browser open with some dentist offices up. Wow. So, it visited all five sites and it was able to find some phone numbers. Now, that's really cool because it actually learned while it was going and it said that Google blocks the automation, so it switched to DuckDuckGo. So, I told it this time, "Don't stop until it actually finds five phone numbers." So, it's going to be a little bit more aggressive and this time I'm actually here to watch it. Okay, so here we are. It just searched for dentist office LA. You can see there's a phone number. I just saw one on screen, so it probably grabbed that screenshot. I see another one on this page, so it's getting pretty lucky here. It's finding the phone numbers pretty quick. What what noticed is it's still going to the site and it's still clicking on the contact button. Even though the phone number was up in the top right, visible, it still clicked on the contact button, which I thought is pretty cool. So this time it got all these phone numbers for these dental offices. So naturally the next question that I'm sure you guys have is how could it work on, you know, things that I'm already logged into? So, let's just find out. So I'm going to go to plan mode and just ask it what would happen. So there are a couple of approaches. We could do persistent browser profile, meaning it can launch a browser using my existing Chrome user data, which already has me logged into school. We could do manual login and handoff in headed mode, or we could connect to a running browser. Okay, so that's pretty cool. It could open up the browser, I could sign in, and then I could just basically keep that signed in and have it do whatever it needs to do. But I do want to try option one, which it recommended in the first place. Okay, so we have the plan, which is going to go into my school community, it's going to go to the wins channel, and it's going to like those posts. So we've got the context, the approach. Obviously you guys know what these plans look like. Let's just see what it did on this first try, and I'm going to go ahead and accept. Okay, so apparently when it runs this, I'm going to have to log in manually the first time. But then in the future, it will basically save that session. So, let's see how that works. All right, it opened up school.com. Okay, it opened up school.com Okay, so what just happened is it opened it up and then it closed it, and then it opened it up and it went to my community and closed it. And now I think it's actually going to let me log in. So let me try that real quick. Okay, so I logged in and then the browser shut down. So now I said, "Okay, cool, I logged in." Hopefully now when it launches up this browser, it already has me logged in. Okay, so there it is. Nice. You can see that I am already logged in, which is great. Okay, so it's trying to find the wins channel. It looks like it found it. All of these look like they are wins. Now let's see what it's able to do here. If it's Oh, I just saw it try to like the post. Okay. It just liked that one. Interesting. So it's like liking them and unliking them. So that's one thing we're going to have to fix. But it is scrolling down. It got to the wins channel, and it is now trying to like posts, but as you guys can see, it's not perfect. So I hope that it's able to realize that it's not really liking them. Yeah, it's basically going like, unlike, like, unlike. And it also just said liking too fast. So let's see if it's able to learn. Okay, there we go. All right, so what I'm going to do at this point is, you know, I've seen enough. I'm going to shut this down. And hopefully it's able to use that feedback to get better. So the login persisted, it found the wins channel, it even found the heart-shaped SVG buttons, but it crashed during the page evaluate for liking. So it's seen all that and now it's going to make the script even better. So I will say with browser automations, I think that this is completely normal, which is why I had a little bit of doubt earlier in the dentist example. But literally every time that you use the script, it's going to get better. So it's about to run it again. Let's see how it does this time. It was able to just find the wins channel. Now one thing I might want to tell it is I might want to sort it from newest posts rather than just going straight to the wins channel. And it went ahead and shut down again. So, it's continuously trying to learn and get better at this script. I'm actually going to stop this and I'm going to try to course correct it a little bit. So I'm going to go into plan mode and I'm going to say So, a couple issues from the first couple runs. The first thing is that it was hitting the like button like four times. So it ended up not even liking the post. The second thing is once you switch to the wins channel, try to filter the posts by newest. So there's a little kind of sandwich bar menu option near the channel filter, and that will allow you to change the view to newest. And then you can go through and make sure that all of the posts are liked. You can tell that they are liked because the thumbs up icon on the post will be yellow rather than gray. All right, so we got the browser open. Nice. It was just able to switch to newest. And now it's going through and liking. It skipped over that one because it already liked it. Yeah, this is awesome. This seems to be doing the trick. It's liking my own post, which is a little bit uh, but besides that, it's actually working. And it's skipping over the ones that are already yellow, which is great. It even just went to the next page and it's still liking. So this would go through and like every single post in the wins channel. So I'm going to shut that down now, just because I don't want it to go ahead and do all that right now. But that worked and it took what, maybe four or five iterations? And now what I would do is I would build a skill out of that. So I could just say, "Hey, run the school like skill." So anyways, I know that that was kind of a simple use case, but the point I was trying to prove is that yes, you can automate stuff in areas where you need to be logged in. And also like school is one of the least automation-friendly platforms ever. So Okay, so I was going to just post the video there, but I decided that I wanted to keep this going a little bit to show you guys what's actually possible with this. So, what's really cool is now that you have these Playwright CLI automations that can do things in your browser, and it can save your cookies. So basically you stay logged in. You can actually start to automate things when you use the desktop app. Now you could also use something like um, you know, modal or trigger and just deploy those automations there. But I like to use the desktop app for my scheduled tasks because I get the more agentic ability. Because when these scheduled runs actually run, it's an agent literally like you see in Cloud Code walking through what's going on. So for example, I have basically created this bot in my school community. So his name is AIS agent. And as you can see, it's posting things like daily news roundups, and it's responding to all of its notifications, and it even today made a happy birthday post for me, which I didn't tell to do. It just knew information about me, and it did this completely autonomously. So I've got it doing things like every day engaging with school wins. So if I click on today's run, you can see that it shows me what it engaged with, and it ran all these commands, and it basically just shoots over this prompt and it invokes the skill to do this every time. I've also got it doing the AI news roundup, and I've also got it responding to all of the notifications in school. Looks like there's actually one running right now. We've hit some sort of error, but because this is agentic, it's going to keep trying different things in order to make sure that we're actually able to get this to go through. And remember how I talked about with Playwright CLI, you have to use the skill more and more or run the script more and more in order for it to understand what to click on and where to submit things, and it has to learn how the actual UI of, in this case, school works. And all of these automations I have running in a headless browser, meaning when I have my computer open, I don't see the tab pop up and I don't see it happen. But right now I told it to go into that happy birthday post and respond to all the comments on that post. And so now it is pulling up this browser. It needs to figure out how do I actually navigate to that post that I made. So it's going to its notifications, and it's just going to look through the notifications and click on those and respond to them. And as it goes through this agentic loop, it starts creating new tools. Because one time someone said, "Hey, AIS agent, can you please vote on my poll?" But the agent couldn't figure it out. So what it had to do is it would had to figure out what does it actually mean to vote and where do I click? And once it did that, it turned that into another script, and then it was able to make sure that its skill was updated so that in the future it was a bit more autonomous and could actually vote on polls. Cool. So what it did here is it made a to-do list of all of the different comments that it needs to respond to. So here's the comment and then here's the actual reply that it's going to make. And now it's launching up this browser, and it's going to individually go through and respond to these. And because it's been doing this so many times, it knows that it needs to hit reply under each comment, type it out, maybe even tag people, and then hit the submit button. So obviously this isn't perfect, right? Like there's still things that I'm working through. I'm sometimes it responds to people twice because it forgot to mark the notification as read. But I basically just told my community, "Hey, I'm experimenting with some browser automations, and if you see AIS agent just kind of like doing some random weird things, just ignore it." So the other thing that I want you guys to keep in mind is that we are using Playwright CLI. There are other CLIs out there that do browser automations, and I've been having great success with Playwright. But I also have been seeing people talk about other CLIs to do this. And the reason why we're using a CLI compared to the MCP, like I said, is we're just going to save a lot of tokens. But here's a little comparison you can see. There's also something like Vercel agent browser or Open CLI, or, you know, there's multiple other CLIs that do computer automations. And I think what you'd be looking for is finding the one that has the best balance of token efficiency as well as performance. I don't care so much about speed, but I meant more like the performance of how actually good is it at clicking around and learning uh, how to improve the script so that next time it doesn't make the same mistakes. I'm going to be testing out other options and seeing which ones I think are the best. But at the moment I've mainly been using Playwright CLI. But that is going to do it for the video. Now the next step would be, once you understand how to create all these browser automations, how do you actually make them fire automatically all the time, kind of like what I've got set up here? So if you want to learn how to do that, then check out my video about scheduled tasks, which you can watch right up here. Hopefully I'll see you guys over there. But if you enjoyed the video, you learned something new, please give it a like. It definitely helps me out a ton. And as always, I appreciate you guys making it to the end of the video. I'll see you on the next one. Thanks, everyone.