Sikuli Tutorial 2: Program visually in python!

Key Takeaways

All right. So, what we've got here, I've got the suli script that we're going to write over here. And then I've just got the browser open. We could obviously start from scratch, have it open up a browser, but here we go. And so, what we're going to go ahead and do is uh start here. And the first thing we want to do is we want to have the mouse come to this address bar. and like click on the address bar. So, the way that we're going to do that is we want it to click. And where do we want it to click? It's going to be um right here. So, that's where we want it to click. Now, just to make sure cuz it might think like the mouse I I believe where it makes that uh crossover is where it will naturally Oh, shoot. Oh, there we go. Okay. Where it will naturally click is there. But if we go Oh, it's like all my screens. Anyway, so okay, you can see where it wants to click, but you could tell it like, no, I really want the mouse to click here or here. Um, the next thing that you might want to do that you can change up, um, is, uh, sorry, this probably looks really confusing because it's showing like I've got four monitors, so it's like this huge mess. Yours should just be a regular square, though. But what it's going to do, I wish there was a way for me to like zoom into my main monitor. Um, anyway, you can see here there's got this similarity. So, it's always a good idea to kind of check this and see, okay, how many similar places are there? And you can require how similar you want it to be. Like if we drop this, you can see that oh, there's a similar to here, here, here, here, here, here, here. But obviously, the most strong one is is this one. But if you raise it up enough, the only one it's going to find is the correct one. So anyway, we'll just hit apply. Okay. And now you can see here in our little image exactly where it's going to stick the mouse to click on this image. So, real quick, since I know everyone's just excited as heck with Sakuli, let's hit run and see what it does. So, we hit run. Oh, we got to save it. Let's see. We'll just run immediately. So, you saw maybe hopefully within um you can see that it move it took my mouse and it moved it and we did click there. So, now that's actually now that we've clicked there, the next thing we might want to do is uh we want to type in something. So, let's say type and in here we want to type um scentex oops scentex.com. So, now let's uh let's actually save this so we don't run into any problems. Let me save it real quick and we'll do save. And we're just going to save this as uh sule toot. Awesome. So now what we'll do, we'll run it and let's see where it goes. Image. I think uh this must have been saving it somewhere. So let's let's retake this image real quick. I guess we'll have to Yeah, let's just redo this image, I think, because it's it's looking for it saves the images in a little bit of a like a file basically. And um so let's say where do we click? So since those um since we saved it in a new directory, I think that's that's the problem is it just doesn't know. So let's do that. And then let's uh edit this target offset. We want to make let's just offset just slightly. Apply. Okay. Okay. So now let's save it. Let's run it. And you can see okay, now it goes there. It types sent.com. Now the next thing we want it to do is hit the enter key, right? And so you obviously can't say like click enter or something like that, right? So now we need to change that up. So how do we do that? So as far as I know, I believe you do key dot um and then like we would do I think it's all caps enter, but you'll have to bear with me. There's also uh say you wanted to do like control um C, you can do like uh key down. Oops, sorry. We're all caps. uh like key down and then do something. But for now, we'll get to that eventually. Um so I think we can actually put syntax.com and then you can say plus and then do key. Oops, got to uppercase that key and then we'll do enter. Plus key enter. And that should take us to the website. We'll see what happens. Let me empty out the address bar here. Uh let's save our suli script and let's run it. Goes here. Hit enter. Sure enough, it takes us to the website. Now, we're already logged into this website. So, let me go ahead and log out real quick. Let's say log out. Hopefully, it'll save us logged out. Let's try it again. Let's run it one more time. See if it keeps us logged out. Hey. Oh, yeah. Oh, so if you have something in the address bar. Okay. So, here's a here's a good thing to run through real quick. So, what's happening is it's looking for this image and this image contains a completely empty uh bar. So, it's probably having a hard time finding it. Like, let me click on this and see our matching preview. Yeah. So, as you can see, it's not finding it. we have a pretty stringent uh match. But I bet if we just turned it down a little bit like here, it would find it pretty quick. But there's a better way to do it cuz you wouldn't want it to find like multiple things cuz this is where suli could go wrong. Obviously, you could use some good logic with it. But what you can do is since like for example this will change like when you're using a browser you know this might might change your forward and backs might change a little bit but for the most part uh what's going to change the most is going to be this the text and the image recognition is that it's quite good. So you can use that to your advantage but also it can kind of mess you up sometimes. So let me show you how to fix this sort of a problem and then I'll pause the video and we'll continue on um in in the next video. So let's say instead uh let's get rid of this click. And the way what you can do to kind of uh remedy this situation is take an image. So let's say okay we're going to click here right or or we're going to do oh shoot we shouldn't have sorry I hovered over this button so that's going to cause a problem. So let me fix this really quick. So one more time click. Wait till it comes up. Okay. So, we want to click in this area. Make sure we don't get any of that blue. But, or actually, let's even do it this way. So, we want to click in this area, but then what we do is we change this target offset to actually be all the way over here, right? So, this is the image it's going to look for, and it's going to click over here. So, that way, um, you're kind of circumventing any issues with stuff being in there. So, let's hit apply. Okay. And now let's put this where it needs to be first of all. Or cut and paste this down here. Put this up here. Save. And now let's run it. And hopefully that'll work out for us. Kind of messed up our URL. Let's try again one more time. There we go. Okay. So now it's actually taking us to sentex.com. Still the same way as before. And if something's in the address bar, let's try again. It might be because it's maybe not highlighting it. But I'll tell you what you can do. Sometimes what you can do is you could click and you can drag over to here and then type the website. So I mean there's all kinds of different things, but Suli is not only for the browser. So and really if you're going to be doing something in a browser, it's sometimes better to use um Selenium anyway, but I'm just trying to get you guys used to using um Suli. So anyway, I'm going to cut off the uh the video here and then we'll continue on in the next video um with logging in with Suli. So, um, as always, thanks for watching. Stay tuned to the next video.