YOLO11 + SAHI = Better Detection for Small Objects! (Step-by-Step Guide)

Key Takeaways

Hello everyone, welcome to this new video tutorial. In this video tutorial, we will be combining YOLO 11, which is an object detection model with Sahi, which stands for slicing, edit, hyper interface. It's basically a computer vision tool that improves object detection especially for small or tiny objects by dividing the image into smaller overlapping patches and then it runs object detection model on each batch. So let's look at the GitHub repo of Zahik. You are already familiar with YOLO 11 which is released by analytics. It's an object detection model. So in this tutorial we will be combining YOLO 11 with SI so that we can improve object reaction for small or tiny objects. So over here you can see we have this SAI GitHub repo over here. So SA stands for slicing edit hybrid interface. It's a lightweight computer vision library performing large scale object detection and instance segmentation. So let's look at this first example over here which is in front of us. So like imagine if you have a big photo or a video uh with tiny objects like small bras or like distended cars over here as you can see over here in this example as well. So a normal object detection model might miss like detections like might might miss cars over here might not be able to detect all the cars very correctly because you can see that uh here you can see we have cars and they are very small objects. So as they are too small compared uh to the full image so a normal object detection model might miss them might miss co detect them. So in this case we will use sahi. So sai solve this problem that normal object detection model faces like normal object detection model might not be able to detect all the objects over here. So in that case we will combine uh sahi with the object detection model. So what does sahib do is cut the image into smaller bars. Like you can see over here like it the sahim will cut the image into smaller bars. Then it will run the object detection model on each small part. Then in the last part it will combine all the detections from those slices and it will generate a final output. Okay. So here you can see we have this complete GitHub repo over here. Like you can see installation step are provided. There are many tutorials available over here and like different demo videos and Google collab notebooks are being available over here. So why you should use Sah? Like you might be thinking why we should use Sah because you should use Sahi with your object detection model because with the help of Sah you can improve uh small or distance object detector detection like you can detect small objects or objects that are far away better with the help of SAH. And uh Sah also improves model performance on higher resolution images. Plus, it works well with any object detection model like YOLO, RFT, TEDR or faster on CNN. And so if you're using YOLO model and it so it might miss small objects in the image or the video then you can integrate Sahi with the YOLO model YOLO model to get better results by slicing the image. So this is a quick intro and about the sahi. So in this tutorial we will combining your 11 with sahi. So let's get started. So now I will just go to the PyCharm community edition and I will just start write the code from scratch. So over here you can see that I have created a project in PyCharm community edition. So these are the two libraries that we will be requiring in this tutorial. So Altitics and Sahi because YOLO 11 is available under ultralitics package. So to install this library you can simply write install r requirements dxt and run this. So now you can see requirements already satisfied because I have already installed these packages. But if you are installing these packages for the first time this will take some time. So now I will just start writing the counter board. So first of all we will import all the required libraries over here. So I will import open CV Python. Uh when you install alterics package, open CV Python package also get installed with it. Then we will import time. Then from sahi we will import autoboard detection model. Okay. And then from sahi dot predict uh I will import get slice prediction. Then from sahi dot uh utils.x I will import download model weights. So I can download uh the ultralytics yolo 11 model weights directly. Then from ultralytics dotails dot files I will import increment path over here. Then from ultra analytics dot utils dot toshot import select device. So these are the libraries that we will be requiring in this tutorial. Then I will just create a class by the name sah inference. Okay. And what I will do is so here we will find detection model over here none. So first we want to load the model. So we have the input parameters as weights and device like if you have the CUDA device or if you have a G uh if you have a CPU. So I don't have a GPU in my local machine. So I will not write CUDA in the device. Instead I I don't have a GPU in my local machine. So I will be uh writing device as CPU over here. Then I will just define the weights path over here. So where I just want to save the YOLO 11 model weights. So instead of this I will just create a directory over here with the name models and inside this directory I want to save the YOLO 11 model weights. So I will just write it over here. models. So first we have the detection model. So we will using the pre-trained YOLO model. Pre-trained YOLO level model. So order first we need to define the model type. So YOLO 11 is available under alterics package. And in the model path we have the path. Now we'll just create another function by the name run. The input parameters will be source. Then we have the weights as the input parameters. Then we have the device uh which will be CPU. We want to view the output image. So we'll just set it by default true. Want want to save the output video. So we'll set it true as well. I don't want to show the confidence score. So we can just set confidence is equal to true. Or if like you might want to show the confidence score. So we can just set it false. Then we will set the slice size is equal to five. So you might be thinking what is the slice size. So now you can see that I have just select the slice size width as 512 and the height as 512 as well. So a complete image or a frame will be sliced into patches of size 512 + 512. The each patch will have a width of 512 pixels and the height will be also 512 pixels. So if you have a image or a frame, it will be split into 512 cross 512 overlapping patches and then we will run YOLO 11 object detection model separately on each batch. Then with the help of sahi or the sahi will merge results from all the patches and it will also take care of the old lumping patches as well. And we want to also say want to save the results in a output video. So I will just write the output video name as output.mpp4. So I will just say cap is equal to cv2 dot video capture. Here I will just pass the input video path. So we add aert to make sure we are able to see the video else. So then we have a save directory. Okay. So here we'll just define the path where we want to save the output video. So I will just create another directory named out video. Okay. increment path and we will save the output video over here. So if it exist okay it's true and if it doesn't exist it will just create this directory. So we have already created this directory. So I will just write save dash directory parents is row as this token that works. Then we also want to save the output video. So we need to define the width and the height. So I will just write in gap.get to gap dash frame weight and we will also define the height. So just type in cap.get cap dash fra and we'll also calculate the FPS as well. So, so the FPS score will roll because I'm writing it on running it on a CPU machine. So, if you just run it on GPU, the FPS score will definitely be high. Okay. Next, we need to find the output video writer over here. So just write video writer is equal to none. If the user select the save image then you will only save the output video [Music] here I will just write video writer is equal to cv2 dot video writer And we will just save the output video into this directory which we have find output video. And we have already passed the output video name as output.mpp4. Then here we will just define the input fps like how many uh frames we will save in a second. And here we will just pass the width and the height of the output video. Going ahead we'll load the model. So like you can see we have the model weights over here. Here we just defend the weights. Give us okay. So now I will just now we'll loop through the video each video complete video frame by frame. So if you are able to read the video then let's write back frame= to cap bit. If you are unable to read the video then this loop will break. Then we have the start time. So we will just click calculating time so that we can calculate the FPS frame per second. this uh the f FPS will tell us how many frames are being processed in 1 second. Okay. So now I will just write over here results get slice prediction. The input will be over here the frame. Then we have a reduction model over here which will be sales dot reduction. So our image will be split into or the image will be sliced into the patches of size 512 with in width and 512 in height. So we have already defined this above. Okay. Like you can see over here slice size. So we have slice size in the first we have the bit and the second we have the height. Now from here we will get the our results like the prediction. So inside this we will have our bounding box coordinates as well. So inside this results we will have our bounding box coordinates class name confidence code. So we not we will now just get those values from these results and we'll just draw the bounding box coordinates at the class name above each of the deducted object. Next we need to draw the bounding boxes. So I will write for prediction the results part object prediction Best. boss one Egg. Then we have the class name over here which will be then I will just draw bounding boxes around each of object using OpenCV Python. And here we have frame. So here we have the top left corner bounding bord bounding box coordinates and here we have the bottom right corner bounding box coordinates and the color of the bounding box will be green and the thickness of the bounding box will be blue. Okay. So once we have drawn the bounding boxes then we will add the class name above each of the bounding box. So we will use cv2. put text to add the class name above each of the bounding box. So here we just want to put the text above the bounding box. So here I will just pass the uh top left corner bounding box coordinates. Just increasing the height a bit because we want to put the text above the bounding box. And we will use define the font over here. The font scale will be 0.6. That would be enough. The color of the text will be green and the thickness of the text will be two. After this drawing the bounding box, we also want to calculate the FPS like how much frames are being processed in a second. So this is what FPS tells us. Frame per second. FPS stand for frame per second like how many frames are being processed in 1 second as I'm running it on my liquid CPU machine. So the FPS score will be very low. It's between 0 to one like this by the current time start time. So we will have to find the start time over here like this. So now we just want to uh put the put FPS score in our output window as well. So we'll use Open CV Python put text function. And the coordinates will be 2040. I've just manually to find the coordinates where I want to just put this text. And the font will be simplex. Font scale will be one. And the thick color will be red of this FPS code. And the thickness will be. So if the user want to see the output video, they will write uh display output video. So if the user want to see the output video then the view image will be set to true and if the save image is equal to true this will save the output video. So, so if the view image is equal to true then I will just show the output video and the name of the video output window will be YOLO 11 plus Sah inference. Here we have the frame output video frame and if the user want to save video frame then write save image and here we have video writer dot write frame. That looks good. And if you want just want to user want to break the loop or close the output window then it we will just if the user press the one key then the output window will close we'll just write dot v key one. So we are adding a 1 millcond delay between the consecutive frames. Now once the window breaks or the loop breaks if the user press the one key then we will release all the frames and we'll also do also release the video writer which we are using to save the output video and we will destroy all the windows. So now what next thing we can do is the user run this script. So now let me just show you. So we need to define uh the video path. So we just need to have few things. Okay. So I'm just checking the outer code. So now we need to have the input video. We need to define the weights. Uh we need to set the device as well. Okay, like currently CPU. But if you have a GPU, then you can set the device as well. Okay, and we just need to define these other parameters as well like view image or show video like all this or save image like all these parameters. And we can just remove this parameter as well. We don't want to find the conference score. So we can just remove it. So let's get started with this. So I will just write deductor. So I will just call the function. So I will just calling the class over here. SI inference is our class. So it's okay. And now I'm just calling this run function over here. This run function. So I will say detector run. So I have the source. In the source I will just define the input video. So let me add an input video over here. So I will just create another directory. I need input video. So now you can see we have an input video over here. So I will just write over here will go into the ebook video and here we have the radiomp. So then we need to define the weights. So here we will just pass the build spot over here. Okay. So, now you can see we have the model weights over here. That looks quite good. Yeah. So but in the weights part we just have to define. So we want to use the yolo 11 nano model. So I'll just define this and here I will just define the device as CPU. I don't have a GPU in my local machine. Then we have uh view image parameter which will set to true. Then we have the save image which will be set to true. We want to save the output video as well. So this is view dimension. Then we have I want to set the slice size as well. So it will be 512 cross 512 like 512 will be the width and 512 will the height you can change it as well. Uh so these are the slicing dimensions and here I will just pass the output video name which will be output dot. So we are good to go now let's run this script and see if there's an error. So we can just fix this up. So I will just write over here Python main. So it's downloading the model weights. So now you can see that uh it's working and now you can see the processing is very slow. So now you can see that we are able to detect the cars and like the detection results look quite good. Now you can see that uh we have implemented we have combined yolo 11 and sahi. So sahi is a computer vision tool that improves object detection especially on for small and tiny objects and what how does s work is that it wides image into smaller overlapping patches and then it runs object detection model on each batch. So now you can see that we are able to detect the objects that are very far away like distant objects like cars that are very far away like you can see over here these cars are very far away and the results look quite good. So the processing is very slow because like you can see that only like uh in 2 seconds one frame is processed like we have FPS score of 0.51 so it means like it took like 2 seconds to process one single frame. Okay. So like we are just that's really slow. The processing on this input video is very slow because it's taking around 2 seconds to process a single frame. So that is the results. So current if you have a good GPU machine the processing will be good. And uh now you can see that the by combining sahi with yolo 11 we are able to detect the cars that are very far away. Like you can see that we are able to detect these cars that are very very far away. Okay. And detection results look very very promising like uh by combining sah with yolo 11 we are able to detect the tiny objects. Um like sah basically divide the image into smaller overlapping batches and then it run detection on each batch. And you can see that we have set the dimension size as 512 cross slicing dimension size as 512 + 512. Uh like yeah the image will be split into batches of size 512 + 512. 512 the width and the 512 will be the height. You can make this my size more smaller as well, but the processing will further decrease like it is FPS score will further go down. But if and if you increase the pack size like from 512 to 1024 + 1024 like this FPS score will definitely improve. So you need to have a balance between you have need to set up a balance because if you decrease the slicing dimension or the pack size very low this will uh increase the processing time and uh the processing on the input uh this will increase the processing time and the processing on the input video will get slow. But if you set the pass size like increase the uh dimension size to 1024 + 1024 uh this will like faster up the processing. But if you uh make the slicing dimension or patch size like 256 cross 256 this will further decrease the processing. But if you increase the slicing dimension size uh this will increase the processing. So now let me just show you the output video complete output video now. So let me just show you the complete video. So I have already uh run this uh so I have I have already processed the this on the complete video. So let me just show you how it works. So now you can see over here uh like you can see we are able to detect the very tiny objects as well. And now this is a complete output video and you can see the results look very promising. Like you can see we are able to direct the small cars and that look very very promising. So now I have also done a comparison with the only the YOLO 11 model like I've done the comparison between YOLO 11 and YOLO 11 plus Sahi. So let me just show you this output video as well. So now you can see over here uh the results look quite good like with YOLO 11 and Sahi the detection has improved. Now you can see that we are able to detect the tiny objects as well. But with YOLO 11 we are not able to detect the objects very accurately like you can see that uh some detections are missing if you only using YOLO 11 but with YOLO 11 and Sah like those detections are also been done like you can see that uh there are some of the cars that are very far away that YOLO 11 is unable to detect but with YOLO 11 and Sah we are able to detect the cars that are very far away as well. Okay. So now you can see in this video like we can see that we have done a comparison between yolo 11 and yolo 11 plus sah and we can see that with yolo 11 plus sah we are able to detect the tiny objects more accurately and like you can see that in in case of yolo 11 some objects are not being accurately directed but with yolo 11 plus I we are able to detect those objects as well. So that's all from this video. Thank you for watching.