Autodistill: Train YOLOv8 with ZERO Annotations

Key Takeaways

I built that model literally in 30 minutes and you can do it too image labeling is slow and expensive process however we needed to train realtime object detection and instant segmentation models like YOLO V8 and yanas well we used to outto distill is new library that allows you to distill Knowledge from large foundational models and transfer it into smaller highly optimized ones it consists of six core elements task base model onology data set Target Model and the Ste mode let's start with task it decides whether or not outo the will automatically build classification detection or segmentation mod depending on the task you select base model the base model is large foundational model that knows a lot about a lot they are very often multimodal it means that they can take different types of input like image and text and can perform different tasks for now outo this still supports only one base model grounded Su but support for clip and many more is coming soon you can also very easily add your own base model and plug it in into outo distill framework base models are slow and terribly expensive too slow to be used in real-time scenarios however they are great for automatic data set annotation such data data set can be later on used to train much smaller and faster Target Model models like YOLO V8 or D are highly optimized to perform a specific task like object detection or instance segmentation on selected set of classes to Define that set of classes we need onology you can think about it as the recipe that can translate instructions understandable by based model into set of classes understandable by Target mode at the end we just use the data set creat with our base model to train our distilled mod today we are going to test out a distill and use it to fully automatically build YOLO V8 model capable of detecting bottles moving on a conveyor traditionally project like that would require several hours of annotation however today we are going to do it end to end in less than 30 minutes here's the plan first we need to create data set in principle to use outo distill you only need images no labels are required we will use grounded Sam base model to automatically create those annotations for us grounded Sam is actually a combination of two models that we have covered on this channel before grounding dyo and segment anything model grounding dyo is zero shot object detection model which pretty much means that it can detect objects on images without any training some on the other hand is prompt model capable of converting bounding boxes into segmentation masks as the result base model will create fully annotated train and validation data sets that we are going to use to train our Target YOLO V8 model the best thing about it is that the whole process is fully automated we'll just execute few commands and that's it okay enough of the talking let's dive in as usual we create a Jupiter notebook that I'm going to use during this tutorial I highly encourage you to open it in separate Tab and follow my instructions the link to Google collab is in the description below but you can also find it in roof flow notebooks repository first thing first we need to confirm that our Google collab environment as GPU accelerated to do that we'll scroll to the first cell of our notebook and run NV SMI command if everything works as expected we should see SMI output with details about our environment like driver and Cuda versions if not make sure to follow instructions in before you start section awesome now it's time to set up our python environment and to do that we need to install few packages each base and Target Model has its own repository and Peep package so when you design your aod distill pipeline you need to install packages associated with your base and Target Model in our case it's grounded Sam and YOLO V8 on top of that you also need to install the base autod distill package as it manages the whole distillation process you can find in installation instructions in autod distill repository along with the list of supported base and Target models in the end we just set up our home constant to make it a bit easier to manage paths to videos images labels and weights and we are good to go to use outo distill all you need to have is a set of images that you want to automatically annotate and use for Target Model training we will store those images in images subdirectory located in our home directory you can see it right now in file explorer if you are following along with your data set make sure to copy your images directly into that catalog however in this tutorial I will start with set of video files and show you how to turn it into ready to use collection of images to start with I have prepared a set of few videos recorded in milk bottling plant each video last few do seconds and shows the bottles moving on production line all videos are unique and each one was taken at different location and from different angle now all we need to do is to convert those videos into images to make it a little bit easier we will use supervision one of the packages that we installed during our python environment setup before we'll start processing we'll divide our videos into two groups test videos and train videos we'll put two test videos on the site and use them for model evaluation later on the rest is going to be converted into our data set supervision will open every video Loop over frames and save every 10th Frame into our images directory here is a small sample of our extracted images we can now pered to the next stage where we Auto annotate that data that part is straightforward but a bit time consuming as I said in the intro base models are powerful but unfortunately a bit slow before we start processing we need to specify objects we want to Auto annotate I would like my model to detect milk bottles and those small blue caps at the top I Define my onology and assign milk bottles to bottle class and blue caps to the cap class instructions on the left will be passed as prompts to grounded sum if you work with your data set make sure prompts are descriptive enough depending on your use case you may need to experiment a little bit if you want to learn more make sure to watch our previous videos we do a lot of prompt engineering experiments there the links are in the description now you just need to sit back relax and wait for the base model to AO annotate your data thanks to the magic of Cinema we can skip that part and take a look at base model results right away we can see that our data set got divided into two parts both containing images and labels similarly as before we can now take a small sample of the images and plot them along with autogenerated annotations well those results look very promising we are almost there now we just need to use our Auto annotated data set to train our YOLO V8 model the training process itself is exactly the same like with regular yolow V8 model I'm just setting up the starting checkpoint and the amount of epo click shift enter and we just need to wait for the model to train once again we will just use the magic of Cinema and speed up that process ah and if you want to learn more about YOLO V8 training we probably have the most popular YouTube video about it the link is in the top right corner and in the description below now it's time to test our distilled model some of you probably remember that at the beginning of the notebook we put two videos from our data set aside we did that to prevent leakage of test data into the training set now we can use those videos to evaluate our freshly trained model this way we can be sure that the model is capable of detecting milk bottles even on video footage that it has never seen before to perform actual inference we'll use ultral litics CLI we just need to provide two Puffs the first leading to our distilled model and the second leading to raw video the inference result honestly blew my mind sure we have some false positives and false negatives however remember we didn't even spend a second labeling those images of course the effectiveness of autod distill is highly dependent on specific project there are some where autod distill will performs flawlessly and can pretty much automate the entire process from start to finish on the other hand there are others where autod distill will need your help it will AO annotate the data but will require your input to clean up autogenerated labels however keep in mind that aod distill is a framework and with every powerful base model it will get better and better oh boy I can't wait for gp4 image input to be released so that we can finally see how well it handles automatic image labeling so remember gp4 is not just a language model it's also a Vision model in the meantime make sure to test out of this still on your data set and let us know in the comments which base and Target Model should we support next you can find the link to aod distill GitHub repository in the description below and that's all for today if you enjoyed the video make sure to like And subscribe and stay tuned for more computer vision content coming to this channel soon my name is Peter and I see you next time bye