Introducing Label Assist: Model Assisted Pre-Annotation for Computer Vision

Key Takeaways

everyone it's brad from roboflow here to share with you a brand new feature that we just launched called label assist now label assist is a really exciting feature because it helps you accelerate your machine learning workflow with machine learning itself so here's how it works normally if you're creating a data set for computer vision you'll have to go through every image in your training set and create an annotation yourself this involves manually drawing a bounding box around every object that you want your model to learn from so here it would be all the cars with label assist you don't have to do that at all you just click this button over here and you can use an existing trained model to jump start the process and you start from the model's predictions instead of starting from a blank image so here i'm going to select the ms coco data set which is a data set filled with common objects from around the world it has 80 different classes and so if the things that you're looking for are involved in that data set you don't have to do anything at all you just come in here and you can jump start the process from these existing classes so in this data set i'm going to be looking for the cars and the trucks so i just select those two classes and to bootstrap my model and off i go so now every image that i start labeling is going to get pre-annotated by the predictions from the model and the only thing that i need to do is correct when the model got something wrong we don't need them teach the model the things that it already knows we want to spend our time annotating the things that are going to help the model do better in the future so here it missed this jeep so i'm going to label that as a truck and this is really good because the next time we go through and train it's going to learn from that annotation that i just added that its previous predictions were wrong but most of the time i just need to look and verify that things are correct or maybe just tweak them a little bit so here it got all these cars just perfect but what do you do if your data set contains objects that aren't present in the ms coco data set that could be a little bit more tricky but not to fear label assist can help with that too with roboflow train you can bootstrap your model with a few hundred manually labeled images then train a model from that so here i've gotten really good results but i've noticed that this model isn't quite doing well on some particular images and then i'm going to add more images from production when i learn how the model is performing poorly and and like what the error cases are so to train the next version of my model i'm going to go in here and i'm going to add more images so i have some more images that i collected from the wild of a chessboard that i'm going to drop in here and now when i go into the label assist tool instead of selecting mscoco i can go through and i can select my labeled chess data set to start from so this loads my custom model that i've already trained with roboflowtrain into the browser to do the same thing that we just did with ms coco except with our custom objects instead of the generic objects that are found in that other data set so here i can select a subset of the pieces but i probably don't want to do that in this data set because i'm actually looking for all of these things so when i select select the same thing happens it pre-annotates my images and as i noted these were images that the data that the model was having problems with so i know that there's going to be problems that i need to go in and correct but here i can just type white queen and just add the annotations for the things that the model missed instead of starting from scratch from scratch you can see it got about half of them i can also go down here and adjust the confidence level so if i boost that down a little bit it'll it'll find and label uh objects that the model was less certain of and this means that i can tune it a little bit so that maybe if it had found this rook over here but it wasn't quite sure enough of it to get a 50 confidence it might show it in the future um and this will make my labeling go really fast so i can go through and i can just double check that all these annotations are correct and fix the ones that aren't instead of starting from scratch and labeling all of these different pieces so that's the label assist the label assist from public data sets like ms coco is available on all of our accounts included for free you just need to sign up for a free account at app.roboflow.com and you'll be able to to jump into the labeling flow and use ms coco to jump start your labeling process if you want access to train to the custom models you'll need access to roboflow pro which is part of our uh paid offerings so reach out and we'd be happy to set you up with some train credits and let you try it out and discuss how this could be used for your business or your project so until next time uh happy training