How to do Object Detection with OpenCV [LIVE]

Key Takeaways

all right hello world it's Raj good to see you guys good to see everybody today how's everybody doing uh I'm going to list some names today all right Ashish against PARTA Dean Sumit Spencer how's everybody doing today yo good morning or good night wherever you are uh so good to see you guys um today this live stream I've excited to do this live stream uh today we're going to uh use open CV to detect a strawberry in an image and it's going to use no deep learning it's GNA use no straightup machine learning it's all open CV um all right so that's what we're going to do and I'm here in a co-working space so that's why the environment is different uh so okay so I'm going to start off with a five Q&A and then we're g to get into the code all right so here we go five minute Q&A hello hi how is everybody doing tensorflow versus psychic learning uh so tensorflow for like 95% of anything you want to do but for like the most simple type of machine learning that has nothing to do with neural networks it's like a linear regress pyit learn there's also TF learn but even that linear linear regression uses deep a deep neural net to do that um they're both the same amount of code but like you know just for like that line of best fit y equals MX plus b like what even is any machine learning psychic learn but then for everything else tensorflow what's your book about decentralized applications is about building apps that are that no Central entity controls and everybody gets paid for their data do you think Google's tpus will take off absolutely absolutely and we're going to see more um Hardware focused on neural networks in the future Google's just ahead of the curve are are you gonna show how to set open CV up um yeah I can do that as well uh yeah I can do that as well are your models deep so deep that I'm not gonna say that TF learn versus Kos TF learn for sure um which algo are we going to use uh uh for this one we're going to use there's several uh they're all open CD alos um and I'll explain them as we go um rap a song in Hindi maybe at the end uh and I'll probably do it in English just to be accessible to as many people as possible how how are self-driving cars going to cope with three third world messy roads that's a great question we need like uh one great example would be India like trying to think about self-driving cars in India is like this is like never going to happen but it has to happen and the way it's going to happen is is just better uh better algorithms and better better data because that's what we use right we have a great algorithm and we've got great data here um what we see um what's the best network for image classification supervised been trying convolutional neural network uh image classification uh probably uh well yeah convolutional nets for anything image related how hype should we should I be for Friday really hype I've been working with a team on this I'm so hype I mean we've been working on the weekend uh to bring what's coming out on Friday so really hype did I miss anything not yet uh detection included uh uh not face detection we're going to detect strawberries uh but this this applies I mean the same kind of logic will apply tensorflow performance on mobile versus open CV um uh performance- wise uh open CD is probably more performant no it is more performant uh because you're not not using the computation that deep learning requires we're not doing deep learning right now there's a trade-off here will it be real-time object object recognition I mean live video feed not real time it's gonna be an image we're gonna read it and then write it okay uh thanks Brian uh one more question and then we'll get started are you going to build a strawberry feature descriptor yes are you going to be using hor Cascade no is this still image recognition yes are you part of Linux Community yes okay so that's it for our uh five minute Q&A let's get started we are going to use open CV to detect strawberries okay it's going to be a image of strawberries we're going to use open CV alone to detect it it's going to be 100 lines of code we're going to do it in 45 minutes uh and uh I'll explain things as we go there's no deep learning coming out um and there's no deep loing that we're doing right now so let's get started I'll start screen sharing and then we're just gonna get right into the code all right here we go screen share time okay I've got two monitors so I can move this one over yay two monitors I don't have to switch back and forth so that's great okay let me move everything as I need to I'll move this over so I can see what you guys are saying all right cool so let me test that yeah everything works okay so let's get started with this um make myself smaller so we can see me and the code okay so here we go so we're going to start off by importing our dependencies and then so this gonna be about a 100 lines of code and then we're going to uh get right into it all right so step one is to import our dependencies okay so let's get started with this so the first thing we're going to do is import um let's see what what do we want to import here we want to import open TV oh right I said I would help you guys uh install open CV right okay so let let me just uh let me just get this so open CV installation uh so it depends on what operating system you're using but if you're using uh buntu it's just much easier if you're using Linux like open TV is just like made to install on Linux uh you can just use apps get and there's like three of these if you're using Mac I would recommend home brw to install open CV or or anaconda uh and if you're using Windows so I've actually never installed open open CV on Windows uh but I assume it's going to be pretty easy there might even be a binary for it U maybe not on this page but um but yeah okay so uh anyway okay let's just get to the code so we're GNA so the first thing we're gonna do is import opencv because that's the library we're g to use for computer vision and then we're going to import map plot live because map plot live is going to help us show our image we're not going to do any actual like like wrapping we're just going to try to show our image okay and we're going to call it PLT um there's that and so now we're gonna import numpy because we're going to do some matrix multiplication uh pretty much anytime we're doing any kind of uh computer vision machine learning uh deep learning we're gonna be importing numpy it's like the library for because everything is really just matrix multiplication in the end right we're just multiplying matrices okay okay so uh okay so we're going to import math in math because because we're going to use some uh things that are not related to we're going to do some math that numpy just doesn't have uh and then we're goingon to import division for some more math okay so that's it for our dependencies uh and we're going to just go ahead and write our highle method and then we'll write our helper methods okay uh so let's go ahead and write our highest level method okay and we'll call it f strawberry okay so let me show you guys the image that we're be using for this okay so the image we're going to be using for this is uh let's see it's called uh it's called berry. jpeg okay this is it and we're gonna and I'm gonna ask you guys to to just throw out a strawberry image and then we'll detect it that as well okay so I'll ask you that later and we'll see if if our code can detect that random one as well okay so this is the strawberry and the idea is we want to be able to segment it we want to draw an ellipse around it uh and then we want to you know distinguish it from this background so I'll show you guys what this looks like um and we're going to be able to apply this to many other images not just this one this is just an e easy image okay okay so and we're going to talk about the math as well so this is our first method it's our main function and we're going to use our helper function to do this the first line of code I'm going to write is I'm going to convert our image into the color scheme that we want okay so that is where open CV's uh convert color function comes in we're going to take our image and our image is that strawberry image that's going to be our input parameter we're going to take that image and we're going to apply this um conversion to it it's called color BGR to RGB now I'm going to explain what this is doing but I want to ask you guys does anybody know uh what the difference is between BGR and RGB I'll be actually very impressed because this took me a quite a bit of Googling to figure out why we do this in the first place I remember when I first started doing this so if anybody knows just shout that out in the next 10 seconds all right image CB2 color bg2 RGB Okay so anybody going once going twice and okay so what happens exact so yes order is one of them exactly yes blue green red and readability exactly the channels are reverse right so right so RGB stands for uh so okay so RGB is red green blue and BGR is BGR is um blue green red so what's the difference here well it's just it's just it's how it's How the colors are stored like the memory scheme of of the colors right and and and the ordering it's all about the ordering and and when whenever we're we're applying some sort of transformation to an image whenever we're applying some kind of we're trying to morph an image in some way the order of the colors matters depending on what's what technique we're using so depend so for the techniques we're about to use um RG uh RGB the RGB color scheme is better than vgr um and I'm I'm gonna explain uh why this is but it's basically the order of of the colors in memory so green so like uh in a so uh blue occupies the least significant area so a bite in a 32bit or 24bit uh format uh so it's it's just about bytes and memory and ordering okay and so that so that's that so that's our first step we want to convert convert to the correct color schem that's our first step once we've done that it's the same image it doesn't look any different it's just that the color scheme and how it's ordered in memory is different okay so that's our first step um and so now that we've converted to the correct color scheme we're we're gonna make sure that it's the correct size so we'll say okay uh we want to get the size of this okay so we're gonna say we're gonna use Python's built-in Max function to take that image and get it shape okay so the the shape of that image is uh the you know the the size of it the window size and we want it to be um uh we want it to be the right right uh size so to do that we're going to uh get the we're going to scale it so we're going to say okay let's let's let's get the let's get the scale of it because we're going to resize it in a second so we're say out of 700 take that uh divide what we just got by 700 why are we doing this because the maximum window size uh uh that we're going to use is 700 by 660 pixels so we want to make our we want to make our uh strawberry image fit in that image so it's gonna be 700 divided by what it already is so it's smaller than that okay so we're going to scale it like that and then finally we're going to resize it okay so so all three lines of code here are to reset are to uh scale our image properly okay so that's a step let me just say step one and then step two right step one step two that's what this is step one step two okay so um so we're going to use open TV's very handy resize function for this okay um so we're going to take our image and then we're going to say um uh so this value is going to be none but this is a this is this is an optional value which we're not going to deal with right now but um so but we want to make the width and the height the same scale so it's going to be a square instead of a rectangle so what's so what's the length of that X so what's the length so we're going to say that's going to be scale and then whatever the 700 divided by that Max Dimension is and the FY is going to be scale right so it's it's the same it's a it's a square the same length same width okay so we' so we scaled our image and it's a square and it's it's it's ready to be uh uh it's ready to be uh uh we're gonna apply our open CV techniques to it okay so that's step two and so now step three is we want to clean our image so what do I mean by that what do I mean by that so let's say step three is clean our image okay so I'm gonna say uh let's blur our image right so I'm going to say image blur equals and I'm going to use open TV's function called g g Ian blur okay what is this so gossan blur is whenever we want to eliminate noise from an image we want to we want to smooth the colors so like think about a strawberry right a strawberry um can have uh you know it's got those uh what are those little yellow dots on the strawberry like the the kind of seed like things whatever they are we wna we want to remove that because we just want to see the the a Pure Clean red image and sometimes when we take an image and we convert it to a different uh color scheme it it's got like little black dots in it we don't want that we want a clean smooth color across the image so we could just say boom this is what it is so that's what that's what that's what this is do that's what this is the goian blur does it takes the color scheme across a gosan distribution and it and it and it Smooths the image across the distribution um so uh you're risking detention to watch a stream live spark chicken thank you so much I appreciate it all right um so so we're going we're going to apply to our image and we're going to uh uh so that's the input image and we want the kernel size what the kernel size going to be well it's going to be 7 by S because you know 700 by 700 uh and so like with the size of the image and then uh how much do we want and then the last one is the last value is GNA be zero which is uh how how much uh we want to filter it which uh we don't really need right now because uh We've the gosting blurt like by default already does that uh so the last the last parameter we're just going to Z because that that's an optional value we don't have to deal with that um so we've so we've got our image and and we we've blurred it but there's there's one more thing we have to do uh we have to convert the color scheme yet again why is that we're going to convert it to what's called HSV format okay um all right so um okay so we're going to convert it to HSV and uh so we're going to use the convert color again and we're going to take that blurred image and and then we're going to add so then we're going to add the and and open CB has a bunch of these uh like conversion uh attributes so we're gonna say take the RGB and now convert it to HSB so we converted it to uh RGB so that we could scale it and now we're going to convert it to another format called HSV uh so that we uh so once it's blurred because now we're going to uh filter by the color okay so what is HSV so HSV form is a different color scheme but basically it separates two things it separates the uh the Luma or the image intensity from the chroma or the color information okay um so we're separating the color from the uh from the from the brightness of it so we just want to focus on color right so that's why we're segmenting these things RGB and BGR They Don't Really separate the the the Luma or intense color intensity from the color itself it's just one it's it's just one thing so that's what HSV is going to help us with so we're going to separate these two and we're going to filter them based on that okay uh yeah exactly uh H saturation value okay so um so that's what we're going to do for that uh and now that we've converted it to HSV um we're going to do what we do best uh do do what HSV is best for which is to um Define uh we're defining a what are we defining here we're defining our filters Define our filters okay so to Define our filters um oh no worries Paulo thanks for thanks for saying that okay so uh we're going to filter by the color so the we're going to filter by the color filter by the color that's the first one we're gonna filter by the color um so remember color is separate from the brightness color is separate from the brightness the intensity of the color is separate from the intensity of the brightness and and so we want to detect strawberries in a certain color range right so there's a certain redness of a strawberry if it's if it's more red than that then it's not a strawberry if it's less red than that then it's not a strawberry but it's in a in a particular range and generally for this we would guess and check what this is so I've got I've got I'm gonna say what's the minimum amount of red and this is where we're using nump because remember colors uh computers recolors as uh thank you 1A thank you 1A okay so so we're going to use numpy's array function to to do this right so remember colors are like you know like if if if if we if you guys used CSS before for anything right 255 255 2 these are these are hex values and so we're gonna use numpy's array function to Define these hex values so we're gon say okay so this is the minimum minimum red value it's because we say 0 180 okay uh and then that's the minimum amount of redness then the max amount of redness is going to be yet another num and we're going to Define this these are bounding colors uh so this is the max amount of redness that we want so we we'll say it's 10 256 and then 256 that's as red as we want to get so anything in that range of redness uh um no I did get a neon shout out uh yeah I mean it's it's all good yeah I mean it's not a big deal I mean I I'm I'm going to I'm going to eventually you know do more stuff with him and stuff no but I was just speaking with Sebastian th recently anyway okay so um uh okay so what okay so we're filtered by the color and now we're going to create a layer with this okay okay so we're gonna create what's called a a mask mask we could call it a mask we could call it a filter mask and filter are the same thing so we'll call it a mask and this mask is like um if we were to take an image and we just focus on one color and we just black out or blur everything else like we don't care about everything else okay so that's that's the mask and it's just to find the color value okay um okay so this so that's the mask um in range uh so we're gonna say okay so what is that value we just find image blur HSV Min red and then Max red okay so Min red and Max red uh so those are our values for our image blur right so that so that's our first mask we we want one more mask and that's why we converted to HSV because we're filtering by not just color but by intensity okay um I did a I did a video for open AI which is Elon Squad AI Squad and Elon watched it and liked it and so like now you know he knows about you know what's good and you know with me and stuff so okay um so minimum red uh so okay so we're going to focus on the brightness now right so this is why we converted HSV so minimum red I got to focus on the code here so we're going to say minimum red to right because we already find a one called minimum red and again we're going to do another array so this is going to be 170 100 and then and this is focused on the brightness okay so that we have a minimum value and a max value and Max red two equals. aray and what's our Max brightness we want 2566 okay so that's our that's our max value make sure I've got my commas there okay cool so let's let's see this is filter by brightness okay filter by brightness okay so that's that um and so now we're going to create our second mask we have two masks to make and we're going to our second mask is going to just focus on the brightness so we're gonna say open CV has this in range function that we we can use for colors for for Hue for saturation for a bunch of ranges that we want to Define okay and we'll say image lur HSV that's the the same image uh and this time we're going to bound it by uh our our uh color our brightness variables okay so now we have our two masks does anybody know what we're going to do with these two masks uh I'll be impressed who who knows okay so shout out if you know what we're going to uh uh do with these two masks take these two masks and we're going to we're going to do something with them okay so what we're going to do with these two and and them no mask plus mask equals double mask spark chicken exactly we are going to do double mask double rainbow double mask we're going to combine our masks we're gonna combine both of them okay so we'll say mask equals mask one plus Mask two that's it we've combined our masks how easy was that okay so we've we've done that for our masks and now we're going to um we've combined our masks and now we're going to uh do some uh that was step four and then step five is uh we're going to segment segment step five is segmentation we're going to segment the image what does that mean step five is segmentation we're gonna we're gonna use those masks to separate the strawberry from everything else okay that's step five segmentation so we'll call this a kernel and let me explain that in a second but so okay so a kernel is we're going to use open CV's get structuring element method and let me Define what that is is let me write this down cbq morph cpse and we'll say this is going to be 15 by 15 uh okay so um we want to Circle our strawberry so we'll Circle it with an Elipse which is like a circle but you know not as per not a perfect circle uh with a with a shape of 15 by 15 and that's that's why we use the the morph ellipse function because we're going to get that structuring element basically kernel is going to have an ellipse in it okay okay and we want to we want to make that ellipse fit around our strawberry so we so we'll segment it all right um okay so we got segmentation and uh now we're going to say uh let now we're gonna do some what's called morphology let me explain that in a second so we're say max. close um morphology examp X so we'll take our mask we'll take our morph clo and then our kernel okay then our kernel um okay so what is this before I explain this let me write out one more line but this is goingon to be called Mass clean because these are both related to each other so let me write this last one out morphology x uh this is a lowercase x got to be perfect about the syntax here Mass closed cd2 morph open and then one more thing the kernel okay so what is what am I doing here so we want to warp the image so uh we want to add that ellipse around the strawberry so uh we're GNA first uh perform a closing operation what does that mean uh a closing operation is dilation followed by erosion it's it's it's process called dilation followed by erosion what is this it's useful for closing small hes small holes inside the foreground of objects like small black points in the object it it helps further refine that um that that that smoothness it it it helps it's like another it's like an it's like another safety check to make sure that it's a smooth uh in this case red layer and then uh uh morph and then mask and then morph open is taking is is the opposite it's erosion followed by dilation so the first one is dilation followed by erosion and the next one is erosion followed by dilation okay um yeah definitely uh thanks Z for help me answer the questions as well uh okay and so it's useful for removing noise so uh both and it's not that they reverse each other they both add to each other okay so um so we got that and so now we're gonna find the biggest strawberry so step six is uh find the biggest strawberry we don't want we don't just want to detect all strawberries we want to find the biggest strawberry so if we have a bunch of them we only want to circle the biggest one okay so we'll say what's the big strawberry contour and is the the shape okay so we're going to get that and so and so we want to get and we're going to get the mass as well so this is the method that we're going to um okay so we're gonna say uh and we're gonna write this method out but it's gonna be a find the biggest contour and that is find the biggest elix so so what our what our algorithm is going to do is it's not going to differentiate it's just gonna say oh strawberry draw a lips oh strawberry draw a lips oh strawberry draw a lips and then this method find the biggest Contour is going to find the biggest and say that's the strawberry we want out of all of them so that's what this method is going to do and we're going we're going to write that in a second um and we're going to use a mask clean parameter which uh to to to do that okay so that's G to find the biggest strawberry uh and now we're gonna um Step seven is to Overlay um we're going to do that uh to do this and uh yeah Short history of Columbia wrote a book traveled around the world for year uh went to San Francisco worked at Tulio as a developer educator best technical writing team on the planet uh worked you know some contract work here and there mobile development did a lot of independent research uh um with a couple um uh distributed systems researchers then some AI researchers that I just knew you know it's just like you know moving around and figuring things out uh and by the way these centralized applications was the best selling software engineering book on Amazon uh in 2016 so but I'm not I'm not like super happy with it even though it was because it wasn't my best work and the reviews weren't like as good as I wanted I want like perfect five star reviews so I'm going to write another AI book in the future probably this year I'll write another book um and I wrote that book before I had any following and so now it's gonna be awesome to write a book uh but I gotta make time for it anyway to get back to the subject um uh okay so so so so step seven is to Overlay the max that we created that we created on the image that's the next that's step seven so we're going to say overlay so we've created these masks for for color for brightness and we're going to Overlay them on our image and we're going to write this function right so we're going to say Mass clean and then the image okay um and so now we're going to uh find the biggest strawberry and now uh step eight is to circle the biggest one because we've differentiated which one it is and now we're going to circle the biggest uh strawberry okay Circle the biggest strawberry um to do this and um uh uh parents born in India I was born in Houston Texas um but I've been to India it was a lot of fun uh fun is a yeah fun is more like a really hard adventure to travel through India like I went to like see where my parents are from and see what my culture was like and it's a beautiful experience and it was uh India is is is an adventure to it's it's it's anyway uh I really like Mumbai though great City okay um uh so what was I doing so I'm going to circle the biggest strawberry so I'll say circled Circle Contour um and we'll say overlay big strawberry yeah oh sh cool cool uh uh big strawberry Contour uh and then so we've circled it uh we'll show it now and we'll write the show function to show it uh and yeah that the show function is going to show our Circle it's going to show our final result and the last step step nine last step is to convert our image back to the original convert back to original color scheme thanks Robert uh thanks VJ uh okay so so convert it back to um our original color scheme CB2 color circled CB2 and now it's opposite method rgb2 BGR okay that's our last step and we convert it back and we'll just return that uh reconverted value okay so that's our main method now we have to write our helper methods okay we've written our main method and now let's write our helper methods all right so um so that's our main method now let's write our helper methods and I'll start up at the very very top okay so um the first thing I'm going to do is Define the color green which I'm going to use in a second I'll Define it as a global so I'll say 0 255 Zer and that's just defines green okay um I'm 25 years old so um so the first thing we're gonna do is write that show method right the sorry the show function remember that show function how we're going to show our image um so we're gonna say take that image as an input get the figure size uh in inches and then we're going to and this is where we're actually using mpot Library this is this is this the reason we imported mpot live uh because we're gonna show images with it so map plot live isn't just good for plotting it's also good for showing images easily in Python um you can do it in one or two lines of code so 1010 uh and then show the image so i' i' I've got the figure size and now I can show it uh image and then the interpolation is miror interpolation equals nearest that's that's that's short hand for nearest neighbor interpolation um uh okay thanks the 13th um I've been streaming for 30 minutes now show nearest okay that's for that stand for nearest neighbor instilation that's GNA help us show our image uh okay so that's our show function and now remember the overlay mask function okay so let me show you guys where where I define that oh and what that's going to do so remember the show image was down here and now the uh overlay mask image where was that was here and that's going to Overlay The Masks we created on the image so we're going to take that cleaned mask and we're g to apply it to the image and this is the actual application process of applying the masks uh to um the the process okay so overlay mask on the image so let's make the make the mask RGB um so we say RGB mask equals CB2 do convert the color and we're going to take that mask and we're going to convert it to uh gray scale just for this uh conversion process so there's a lot of as you can see there's a lot of color schem conversion happening here and and in this case we're converting it to gray scale uh because well let me let me write this out and then I'll explain why we're converting it to gr scale the the the the second line is to um add the weighted sum of two arrays and let me explain in a second uh let me just try this out and then we'll return the image okay so what's happening here so uh so what ad weighted is doing is is calculating the weighted sum of two arrays all right in this case is going to be our image arrays we can think of our images as arrays um and so if we just think of uh the uh both images as array arrays when we add when we add them together we're adding the weighted sum of the of the image values like the sheer uh the numer value of images if we just think of images as just raw numbers that just build up into an image that you can see if we add the weighted some of those the mask and the original image we'll get the mask overlaid on top of the image and we're converting it to um uh the The Mask to RGB uh because uh the original image is in RGB so we'll just well because the mask is going to be in Gray scale we'll convert to RGB so they're both so they're both going to be um they're going to be there RGB to gray uh right cuz it's because we're not converting the image we're converting the the mask from gr to RGB okay and then we're going to return the image so that's that how many helper functions do we've got we've got we've got two more helper functions and then we're good to go okay so those are our first two and now we're going to do our next two helper functions so the next one is going to be find the biggest Contour right so we've got our Contours or ellipses and we're gonna detect the biggest one out of all of them and we Define this function right here where did I Define it right here the biggest strawberry it's going to return the biggest ellipse for that strawberry as well as the mask for those strawberries okay so let's do that right now so to find the biggest Contour first we're GNA make a copy of the image right we want to we want to retain the the the original image um and we're going to retain the original image but we want a copy of it so we can modify it and this is this is where opencv's uh image copy uh function comes really comes comes really handy we could copy the image but then keep the other one separate so now we're going to we're going to get those Contours so how do we do this well luckily for US Open CV has a function called find Contours so given an image given an image um and these two values it's going to find those Contours I'm going to explain these values in a second um so the first is retrieve list RT RR list which is Shand for retrieve list and the next one is called chain approx simple okay so chain approx simple what what what are we doing here okay so what we're doing here is what this what this function does is it gives us all the Contours and the Contour approximation com uh and we want to uh uh so we're going to use the retrieve list to get the Contour approximation that compresses the horizontal vertical and diagonal segments and leaves only with their end points and so that's what the approx uh approximation approximation simple function does we only want the the the ones that get we only want their endend points we don't want all the Contours so it limits what we're contouring so for example for an upright rectangular con Contour it's encoded with four points um and we're GNA get a list of those Contours so that's what the retrieve list does it's we want a list of those Contours okay um uh okay so so that's what that does and we're gonna get we're gonna get the Contours and we're gonna get a hierarchy of those uh which is uh the the the chain of Contours from the from the greatest to least so the from you know the the size of all these Contours or ellipses okay um and so now we want to isolate the largest one now that we have a list of them let's isolate so we get just the largest one so say Contour sizes we want to get a list of all those sizes and we're going to use um uh we're going to use open CVS function to do this um Contour the find the Contour area and all right um Contour Contour or Contour in Contours okay so for every Contour in our list of Contours get the area of of that and then store it in our Contour sizes uh array okay that's what that does let me make sure I have the syntax correct here I do have the correct syntax Okay cool so now we want to get the biggest one we've got a size of all of them and so now how do we get the biggest one luckily for us luckily for us um python has that built-in Max function we'll just get the the max value of our array uh which will automatically detect the biggest value in our array of contour and oh let me see this let me write this out Lambda um X and x0 um and then one so we want that first value U for so yeah so then for the key we just want that so once we've um we've got we want the max value so let me write down Max um and once they're ordered we want the one we want the the Contour in the first position that's where that that's where that uh one value comes from uh and then which is going to be because it's ordered right so we'll get the biggest one at the beginning of the array and then let me say let me say what this is we're isolating the Contours we're isolating the sorry we're isolating the largest Contour okay um uh okay so that's isolating the largest Contour um and uh so then our last step is to return the the biggest one return the biggest the biggest Contour okay um line 25 what do we got here Contour hierarchy CB2 toine Contours image through yeah we're good here right oh yes Contours thank you aush Contours hierarchy and then uh here okay cool so now we're GNA return the biggest Contour so let's get that mask we'll say n zeros and then get the shape of the image and then 8 so we're going to get the the mask of via the shape and then we're going to draw the Contours and we're going to return it we're this is where we actually draw the cont the the Ellipsis on our strawberry uh and then I'm gonna I'm gonna demo this so hard for you guys in a second it's gonna be awesome we're gonna try multiple images we're gonna get all the beautiful uh Contours to get work to to show up okay so we're going to draw those Contours with the range from using the mask from the biggest cont uh using the biggest Contour uh in the range up till 255 uh terms of the color scheme and then we're going to return it okay return biggest Contour mask we have one more method to write okay Circle Contour this is where we actually Define um this is where we Define uh the actual Contour itself so we'll say we'll say the image Contour okay um this is okay so this is where we um uh this is where we uh Define the shape of that ellipse this is where we Define the shape of that Contour um so we'll say okay so get the bounding get the bounding ellipse first this is our last function it's G to be about five more lines and and then we're done we'll say we'll get our bounding ellipse say image with ellipse and then I'll do I'll demo it and then I'll do my last five minute Q&A and I'm GNA use a copy of the image because we want two copies of our image we're gonna right so then um we'll say the ellipse is um we're going to fit our ellipse to our uh contour and then we're going to add it and by add it I mean take that ellipse function um to take the image with the ellipse so we're gonna take that image with the ellipse on it um and then add our fitted ellipse so it's a a size a better size we're gonna say I want the ellipse to be green um uh with uh value of two because we want uh that that's the size of the the length of the the width of that that contour line um and then TV2 do TV AA um and then we'll return it okay and then um image with ellipse okay so let's compile this baby um and see if it works we'll just compile okay so uh let let me just see if this works oh no no no I forgot something guys I forgot something we've got to actually write that that last code to to read in the image I never did that so it's three lines read the image in three lines so what is that we we'll get an image using uh open TV's image read function and the image is called yo. jpeg which is the image of the strawberry I showed you guys at the beginning we'll take the result um um all right CH Carmen car Carmen okay so we'll say uh find a strawberry and then using that image and then write it that's it okay so now we're going to demo our code write the new image uh you and I'll write it using open TV's image WR function yo 2. jpeg we'll call yo2 the new image and use the result as the parameter for that because that's our newly image okay so let's let me open the berry so this is our initial strawberry right this is our initial strawberry and now once we up run this code which I've called fun. Pi once we run this code which I called fund. piy okay let's see we've got an error here find Strawberry image so on line 100 I said find Strawberry image uh oh what am I doing here oh right equal sign yes here it comes guys here it comes so uh let's on future import division oh future import have to happen at the beginning of a f I forgot about that okay so remove that or add it there let's try that um okay so this says Okay so the problem with that is um okay so this is a weird error so okay hold on let me make this bigger so you guys can see um so python uh FP right oh right it's not Barry it's uh python uhp right oh yeah so um interesting okay cb. color BGR to RGB and find Strawberry okay no no no that's definitely correct um let me just remember what the deal is oh right we got to convert Barry not yo we'll call this Barry too so it was a wrong image right okay so that that's what it was all right so let's let's run this this is going to work now global name biggest is not defined okay let's see where that is um debugging time exactly debugging is fun as hell sometimes um it's starting to become more fun even when I'm live because I'm getting more used to this okay so what is so biggest let's find biggest biggest is not defined what are you talking about I Define biggest obviously I didn't because it's not there but okay so I said biggest um and right so where should I Define biggest so biggest um oh biggest Contour not biggest uh that it's biggest Contour Contour okay let's try that okay interpolation on line 14 unexpected keyword interpolation image interpolation PLT oh image show not show image show okay let's try that y okay that worked I mean well let's see if it works so it's called Barry 2 okay yes how cool is that so this our original Barry and this is the one that it detected okay so how the hell did it do this okay it detected it right so to do this we here's what we did here are the steps and now and now I want you guys to find a strawberry image on the net and just post it in uh in the in the uh chat so I'll try it on that image as well while I explain this what we did was we first converted the color scheme to from BGR to RGB so that we could so that we could scale our image properly the next step is we scaled our image so it fits in the window size that map plot live requires uh which is you know under 700 by 700 it's a square image we turn our strawberry into a square image then we cleaned our image using the goian blur function which smooth the image so that it's a it's it's it's a one uh one color scheme uh that we can just focus on like red instead of you know little black holes and yellow and all that and so then we Define our filters what's what's the minimum red what's the maximum red um and then what's the minimum brightness and the maximum brightness so we use two filters we combined them we applied them to the image we segmented the strawberry from the rest we got a list of strawberries in this case we only had one but this would apply to multiple strawberries as well we added a mask we overlaid the mask and then we circled the biggest one and then we showed it and then we converted our original image back and that wasn't necessary but we just let's just convert it back to BGR so that we could perform other um processing tasks on it so okay so that's the code for that let me try out one more image okay strawberry image uh and we'll do several let's pick uh let's pick this one okay and we'll save that and we'll go to our um uh we'll move that to this uh here and we're going to rename so then we'll open this file and we'll say what was it called what was it called it was called yot test. jpeg we'll rename it yot test. jpeg we'll use JPEG and then in our code we're going to uh exactly yo test. JPEG and then we'll say yoest 2. JPEG and let's see what happens well I've never tested on this code in particular uh let's see what happens here fun. py okay and now we'll open yest 2. jpeg do with it okay so these okay so it just circled the entire thing uh but and it's because these these strawberries were bunched together so well we can further improve on this to make sure that it only right um segments the best one our code is ideal if the strawberries aren't touching but these are these are touching so that's that um uh and there's there's one more thing I want to say about this so let me let me start screen sharing again let me stop screen sharing get back to um stop screen sharing okay so so yeah I mean open CV it's been in use for so long you know there's so many built-in methods transforms different types of you know all sorts of research has gone into open CV U to to do that but deep learning now is slowly taking over so if we had and let me just show this right here so it's like it's like it's like if this is deep learning can you guys see this oh my God it's backwards right I forgot about that um how does shiftman do it where it's like not backwards interesting uh okay so I can just do it with my hands I do that anyway so it's like deep learning was like uh so open CV was everything it was all of computer vision but then deep learning showed up and now deep learning slowly taking over so right now you can use a little bit of open CV and a little bit of deep learning but eventually deep learning will just take over and it'll all be deep learning uh so like right now what I mean for by that is like well we can use open CB to segment a strawberry and then we could use deep learning to identify it as a strawberry so we could say okay this is a strawberry in a picture write a circle around it and then use deep to detect well what is the name of that thing okay so he say we could Define the shape and color of something and then we could say we use deep learning to to recognize what that is um and also deep learning is computationally expensive right so sometimes you want a quick and dirty solution you just want to segment an image and you don't want to have to and U although deeping learns features you just want to encode those features yourself because it's something simple you're doing open CV is great for that um so yeah so that's that uh and let me do a five minute Q&A at the end and then we'll and then that's it for the live stream any any other questions oh it's only backwards for me you're right right you're right I forgot about that um yeah shipment is awesome uh will you make a Ser Series where you make build a robot using AI yes that's coming up uh I'm gonna use a drone uh will you make any Google search algorithm that's a great idea um that's a great idea Aur I never thought about that I should do a search algorithm um what was your big announcement last Friday wait two days it's coming out dead hype it's coming out this Friday C hell or high water it is coming out this Friday um you should release more videos I I'm on that the thing is I don't want to I don't want to I'm never I'm never gonna let anyone else write the scripts for my videos I don't care who it is I don't care who pays me how much I every single word that I speak will come from my heart my soul my mind so if that doesn't scale it doesn't scale but I will never speak the words that someone else tells me to speak okay I promise you that uh make a search algorithm that would find the most complete comp complete answer to any question um yeah I'll add that to the search algorithm video I make it please make a dedicated video for Q&A section A I'll do that in the future can I get a shout out hi JD can you recommend can you recommend cntk um I I wouldn't recommend cntk actually I would recommend tensorflow uh yes please support me with patreon I'm not for sale exactly thank you JD is there a simple way in open CV to get the total variation for Hue and saturation um uh yes there is uh and I um yes there is um but I don't have time to show exactly that uh thank you Enzo uh how to use neuronet and open CB I'll do more on this stuff later uh two more questions and then we're good to go okay two more questions uh can you make a video on motion tracking in open TV that's a great idea that's a great idea um I can do that Alejandro one more question um where did you learn all these from how much soilent do you drink uh I stopped drinking soilent I used to drink it a lot um where did you learn all these guys it's um so I think a lot of us are bounded by the idea of we have to learn a certain way and it has to be in this exact format for us to tell ourselves that we now know this but look if you if you like you know read if you really take a day to read like I don't know Andre karpathy is that famous lstm blog post from start to finish and you're in every time you see a word or phrase or equation that you don't understand you Google that and then you go back to it if you complete an article like that like a very heavy article You Now understand at a high level how an lstm works you don't necessarily need you know like a four-year degree to say now I know how lstms work so we have to we have to shift how we think about learning it's not so much about about having to do a certain structure it's about telling yourself and believing in yourself that you can you you can know this you do know this and you can can build something with it and so that's what I'm here to help you with I'm here to inspire you guys guide you guys on this journey that we are all on okay we're all learning okay so um one more research intership or internship in a startup uh that's a great question it depends what you're doing in the startup if you're doing research stuff uh me personally I would do a research intership just because I'm interested in research um cool so that's it Reddit AMA that's coming up someone has to like request me right like request me to do an AMA I'm not just to say hey I'm I want do an am okay so if rdit contacts me or somebody eventually I will I know the CEO he's a cool guy okay um so that's it for this uh Q&A and for this live session I love you guys so much thank you for watching um uh something cool is coming out on Friday it's it's happening this Friday come hello high water I I made I'm making this promise to you guys it's gonna be awesome um and thank you so much for uh for doing this for for being here okay so for now I've gotta find a place with better lighting because I love natural lighting so thanks for watching