Community Spotlight: Improving Uno with Computer Vision

Key Takeaways

hey there it's joseph from roboflow today i'm going to show you a community spotlight that means someone that used rubbleflow to build something super cool if you want to feature your project on the roboflow blog on the youtube channel and share with our thousands of subscribers and our email newsletter and other places be sure to use the link in the description to submit your project so today i'm going to be highlighting something that was a quarantine project from a software developer in wales united kingdom adam crawshaw found himself playing a lot of uno during quarantine and if you're like me you might think that how would you use computer vision to make uno be more fun it actually stands in the details for starters uno isn't won by the first person to get rid of all of their cards in fact buried in the official uno rule book which don't worry i also didn't read is this rule here which says the first of 500 points wins and the way that you earn points in uno is you get rid of all your cards and then based on the number of cards your opponent still has you sum up each of those cards where each card is worth a different number of points for example the skip is worth 20 points the draw two is worth 20 points and then any of the zero to nine cards are just worth their face value so a card with a four is worth four points for example so our uh subject here adam found himself playing a lot of uno and get a little tired of counting all of the cards uh that he still had in his hand or his opponent still had in their hand and so he used computer vision to automatically identify the value that a given card was and then sum them up his results are truly stunning look at this run in real time adam here lays down each of the cards and you can see the computer vision algorithm identifies the number on the card and on the right you can see it's saying the probability for each of those cards in real time and doing the sum score how did he do it well let's do a little bit of an overview here first and foremost adam had to collect a bunch of data he collected a bunch of cards specifically he took about uh four photos of each card so four photos of the three four photos of the draw two etc etc and then he labeled the number in the upper left hand corner critically adam did not label the number in the upper right hand corner or give me the upper left hand corner and the bottom right hand corner but only the upper left hand corner because he didn't want to count a card twice he only wanted to counter the upper left-hand number once he had four versions of each of these cards meaning 41 source images and all of those labels he used a little bit of open cv to lay cards on top of each other and put them in a variety of different backgrounds so for example he put these three cards on this different textured background in fact he grabbed a bunch of different textured backgrounds from the oxford describable textures data set and voila he had backgrounds like this all in all he created a data set of 8992 examples of all these labeled cards with that amount of data he was ready to train a model but not so fast with that raw data he had to do a little bit of augmentation and pre-processing specifically he wanted all of his images to be 416 by 416 and augment the images to say uh randomly cropped to zoom in on part of the images to do rotations and flips and things like this so that his data would uh be more representative of the problem that he was trying to solve i mean you get it adam could be in a room that might be brighter or darker if that room's brighter he still wants his machine to work or his phone to work just as well on detecting the cards if it's darker the same thing is true so he used roboflow like random crops brightness changes etc to improve the variation of his data in his data set now adam actually had nice things to say about roboflow in this process because he had his data on roboflow but wanted to train in a variety of different places he tried collab notebooks he tried managed notebooks on google cloud platform and adam had something to say like quote the data is right there it's in the right format and i know it's all there it goes from god knows how many lines of code to whatever it needs so that my data is easily available in any environment that's right with rovell he was able to export to the yolo format to tensorflow format to pytorch to try a bunch of different models now adam tried a few models from the roboflow model library like yolo v3 and momonet ssd v2 actually when i caught up with adam recently he said it was recently trying yellow v5 in comparison to yellow v3 though this post is written in the context of using yolo v3 for these results he deployed his model using tensorflow lite to his android device and with that it was able to run on device and create the stunning results that he shared with us above wow really really really well done of a project adam hats off to you and thanks so much for sharing your story reflow with roboflow if you want to recreate adam's results adam was kind enough to release the open source data set that's right if you go to public.robo you'll see the uno cards dataset you click in and you have all the 8992 images in the v1 version with 21 582 images that adam augmented you can go ahead and fork this data set to your own account or download it locally now adam was kind enough to make the data set mit licensed which means you could build a commercial application that scores uno cards automatically but it's not just any mit license adam was careful to say i want to use the hippocratic mit license which means do no harm so only make well-intended uno games now with all that said please have a look at the post links in the description and if you want to share your story your community story of how you use computer vision fill out the form link is in the description cannot wait to hear what you've built with roboflow and computer vision thanks