An Introduction to Google Vision API | DataHour by Ravi chaurasia

Key Takeaways

you can start okay uh so good afternoon good morning good evening everyone across the globe so myself and I'm currently working as an application development team lead at Accenture I have more than five years of experience in it I've worked on multiple Technologies like RPA AI ml DL and in fact dictator Technologies so I I want to work in a field wherein I can solve real world problems with the help of intelligent Automation and when I say intelligent automation it means the integration of RPA with the artificial artificial intelligence so that's where we can try to reimagine the entire business across the globe okay so without wasting much of all of us all of our time let me quickly share my screen and start off with today's session I I'm not able to share my screen soon could you please help me sorry about that now you can like share the screen okay so let me quickly share my screen is my screen visible yes okay thank you Okay so okay so here I have logged in inside uh cloud.google.com okay so this is a the official page for this Google Vision API yeah so whenever we say if like at AI so it stands for artificial intelligence as we all know so nowadays across the globe everyone everyone everywhere artificial intelligence is used correct so it has a lot of functionalities and it is helping a lot in order to solve the real world problems so whenever we say Google Google views an API okay so it's a product launched by Google which gives us the functionality of all the artificial intelligence related stuff and we can use it very easily it's very easy okay so for example whenever we upload photographs in Google Google uses internally Google Vision API to organize the different categories of the images and also it on the basis of categories it's certain it's it's most of the most of the times it also gives you the recommendations correct so all these things are internally working behind the engine and that engine is reason AI so you can go go you can see the in the screen like uh let me just show you yeah so Google offers the three reasons API so one is vertex AI Vision so whenever you want to work on the real-time videos okay so you can directly use these vertex AI vision product from Google the second is custom machine learning models ml model so for example if you have all if you have built your model by your own and if you want to further optimize that model okay you want to you want to further improve the accuracy of that model so with the help of custom ml model you can do that and the third category is reason API so this is the API that today we will be discussing about so this is a very powerful pre-trained machine learning models through rest API and it will help us it has a lot of functionalities okay so today we'll be going through some of the functionalities and we'll be seeing live how to write the code and how to further leverage these functionalities in our day-to-day activities okay so so when I say V is an API so anytime if you want to extract the data from a given document and this document can be anything okay it can be digital it can be handwritten so for everything for every kinds of document this Vision API will work okay apart from this documentation related stuff if you want to detect the options from a given image or if you want to classify the images so in fact that in that case also you can use this reason apis and also for example if you want to to identify the logo if you want to identify the places so all this features bundles up and with a single product that is reason API you can you you can use all these functionalities okay now so this is a kind of documentation I will just give you a small example over here so let's try to use a file okay so for example the same file I'll be using so you can see I have just uploaded a phone okay and if you go to the objects you can see it is predicting that it's a mobile phone and the accuracy is quite high that is 92 percent okay when I go to labels here you can say all the other informations like it it falls in the telephony category it's a communication device it's a gadget and basically it's also detecting that it's an iPhone and the accuracy is pretty much very pretty much high that is 86 percent if you go to logo 6 and here you can see it has successfully determined like detected the logo of of an Apple phone and the accuracy is 99 percent now let's go to the text section so here you can see all the texts that were there in this particular phone has been successfully extracted out iPhone is designed by Apple in California assembled in China model okay so and also in properties also you can see the color combination this in in this entire phone you can see the maximum color maximum color used is this one that is 67 percent and then these are the various color combinations okay so this is how you can just try to play with this coming further like so now this is a documentation okay if you want to go through this entire stuff Google has provided a very good documentation on this it consists of vertex AI visa and documentation Auto email documentation as well as views and API documentation so today we will be just going to to freeze an API documentation before that let me just show you some more stuff related to pricing and all Okay so let's open the pricing for vision API so you can see like if you are a beginner if you want to learn then it's free of cost so initially Google will be giving you 300 in your account you can use that amount and you can you can learn all these informations like you can learn all these features of Google Vision API and post uh three month subscription like if you want to perform any of the perform any of these functionalities then you need to pay certain amount but for learning purpose yes definitely it's a very good choice and it's a very good option so you can go ahead and you can learn at any point of time okay now let me go to the documentation part again sorry yeah so today we'll be we'll be working on Vision API and let me just open that tutorial yeah so you can see if you want to further drill down into reason API everything is there okay for example if you want to perform OCR so you have all these codes ready readily available in this documentation section you can see here like whether you want to write code in go language Java node.js python every language is there and every code is there you just need to use this code and you have to just do certain modification and you'll be ready with the products okay so there are a lot of functionalities like you can detect the text in the images you can detect crop ins you can detect faces from the given images you can detect image properties you can detect the labels landmarks logos you even you can detect multiple objects as well so there are a lot of functionalities you can go one by one I'll not take too much of your of your time uh in exploring all this documentation by part because it's very easy anyone can go through this and you can learn out of this so uh so today I'll be I'll be using python as my primary language in order to perform some of the some of the features of this Google Vision API okay I don't know the best part is like it's very easy it's very user friendly so before so the the first step in order to start with this is first of all you should have a Gmail account you should have the Google account okay once you have the Google account you need to look you need to log into console.google.com let me just logging there so console.google console.cloud.google.com here to launch this website so here you can see this is my dashboard okay so here you can create your own project so I have created two projects okay you can go ahead and you can create one project using this new project and you can you can give the name and then you can create the project okay so I have already created a project so let me just show you how to create a project so here I have just given the project name my project so let it be the default name and then I'll create I'll click create so you can see uh one project has is created okay so this is the first step now the second step is you need to enable your vision API okay so for that you can go to API and services and then you you can go to library section and inside Library you have the option to search here you can search for Google Vision API okay so this is the extension so you just need to click on this on this extension and then you need to enable this so right now it's already enabled because I have I have enabled it so you can click on API enabled so once this extension is ready now you need to set up a service account okay let me go back to this dashboard and in order to create a service account you just need to go to Crane Credit in sales okay and here you can create a service account okay so basically you can see here one one option is here manage service account I have already created a service account but again I'll just show you how to create a service account you can go here and you can click create a service account okay you can click create service account and here you can give all the information and then once you are click once you are done with once you are done with the creation or service account you just need to go back to your main screen credentials so once you have the service account okay then you need to create one API key so if you want to create API key you can just click on edit option post this you have the options of keys okay you can go here you can navigate to keys and you can you can create a key okay you can add a key and then once you have that key ready so this key will be a Json file okay so I have already created two keys if you just click on key create new queue and it will prompt for a pop-up just click on Json and then you you need to create it okay so that's all I have got this Json file so this is this is my key so this key will be using forward in this entire demo so once we have the key ready okay let me directly have so this is my project folder for today okay I have created a folder Google Vision API and inside this folder I have placed that particular key okay so this is my key okay so this is my key that I have placed the one that just now I downloaded it I had to place it in my project folder and then I'll open Visual Studio code foreign code as we all know how can I move this anyone knows how to drag this okay are you there yeah uh how can I move this what actually like don't understand what you're saying I want to move this address like title like this stream sharing one where it is creating a problem you can move it downstairs like you can pick it a view options okay okay so yes I can do it thank you okay cool so let's start uh building this application okay so I'll I'll quickly go to my project folder and here I'll create a file okay so for example let me just create a file extract data dot py okay so I have created a python file I'll quickly import all the packages so just remember one thing uh we once we are done with the service account we just need to we just need to import one of the package okay of python and that package name is uh we install so this so peep command is used to install any of the packages in Python and the package name is Google Cloud Vision Okay Google Cloud vision so I already have this package available so no no need to install it but yes you if you are trying to do this you need to have this package Google Google Cloud within all this documentation part I'll give you uh I'll share it with you with you all okay so let me just quickly import the package import so first of all I'll import the OS model OS model and I O module and then from Google so I'll just show you one one code okay and then rest code will not take too much time so that everyone uh follows this how to write code and how to use these functionalities okay from it will it will not take too much time from Google dot cloud import Vision so we'll be importing the vision package from google.cloud okay once we we are done with this so we'll be using certain more packages like pandas in order to work with the data frame so I'll import pandas and then once this is done now what I need to do is I need to just set the path okay so OS Dot environment and then I'll be passing the Google credential over here the one just now I downloaded it so Google underscore application underscore credential I want everyone should follow this because uh you need to know all this all these steps okay then only you'll be able to work on this features credentials and then I'll be I'll be passing the path of that file so this is the file that just now I have downloaded it so I'll just copy the path foreign dot Json so now I have already set up The Path okay now I just need to create the object of our Google image image animator client so I'll just create object client is equals to Vision Dot image I know annotator client okay that's all so once I am done with this I just need to create a file path so the file that I'll be passing so I'll just create a variable file name is equals to I'll give the file name so here we have some of the files so for example let me just show you one file that is uh so for example this is one of the code okay so I'll use the same same code so the name is code dot jpg and then I'll be passing the file path we file path is equals to F column and then I'll use the same path here so once this is done and we just okay so I just need to replace to get the absolute path replace it once this is done I'll just I'll just pass the file name so we file name yeah so this is I'm done with the file setup now I'll just read this file so we will use the i o package of python with IO dot open and I'll pass the file file name here we file path and I'll I'll do a read operation basically as image file as image underscore new file for example and then I'll be using a content is equals to Image new file dot read so I've just now redid the sorry instrument Dot so I have just rated the file so I'm done with this now I need to create an instance of uh this image class Okay Google image class so I'll just use image is equals to v's and Dot image and then I'll just pass the content content is equals to content okay so you can see here I have just created a class of Reason vision and I have passed my image this content with the argument contained equals to content so I am done with this now I just need to fetch the response so I have already called created the object from of this class and I just need to get like I just need to call the response method so I'll just use response is equals to client Dot text detection so this is we are using in order to detect the test text whatever text we have in that document will be able to extract extract it okay Direction and then we'll be passing this image object image is equals to image so once I am done with this I'll just try to click the rest print the response okay let's see what it gives print response let's save it and let's try to run it so it's account.json okay yeah so you can see like I have already extracted all the information from the from the document let me just try to print those documents so that it will be easily it will be easily readable okay let me just click now use a panda data frame so I'll using a data frame DF is equals to PD Dot data frame sorry and I'll create two columns over here using pandas data frame columns is equals to let me create two columns one is a item and then we'll be we'll be creating a description okay so I am done with this and let me just uh extract all the annotation from this response object so text is equals to response Dot we'll be using the text annotation okay so all so this is done now I'll just try to print this text and see what what it gives me as an output so you can see the entire text is giving the entire text are getting extracted now let me make it more clear so let me just go through the text for text in text so I'll create one file okay and then from this file only I'll try to just uh edit this file and I'll try to show you all other features so just bear with me for a few more minutes because I really want that you all should learn how to write these codes okay and then I'll use the data frame this question I'll append all the all the elements append I'll create a dictionary of the object and then I'll just pass that values item is equals to text Dot local okay so basically I'm just trying to say that this is the English statement and then I'll pass in the description is equals to text Dot description and I'll just ignore the index value I'll set it as true yeah now let me print my data frame foreign looks foreign method is deprecated okay so I'll just try to remove that warning so I'll just import the warning class import warnings and then just warnings Dot simple filter action is equals to ignore I'll ignore the future warning so this is all set yeah now I'll run it print then I'll just print the first element of zero yeah so you can see here so this was my image uh code.jpg okay so this is my image and you can sorry not this one this one and you can see here it has successfully extracted the element if your if your dream Stones case scare you they are too small and then Richard Branson so this is how you can extract the data from a given image okay now let me just quickly show you some other stuff so this is all when we use uh the Google Vision API if you want to extract the data from the digital document now let me just show you I have some handwritten document as well as well okay so you can see this is a hand written document I have just downloaded it from Google and I also have few more handwritten documents like you can see here I have this one and then this one and also I I have few more my name is Ravi Kumar chaurasi okay so so this this is a code base okay that we'll be using across our across our demo so you can see I have already imported all the necessary packages I have set up the environment variable I have read the file I have created an object of image class class vision and then I have created a response from Clyde client or text detection and then I have just printed all this doc printed all the information that I'm getting as part of response so our code is ready now let's go to the handwritten part okay so I have already written the code for handwritten one okay so what I have done here is I have done the same thing I imported all the packages I have set the environment path I have created the object of our image animator client and then I have passed the file name so here I have passed the file name is handwritten four dot jpg so I have this file ready with me it says my name is okay yeah and then I have read that file and I have created the image instance that is vision.image and this time the only change that I have done is like instead of so you can see in extract data what I have done I have just uses the function client.txt detection and I pass image is equals to image but while extracting the data from the handwritten document we just need to pass client dot document underscore text detection image equals to image okay so this is the only thing that this is the only change here we are just calling the dot text detection and your document dot text detection that's all now let me just print this document and see whether we are able to get the output or not yeah so you can see uh I am able to extract the output and the name is and the output is coming as my name is Ravi Kumar chaurasia so this is what it was there in that in the image you can see my name is Ravi commercial so it is successfully extracting all the information and everything like word by voice word by word it is able to extract properly now if you want to see the confidence okay confidence ratio whether it is able to how much confidence is it giving you so for that also I have just written the code I'll just enable it so what I have done is like I have just uh called the text annotation dot pages so this will give me the confidence of all the functions of all the extracted documents okay let me just print and show you print pages yeah so you can see here I am able to get the confidence value for all the text so for example the value is let me just scroll to the top so this is the information that I'm I'm getting from response dot full text underscore energy annotation dot pages okay now from this we need to extract the individual confidence of all the text so for that also it's just a way to iterate through this entire list entire dictionary and then we need to extract the data let me just uncomment this piece of line so from here I'm just extracting the confidence of the block and the paragraph okay and then also the word text let me just run this yeah so now you can see uh the output for my is the confidence ratio of my 79 percent my name is 84 percent is 99 Ravi 97 percent Kumar 83 percent and the paragraph confidence is 90 is approximately 94 percent and then the word text like chaurasia is 93 percent so you can see the confidence ratios are also also are also very accurate and it is able to extract the data properly now let me just show you some other stuff let me just uncomment this piece of code and let's take some other images so let's take this this images okay so handwritten dot handwritten two let me just change the image and let's run it yeah so you can see it is able to successfully extract the image the text out of this document also so my document is here yeah so this is what magic link handle handwriting looks like when it is written by hand so you can see even though the the handwriting is not so clear also but still Google Vision API is able to extract all the data from this document correct I have one more document let's try with that also so this is a handwritten document again let's try to see whether we're able to successfully extract the data each and every data from this hand written document or not I'll just change the image name and let's see let me open the image yeah so you can see I comma Suresh padala comma hereby declare that I comma Suresh vanilla comma hereby declare that all the information submitted by me in the application form is correct true and valid I will present the supporting documents as and when required so you can see it has successfully extracted every information this document correct you can see the accuracy of this Google Vision API no none of the words are incorrect while extracting it so this is how you can Leverage The API and you can use it in your existing feature existing process okay so this one with respect to extracting the doc extracting the text from the handwritten document now let me just show you how to EXT how to uh extract like how to detect the object from a given picture Okay so for that also I have written the code okay you can see here I have a sample input that is study room so this is the images okay so in this image we can see we have chair we have table and then we have a plant so these are the objects which are clearly visible so let's try to use the Google Vision API and try to see whether what are the different objects it is able to extract from the even images okay so let's open the code now yes again so here also pretty much the things are very same everything is same I have imported the package I have just ignored the warnings I've said the environment path I've created the object of an emo image annotated client and then I have my file file path and I have passed so study room dot JPG file I have read the image and I have created an image instance of of the of the client okay and then and then once I am done with this let me just disable this particular piece of code and let me print the localized rotation so here you can see the only change that we have done is we have changed the file name and we have changed the function so in the earlier code the function was client dot document underscore text under support detection correct and here the only change is client dot object localization so we have just changed this function and rest all things Still Remains the Same so you can see how easy it is to use and on python file it is foreign also let me just debug the code somewhere I'm getting submitted that's okay yes papa is creating a problem okay so now it is running fine this thing will show you the outfit thank you for this property is creating finishes yeah so you can see here uh in that in this image basically the study room dot image I the Google vision is is able to successfully Identify some of the objects like you can see here this a chair is there and the equation is 67 percent a house plant so this is a house plant and the equation is 66 percent and then there is one Cabinetry okay so this is the Cabinetry and the accuracy is 61 percent so this is how you can uh leverage the function uh localized client dot object localization function and then you can detect the different objects that are present in the given image okay now let me show you one more feature of this and that is to identify the places okay let me open my code base let me drag this Json down just creating a problem okay so I have two pictures like you can see this is which is situated in Delhi and then I have a picture of Taj Mahal that is in Agra okay so let's use the fish the Google Vision API and try to see whether Google with an API is able to extract is able to identify this particular place or not okay okay so now pretty much the code is still the same we have imported the package we have set the environment variable we have created the object of image annotator client then we have passed a file name and then we have read the image file we have created then an instance of the image class and then here the only changes here we have used client dot Landmark detection that's all and everything remains same with these are the couple of functions that is provided by Google Google vision and then I have just created a data frame wherein I have I am storing the description location and the score so description is nothing but the place name the score is the confidence score and the location where exactly it's situated okay let me just run this code okay so sorry it has ran the next places yeah so you can see it has successfully extracted the information like this is a Taj Mahal and also it has given the geographical location latitude and longitude and also the score and the confidence score you can see here it is 94 so the the first score is 94 and then again it has just extracted uh the second value but still both the values are same and the confidence ratios are is quite high that is 95 percent I'll just try to use some other image as well let me just now change Google Chrome like Taj Mahal with without Minar and save this file and then I can open it yeah you can see it has actually extracted Minar latitude longitude though the confidence ratio is not so high that it is around 81 percent but yes we can set the threshold value that if it is more than 70 then we can consider it and here in this case it is giving me the output as the confidence ratios at 81 percent so you can you can just see how good this API is and how easy it is to use so I have just for this demo purpose I have used four of the function to extract the images from to extract the text from the from the digital image then second was was to one was to extract the images extract the data from the handwritten document the third was to detect the object and the fourth one was to detect the places so this is how you can just go through this documentation documentation and you can learn all all other stuffs let me just go to the documentation once yeah so you can see here many other functions are there detect crop hints detect faces so face is also there so if you use this particular feature you'll be able to depict the emotions of the page like which which picture which which person is happy which one is who is sad who is emo who is crying even so all these things you'll be able to use it if you'll just try to use this feature and and run this code okay so everything is there it says that you need to Plug and Play and rest all Remains the Same and then also let me just show you a few few more steps okay yeah and whenever you are trying to work on this API okay make sure that in Billing section your billing is enabled okay otherwise you'll face some issues I also faced the mission so I don't want like uh you all should face issue am I doing it so you can check whether your billing account is active or not otherwise you will not be able to extract you will not be able to use this feature and then make sure that I'm sorry yeah so this is the only library that you will be requiring as part of Google Vision Cloud Vision API okay you just need to enable this and then you you will be all set with the code base and you can try to learn this feature of Google Cloud so I think I am almost done so that's all from my side so just try to explore this particular package of Google and let's see how we can further leverage this and use this in our day to day life thanks you thank you for joining that's all from my side and if you have any question then yes I'll be happy to answer all your questions before we start to an error we just let me put purple and then we can start with q a yeah sure yeah I've put up the poll you can start the Q a uh first question is Google is an API free so yes it is free up to certain time like you can just launch Google vision.api.com and in that they are giving you 300 US dollars for free you can use that and post that if again if you want to use it you need to pay it but for initial 90 days it is free you can use that you can consume the entire 300 USD next is like what if we have other language written other language written document like Hindi or Urdu yes in fact that also you can be able to do it so the only thing is like first of all you need to extract the data and then you can use the natural language processing package of python so we do have few packages like we have nsdk and then we have text block so these are the two two majorly used packages in order to do natural language processing so with this features you'll be able to extract any any data from any of the language okay so now is how Vision API is different from opencv so again opencv is all together a different product and reason API is a product that is launched by Google and it is very easy to use and the confidence ratios are pretty much pretty much high in this Vision API and also in view the name through with an API you hardly uh need to know the coding okay it's very easy to use it is user friendly it's just that you have to call the function that's all in in opencv you need to write a lot of code correct so this is a kind of uh kind of very easy two very easy package that you can use it next question is can you share the source code of file yes definitely a position I'll be sharing the link that will have the course as well as the input sources so for Real Time object iteration which service needs to be used so you can use as I already mentioned in the very in the very initial Stitch let me just show you the package name just give me a few seconds so there are three features that Google Google provide one is vertex AI Vision second one is custom ml model and third one is Vision API so if you want to deal with videos then you can use vertex aiv AI vision now next question is again uh the same no it is not free we need to make sure that it is yes it is free but the only thing is like you will be you'll be getting 300 so once that that is exhausted you need to pay it and it is free for 90 days uh yeah so that's all questions we have thank you for all your questions it was a great session thanks a lot I hope you can conduct more suggestions like this and like thank you everyone for joining in if you have any question you can always connect to Ravi or any doubts you can always connect with the analytics with your team and we will provide you with all the resources and through mails So yeah thank you very much for joining in yeah thanks everyone for joining if you need any help or if you want to know further anything you can connect with me and thanks once again and have a nice day bye goodbye everyone bye oh