Build a Media Analysis Dashboard with Python & Cloudinary

Key Takeaways

hi everyone I'm Patrick and in this tutorial we built a media library app with python and also a dashboard that can automatically tag and analyze all your images and videos I have so many images on my phone and I thought it would be really cool to automatically tag them so that I can filter them accordingly and also analyze them to get an idea of what I like to photograph so the images will be uploaded and hosted in the cloud using the cloudinery API and then here in our app we can display them so here you see the tech clothing then you can also select all other ones and then also the most common combinations of tags so let's select for example the coffee cup and now you see it's displaying 12 images with coffee inside and this is the first part of the app and then we also build a dashboard to analyze the text so here we see for example a pie chart with our top 20 tags so the top one is clothing then we have man human face woman also sky story persons I recently visited New York then we have the top five tags as a bar chart and also the most common combinations so the top one is clothing and man and then in the top five for example we also see men and women so apparently I like to take a lot of photos with my friends so this is the app we're going to build and in the first part you learn how to work with the cloudinary API to upload and tag your images it also works with videos and a quick disclaimer cloudinary sponsored this video so a thank you to cloudinery for sponsoring me and then in the second part you learn how to build this app with streamlit and also this dashboard with plotly so this is a follow along tutorial so nothing will be skipped you can just grab your laptop and follow me here if some parts are too fast and the whole code is also available on GitHub I put the link in the description and all you need is a folder with some images that you want to analyze so let's get started alright so first let's set up cloudinery and what is cloudinary so it's a very powerful media API where we can upload and store our images and videos then we can also transform them and optimize them on the Fly and then deliver them to all our apps and websites for example if you want to build a jungle or flask website then you can use cloudinery to host all your images and you can also use features on top like this Auto attacking that we use in this tutorial then it also offers lots of different Integrations and you can also manually manage your assets in the media dashboard in this video of course we want to work with the API and do everything from code so you can get started for free for this you can go to cloudinery.com or use the link in the description so let's sign up for free so after signing up you should see your dashboard with the API credentials and also your current usage and everything we do in this tutorial can be followed with the FreePlay plan then you could also click on getting started and configure this for many different programming languages in our case of course we want to work with the python SDK and then the core will be the media library so here you could manage your assets and you could for example here upload them and then also delete them again of course we want to do this from the code later and if we click on this then here you should see a URL and this is for example what we would access in our application to display the image this is where it's hosted and what's really cool is that we can get different Transformations on the fly like this circle version just by using a different URL and this makes it super simple to get different optimized and transform media for your application so now we want to set this up with the python SDK so for this you can find the python some quick start documentation guide so let's follow this so to set this up we want to install the cloudinary SDK and also python.nf to store our environment variables in a DOT EnV file so let's go to our terminal and now we want to say pip install cloudinery and also python.anth and hit enter and now we want to create a DOT e and v file and to get our API key we need to go to our dashboard and grab this one here and copy this in here so this will set the cloudinary URL environment variable and now let's create a file that we call cloudinary service dot Pi here we Implement all the helper functions to work with the API so let's follow the quick start guide a little bit more now let's copy everything from here into our our final so first we want to load dot end and load our environment variable variable so let me actually get rid of all of those comments then we want to import the cloudinary modules then here we don't need Json then now we want to configure the cloudinary service so here secure equals true and now if we run this then this should print our Cloud name and the API key so let's run python cloudinary service dot pi and now this works so now we can start working with the API the first thing I want to do is upload an image so let's get rid of this print statement and Define a function upload image this gets a file name and also an optional folder because I want to store all the photos in a new folder that I call my photos so here first I want to get the file stems so for this I import pathlip and then I say pathlib dot path and then put in the file name and then dot stem so this will get rid of the ending dot jpeg for example and now only have pancakes for this example and now we call the API so we say result equals cloudinary dot uploader dot up load and now we put in the file name and then we can also Define different Arguments for example we say the pop lick ID equals the stem and then we can find this again and we also specify a folder equals folder and then we can return the result and this is everything we need to upload this so let's call this and try this out so here we say up upload image and then we can simply pass in the file name so let's try pancakes dot JPEG and then let's print the result so let's run this and see if this works and here I have a typo jpeg so let's try this again and you see we get the results so if we make this a little bit larger then we see this is a Json data so now a dictionary so we get the assets then here the public ID so this is why I wanted to get rid of the ending and then here we also have the URL that we can for example open and now this should open this in our folder so not in our folder in the browser so now this is uploaded and toasted and also if we now go to our media library and then to folders then you should see it created a new folder my photos and in here we have the uploaded picture so this worked then here you would also find the different Transformations that you can access with a different URL and you can also click on metadata and here you see all the available metadata and here you can also see the available tag X so in this case it didn't add them automatically yet so we could do this manually but of course we want to do this from the code now and for this we have to enable a add-on so we have to click on add-ons and there you see all the available add-ons that you can use for example there is one to automatically remove the background and you have one for video moderation video transcription translation and a lot more you could also use the Google auto tagging but in this one I want to use the cloudinary AI content analysis add-on this has powerful AI based object detection and content aware algorithms such as object aware cropping and disorder tagging and in order to use this we have to first enable this so we can select the free plan and then we have to add these two arguments when we upload this so we have to define the model and then the auto attacking threshold which defines at which confidence level this tag is added so let's click on read more and have a quick look at the available models so in our case we want to work with the open images model which is Google's open images data set model that contains 600 Channel objects so let's go to our code and let's define a second function that we call Define uploads and Define uploads and tag image and here it gets the same parameters and now at in addition like we've just seen now we want to add the um detection model so we call the detection parameter and the model is the open images one and then as Auto tagging threshold so for this one we use a very low one at 0.25 and now this is everything we need to upload and tag an image so let's call upload and tag an image and let's use the second example which is this one here so let's save this and then let's run the code and it worked so again we printed the result now here you should also see the key tags and here it lists all the text that it found for example the clothing then it also puts the corresponding bounding box and also the confidence level so yeah this worked and also if we go back to our media library so let's go to our media library and open the new image so there it's already hosted so now we can click on metadata and now as you can see it puts the tags clothing human face and man so yeah this worked so now we can upload and tag the images now the next step is to do this for all our images in this folder that I prepared so I imported this from my iPhone so let's create another helper function define upload folder and here we want to iterate over all the files and let's also keep track of how many we could successfully upload so let's import OS and now we can say for file in OS dot Lister and the folder is called photos then let's also sort this to see it in the same way that we are seeing here and now we want to check if the file is supported so we can say if pathlib dot path and then put in the file dot suffix dot lower if this is in all the supported files then we can go ahead so we want to Define this up here as a tuple so here we can put all the file endings that are supported dot PNG dot jpeg dot jpeg with an e and also D dot hike so this is what the iPhone uses here as you can see a lot of those images are in the hike format and this is also supported in cloudinery so if this is supported then we can go ahead and we want to wrap this in a try accept block so now we can call upload and tag the image and here we have to be careful so we first have to use photos and then plus file and then we say n plus equals one and then we can catch the exception as e and then we say print let's say failed 4 and then the file and then we also want to print the exception and when we are done we can print um n photos uploaded so now let's save this and then down here we can call upload folder and let's run this and first one small change I also want to print the file so that I can see the progress so let's call python cloudinary service dot pi and as you can see now it's starting and all the images are uploaded so now we have to wait until this is finished all right and it's finished so 449 photos have been uploaded and if we have a look at our media library then you should see all those new photos and all of them should have text baked goods fast food hamburgers so yeah this is working so now the next thing I want to show you is how we can search with the API and then retrieve the images and text again so the first thing I want to do is to Define find a function get all tags and now we search for all the available tags in the API and there are different ways how we can do this so there are different available apis for example we just work with the upload API then we can also use the admin API and the search API and also the transformation URL API so in our case I want to show you how to work with admin and search very quickly so to get all text let's define a empty list all tags and now we call tax equals cloudinary dot API dot text so this is the admin API and here we can Define arguments so we can say Max result equals 100 so I think by default this is 500 no the default is 10 and it can take up to 500 but we also work with the next cursor so we can leave 100 and still get all the available tags so first we extend all tags at all text Dot extend with the new text and now we only want to access the tags field then let's get the next cursor by saying next cursor equals all sorry text Dot get the next cursor this can also be none if we already reached all results and then we use a while loop and say while we have a next cursor and then we do this again so we can copy and paste this in here and now as another argument here now we use the next cursor parameter and say this is now the next cursor then we again extend this and update our next cursor and in the end we want to return all tags so now if we call get all text and now assign this to a variable or text equals and now let's print all texts now this should work so let's run this and see if it works and the result is here so here this is now a list with all the available tags in my media library so this is the first thing I want to show you now the next thing is to search for images so let's define a function search image and here we work with the search API so we say result equals cloudinary Dot search and now here we can define an expression so we can say expression and this is a string so we say resource underscore type and then colon we want to search for images and the text equals let's try let's have a look at them let's try the text equals wine and then we can also do some more so for example we can say sort by and then we want to sort by the public ID and this should be in descending order and then in the end so here for example we can do more for example Max results and in the end we want to call execute and now we want to return the results so we can say return results and then let's print this or let's call this so let's say result equals search image and then we first let's print results and then we can access the field total count and then we can iterate over this so we can say four results in results and then the key resources and now we can print for example the result URL so let's comment this out and also this out and then let's run this again and it worked so it found three images with the wine tag so let's open one of those URLs and see if this is correct um it's actually not displaying it here so but it's downloading this so I think this is because of the format but there's one very cool thing that that we can do with cloudinary so we can actually change the format also on the Fly by simply changing the ending so let's edit the URL and let's cut off the ending and then add the jpeg ending so now if we run this again then you should see now the image is with the jpeg ending and now if we open this then our browser should display this so yeah here we have the image and we see correctly it contains a wine in the image so this is working super cool now finally I want to add one more helper function and here we want to Define get all images with text and here instead of using the search API we again use the admin API like here and return all Resources with text so we say all resources equals an empty list in the beginning then we say result equals and then we say cloudinary dot API dot resources and now we Define our parameters so the type equals of upload then the re Source type equals a image this is the default I think then we say the prefix equals my photos so it should only look in our my photos folder then we also want to include tags so we have to say tags equals true and again we use this Max result equals 100 and then we um extend our all resources so we say all resources is dot extend the result with the key resources and now we also get the next cursor so we say next cursor equals result with the key let's use the dot get and now the next cursor so this can also be none and then again we do our while loop so we say while we have a next cursor then we do this again so let's again copy and paste this and here let's indent this correctly and then here we want to also now add the next cursor parameter and this is the next cursor and now in the end we want to return this so here we return all resources and and then down here let's call this all resources equals and then we have get all images with text so let's print all resources and let's also print the length of all resources because this is a very long array so let's run this and it worked so as you can see we have 451 images in our my photos folders or the 449 from this folder and the two that I uploaded in the beginning to demonstrate the upload API so yeah this worked and now we have all the helper functions that we need so now we can Implement our app and then display the images and analyze the text a little bit so let's create a new file app dot pi and in order to build our app we want streamlits pandas and plotly so we can install this with Pip install streamlit pandas and plotly and now we can import everything so we say import streamlit as St then import plotly dot Express SPX and import pandas SPD then we also need collections in import counter and we also want from either tools we want to import combinations to build some combinations of the text and then we also import the cloudinary service so first let's define a helper function get images with text and here we simply return cloudinary service dot get all images with text and the reason I'm wrapping this here again is because now I can decorate this with at streamlit cache and now this will be cached so this will only be called once or if something here changes so let's call this and say all images equals get images with text and now this will be the whole resource so I only want the text so let's save them in a list all tags equals an empty list and then let's go over this for image in all images and then we can extract the tag by saying image and the key tag then we can also check if not tag so actually this is plural text so text equals image text so we check if not text if this is an empty list or if person in text so in my case I want to filter out this tag because it already contains human men and women so so I don't need person in addition so here in this case I say continue and otherwise we say all text Dot extend the text then we also want a separate list to create a list of lists so here we say all tags lists dot app pins the text so this one here has a list of the tags for each image and this one is only one list with all the tags inside so now we want to create a counter object out of this so we say counter tag counter equals a counter of all the text and now let's print for example the tech counter and let's also print the all text lists so that you can see the difference between those so let's run python app dot pi and here we see the list of lists so for each image we have a list with all the tags and if we scroll up then here we should see the counter object so the tag clothing has 124 count so 124 images with the tech clothing this is the most common one then we also have man human face women so that's why I wanted to filter out person for example and now let's continue so now let's create combinations of those so first this is an empty list and then we say for tag per image in all text lists and then for comp in combinations and now we create combinations of the text so this should be plural text per image and we want to create two combinations and now we say comps dot appends the comp and then we want to create the most common one so again we say most common comps and then again the counter object with the comps and then we can immediately call the most common one with let's say the 20 most common combinations so let's comment this out again and then I also want to create lists for displaying so I want the sorted text equals and here we use lists comprehension so we say item for item in sorted and here we put in tag counter dot items and as a key we want it to be sorted according to the counts so here we use Lambda X and we return x one so this is the count because Tech counter gives us a tuple and here we put a minus because we want to have this in descending order and then let's also create the sorted tag strings this is only for displaying so here again we use list comprehension with an an F string so first we want the item zero so this is the tag name then a space then parentheses and here we put in the item one so now this is the count and then we do this for item in sorted text so now we have all the objects we need so let's create our streamlit app so here we can say if underscore name equals equals underscore main then we want to display the widgets and we want to display two different options so this is a tuple and the first one we call this image gallery here we display the images and in the second one we want to display the statistics so image stats then we say selection equals stream let see leg box and we call this menu and here we put in the options and we can also move this to the sidebar like this and then we check if the selection equals equals image gallery so this will assign the value to this variable so in this case we want to display the image page so let's call a function that is called image page and also let's give this a title and here we say the title is the image gallery like so and otherwise we say else we also give this a title and in this case we call this image stats and then here we want to call a function that we call let's call this stats page so now up here of course we have to Define this so let's say Define image page and for now we say pass and then we also want to define the stats page and let's also call pass so now let's start by implementing the image page so in the image page let's first only call a function that we call show images and here for now we put in all images and later we want to also select only the tags that we want so let's define this as well Define show images and this gets the images here and we want to display this in three columns so we say columns equals streamlit dot columns and we say three and then we iterate over this or over the images so we say four index and image in enumerate images this gives us both the index and the item then we Define the column index so column equals columns and here we say index modulo 3 and now we say URL equals the image with the key URL and then we can create a column like so so we say with column and then we say streamlit image and call the image widget and here we can use the URL so this is the URL that we get from cloudinary and let's also give this a the link below so we say streamlit markdown and as an F string and here we use the let's display the text link and the actual link here will be the URL and before we do this let's make one quick modification that we've seen before so if the URL ends with um dot hike then we want to change this and say the URL equals the URL from start until minus five and then let's add the ending dot JPEG and now to run the App instead of saying python app.pi we want to say streamlit run app.pi and this should start the streamlit app on your localhost and it worked so here we have our app that is up and running and at the sidebar you see we have those two options and for now we display all the images here so now let's add a selection to select only the images with the tag we want so let's add another select box and we say tag equal those streamlit select box and as label let's say select tag and here we put in the options so this can be a tuple or a list for example so let's say the options equals the sorted text strings and for Simplicity I only want to um to display the 20 most common tags and then I also want to add the combinations so we say four item in most common comps and then we say options dot append S A F string and here we use the same formatting that we do here so basically we first want to display item zero so this is the first tag then a comma then item one and then a space and then parentheses and then the item one so actually this is item one is the count and here we have two so we have to say item 0 0 and item zero one and now we can display this so um let's try this and refresh this so if we go to the page this should automatically be included and it worked so here we have now our select box with all the text and then the count and in the bottom we also see the combinations so for example the most common combination is clothing and men and clothing and human face or human face and Men clothing and woman um yeah so this works so now after we selected this tag then this is the exact string so now we have to convert this to the actual tag again and get rid of the parentheses and the commas so here we check or first we say index equals and then we say tag Dot find the opening parentheses this is always included and then we say the tag equals tag from the start until this index and also -1 to get rid of the space and then we check if we have a comma in the tag this means we have a combination if this is the case we want to say Tech 1 and tag 2 equals Tech dot split at the comma and then we say tag one and tag 2 equals tech one dot strip so get rid of the white space and tag 2 dot strip and now we want to filter them so now here we say the images with tag equals and now we use list comprehension and we say image for image in all images if tag one in image and the key text and also and if Tag 2 also in image text and then we want to display the images all images with the tag and otherwise if we only have one then we can do the same so here let's copy and paste and here we say image for image in all images if the tag in images text in this case this is all that we need so let's refresh our app and try this so let's for example choose the text skyscraper and this should automatically refresh the page and as you can see here now we only see the images with a skyscraper in it so pretty cool so now our image gallery where we can filter according to the text works let's for example also try food and now we see all the images with a food so this works as well so so now we implemented the image Galleries and then next thing is to implement the image stats page so let's go back and implement the stats page function so here I want to display three plotly charts with some statistics a pie chart with the most common text a bar chart with the top text and a horizontal bar chart with the most common combinations so first let's only consider a minimum tag number so let's say Min text number equals 20 by default and then let's filter this so let's say filtered text equals until we use dictionary comprehension so we say key and value and then four key and value in sorted and here we say tag counter dot items and then again we use the key equals Lambda and Lambda X and we return minus x one so we sort this by the count and then we say if the value is greater or equal then the minimum tax number then we can say labels equals filtered text Dot keys and the counts equals filtered text Dot values and then let's also convert this to a list again so this is a list and this is a list and now let's create the first chart so first let's display a small title with markdown syntax so this is a H4 heading and then we say top and then Min text number text and now here let's create a data frame out of this so we say this is a pandas data frame where we say list and Here We Sip This Together the labels and the counts and then we also want to define the columns so we say columns equals and the First Column let's call this text and the second one let's call this counts and now we can create our plotly chart very easily so we say figure or equals p x dot pi and we put in the data frame and the values that we want to consider for the plot should be um counts and the names should be the column that we called text so now we have the pie chart so now we can also um slightly modify this by saying thick updates update underscore traces so I want to change the text info a little bit so we can say text info equals and here we want to display both the label plus the percent sign then also we say fix update layout um up date underscore layout and we want to give this a width of 700 and a hate height of 700 as well and then we can say streamlit dot plotly chart the figure and now if we go to our app and refresh the page we should see our pie chart and it worked so as you can see here we have a pie chart and we can also hover over this so this is interactive and as you can see clothing has 17 of the tags are with the tech clothing so yeah this is pretty cool to analyze your photos so let's add two more plotly charts here so first we want a bar chart so let's copy and paste this and also this so here we want to say the let's call this top 5 text and here we want to sip only from start to five and also for the counts from start to five then we use a plotly bar chart where we say x equals the text and y equals the counts and then here we don't need this and for the layout in this case let's use 800 and 500 this seems to look good at my computer and then let's also copy and paste this one more so here we want to display the um most common combinations and for this we need to get the combination so let's extract them by saying the labels equals a list comprehension and as a string here x 0 and then four x in most common combinations and we do this in reversed order from start to end with a step minus one so this should actually be after the closing parentheses and then the same for the um for the counts so we say counts equals and the count is x 1 4 x in most common combinations and then here we can get rid of this and get rid of this and the columns are called combinations and counts then here this is again a bar chart with x equals counts and y equals the combinations and now here we say the orientation equals horizontally then for this layout we give this 800 and let's say 600 and this is all that we need so now let's again refresh our app and let's scroll down and here we see a value error value Y is not the name of the column in the data frame so let's have a quick look at the um second one so here this should be a Capital C for count so let's again refresh our page and now we see the bar chart with our top five tags clothing men human face woman and skyscraper and the most common combinations are clothing and Man Clothing and human face human face face and Man Clothing and women and so on so yeah I think this is pretty cool to analyze all your photos and yeah this is all that I wanted to show you and Implement for this app in this tutorial so yeah I really hope you enjoyed this tutorial and by the way you can get both light and dark mode just by changing this here in the settings so if you enjoyed this tutorial leave me a like and also check out cloudinery with the link in the description you can get started for free and then I hope to see you in the next one bye