Automating WebScraping Amazon Ecommerce Website Using AutoScrapper

Key Takeaways

[Music] hello all my name is krishna and welcome to my youtube channel so guys today in this particular video we are going to do an automated web scrapping using auto auto scrapper for the e-commerce websites like amazon and flipkart now in this example i'll be taking amazon and this will be an assignment to you just try to do the similar thing with respect to the flip card now if i go and search for the amazon dot ins website right and probably if i search for probably i'll just write it as headphones okay i'll just write as headphones now when i'm searching for this i should now these are all the information that i'm actually getting and my aim is basically to take out the title probably this amount information how many number of stars you know so all these particular informations i actually want so how can i do this with the help of auto scrapper now what is this auto scrapper if i go and show you it is an amazing librarian yesterday also we discussed about this one video i'd actually uploaded it is a smart automatic fast and lightweight web scrapper for python it's pretty much awesome guys and just tell me which all web scrapping two libraries you used to use before okay but this is pretty much amazing when compared to the other one so let's go ahead now first of all what we are going to do over here is that we are just going to import from auto scrapper import auto scrapper but before that just try to install auto scraper right so pip install auto scrapper okay so scrap i'm making a spelling mistake okay so this probably requirement is already satisfied because i have done that particular installation now quickly let me make some cells also now i'm just going to import from auto scrapper i'm going to import this auto scrapper itself this is the class that we are going to use now let's consider the amazon url first of all which url i want to search and remember guys i am actually making this pretty much generic right now i've just searched for headphones you know now i'll be taking all the necessary information like title probably price probably uh how many number of ratings are there with respect to each and everything later on you search any kind of products you will be able to get the similar data through the web scrapping technique so i'm just going to take this entire url okay i'm going to paste it over here perfect i've done this now from that what all what are the important information that i want to recreate so i'll just create this kind of wanted list let me consider one product at least okay suppose i want to consider one product so i'm just going to copy this paste it over here okay i really want the price definitely price is important let's take the second information probably my title okay so the second information will basically be title so i'm just going to copy and paste it again over here perfect now the third information that i also want is basically my information regarding to stars okay so i hope i was taking this right so you can take any it is not like you have to take that on me so this is all information i actually want from this i'll just execute this so this is my amazon url this is this now let's go and try to build an auto scrapper with respect to this specific url okay now first of all i'll just try to initialize scrapper is equal to auto scrapper and here i'm just going to use this and then i'll go to say result is equal to scrapper dot build and the first parameter that i have to give is url and the second parameter is my wanted list now from the wanted list i'm actually taking out three main important information the price the title and the third more information is something called as uh the star ratings okay so this will be my let's print my results okay and let's execute it okay url deny not executed sorry okay sorry it is not amazon url i just wrote it as url okay so just execute this now here is the entire information that we are actually getting right we are getting the entire information now let me do one thing quickly now i will oh i know that i'm getting the price information i'm getting the title i'm getting the star ratings over here you can see these are the very star ratings that i am getting two six one six one seven six eight and all okay so we'll try to group this let's see how to group it now in order to group it what we'll be doing is that and just observe this particular information initially we had 799 2490 let's see so uh we had 799 to 2490 2499 different different prices are here automatically and from that you are actually able to get that information right so just try to see which all information are coming out okay two four nine zero four nine nine one four nine nine everything is there now we are going to use this scrapper quickly and then i'm just going to search for get result similar okay scrapper dot this is a function which is called as get result similar again i'm going to use this particular amazon url and i have to make is that there's one more parameter which is called as grouped grouped is equal to true because i am taking out three main important information so it should be grouped based on that okay so once i execute it you'll be able to see these all prices are my one group uh these all prices are my another group see there's different different prices we are actually able to get and we are able to get this information see different different prices why probably because here you'll be able to see there is another sponsored ad then there is one more this kind of ad okay and then here you can see that this way also the product is right so different different products you will be able to see in different different way and it is actually able to take out those information okay and remember guys if probably it is in this way you'll be getting multiple groups of prices okay that also may be there but let me just see what i'll do is that guys quickly from this i really want to take out my uh the title of all the information probably i i really want to get the title of all the products that i actually have and remember sometimes this may also not work that much well you know me you may have some kind of problems let me just show you some more things suppose i am going to search for iphones okay because there you are not able to get all the products in a proper way this page looks pretty much suitable because all the products are listed in this so i'm just going to use this particular url okay let me make some changes because previous one had too much of extra information okay so i'm just going to copy and paste it over here okay and then uh probably i will just go and pick up one information like this so this will basically be my title okay now i'm just going to put this information over here as my title and remember guys your page needs to be really really kept well it should be all the products needs to be seen in a proper way then only it is better that will be able to extract that particular information okay and uh probably i'll go and pick up this particular information 58 400 let's see this is basically my price okay and then i also have one more information which is like 24 406 ratings so i'll keep this information now let's try this probably this should work well okay now here i am able to get some better information when compared to the previous one but here again some problems are again there but let me just execute this now okay now here you will be able to see with respect to the groups i think uh this also has multiple groups of pricing but this is pretty much better than the other one now let me do one thing let me just take the allies and save all the information of the title together okay so what i'm going to do over here is that i'm just going to copy some code so instead of copying what i'll do i'll just write scrapper dot set underscore rules sorry it is single scrapper dot set underscore rules underscore last now let's take my alliance information in the form of key value pairs now suppose i want to take this rule of s i okay because this is having all the product names so i'll write like this and i'll say okay this is basically my title okay and i'll i will be able to get this information then what i'm going to do is that i'm also going to make sure that i keep this rule whenever i want to apply it for other pages also so here i'm just going to say keep underscore rules and here i'll be writing my rule name that is this specific rule right now i'm just focused on title later on we'll try to take some more information okay now this is my other information and then i'm going to say scrapper dot save and i'm going to save this as my amazon search so this all information will definitely be there let me just execute it now let me go over here also guys let's not focus on the ratings also okay let's focus on these two things that is the price and other things so here are all my information okay and then probably i'm just going to see for title okay sorry uh just a second so this is my information just a second guys so let me see because the group name will obviously change you know um save with extra cost is this this information is there okay this is my group name okay now i'm just going to save this group name save this and i'm going to save this as my amazon search now let me write like this scrapbook dot get some results similar and here let's take some another url suppose i am going to search for mi phones mi phones uh something i want to search like this now mi phones these are my entire url so i'm just going to take up this specific url and let's see all this information because this is also now properly given right so i should be able to get the title information at least okay and there is one more uh i have to make sure that previously i was using grouped is equal to two now there will be another uh parameter which is like let me just see one more parameter will be there which is called as group by alliance group by allies is equal to true just make this as true so that whatever grouping technique we have actually used in that way only this will work so here you will be able to see all the title information now in order to check out all the title information i'll save this in my result okay probably this will be my results i'll execute it and if i write results of probably title okay then i'll be able to see all the titles all the titles now let me do one more thing let me add the price information also so this price is also given let me see what is the uh price grouping okay so obviously this is my price grouping you can take any one of them okay and i will be writing over here as in this particular grouping technique this will basically be my price okay and the next keep rule i will be saying the same thing where i'll be adding this specific thing over here okay now let me just quickly see this rule h rule underscore h9b let me just execute it once again guys i think uh the entire thing is gone so i will just execute this okay fine this is done group is equal to true let me execute once again okay because i i made sure that i want to add price also okay so this is my result okay this is my result with respect to iphones uh let me do one thing let me execute it i'm getting all the information regarding my iphones uh i'm just going to say group is equal to true so this is my all group information probably let me see where is the group information okay now let me add this parameter over here for price and this parameter over here always make sure that whenever you are making any changes start the execution from starting itself over here and this will be my information and this will be mine perfect i'll do this once again now i'll try to get the similar results and make sure that i'm giving group by alice is equal to true now if i go and see my title i'm getting that information if i go and see my price with respect to that particular product i'm actually getting all this information so 13 triple nine probably somewhere it may be 13 triple nine right remember uh 13 triple nine is coming up all the information is basically coming up with respect to this right so pretty much good you are able to get this particular information and this is how we can actually do this web scrapping i'll also make sure that in the upcoming video i'll try to create a flask app where i'll be able to see this kind of amazon products in search of a keyword you know just by seeing or hitting an api okay now your work will be let's just try to have a look onto this flipkart website okay and try to do it and in my next video also what i'll do is that i will try to scrap stack overflow so one assignment that i really want you to give do is that try to web scrap flipkart urls also just by searching some products over here okay so just by searching it you will be able to again scrap this information just try to do it and let me know whether it is working or not so i hope you like this particular video please do subscribe the channel if you're not already subscribe i'll see you in the next video have a great day thank you bye