Web Automation using Selenium | Hands-on with Python & VBA | RPA & Web Scrapping | DataHour

Key Takeaways

good evening everyone so I will be covering today the topic of license free web Automation and like in today's world there is lots of Buzz going around robotics or robotics process automation so let's understand that what is this robotics process Automation and why way big automation is so important uh concept of like one of the concepts of robotic robotics process automation and also like with wave automation what we can do and what we can achieve what are the benefits and um like even like with the script based license free web automation what are the benefits that like we will see in the like some in some Hands-On coding so let me share my screen uh so let's see like uh what we can understand from this topic okay so the topic is license freeway Automation and our agenda for today is uh what is RPA and different Market players or for RPA what is web automation uh web scrapping versus web automation so we will understand what is web scrapping and what is web Automation and why it is important both in RP and data analytics simple script based web automation using python plus selenium so we will see some coding in this like how it works multi browser automation using multi-threading technique with acelenium so actually python python has that feature it's suppose multi-threading so selenium is one of the library of python selenium comes a selenium can be integrated with the other programming languages as well even it can be integrated with JavaScript so we will be using the selenium Library here to do the web Automation and we will use the multi-threading techniques of python to see like how we can launch multiple browsers in simultaneously simple wave automation using VBA plus selenium so this is a bonus topic which I have Incorporated it's Incorporated in this session so we will see like how we can incorporate selenium library with VBA code as well okay and lastly we will take the question answers around so I will ask you like if you have any doubts you can share your doubts with me okay so before going into the session uh this is a brief about niacom so I am having 16 years of Industry experience and I have worked in plethora of Technologies like I have started my career as a software developer and then with the you know like with time I have Rose the ladder and now I am leading uh like some of the teams in Accenture uh I am also leading like The Innovation Center of Excellence Automation and Innovation friendship center of excellence can say like half a ticket and also in AI Abel so cognitive intelligence uh cognitive automation uh then robotics process Automation and you can say like uh uh like a subset of digital transformation so um so and also I am also working as a industry Tech Force like maintaining industry Tech Force as a speaker mentor and the gear selection uh I am doing a doctoral research on uh risk management like building or researching on building a risk management model for sustainable smart cities uh which is enabled by AI IIT and blockchain and a couple of my white papers are being acclaimed and recognized uh both in Accenture and in uh in our confidence International confidence of business and security uh my hobbies include I like traveling I like trekking I like off-road biking in 2016 I have gone like I have visited solo to ladaf like from Delhi to lazak and back to Srinagar and I have traveled my traveled in my bike uh like alone so it was a great experience and like some of the hardships though I have you know like Crossroad but it was a great experience okay so this is all about me so going forward uh let's uh dive into the our topic to them uh now lots of you know like buzz is going around robotics process automation now what is this robotics process automation let's understand from the official definition of it okay so robot expresses automation mimics the action of user that user interface level okay so developers actually need not to know about the interacting systems uh they will just you know like uh they will like whatever like the whatever the uh the developer need not to know about the interactive system please uh the underlying systems like in what the system is developed like whether it is developed in Java whether it is developing.net whether it is developed in Python you need not to know that it is basically the bot will be mimic the human interaction with the waveforms or the website and uh it will just in that the user interface level only it will be the user interface level so these kind of softwares are actually enabled by the power of artificial intelligence machine learning and it automates high volume repetitive mundane tasks without human International okay so in RPA we create Bots also known as digital workers so these Bots are also referred to uh as the digital colleagues and and it mimics human actions so that is what what I said before okay now some of the market players which are there in RPA in current test uh we have UI path we have automation anywhere uh we have blue prism uh we have worked pollution and some other tools like Vega Vega is there and four packs and uh another like ABN are I don't remember the name now because some other markets layers are there but if you see what is the commonality of all these Market players uh they are actually licensed because you cannot go license free uh and like you cannot use this you know like any of this tool without any license so license when I talk about license it means it is incurring a cost so and sometimes what happens is like uh like in my industry experience I have seen that the Bots are built in blue prism and then they are again migrated to automation anywhere um because automation annual license cost is low so again it you know like involves Redevelopment uh like rebuilding and then again you know like the migration cost as well so and also there is a like support team who will be support 24 cross seven the RPA servers so this cost will be there uh like when you are going with these tools now uh like for data analytics or even you know like for that even for a large scale automation what we have seen is we can actually go spread this instead of using this license tool we can go script this like we can use Python and python has some extra has some uh like extensive uh comprehensive like uh like available libraries like we should print use to leverage anything actually to do anything so uh let's dive into that so this is a way of automation so web automation is actually one of the uh most uh or you can say like um it is the um it is the characteristic one of the one of the characteristic of the RPA because in most of the processes we need to do web automation because we need to interact with the websites we need to interact with the firms we need to interact with uh different you know like Legacy systems wherein we need to page data uh for for our purpose like for building a report or for you know like uh for building interactive dashboards and um like even for you know like data entry operations uh so it is one of the you know like important feature of RPM now uh when I talk about data so if I say that if you talk about uh any recipe actually so data is the main ingredient right so nowadays data is very you know like um it is the most important thing a industry can have and everyone is seeking for data even for you know like doing the sentiment analysis uh like uh we sometimes perhaps straight to Twitter data uh Facebook data to understand that uh how the messages are coming or what is the you know like internal the intent of those messages right so lots of work actually happening in those areas where we built AIML models on like creating those data so that is about data so going to the web automation so now what is web automation so as I said like when we need to page data from a website or when it need to post data in a website so that is actually web automation okay so web automation if we go by the uh official definition of web optimization which is a concept of letting software robots perform predefined action tasks and processes on a web browser or web application okay so suppose for example um for example like you know like generating an invoice sales invoice okay so we need to give some invoice number or the customer number and then we generate the receipt of the that customer whatever he has he or she has bought so here we are actually interacting with the website or or any web platform or web form and we are generating the report so uh this kind of task okay so as I already said the tasks are essentially repetitive one then and does not require any decision making ability now what does it mean that it does not require any decision making ability that means it does not require any human uh human intervention like I have to think on this process I have to validate this process uh because bot doesn't have that ability it can only mimic the task like if I am going to the website I am logging into the system I am giving my credentials like I am entering my username I am entering my password I am clicking the enter button so what is happening is that bot can understand that what what are the tasks I am doing going to the website so it can meaning those tasks but suppose uh going uh going to a website I generated report and on that report I am doing some analysis I am understanding data I am doing some analysis and I am understanding that on this uh like there are some conditions where we I need to validate something or I need to take some necessary actions so on those conditions uh what happens is there's some human intervention is required so there are some human decision making ability is required so those things like for uh those purposes RPA is not applicable so RPA is applicable when you are mimicking any repetitive mundane task which a human does going to the website okay so that is uh like what we can do in the web automation okay now some examples can can be like travel claims purchase of invoices purchase invoices current payments like if you want to you know like generate these reports reconciliation of accounts like bank accounts ledgers Etc so these are all like real-time industry you know like industry examples accounting periods and other entries so when you are doing some entries Into The Ledger system then payroll calculation so this is also one of the important uh work work process actually where we need to calculate the payroll so that is also one of the important thing okay then updating variables affecting statutory wages so these are some of the you know like common examples or processes uh which where we need to interact with the website we need to enter data or either we need to page data so yeah so one more one and another example is data extraction of the Mis report and dashboard so these I have already told mentioned okay now let's understand that what is wave scrapping and what is web Automation and what is the difference like when we will use web scrapping and when we will use web automation okay so essentially there are many web scrapping techniques and tools available uh even python has some good libraries to do web scrapping okay so web scrapping refers to the extraction of data from a website okay so when we only face the data from a website and this information is collected and then exported in a format of a report or in in the spreadsheet okay so this is essentially web scrapping means when we are going to the website we are extracting where we are scrapping the data out of it okay so this is essentially the web scrapping means now not necessarily means that we need a RPA tool to do web scrapping because there are lots of you know like tools available already the lots of libraries available already we can use scripting languages to do web scrapping now web automation wave automation is the interaction with the website but it is associated with both paging and posting of data so here we are not only fetching the data we here we need to like we are doing some data entry as well right so that is the basic difference between web scrapping and web automation like where we are talking with the website basically it is not a one-way thing that we are going to the site and we are fetching the data here we are doing the both we are creating the data as well as we are posting the data so this is a basic difference now if you see like this this uh topic or this uh you know like this concept is very important in uh two days you know like RPA scenario uh because if I like I have actually taken a report like how the you know like Market is going and the predicted uh market share for RPA by 2024 will be 3.5 billion so it is quite high so uh from there we understand that why we need to you know I understand this topic in data done and because it is one of the most important feature of any like robot expresses automation so we need to understand uh this topic in detail and also web automation like from my experience I can say that it is uh one of the most crucial thing where people face most difficulties because uh because website doesn't react or in like you know like doesn't um the outcome is not as expected because sometimes the website gets low sometimes it is fast so there will be always you know like problem in giving Optimum delay to the bot maybe bot has run and executed the next line of code but the website has not loaded at all even giving the optimum delay there is a problem uh like lots of times people faces where uh you know like where the bot cannot you know like access the controls uh or the website has been very slow the bot has executed and gone to the next line of code and it is not getting the required control so all these problems will be there lot many times we have seen in our like in my industry experience I have seen that it is a most painful area like where we face in the robotics process automation because if you say like uh like uh you know like automating Excel automating sap or automating other uh you know like other dashboard or you know desktop application are still easier than the web of automation because website has its own you know like um loading time then uh like sometimes a website fails it doesn't load at all so it is a very painful area and it is an important area Okay so going forward so here I actually I have you know like taken some hands on coating uh uh like um for to show like how we can do simple web automation using python uh and here I am using selenium library and a chrome driver so Chrome driver I think is uh like is installed in everyone's system so it generally we are using Chrome Orange but selenium is compatible with uh most mostly all kinds of drivers like a chrome or Mozilla Firefox so any kind of driver you can take okay so going forward okay so here are the sum requisite so I need python like we should have python installed in our system to run this code uh then we should have selenium Library I will tell you how to install selenium Library it is very simple those who are already working on python they know like how to install external Library and uh another installer we need is Chrome Driver Auto installer so for uh like with this Library actually we don't have to give the explicitly we don't have to give the path of the Chrome driver like where it is situated in our machine so it actually automatically handles that okay so let me just go to the port okay so this is a visual studio code uh the ones who doesn't have you know like the introduction of visual studio and python so this is a visual studio code and uh this you can uh like this you need to install this is a very lightweight tool or ID where we I generally write the python coding but there is other IDs also available so you can use that as well uh I am using visual studio so I have opened that and here I have created a folder so where I have taken some of the codes so with the interest of time I have already kept some code to show in the demonstration so uh let me just open so the here okay so now for selenium so python is already installed in the system you can go and install python python is uh like you open source software and it is like not it is free so you don't need any post for to install python uh then for installing selenium you can use this command peep install selenium so for installing any driver actually means we use this command since now selenium is already installed in my system so I am not running this but you can just use this command and put enter so it will actually install the selenium in your system so likewise uh likewise you can install a chrome Driver Auto installer also so these two libraries we will be needing essentially to do this or to show this uh demonstration okay so I have already installed both the things I increase in my system so I am not installing it again okay now let's see uh let's see a piece of code okay A simple piece of code let's understand so if you like here I will be opening youtube.com okay and I will be searching uh like I will be searching blockchain so I want to see all the tutorials related to blockchain and I will be uh like uh just you know like click the search button so if my if I open my Chrome I will explain the uh like each line of this code okay let me open youtube.com so the YouTube is open now what we need to do is see we actually we need these two you know like these two uh controls one is this text box where we will type and another is this button so this button will actually do the searching okay so to access this control what we need to see is we need to inspect this element so we need to see the HTML uh ID or the name of this element okay so we need to inspect this page basically okay so once we inspect this element so it shows me this curve in the inspect panel okay if you see here in the right hand side a inspect panel has opened and here you can see the ID of this ID and all the details of this control specific control so here I have used a text box okay so this is the text box so if you see the input tag is there so which specifically says that it is a text box okay and there are certain other things like ID search so the search is its ID then the name uh or the internet the tag is there okay and other things also there like last name or something is there okay so all the HTML components are visible here okay so now I can take either ID or I can take the name to search this or to access this control okay so I need to search the blockchain so that means I need to access this control first and I need to type blockchain here and then I have to click the I need to click the search button okay so for this part purpose I need to access this control first so I can access it by either by ID or by name now often it may happen that you really need to inspect this webpage that if that parameter which you were uh like you you are considering whether it is a unique parameter because sometimes what happens is maybe this search ID will be there for the other other you know like other controls also so so actually what cannot or the selenium Library code or the web driver cannot you know like access exactly this control okay so it will have that ambiguity so we need to think we need to see a little bit the whole page and we need to see that what can be my unique parameter over here to control these uh to access this control okay so here I have taken search query so I have taken the name parameter okay and for the click button again I will go here okay and I will do I inspect element so it will give me for the button as well so here is my button as you can see the button tag is there okay and the class is there ID is there so I will take ID for this okay so I am taking ID for this and I am taking the name parameter for this okay so now let's go back to the port now if you go back to the port the code is very simple so these are the some of the libraries I have imported from selenium so one is a web driver this is the this is actually the most important uh like Library what I am importing from selenium and some other libraries are there okay so first what I am doing is I am opening the Chrome so this crease of code this line of code actually opens up from so here you can give the path also but I am since I am using already this Library so for that explicitly I don't have to give the path I can just uh you know like write this line of code and it will just you know like open my Chrome browser okay so it will launch the pro browser okay and this line as the name suggests it is maximizing the window okay now then I am just it is a very simple line of code I am getting this uh I am just getting this URL so here in case I am opening uh youtube.com so I am getting these URL then I am putting some slip so actually this is not required but generally you know like the wave loading time is uh handled by the script but still you know like because it is very fast and we cannot understand only what happened so I am giving some weight parameter like some weight weight into the uh to the execution and like it will wait the what the code will wait for something like 10 seconds and then if it will go to the next line of code okay so as I said here again I am just finding the element by dot by what by dot name so if you type here you can see various things over here so if you see class name create the selector ID link test name tag name XPath so anything by anything you can search what you think is a unique parameter for that HTML control okay here I have taken name as I already said so by name I am putting the name value that is search query okay so the name ID and I am sending the saint Keys like this is a this is a function to send anything like type anything on that text text box so I am saying using this method and I am searching blockchain okay again I am giving a two minutes wait so I just want the port to wait for two seconds sorry not two minutes two seconds and then I am going to the button so I am accessing my button and here this time I'm giving the ID so by what I am searching I am searching by ID and this is my ID of the button that I have taken from the website and then I am doing a button click and then I am again waiting for 10 seconds just to see like what is happening so if I run this piece of code let me run this port the Chrome browser is launched and we have youtube.com as well so I am searching blockchain okay then it is searching like it is searching that it is clicking the search button okay so it is giving me all the tutorials related to blockchain so this is very simple so you can do this actually and automatically the browser gets uh you know like killed after 10 seconds okay so we don't have to explicitly close that uh so that is how it is happening okay but even if the browser is not getting close there is a function for WT or Webdriver dot quit so you can use that function to just create the browser when the work is done okay so for the timing I am hashing it because commenting it because I don't need this because anyhow I am not using this code snippet inside a function so it will automatically close the browser okay so it is a very simple code and as you can see like it is like the only like you know like four or five lines of code which is you know like I am doing to automate a YouTube search okay so this is very essential or very you know like helpfully doing any kind of web automation now we will see some other other examples also uh let me go to this code okay so if you see this this is actually a address verification site of us so I will first open this site to see what is there in this site I am closing this okay so if you see this site we are sending like you know like input giving the input address as an input so we are giving street address CT State then zip code and then we are finding that whether this address is valid or not so it is basically uh Us address verification site okay so here we will you know like do the entire automation like just you know like the bot will do automatically or the code will automatically execute and it will put the address street address City and the state and the zip code and it will also click the find button okay so this is this is again a simple very simple coding so what we need to do is again we are you know like accessing these controls by doing a inspect okay okay so say here uh it is giving me yes so this is the input and this is the IDT address okay so here again we have accessed it or access this control with the ID you can access it with the name also because both the things are given okay so I have access it with that ID so likewise you can get all the IDS of the necessary text boxes and the button controls what you need to uh just uh you know like uh handle over here okay so I will close this I will run the code so if you see the code is pretty simple the same thing the same libraries I have imported and the same way I am you know like installing the Chrome launching the Chrome uh installer or sorry for Chrome browser and I am maximizing the window then I am getting this website then I am waiting for some time I'm telling the port to wait for five seconds then I am accessing this control straight text box and I am putting a you know arbitrary address over here okay so I am putting an address I am quoting CT I am putting State and then I am putting zip code and I am seeing like clicking the button whether the address is valid and then again I am waiting for 10 seconds so the code is very simple as I have shown in the YouTube demo also so the same coding only we have instead of one text box we have four to five text boxes to uh for our data entry okay so let me run this code so after every input I have given two seconds DeLay So to understand that whether the code is actually you know giving the input put in the text boxes otherwise it will just run in the claims of like claims of a second only fraction of second okay so here I have my address it is validated okay so you can see that it has clicked the find button and the address is validated okay so this is how it is very simple so it can be you know like used on any kinds of website and like no matter what like what is the complexity of your project or what scale of web automation you are doing you can or what you know like level of web scrapping also we are doing it is library is very helpful to do web scrapping and also the you know like data entry so uh this is uh these are the two examples some other examples I have which I want to show uh okay so let me open this so another important you know like this is also a very uh our problem statement actually for us which we get often where we have the website where multiple eye prints and nested eye prints are there so if you use any RPA tool like uh I have some experience with UI path automation anywhere and we have seen that actually the bot has spelled in you know like accessing the iframes and nested diagrams so I have found out a website where uh there is a use of some iframes so if I open this website it is actually a w3school website where they are demonstrating iframes so they have to incorporate iframe for that in the site as well okay first I will close this okay so see this is actually they are demonstrating the iframe like how item works so likewise there are many Legacy systems where we have this problem like where we get multiple iframes like there are nested the problem of nested diagram like one item will be inside another iframe and that time I accessing by ID or accessing by expat or accessing by name won't work because it won't be able to even selenium won't be able to access the controls in inside the iPhone okay but even or like even other tools like you know like RPA tools they also you know like fail in you know like where there is a complex eye print problems okay now if you see this uh inspect these elements you will see there are iframes okay let's move here since take this element okay now if you see uh this is a div this is not a iframe here you can see there is a iframe okay so this is a iframe okay if I go inside this again you can see another right frame okay so there is a iframe inside an iframe okay and this I frame is inside the main page okay so there is a problem of nested diagram so here you can see actually the structure of your HTML code okay many a times happen that we try to access the controls by ID or by CSS class or by name but we cannot find the control and it gives an error that time we need to see the structure like how it is structured in the htn1 so if the iprm is there basically then you won't be able to access in a simple way so selenium actually makes it easy we can use a simple line of code only one line of code to switch inside the iPhone and then we can access this code okay so let me just show you the port but actually it becomes many like very complex in any like if you use any RPA tool okay so here see only see we have to use this line of code so when we have to switch inside the I them so no matter how many items are there if you see I am just enabling this code it is commented out okay if you see I can put frame reference so I can either put the frame ID or the frame number okay so here I have put the frame number so I am going inside the first frame and I am trying to get this header so if you see this is I think this is H1 but let me see what is the element here okay so this is H2 so I am trying to get this header and again it is inside the iframe okay so if I normally try to get this header I won't be able to get because it is inside the iPhone so let me just run this code without this line so I am not switching before switching I am running this code just to see like what happens whether it can access the header H2 sorry I forgot to save the file I've been closing this because I forgot to save the file okay if you see that no result has come because I am printing trying to print this text okay but since it has you know like even either it couldn't access this Ctrl H2 okay because it is inside the iPhone okay so with this one line of code enabled I can actually go inside the item and I can access this tag okay so you can see this text has come remove the iPhone border so this time actually it has I don't know whether this is visible uh just let me know anybody if you can so if you see I have printed this text so this is my text actually remove the iPhone border okay so this time it has printed this text so this time my control is accessible okay so wait selenium it is very easy to you know like switch to and flow inside the iframe and outside the iPhone so in this way we can you know like solve this very complex problem because actually it takes lots of time when we do not understand that why the bot cannot access this control even with the RCA tool it is a problem okay so this is very easy with selenium okay then another uh example I have is uh and I think this is this is also a very like a great example uh where we can do some multi-threading okay so python actually supports multi-threading techniques and here actually I have created two threads so if you see I have created two threads so here T1 and T2 so these are basically two like the objects of the thread class so I have created two threads and I have started these two threads so what this threads are essentially doing these trades are essentially validating address so already I have this line of code and I have shown this demonstration like I am going to this site and I am validating the address right so here again I am validating the address but I am launching two browsers okay so in one browser I am validating this address 11610 Memorial uh Pkwy South and in another browser I am validating another address okay so this is on a different address okay so but both the browsers will be open simultaneously because I am using multi-threading here okay and uh you can do like the bot can execute you know like the both the you know like browsers multiple times like it can process both the browser at the same time multiple browsers at the same time so this is also a very essential thing where we need to you know like there are some time it happens that we need to go to a website we need to extract data and then we have to take the data and insert in another Legacy application so those times we is very useful where you can you know like run two browsers at the same time okay and here actually I am opening the same browse same site basically in two different browsers so let me just run this code it is very uh like different like it is a very essential like an important feature of uh python plus selenium so it can be leveraged in a great way actually uh depending on the use cases so if you see already the two browsers are open okay so two browsers are open already as you can see okay and both the browsers are actually you know like doing the address verification but different addresses see this is a reprint address and this is different address let me go to the other browser okay both the addresses are validated so if you see this is one browser and this is another browser okay let me run this code again closing all the Chrome browsers then I think we will have a good understanding okay so now I have closed all the Chrome browsers and I am running this code again so if you see the two browsers are open okay simultaneously okay and this is actually a very you know like a very important technique which can be leveraged in a great way like if your use case is such that you need to do some multi-processing then you can use this technique because no RPA tool gives you this facility you don't like to do uh like to execute uh like launching both the browsers at the same time okay so here I can see that both the addresses are validated okay so this is one thing and another thing is that one small example is there okay where we do some uh where we have taken some headless mode okay so the browser will open you will do the work also but you won't be able to see it in the uh like the seat that in the system that the browser has actually opened and you are you know like doing some data entry wall this is very useful when you know like when uh we have the client sensitive data and it we don't want to export the data to the outside world okay so there we can use headless Elysium mode where we can do just in the back end but at the same time if any error occurs or anything occurs you can take a screen capture so if you see I have taken a screen capture in this path and uh already I think our image file will be there in this path so I will go and delete this image file so in front of you I am deleting because I want to see that whether it is generated again okay and the same YouTube code I have used but this time I'm searching analytics video and I am taking I am taking a capture of that image capture of that site because I cannot see the browser open so it is actually running in the back end kind of it is running in the back end it is running in the Headless mode okay so only thing is that we are doing the Headless equal to true so we are putting that option for the Chrome driver so if I run this code again it is a very uh like it is a very important technique or like very essential like when you when you have the client sensitive data it will be helpful okay so here actually I am not understanding whether at all the browser has opened because browser has opened and worked even I am not understanding that whether it has worked okay so if you see that I have printed also the driver.title that means the website title basically so this has given me the print like analytics with the YouTube and let me see whether any image has been generated okay so there is an image let me open this see so you didn't understand even that the browser has opened and walked and everything is done completed but it has done like it has been done in the back end and if any error offers you can take the image as well you can capture the email so this is another essential you know like a feature of selenium which you can you know like incorporate depending on your use case okay just two more minutes I will take so similarly we can you know like access selenium through a VBA code also so VBA is very uh you know like uh like mostly used widely used in you know like any automation so inside the VBA code also you can do web scrapping or web automation using selenium here I have used to uh like for that actually uh I will go back to the slider later so I will just tell two things that is where you need to like install the selenium basic tool tool for VBA so this is a site actually where you can go and install this and chrome driver is uh I will show you how to install the Chrome driver and keep in this path so you have to ins like download the Chrome driver uh based on your version and you have to just you know like save that in this part okay so let me just go to the code uh first I will run the code to show you how it is working okay so once you install the selenium basic here you can uh actually you know like refer that going to tools reference and here the library will show so I have selected the selenium type Library okay so this is already selected over here now here what I am doing the uh I am going to Amazon Amazon and I am searching for cameras okay and I am searching for cameras and whatever camera I am getting over there I am just downloading this or scrapping the data the name of the camera and the website of the camera the link of the camera and I am incorporating in this page okay so little bit I have elevated it so okay that way I can see the site okay so here as you can see that this camera sheet I have prepared okay and there are already some data so what I will do is I will delete all this data to see how the code is working okay so now I don't have any data and I'm saving the file and keeping like this okay now I am just running this basic coding and again it is the same coating as I have shown you python we are finding the element by ID so here I have just put the search text box and the button click and whatever like the cameras I am getting so I have actually uh taken all the eight tags so a tags is giving me the link of the camera okay so I have taken all the eight tag and I have Incorporated the link in my Excel okay so if I run this code so it will actually automatically close the browser okay now let me go to the page and you can see all the camera names and the link has been Incorporated so this is also again another essential you know like way of you know like doing the web scrapping for the VBA itself from the VBA code and here you are using the basic selenium Library okay and another way of doing the web scrapping from VBA is using the query table so I have shown both the ways so there are actually multiple ways I can download the HTML and read that HTML and purse and take the data also uh so here I have used a quality query table so this is a site where it is giving me country plus Capital so I will just run this code and it is populating in this sheet one so I will delete all these data okay so now now my Excel is clear clean okay now I will just run this code piece of code I think it is very fast actually I think it will be done see so all the country and the capital is populated so again it's a very like good technique you know like to scrap any website so I have actually scrapped this website and I have taken the country and capital so all the countries and capitals have been populated so this is the session uh most of the session I will take the Q a only now uh just over one thing I want to share with you guys like if you face any problem in with the Chrome driver uh just open your Chrome here and see the version so mostly with the pro Moto instiller we won't you won't face any problem but uh if you are accessing the Chrome driver from your VBA code uh then and if any problem is there just open the Chrome driver I will grow here see the settings okay so I will go here and see the version of the Chrome so it is actually about so here you can see the version of the Chrome okay so this is one zero seven dot zero so I will go to the website and I will search Chrome driver download okay so you can do this for the other drivers as well so this is my 107.0 I will download this so you will get a exe file which you essentially need to keep inside the selenium basic like where your selenium basic is installed for VBS so you have to take the exe and you have to keep it here so this is a path like in your user in your user folder there will be update and open and the selenium basic uh where where it will be installed so you need to just store this file there then everything will be fine so most of the times the version problem only happens but other than that it's run smoothly yes so I am done with this session uh any questions okay what is the difference between Automation and RPS okay so automation actually basically it can be anything the process automation when we talk about uh suppose like when you are building the website also you are you know like uh like fetching the data from the database uh you are you know like taking the data entry from the users so all these things are automating even you're automating like the file creation you are writing like opening a file you are creating a file you are writing something in the file then you are closing the file so that is kind of automation but RPA stands for only for those stuff which are high in volume we are which are repetitive and featured like mundane process like it's a it's kind of a boring work like day to day you are doing the same right suppose for example uh you are opening your laptop and first thing you need to do is you need to read the emails from the clients so based on the email uh email emails then you need to take certain actions and those actions are already predefined you know where I have to go I have to go to this site I have to do some entry then I have to generate the report then the report I have to mainly take into the client so this set of stars is like very much repetitive and you are doing it for high volume of data maybe three thousand four thousand transactions you are detailing everything so there comes actually the RPA like the robotics process automation okay so this is uh this is clear so this is the difference then how RPA is different from web automation like selenium okay RPA if you take RP is a superset so RPA is not only about web automation but it actually handles lots of many other automation it can be file automation it can be a dashboard automation it can be desktop automation uh even it can be Excel automation email automation so lots of automations you know like club together in a single process called RPA where Bobs are actually you know like mimicking the Human Action wave automation if you talk it is a subset of RPA so it is one of the features of RPA but it is not everything about RPA but yes it is a most complex thing as I already said because lots of the problem you faced in wave automation only because sometimes a wave loading factors are taken into account and where you need to you know like uh really you know like handle those website very critically um so what's often fail in web automation so that's that that is why where water mission is very important for RPO okay is RPA not possible beside website automation yes it is possible depending on your requirement if your requirement does not actually have any website interaction then you don't need website automation but if if your requirement is that you have to interact or handshake with different Legacy websites then this this website automation is required this is the you know like parts and parcel of the process so you cannot eliminate and go with the RPA okay so if you want to if you are giving an end-to-end automation to your clients then uh and website is like interaction with the website or any Legacy site is a part of the process then website automation becomes important in that but if your process doesn't require website automation then it can you can eliminate that it is not required so it is depending on the process okay the manual process what you are dealing with okay have you used rice Raspberry Pi for the creation of any RP offering no I haven't used Raspberry Pi but uh I would like to know or learn about it uh so maybe if anybody is interested can give a session on that maybe I can uh you know like um attend that okay so I am a little bit late if you can give a give a quick recap so okay so here we talked about I don't know like from where you have missed uh here we mostly talked about the web automation uh how we can do web scrapping and web automation using python selenium VBA code like full strip based automation without using any license okay so uh those things we have seen and we have seen some hands on coding where we have seen some examples like where we can go to the actual websites and do some interaction with the websites okay if you have input data for 10 addresses in Excel it is possible yes you can open the Excel that is also possible in Python there are several libraries you can use Open PI Excel you can Loop through the data you can read the addresses and you can put the website and the same Loop you have to run again and again putting all the addresses so this is the one of the actual use case we have also dealt with and this is very much possible you can do it you can try it out with the same code only thing is that you have to put the code inside of for Loop okay does hand selenium handle simple web automation example like selenium handle both simple and complex way of automation where you eat you don't need any human intervention so there are some websites where if you log in it needs to wear or multi-factor authentication where the OTP comes to your phone and you need to give the OTP right so in those you know like processes you cannot you know like actually you cannot handle with any kind of Library not only with selenium but not with any other Library you cannot handle because they are some kind of human intervention is required but yes like other than that where no intervention is required element selenium can work perfectly you know like it can work great and you have seen like some of the features are there which are actually missing in you know like most of the you know like Market leaders of the RPA so yeah yeah that is wonderful action using selenium okay can wave automation can be implemented without the need to open red browser to avoid system overhead so these we have already shown we can go headless where actually the website is open and the work is done but we cannot set a website it is open like the browser launched we cannot see but anyhow anyhow we the edit offers or anything happens we can take the screen capture so that is very much possible okay then what if we paid something but put the same in some same in other thread can we please share some example here so suppose you are facing some data uh so but right now I cannot remember the example but I can take a customer invoice so suppose from a website you are taking the customer IDs okay so you have several IDs so you are taking the list of customer IDs so you are taking the list and you are saving it in the file and you have opened the other thread as well so that thread will just uh hold on for maybe fraction of seconds say two to three seconds till the file is created with all the customer IDs and then you can put a for loop on that file and you know like with each you know like with each ID you can search in the second site uh like anything like the invoice record or any other information so this is very much possible you can do that only thing is that you need to handle little bit with the time or you can put a while loop that while this file is created you can with uh till that time it will just launch the site and open open and keep and then as soon as a file is spreaded it will open the file and read the data and accordingly it will give the input as the customer ID to the website so that is how it can work okay assuming you want to pass you want or bypass automated authentication can I use selenium to refer to an open browser and respective session to automate tasks from an already open browser and session uh assuming you want to pass automated authentication can I use selenium to refer to an open browser uh selenium I don't think you can refer to or you can access to open browser selenium like you have to open the browser freshly from the code itself if already a browser is open uh I don't think you can access that browser with selenium but this thing I need to check I am not sure about this but this is a good question actually I can I can check on this but as far as my experience you cannot access already open browser okay web strapping is a part of way bottom optimization is a part of web optimization this is a question like I didn't understand this question so I will go forward with the other question do you have your YouTube channel where you have shared some tutorial videos no I don't have but I am releasing my first book on this so maybe it will be released within within a couple of days so there then I can share the link with analytics with there as well what is the importance of path for your Chrome driver is it depends on any specific language no it is not required actually if you are using python because we have the auto installer so with auto installer you don't require to uh give the explicit path of the Chrome driver but for the Excel VBA code you will need to give the path and there also actually you don't need to give the path you just need to put the driver in the required you know like folder the selenium folder then it will work accordingly how to integrate the python selenium with JavaS