Tutorial 12- Name Node Architecture

Key Takeaways

hello everyone welcome back so in this session we are going to talk about what is name node and why name node is important so fine let's try to understand that so basically as we know that whenever we'll try to store a data inside HDFS file system so first of all you will have to encounter a name node or whenever you will try to execute any kind of a job any kind of a processes so you will have to encounter a name node and name node is a one which will try to provide you a meta information meta information about I can say a file system which you have stored inside your HDFS and apart from that so named or is someone which try to store a in memory mapping of each and every blocks so let's suppose if I am trying to execute some jobs so there will be a client who will try to initiate I can say a job so fine I am trying to execute some job my first point of contact will be a name node I can say so it will try to go to a name node and it will ask for the information because whenever we are trying to execute a job means what so simple meaning is so you are trying to access some file and then you are trying to perform some kind of operations on top of that file but this file information will be available inside a name node so if a client is going to execute some kind of a job as some kind of a processes so it will ask name node that okay so I am trying to execute this file so whether you have an information about this particular file or not and in a response so name node will look into its a meta information I can say or in other word so I can say it will look into something called as a FS image so FX image is nothing but it's a file system image so it will look into a FS image which always try to store a meta information about each and the data set that you have stored inside the HDFS system and then it will respond back to the client that okay so you are trying to execute basically this particular file and this particular file is available or this particular file is not available so this is what a name node is going to respond back so as per the roles and responsibilities of this name node so it is store a meta information and it is going to a store I can say in memory mapping of blocks I can say so fine so it is going to store these two information and based on that so it will try to initiate a job so once client will be having an idea that okay so this files is available and like a name node will be able to respond back to the client so then it will ask name nor that okay so if this file is available now I wanted to execute a job so according to Hadoop one dot X architecture so name node will try to initiate a process so it will ask your job tracker basically so name node is going to ask a job tracker to initiate this entire process in which it is supposed to execute a data set so basically job tracker roles and responsibilities is to find out that okay so in which node my data set is available and which node is available as of now for the execution so fine job tracker will be able to extract all of those information from a name node that in which system my files are available and then so suppose like we have a multiple system over here so we have a multiple system in a place so this is called as a data node so let's suppose I have a multiple data node or a multiple system which is actually having a file inside this one so job tracker will ask each and every task tracker I can say so you will be able to find out something called as a tea tea or something called as a task tracker associated with each and every data node so job tracker will I find out that let's suppose a data set is available with this data node so it will try to ask this trash tracker that okay so whether you are available or not at this point of a time so if it is responding back by saying that okay so I'm available at this point of time and even data files are available with me so it will try to send this information saying that okay so let's try to execute it if this files are available and if I'm talking about this data node so data node is nothing but it's a system I can say system which will be having a ramp which will be having a hard disk as well as which will be having a CPU computation power I can say or GPU computation power I can say and task tracker is a local entity which will try to monitor a respective data node whatever task gets executed inside this data node so basically trust tracker used to monitor each and every data node and it will try to send a response or I can say it will try to send an acknowledgment acknowledgement or a feedback to a job tracker by saying that okay I am trying to execute this file what I'm trying to restore this kind of file so this is what a trash tracker is going to do and data node is basically someone which will be having all the processing and computation power and data node is someone which will try to store each and every information and each and every data set so this is architecture we can define for Hadoop one dot X whenever we'll talk about a Hadoop 2 dot X so you will be able to see some changes inside this architecture so basically this entire architecture is called as a master/slave architecture so if I am talking about this upper one so this is called as a master and if I am talking about all of those combination of a task tracker and a data node so it's called as a slave I can say so master is going to command each and every meta information as well as master node will try to initiate each and every job inside this entire cluster or inside this entire system and a slave node will be a worker node which is going to work actually which is going to process your data set which is going to execute your data set there will be a communication in between a master and slave through a task tracker and a job tracker so job tracker will keep on taking a response and task tracker will keep on giving a response to the job tracker and then accordingly so if that is some file manipulation which it has done if it has created a new file so it is going to update your name node basically it is going to update your meta information' but not directly so in our next section we are going to talk about something called as a secondary name node and we are going to talk about a primary name node so in that one so we'll try to understand that how it updates our meta information on a timely basis I can say but as of now you can try to understand in such a way that so trash tracker sends a heartbeat or trash tracker since our update to the job tracker and accordingly so job tracker try to update our meta information and in this way name node keep itself updated with each and every changes which is going inside a slave node so this is a architecture of I can say a Hadoop one dot X so thank you so much guys for watching this video in a next section so we will come up with some of the new concept with respect to Hadoop architecture