Linux for Programmers #10 - Grep Command & Regular Expressions

Key Takeaways

[Music] hello everybody and welcome to another linux or programmers tutorial video in this video i'll be covering the grep command now the grep command is extremely useful it's used very commonly and it actually exists in pretty much all linux distributions now what grep allows you to do is search for a pattern of specific text in a file in console output and in a bunch of other things as well now it's very useful there's a ton of different uses for the grep command and as like all the other videos this is just going to be kind of a brief introduction to it you will have to do a little bit of your own research and and find out a bit more about the grep command because i really just can't cover everything in this one video so with that said let's dive in after a quick word from the sponsor of this video and this series which is lenode now lenode is the best company to use when you're hosting a website app or some service in the cloud if your app runs on linux it will run on lenode and lenovo has great support 24 hours a day seven days a week 365 days a year as i've mentioned i have been a customer of lenod for over two years i love their service i use them for all kinds of different projects and i just love that they have cheap and affordable servers that i can use for some basic hobby projects too and i don't have to kind of break the bank if i just want to get something spun up and use it for a few hours or a few days so anyways claim your free 100 credit towards the node by clicking the link in the description and one last reminder to make sure that you sign up to get access to the last five videos in this tutorial series this is video 10 if you want to see the other five videos and you don't want to have to wait for potentially a long time then make sure you sign up at this platform again there will be a link in the description now lastly there is a text guide for this tutorial video this comes from lenode this is a very helpful guide talks about a lot of the things i'm going to say in this video and just as kind of a cheat sheet for some of the syntax and regular expression syntax and all that so anyways make sure you check this out there will be a link in the description to that so with that said let's talk about the grep command and actually before i do that i want to quickly show you what i've set up here on my system because i've created a bunch of files and a large text file that we're going to use to kind of test out this command so i'm in this folder called files and if i ls you can see that i have a bunch of files created here now i've also created another file this one is called grep.txt and inside of here you can see that we have a bunch of text and you don't have to worry about the contents of this text or anything like that you don't have to recreate this just understand that i've generated a ton of text and a bunch of empty files i just wrote a really basic script to do this so we had some kind of meaningful output for our grab command in this video so anyways let me show you grep so the syntax for the grep command is simply grep a pattern and then a file that you want to search for this pattern in now this is kind of the most basic usage of the grep command you can also use this to filter console output which i will show you in a second you can do a lot of other things with it as well but specifically if we wanted to search for a pattern in a file then what we would do is we would type the pattern so let's say i want to look for the pattern which is a and i want to find that in the grep.txt file i would type grep a grep.txt hit enter and this is going to show me all of the lines that contain this pattern on it so let me go back here let me just clear the screen and i'll explain this more in depth so this is the command that we just ran so what this is going to do is it's going to find the pattern a in this grep.txt file and whenever it finds this pattern is going to print out the line in which this pattern appears on now notice i keep saying pattern the reason i'm saying pattern is because we're not just searching for this absolute text we're searching for a pattern we're going to match whatever pattern we type here which can be a lot more advanced than this with all of the text in this file and if we ever find a match we will output the entire line in the file that contains that match even if only part of that line was the actual match so for example you could see here that only a is what's actually matching and that's highlighted in red but we're showing all of the lines in which we actually find a right and you'll see that some of the lines we have two matches because there was two a's so just keep that in mind it will show you the line in which these matches occur now as i said at the beginning of the video what grep will take or what it will look for is a regular expression pattern now i'm not going to get into depth about regular expressions because they're very complicated and there's a lot of stuff that you can show but i will show you some very basic regular expression syntax or patterns that you can write so you have an idea of what i mean so if i type grep a and then period what this period means when this occurs in a regular expression pattern is that we're going to match any character or any symbol with this period so when we talk about the pattern a and then period what that means is we're looking for anything that starts with a and then has any other character afterwards so you'll see when i run this that we get zed and then a and then any other character right and we have all other letters and then a and then any other character because we're looking for a followed by any character right now if i change this and instead i say a dot a what this means is we're looking for any pattern that starts with a has any character and then ends with it so now when i do this you can see that these are all of the patterns that we get so a dot matches any character or any symbol just keep that in mind because that's probably the most useful one or the most common one that you will use now another one that you can use is these square brackets now what these square brackets will do is look for one symbol or the other so if i type a comma b and then dot a what this means is look for anything that starts with a or starts with b that's what it means when you're inside of these square brackets then has any symbol and then ends in a so if i run this notice that we get a bunch of stuff that starts with a that starts with b and that has any other symbol and then ends with a so that's what that pattern did so i just need to take a quick pause and thank the other sponsor of this video which is algo expert i'll go expert is the best platform to use to prepare for your software engineering coding interviews and i actually work there as an algorithms instructor so if you want to check out the questions i've created on algo expert head to the link in the description and use the code temp with tim for a discount on the platform so now i'll show you a few other things that you can do with grep so let's go back to our command here so let me remove this now usually when you do a regular expression pattern you're going to put it inside of quotation marks the reason you do this is because there's some symbols that you type that will just kind of be messed up or interpreted incorrectly if they're not inside of quotation marks so keep that in mind if you're getting some types of issues when you're doing your regular expressions just just put the expression or the pattern inside of quotation marks anyways the next thing i wanted to show you though is the or operator so if you want to search for one pattern or another pattern then what you do is use the or operator so the or operator looks like this it's a forward slash and then a pipe at least i think that's a forward slash i don't know if you call that one the forward slash of the backslash someone can correct me in the comments but regardless you put a pattern on the left hand side and a pattern on the right hand side and it will look for either of those patterns so if i want to look for a dot b or i wanted to look for b dot a then i would do this and it's going to show me all of the patterns that match either of these so we're looking for a pattern that is this one or the other one hopefully that's clear but that's the or operator there's a few more things that you can do as well i don't really want to get into it a ton you can match the beginning and the end of lines so for example say i wanted to look for a b and then i wanted this a b to actually be at the end of a line i would type a b and then the dollar sign and what this means is match this pattern here so a b with the end of the line so when i do this it's going to find all the a b's that are at the end of the line not at the beginning of the line and of course if a line is just a b it would be at the beginning and the end of the line but you get the idea right so it's just showing me once at the end if i only want to match the front of the line or the beginning of the line i use this little hat so that's what's above the six or shift six so grab hat and then a b and what that does is show me all the a b's at the beginning of the line so you get the idea these can be really complicated you can make a very advanced regular expression patterns again not going to get into those too much that's all i wanted to cover so now let me show you a few more options that we can use with the grep command so first of all sometimes you don't just want to know the line that this pattern is on like the contents of the line you want to know the line number so if you want to know the line number what you do is you type grep hyphen n standing for the line number that's pretty much what it means and then you are going to type your pattern so maybe we'll do abc and then the file in this case we do grep.txt and we see that line 32 is what has the pattern abc now even if we did say a b and then dot this will show me all of the uh all of the line numbers for all of the different patterns so that's pretty helpful now there's another thing you can do which is hyphen o now what hyphen o will do and i believe this is a lowercase o not a uppercase o is it is just going to show you the matching pattern not the entire line now sometimes like i don't know why you would exactly want to do this but sometimes you don't want to see the entire line you only want to see the pattern that matched so for example if i were to type a b and we didn't have this hyphen o notice how it's showing me all all these other characters before it if i didn't want to see that and i just wanted to say c a b then what i would do is i would put hyphen o here so grep hyphen o a b grep.txt and now it's only going to show me the pattern it's not going to show me all of the letters before that now this is kind of a poor example because you know why would you want to do this but for example if we did a dot now we'll see just the pattern right we won't see all of the uh all of the characters that are not a part of the pattern but are on the same line so hopefully that's clear that's what hyphen o does just shows you the matching pattern then if you type hyphen c and this is uppercase c and then you type some number this is actually going to show you lines after the matched pattern so say i wanted to see like four lines after the matched pattern what i would type is grep hyphen c four and then we'll go a and then dot and then a and now it's going to show me four lines above and four lines below the matching pattern right so it shows it to me on both sides and so i had mis-explained that previously but it does show you on both sides of the pattern so this way you can kind of see what's around it now you can imagine that would be useful if you're searching for something in like a python file or in some you know programming related file because you could see maybe the block it was nested in and stuff like that so that's kind of the basic commands and options for the grep command of course there's more that you can do with it and the more you know about regular expressions the better you're going to be with grep but what you can also do with grep which is really useful is you can search for files you can look in directories and you can filter the output of a command so the most useful one is filtering the output of the command so i'll show you that first so if i type the ls command and then i type the pipe and then graph and then the pattern that i want to search for what this will do is it will find this pattern in this output so ls has some output right so ls command will run and then grep will search through the output of ls and show me all of the matching patterns so if i say ls and then grep and then a b it's going to show me all of the file names that match that pattern now it's kind of confusing because my grep.txt and my file names have the exact same thing but remember if i go ls i have a bunch of empty files so what i did was i did ls and then pipe and then grep and then let's just go lh and now it only shows me files that have lh in them right and i can use the same pattern i would use previously i could match the ending of the line so i would do lh like that and now it only shows me files that end in lh so you can use this on any command you don't just have to use it on ls for example we could say like cat and you know grep.txt and then bar and then grep and let's say a b and then dot and notice is going to give me the same thing right it's going to filter the cat command the output of the cat command and just give me all the matching patterns so another thing that we can do with the grep command is we can actually search a bunch of files at the same time so for example i just have changed my directories so now i'm inside this directory my home directory notice we have abc.txt a dot txt ab.txt abcde right so let's just nano into abc.txt let's add and hello and let's escape let's nano into a.txt and let's add and hello and escape and now what i'm going to do is i'm going to cd dot dot and i'm going to look inside of this entire directory so inside of my entire home directory and find all of the strings that are hello so what i'll do now is i'll just type grep hyphen r hyphen r stands for recursive so if you want to look in an entire directory and search all of the files in a directory you do hyphen r then you do the pattern you want to look for so in this case i want to look for hello and then i want to look in what directory i want to look in the home directory now i think this is just called root actually let's do grep hyphen r hello root and now it's going to show me all of the strings hello in any of the files that are inside of this root directory so notice inside of test2 we have low a bunch of times inside of abc we have hello inside of biminfo we have hello right so you get the idea that's what it's showing us so that is kind of the idea behind how grep works you can use it to search for text inside of a file you can use it to search for text inside of a directory so it looks through all of the files you can then output commands or you can then filter commands using grep so you saw when i typed cat i could filter that when i typed ls i could filter that there's a whole bunch more uses for grep as well but super useful and again definitely something that you need to know and be comfortable using so anyways i hope this video was helpful to you if it was make sure to leave a like subscribe to the channel and i hope to see you in the next linux for programmers tutorial [Music] video you