Regex is HARD!

Key Takeaways

I'm going to cover three reasons why you should be careful using reg access in your code regular expressions or reg xes are really powerful tool to help you see if a string matches a certain pattern they're widely used for validation of forms for example if you enter your email address somewhere then you can use a regx to check that it's actually a valid email address and lots of websites lots of applications are using that but if you use reg AIS wrong it can actually do way more harm than you might think by the way this is a new series that I'm starting of shorter bit more focused videos that I'm going to publish on Tuesdays we haven't really thought of a proper name yet for the time being I'm just going to call these Tuesday tips with the rist that actually we decide to no longer run these on Tuesdays and then we have to rename them but Tuesday tips let's stick with that if you have any other suggestion for how to name these videos feel free to post it in the comments the first reason reason why you should be careful with reg AIS is that you can trigger a so-called redos attack here I have an example of a simple main function that has an email address and it calls the function validate email and validate email what does that do well that's a uh simple function that matches a regular expression with a pattern in this case we're doing this for email addresses when we run the main file you see it's actually really fast 0.0022 seconds to perform the match but actually there's a huge problem with the RX that we're using in this particular case if I change the email address to something like this lots of A's let's save that so now you see when I run this code it doesn't actually stop at all so I have to cancel the program like this the reason that this happens is called backtracking and that means that if the input fails to match that the regx checking engine goes back to the previous position to try again and the engine will try this over and over again to explore all the possible paths and the particular regx that I'm using here is inefficient so it creates this really long Loop to go through all of the possibilities and because not everybody is nice on the internet we can't assume that everybody will put in nicely formatted email addresses in your form so you have to be careful and that also brings us to reason number two which is that readability and maintainability of reg xes is hard so here you see a couple of different reg xes for email addresses now before I continue pause the video and ask yourself which which one of these reg xes is actually a bad reg X is it one two or three the answer to the question is the first one is bad and reg xes 2 and three are good now if you didn't catch this don't feel bad about it because reg xes are actually really hard to read and it's really hard to guess which ones are good and which ones are bad here you can actually pretty clearly see the difference between the performance of these R xes so I have the evil pattern which is the blue line and we have the good pattern zero and good pattern one which uh you don't see this one because it's overlaid by the green one because they're both like really fast but you can see clear difference between a good pattern like this one and the bad pattern like this where if the length of the string increases the evil pattern the bad pattern actually increases a lot in time in order to compute that particular match so that has the possibility to make a big difference in performance of your application if you're using a lot of pattern matching the third reason that you need to be careful with reg xes is that they can lead to false positives and negatives for example if your email address validation Rex is not perfect it might actually let through invalid email addresses that are done stored in the database leading to all sorts of problems with people not being able to log in and especially if you use the email address later on your application you're going to assume that it's going to be correct so that can lead to all sorts of problems and then when your application crashes because of that it's going to be really hard to debug because it's it's not a problem in the code it's a problem in the data now of course this is something that you should take into account when you send an email and probably an email service will raise an error if for some reason the email address doesn't work but it can still be hard to figure out so in summary you have to be careful when using regx one small additional modification to the pattern can actually have a huge effect on performance so two recommendations the first is use a pre-existing and validated regx don't just randomly use something that you found on the internet or on stack Overflow or something make sure you're actually using using high quality reg xes the second thing you should do and that's specifically to avoid these redos rega Deni surface attacks is to make sure you have some additional input validation for example you could put a maximum on the string length to avoid these kinds of attacks and that way you have an extra line of defense that helps you avoid performance issues with regex validation I hope you enjoyed this short video for the little dashboard that I made here to compare different reg xes I actually used Dash and that's a really nice tool to create these sorts of dashboards if you want to learn more about how to build an app using Dash I did a video about that a while ago and you can watch that right here thanks for watching and see you soon