HTTP Caching with E-Tags - (Explained by Example)

Key Takeaways

HTTP e-tag is a mechanism to validate the web cache in order to improve performance in this video we will explain how etags work and will also explain their pros and cons and answer the controversy re tags bad what's up y'all this is hussain NASA from I geometry where we discuss software engineering by example and let's get to today's video eat ax this is a very interesting construct etags are have been built in order to improve the web caching and improve performance in general so so the definition is I would basically the client let's look like a second look at the client here could be a browser most probably it is a browser or an HTTP client that you wrote your own either javascript electron I was a desktop application anything that uses HTTP in general could be in potential HTTP client so you would make a request who says hey you know what give me this image or give me this resource right using git or our post or any other HTTP method and then the boat server says ok I'm gonna give this to you and here it also by the way this is that etag associated with this resource if you want you can use it keep it keep it handy because next time you request the same resource just tell me that etag and i'm gonna tell you if it changes or not so what does this do but by just using this mechanism now we have gained a lot of performance right because the user out of the web server doesn't have to respond with the whole resource like if it's an image and that client says you know what don't don't give me an don't give me their image if it doesn't change just just tell me that it didn't change so that's essentially it's very simple construct which is very simple contract but it causes a lot of controversy and a lot of problem people companies have been using it for bad things so we're gonna explain that a minute or so but let's go to their example here so I am an HTTP client a browser and there is a wrist endpoint here and I'm requesting user Bob hey just give me Bob right could be any resources but in this case it's a rest endpoint that says user slash Bob just give me a formation Bob are like user ID a name a general like when was the last time you looked and or what movies he watched anything right it depends on the web server and then you would record the server says hey by the way this is the Jason I didn't I cut down the response here but this is the Jason response and here's the e-tag associated with that Bob right and then it's a client responsibility whatever the browser or the HTTP client that you write - next time you request the same bob user you do the same thing but you also add a nice header if none match and you specify that you know as a client Bob is associated with this tag you you did the work to link those tube and source together the resource and the tag and then you wouldn't make a request and use as they said hey by the way if you change please tell me that it is Joe I can make the decision and move on ok so the server in this case says by the way Bob didn't change that resource that image that file that downloadable file that anything really any resource it didn't change just tell me that didn't change so this is a very thin and a very fast response right the user that the server doesn't have to first consume that extra memory to build that resource it doesn't have to consume CPU resources to build that resource it doesn't have to send all this big stuff through the wire because hey this didn't change so why why send it to begin with right so that improved performance and then just quickly responds and the client can move faster so the broth performance so it's a caching as you can see it's just built in caching and it start building your own caching is that server you can just utilize that attack right and yeah just like it tells you that information and attack that's how etag works very simple all right however so these are the pros the pros are obviously fast response you get faster response you get less bandwidth you don't have to this saturate your bandwidth with stuff that you already have right thus obviously four four four four countries that has less bandwidth or four companies that has less bandwidth or clients that has less bandwidth you utilize this so you get even with countries that has a huge banner but this is actually really good you don't have to send something that the client already has all right if it doesn't change its tell me and and and this is really good if if you are implementing your own application your own wrists client right do utilizing the e tags are very critical here so apart you what it does like Apache server if you have the web server core bacheior iis and does this on algorithm to build that etag right this brings us to the cons really right so before we go to cons one more benefit on this is the pros is couldn't consistency in the databases people have been using this as it are debit transaction so I can make a request and guess what it says like you know how I'm gonna reference here the the video we did and on concurrency and that at the DBMS in the relational DBMS is like how multiple transactions if they are not modifying the same resource they can execute in parallel so you can actually use etext at that right how so he says ok you know what I'm instead of doing again i'm gonna i'm i'm changing bob i'm updating certain information above but i changing but on that moment on that attack right because I say I'm changing name is bad ideas like what do we change it Bob let's say I'm changing the nickname right Bob nickname so the original nickname is this right at that at the moment you got that etag and then you say by the way I'm changing bug nickname to something else and that's the e-tag that's the moment I want to read from okay if that the attack doesn't match fail please go ahead and failed that means someone between my time reading that resource and posting that transaction someone changed it so if if that etag is bad just please go ahead and fail so that's another way to control concurrency and then that consistency essentially in the database transaction so let's go through the cons right so as we said II techs are generated by the server weeks kinda as administrator have controls over that we can control how he tags are generated but by the by default apache like tomcat server or iis or other web servers as well generate their own etags based on some algorithm right so the file size the date modified maybe the server that is located on and analyze a problem right a lot of people a lot of website says you know what please don't use a tag because it's causing problems or what what problems are is it causing let's explain it so one of the problems that etags are closing is when it is laying behind a load balancer right so this is just quickly moved this o below here so what it does here is if your what if you have like a load balancer set ups and this is very very popular setup because you have a load balancer a cluster and you have multiple servers and you're gonna request user Bob all right so the first requisite because it is stateless you go you're going to web server one and though observer one it's okay by the way a hair's bob by the way and here's the e-tag for Bob I generated to be ABC I just shortened that just obviously longer so web server one generated the resource for Bob to be ABC and it returns to the client so clients just saved add to be ABC but then it says ok I'm gonna request that game resource Bob and I'm gonna add that if non-match ABC right that's exactly do it right but guess what now the lovebirds are decided to move me to observer - okay I'm expecting Bob will didn't change however whoops over to generated and another te tag and he says ABC is not my attack sorry it looks like Bob has changed bob user did not change it didn't anything that nothing happened but the wayy tags are generated our per server and that is a problem to a lot of countries I mean companies write one website and one server generated it again for that for a resource a and then another attack for a resource a four on another server that's a problem because now attacks are useless and actually they are an overhead to begin with because if I am requesting the same resource and it's generating these like every time and it's a big problem right because now I'm equipping first I'm requesting the same resource it didn't change yet I am adding extra bandwidth right and I'm adding an extra e tag header which is absolutely useless in this case right right so how do you solve this problem this problem is not hard it's it's solvable right but a lot of people just don't want to deal with the headache right that you tell the server basically hey there is this is a good configuration so you have to spend time in the configuration of Apache or I is so the web server says you know what if you generate etags they should be you know the same across all servers right so I'm gonna link out description below how to do that in Apache and I is so it depends right so you have to configure your Apache servers to always generate a fairly certain resource of it didn't change use things that are doesn't rely to that it doesn't depend on the server you're on essentially right so let's talk about the cons that's the first cons right obviously the first cause is the load balancing problem but can be solved with the what server if you configure your server correctly another what another disadvantages obviously if you're writing your own client here right if you are having browser you don't really feel it right but if you're writing your own application as software engineers we write a lot of HTTP calls right and then as you have to make them etag aware because that's the nothing comes up free right so we have to check like Oh read the header and if it's etag if it's the same then request it if it's not then delete that cache and you have to manage all that to yourself so obviously it's harder to write there are benefits but it's harder to write so it depends on you right so I'm gonna actually ask the question after that but last one is and it's really really bad right because some companies like Hulu have been using etags to attract users so how how do they do that so one way you can do that is is basically instead of using cookies because cookies people basically can delete them etags cannot easily be deleted you know because they are managed by the browser right the browser said oh why would the user shouldn't care about anything this is a it's a known thing right so if I request an image I the browser I am Chrome or Firefox I'm responsible to basically keep requesting the same image and I'll take the job for you I'll take care of this you don't have to worry about it however companies or web sites like Hulu and others have been that was a long time ago they've been using it to track users and so they wrote a special attack generator at the web server attacking server to always respond to the server by hey it did not change it's always responds with the server will this - always responds with not modified so this way the client will keep always with will always keep sending the same etag okay what's the benefit of that if the server always returns this like for a certain request certain page let's say user index dot HTML if that page the home page always returns not modified and the whole size okay I'm not gonna change the side really we're going to track the server and and then we're gonna track the client and then you're gonna return that results and then II tagged right will not change so guess what the client will not change that either so they will say oh it did not change I'm gonna keep sending that and guess what this judge you just has established an identity for that session for that client now even if you close a browser you come on restart it and you opened again you're gonna you the browser will always send that etag because it has it somewhere right it's cached right and then you're gonna send that request and the server says yeah no it didn't change so now the web server can use that attack to track you it doesn't have it doesn't know your name doesn't your nor information but it knows your behavior Oh dad Oh history's more hey okay he's watching house of cards he's using are you watching I don't know a fixer-upper all these shows now ok he's watching all this stuff so now they built they don't need to know your name even if you're not signed in they don't care right they just use that e-tag that etag is someone lives in Minnesota okay and he likes to watch Game of Thrones everybody likes to look at you and so that's not an argument here but yeah comment if you know this is don't disagree okay but yes so they can use that eat a gif it's always the broads are gonna always send it then that's your identity essentially right and that's bad so the lawsuit has been filed for them because it's cannot be purged and then I think chrome try now included a way to purge even etags in their system all right guys been a long video hope you guys enjoy it under gonna shut that question to you what do you think do you think II you are you even using etags in your have you ever used it is it useful do you see it do you see it like using etags will improve your application performance in general what do you think I mean I personally have used it ArcGIS Pro actually uses etags and the if you user actress produces and is very applicable uses etags to cache resources right and yeah guys I personally didn't develop an application using e tags but I don't mind it if I use it in a correct way I can still save myself a lot of trouble coding at the server caching mechanism and instead using that built-in HTTP mechanism alright guys you stay awesome let me know what do you think in the comment below and I'm gonna see you in the next one if you liked this video consider subscribing and like this video if like it add any ask any questions you want and check out the other contents of this channel IgM trig walls discuss software engineering by example a lot of cool stuff here to become a better software engineer and I'm gonna see you in the next one have a good day