The HTTP Protocol: GET /test.html - web 0x01

Key Takeaways

[Music] in the last video we looked at what HTML CSS and JavaScript is and how the browser uses that to rendom display a web page in this video we will now talk about how the HTML code gets to your browser in the first place so we will learn about the URLs and the HTTP protocol let's start by running a simple web server on our local machine depending on what kind of language you want to use for development there are different options for running a production or development server for example if you want to write PHP code there are all projects like xampp that sets up an Apache with a database and PHP enabled for development purposes or if you want to write a python web app in flask it will come with its built-in debug web server but let's start last video we wrote a very simple HTML page and open it in the browser from our local file system now let's try to serf this HTML page via HTTP from a web server instead recent versions of PHP as well as python have cool features to run a simple HTTP server so a server is just a program that gives you a file if you ask for them let's start a simple server with PHP if you don't have PHP installed yet install it with aptg install PHP 5 CLI on Ubuntu or whatever matches your operating system then write PHP minus capital S followed by an IP address and a port where to listen from in this case we only want to listen on one 27001 which is a special IP referencing Local Host meaning this IP points at your own computer and then Port 880 Port 80 is the default port for webs ites because people on the internet agreed on that but generally you can run a server on any port you like a port is just a number inside of a TCP packet so when the network card and your operating system receives a TCP packet meaning it's just a bunch of bites it will look at the port number inside that packet and then it will check in a table if a program is listening meaning if it's waiting for a packet on that particular port then the operating system will give the content of that packet to that program and we registered PHP to listen on port 8080 so now let's open this in the browser we want to load a site via HTTP whatever that means then the IP address which is Local Host and pointing to this local machine um and the port 880 hm not found the requested resource slash is not found found let's look at the terminal we can see here that on that day this IP address tried to access slash but there is no such file when we start the server it displayed to us that the document route is here and we know we have the test HTML file in here so let's try to access test.html cool that worked and in the terminal we can see that this IP performed another request test this time to test.html we can also see that we apparently tried to access favicon.ico we didn't type that in the browser well see see those tiny icons for websites you visit well those are favicons and people on the internet agreed that file is supposed to be called favicon.ico by default so the browser is checking if you have such a file so that it can display a neat icon for you now let's open the browser developer tools again and go to the network tab then we reload this page here you can see now all the requests the browser is doing to display this page in this case it's just two simple requests to test.html and the image but if you do the same for example on Reddit this is going nuts it's loading so many resources so let's look at that test.html request so first of all it accessed the file with the name test HTML and the status return code was 200 this means everything went fine for example earlier we saw the arrow status code 404 which stands for not found there's a huge list on what certain status code numbers mean now let's click on it let's first check the response tab this is what the server returned to us after we asked if we can get the file test.html that's just the content of the file we requested but how did we actually ask this server to give us test.html let's do an HTTP request by hand we use a simple tool called netcat for that netcat is a simple Unix utility which reads and writes data across network connections using TCP or UDP but why would that help us with the HTTP protocol well HTTP is spilled on top of TCP the TCP layer is responsible for the ports and TCP sits on top of the IP layer which handles the IP addresses IP addresses identify the receiving computer in a network and the ports are used by the operating system to decide to which application it should forward the content of the packet so the web Ser will handle the content of that packet and the content will follow a special format which is HTTP so this means we can use netkat to send a TCP p packet to the web server with any payload we want and we can decide that we want to talk HTTP now let's request the test. htme page first we open our connection to the IP and port and when we type something that text will be sent as a TCP payload to the web server to request test.html we have to do the following we start with the HTTP method get so we tell the server that you want to get a resource followed by a space and then the resource path in this case SL test.html now some Webers want to follow the HTP protocol more strictly but in this case this is already enough to finish up this request we have to indicate that with an empty line and we get immediately a response back from the server so it responded to us with code 200 everything is okay and there are some value pairs called HTTP headers for example informing us that the request resource has the content type text HTML and how long the content will be followed by an empty line and the content of the resource now if we were a browser we would read that content type and we would know we have HTML and then take the file content and render that in a window so HTTP is literally just human readable plain text that is sent between your browser and the web server and yes that makes it super inefficient why not use binary but that makes it super easy for us to understand now let's go back to the developer console and check the first tab of the request the headers here you can see the response headers from the server again those key value pairs before the content of the file but you can also see request headers so the browser also added some key value pairs to the HTT P request and did not only send this one line we did we can also see this if we quit our web server and use netcat instead to listen on that Port so we can see the content of the TCP packet and now when we refresh the page the browser will send the HTTP request again to this IP and Port but this time netkit will display us what was received we just want to see a full proper HTTP request from the browser and here it is you can see it's a bit longer than what we sent the first time for example it sent a user agent with chrome so the web server knows that uh web browser Chrome is requesting the file and can act accordingly for example if the user agent were phone it might want to send a special mobile version of the file instead it also sent an accept header telling the web server what kind of content types it can work with or what language the user prefers oh and with netkit we can also send back a response so we can just sent here some text not including any HTTP header the browser will make sense of it so HTTP is super easy right a browser simply creates a connection to a server which is another program that has a purpose of providing files when requested and the browser can ask for files or Resources by using this plain text human readable HTTP format and the server will respond with the content of the requested file in the next video we will write a little bit of shitty PHP code which can be used to dynamically create files that are returned by the web server instead of only serving static files from the file system and after we got that knowledge we can then soon head slowly into security related topics [Music]