I have been interviewing recently, and as most devs know, the breadth and depth of questions you can expect to be asked during an interview is pretty much immeasurable. With that in mind, I have been probing recruiters on what I can expect to be asked during technical rounds. Sometimes I find it hard to maintain all the possible categories of questions I can expect straight in my head, let alone the questions themselves. I tend to forget things under pressure, like that [_|_](https://stackoverflow.com/questions/9553628/piping-and-redirection)
in bash is for processes/programs and [_>_](https://stackoverflow.com/questions/9553628/piping-and-redirection)
is generally for files. In service of wanting to retain the information better and force myself to write, I have decided to start posting walkthroughs of what I brush up on. I tend to require at least 3 sources to get a complete, correct, and well-written explanation of a topic. I would like to consolidate that for other people.
This week, I went over the HTTP request lifecycle. This is a pretty broad overview, and it is for a simple HTTP 1.1 request, not a persistent connection, but it could act as a good jumping-off point for HTTP/2 and persistent connection requests. I will not cover too much about the various HTTP methods because that could be a post on its own, and because they are close enough to many developers’ workflow that the reader could probably recite the basics already.
Note: Medium doesn’t support footnotes, so all footnotes link out to the original dev.to post
Depending on how in depth you want to get, much can happen during this step depending on the application making the request. I am going to proceed on the understanding this request is being made by a browser, as opposed to cURL, an API client like Postman, or some other app.
Your browser extracts the “scheme”/protocol (we have established that this will be HTTP), host (www.example.com), and optional port number, resource path, and query strings that are specified in the form <protocol>://<host><:optional port>/<path/to/resource><?query>
. An example is |http|://|www.example.com||:5000||/mainpage||?query=param&query2=param2|
Making GIFs is hard
Like the processing done locally, resolving an IP from a “DNS server”² is a sequence that includes many steps, and includes failovers if the first request fails to return an address.
I have no idea whether this dot is actually on the shortest path
Now that the client has an IP address, it can send an HTTP⁵ request, right? Almost, but first, since the request is sent over TCP ⁶, which is a transport layer protocol like UDP, the client must open a TCP connection.
SYN
⁷ “control”⁸ packet to the server. The client sets the sequence number for the first packet to a random value purposely, in service of security. We’ll refer to this number as x
for now.SYN-ACK
message⁹, which contains an acknowledgement number for the original message that is always x+1
, and a new sequence number for the response itself, which is another random number y
.ACK
message¹⁰ back to the server with a sequence number equal to x+1
, which should match the SYN-ACK
message’s acknowledgment number and ensure that our data is being delivered reliably. The ACK
message’s acknowledgment number (since it is acknowledging the SYN-ACK
) is set to one more than the received sequence number, or y+1
.Wow, that was a bunch of steps! But now that the client has an IP address and a TCP connection, it can finally send an HTTP request! Except…..no I’m kidding, we can send a request for real this time!
name:value <CR><LF>
. Two consecutive <CR><LF>
pairs indicate the end of the header section. The only mandatory field in an HTTP request is HOST
, which contains the domain and port that the request is being sent to (domain.com:8080
), although in some cases the port can be omitted. Outside of the host field, common standard HTTP header fields include Origin
, Accept
, Accept-Encoding
, and many more. The request can also include any non-standard header fields, and historically non-standard fields are indicated by prefixing X-
to the field’s name. The body content of an HTTP request is completely optional, but often contains something like form data or JSON.ACK
message to the server. This goes on until the response is (hopefully) fully loaded.We’re almost there!
FIN
packet at the TCP level, to which the server responds with an ACK
, and then generally sends a FIN
of its own, which the client responds to with its own ACK
signal. The client then waits for a brief timeout, during which it cannot accept new connections, to prevent delayed packets from previous connections arriving during subsequent activities on the port. This four way handshake¹² signals the end of the TCP connection.All of the above is, again, only a description of a simple HTTP transaction, not a persistent transaction, which would maintain the same connection for multiple requests, but that model doesn’t differ heavily outside of behavior that occurs once the TCP connection is open and the first control packets are exchanged. It also doesn’t cover parallel HTTP transactions, which spin up additional connections for subsequent requests once the first request’s connection is established, but again, the lifecycle doesn’t change much. Primarily, these other approaches help mitigate something you probably noticed as you were reading: the handshaking spin-up and tear-down procedures of a single TCP connection can be very time consuming, and developers have the ability to avoid re-performing them or performing them consecutively.
Well, that is it, that is the multi-layer lifecycle of a single HTTP request! Thanks for reading, please let me know what you think, and if you feel that any corrections are needed please reach out!
Originally published at dev.to. List of sources can be found at the end of the original post.