I’d like to start a series of posts talking about networking concepts. Nothing extremely complex and just for fun. I thought it would be a good idea to start talking about traceroute as it’s one of the most known troubleshooting tools. Hope you enjoy the reading. Traceroute. Any of you working with networks have used traceroute at some point to troubleshoot a connectivity issue, but how much do we really know about how it works and about interpreting its results? Most of you know that traceroute “uses ICMP”, but this is a very broad statement and not always 100% accurate. Why do I say that? Well, let’s start by analysing one of the most common traceroute implementations, tracert.exe from Windows. tracert.exe This implementation of traceroute in its most simple usage takes just one parameter: The destination IP address we would like to trace our route to.What does under the hood when we hit enter? It immediately sends 3 ICMP packets with Type 8 Code 0, also known as Echo Request or more commonly Ping, to the destination IP address. tracert.exe These 3 ICMP packets have a particularity though: Their TTL value is 1. that gets decremented in 1 every time an IP packet is moved from a subnet to another. In other words, every time a router forwards a packet, it decrements the TTL of the packet. The TTL is an IP header Expanded IP header in Wireshark So sent 3 pings to the destination IP address, with a TTL value of 1. Where do these packets go first? Well, given that the destination IP address is not another host in our network, our computer running will look up its own routing table to make a decision on where to send this packet. For most scenarios we will have a computer with simply a default route, which then points to what we call a default gateway (because it is the gateway designated for the default route). tracert.exe tracert.exe The packet reaches the default gateway, which would be a network device with routing capabilities. Probably a router or a firewall. This router will look into the packet headers to decide what to do with it. In normal situations I would mention now that the router will check its own routing table too, but there’s one thing the router will also check apart from the destination IP address: The TTL header. As we have mentioned before, the TTL header has a value of 1 and a router will decrement it when forwarding the packet, so in our case the router will have to forward the packet with a TTL value of 0. Is that even possible? Well, a broken IP stack might want to do it, but the short answer is . Once you have to forward a packet and decrease its TTL value to 0, instead of forwarding it you will and send a notification back to the . Who is the in our case? Our computer. no drop it original sender original sender What kind of notification will the router send back to our computer? If only we had some sort of standardized nternet ontrol essage rotocol! I C M P That’s right, the router will send back an , also known as “TTL Expired (or TTL exceeded) in transit”. This ICMP will also contain the original dropped packet as its payload, so your computer can identify the packet that has been dropped. ICMP packet of Type 11 Code 0 Well, that’s all fine, but how does take advantage of this? Easy! ICMP packets are of course IP packets, so they have IP headers, right? What useful information could use from the IP headers? . As the TTL Expired packet is a new packet originated on the router, the source IP address is the IP address of the router, so will read that IP address and identify who is the first hop: tracert.exe tracert.exe The source IP address tracert.exe As you can see in the screenshot above, our computer sends 3 pings to (with TTL==1 that can’t be seen in the screenshot, but you can see in previous screenshots on this article) and gets 3 TTL Expired packets back from . That’s how identified the first hop and how it measured the latency between us and that hop (time difference between sending the ping and receiving the TTL Expired packet). The process is the same until the TTL is big enough to reach the destination without expiring in transit, in which case we will get a reply to our ping (Echo Reply ICMP). 192.168.1.103 8.8.8.8 192.168.1.254 tracert.exe So yeah, we could say that “Uses ICMP”, because there’s no other protocol higher than layer 3 involved. tracert.exe Request timed out at hop #2. What’s going on? Request timed out. It’s not rare to see this message on our output. What does this mean? In a nutshell, it means that we never got a TTL expired ICMP packet nor an Echo Reply ICMP back for those pings sent. Reason? tracert.exe - The router is configured to not send those replies for security or economy reasons.- Either our pings got lost before arriving to the router or the ICMP replies got lost in their way back to us. In the above screenshot, I’m inclined to think that the router at hop #2 is configured to not send those ICMP packets back, just because the rest of the hops seem to be in good health and reporting back, so I would discard heavy packet loss in the path (see? We have started getting something useful out of ’s results). tracert.exe Fun fact: After hop #9 we have crossed the Atlantic Ocean (+100ms). There are other occasions where the messages do appear after certain hop and we never get a new hop IP address. In those cases, like in the screenshot above and just after hop #14, there is a device dropping the ICMP packets. In some occasions you’ll be missing also a bunch of results until the very last one: In that case, the intermediate device is dropping ICMPs that are not pings (echo request and echo reply). Request timed out In this case and with only this output, we can’t say for sure how many hops there are until destination and would keep going until hop #30 (by default). tracert.exe How do you go past this? You can’t. — Yes, is a bit different, but it’s not going to do the job on many scenarios. Let me explain why: But I’ve heard about tcptraceroute ! tcptraceroute sends TCP SYN packets to the destination, starting with TTL == 1 and increasing its value every time after getting a result. Still, we have to receive TTL Expired ICMPs to be able to identify who dropped the packet — we won’t be able to get them if a device is dropping ICMPs. Tcptraceroute Why then? Because if the final host doesn’t reply to ping, it might reply to your TCP SYN packet. Or a firewall in between might let TCP traffic inbound but not ICMP inbound (and permit anything outbound, including TTL Expired ICMPs). tcptraceroute There’s one more detail I’d like to highlight from the screenshot above. The theory is that tracert.exe hops should be increasing its latency, even if it’s less than 1ms and doesn’t reflect on the output, but how come does take longer to get to hop #6 than hop #7? ICMPs surely don’t travel back in time. With what we know now, that each hop is measured independently and the latency is how long does it take for us to get a TTL Expired ICMP back after we send our ping, having higher latency on a farther hop doesn’t sound that crazy. From the above screenshot I can’t be sure, but it’s probably one of these two: : Some network administrators, especially at ISPs with busy networks, configure policies to reply to these ICMPs with a very low priority so they might be very fast forwarding your ICMPs to the next hop, but when they are the ones that have to reply to them, they will probably delay the response a bit further. ICMP Throttling : A device in the path might have been more overloaded than when we tested hop #7. Network fluctuations The latency difference between #6 and #7 is not that big, but in some cases you might see up to 100ms delay on closer hops. Don’t get fooled by this, there might be no issues at all in the path even with those weird results. Fun fact: Most traceroute implementations in Linux use UDP by default. So, again, traceroute doesn’t necessarily “Use ICMP”, but it surely relies on it :-) If you think this article has been either useful or entertaining, please don’t forget to click on the tiny green heart below to recommend it to other fellow readers. I would also be extremely grateful if you could share this among your friends and colleagues on your favourite social network. You could also take a look at Wormhole Network , a company that provides on-demand secure private networks as a service. It may be of your interest. Thank you!
