Overview
The internet is a network of networks. Millions of independently operated networks (ISPs, clouds, universities, companies) agree to exchange traffic using a shared set of protocols. Data travels as small packets, each carrying a source and destination IP address, and routers forward each packet one hop closer to its destination.
Communication is organized in layers. Each layer solves one problem and relies on the layer below: the physical link moves bits, IP moves packets between machines, TCP or UDP moves data between programs, and application protocols like HTTP define what the data means. Understanding these layers lets you reason about latency, failures, and where a design decision actually takes effect.
A letter is split into numbered postcards (packets). Each postcard has a destination address (IP) and an apartment number (port). Sorting offices (routers) do not know the whole route; they only know which next office gets closer. Postcards can take different routes and arrive out of order, and TCP numbers them so the receiver can reassemble the letter.
When to use it
- Explaining why latency exists and why distance matters.
- Debugging connectivity: is it DNS, routing, TCP, TLS, or the application?
- Choosing where to place servers, CDNs, and load balancers.
- Understanding what can fail between a user and your service.
Where it shows up in interviews
Recognize it when: an interviewer asks why a global service is slow or how to make it faster.
- Design a global video platform
- Design a multiplayer game backend
Recognize it when: what happens when a region, ISP link, or data center goes down.
- Design a highly available API
- Design multi-region failover
Where it is used in real software
Autonomous systems (ISPs, clouds) announce which IP ranges they can reach using BGP. Misconfigured BGP announcements have taken major services offline for hours.
About 99% of intercontinental traffic runs through undersea fiber cables. The speed of light in fiber (about 200,000 km/s) sets a hard lower bound on latency, around 70 ms for a New York to London round trip.
AWS, Azure, and GCP place regions near users and connect them with private backbones to reduce hops over the public internet.
Key terms
- Packet
- A small unit of data with headers (addresses, protocol info) and a payload.
- Router
- A device that forwards packets toward their destination based on routing tables.
- ISP / autonomous system
- An independently managed network that exchanges routes with others via BGP.
- TCP/IP model
- Link, Internet (IP), Transport (TCP/UDP), Application (HTTP, DNS, SMTP).
- RTT
- Round-trip time: how long a packet takes to reach a destination and come back.
How one packet crosses the internet
- 1Application creates data
Your browser produces an HTTP request.
- 2Transport layer adds ports
TCP wraps the data with source and destination ports and sequence numbers.
- 3Network layer adds IP addresses
IP adds the source and destination addresses. This is what routers read.
- 4Link layer delivers to the next hop
Wi-Fi or Ethernet frames carry the packet to your router, which forwards it to your ISP.
- 5Routers forward hop by hop
Each router looks up the destination prefix and forwards the packet toward it, typically 10 to 20 hops.
- 6Destination unwraps the layers
The server's network stack removes each header and hands the payload to the program listening on the destination port.
STEP 1The browser creates an HTTP request; TCP and IP wrap it in a packet addressed to the server's IP.
Latency by distance
Light in fiber travels about 200 km per millisecond; real routes add 20-50% overhead
| Route | Distance | Minimum RTT | Typical RTT |
|---|---|---|---|
| Same data center | < 1 km | < 0.1 ms | 0.2-0.5 ms |
| Same city | ~50 km | 0.5 ms | 1-5 ms |
| US East to US West | ~4,000 km | 40 ms | 60-80 ms |
| New York to London | ~5,600 km | 56 ms | 70-80 ms |
| US to Australia | ~15,000 km | 150 ms | 180-220 ms |
NOWRoute: Same data center | Distance: < 1 km | Minimum RTT: < 0.1 ms | Typical RTT: 0.2-0.5 ms
No amount of server optimization beats physics. For global users, you reduce latency by moving content and compute closer (CDNs, edge, regions) and by reducing the number of round trips.
Implementation
# See the route packets take (hops and per-hop latency)traceroute example.com # macOS / Linuxtracert example.com # Windows # Measure round-trip timeping -c 4 example.com # Show your machine's IP addresses and default gatewayip addr && ip route # Linuxifconfig && netstat -rn # macOS # Inspect each layer of an HTTPS requestcurl -v https://example.com # DNS, TCP connect, TLS handshake, HTTP exchangeComplexity and performance
A physical floor on latency.
Routers between client and server.
Typical maximum packet size on Ethernet.
Trade-offs
The internet shares links between many users (packet switching), which is efficient but gives no latency guarantees, unlike old telephone circuits.
IP does not guarantee delivery or order. TCP adds reliability at the cost of extra round trips and head-of-line blocking.
Variants and related techniques
A 7-layer reference model (physical, data link, network, transport, session, presentation, application) often used in interviews.
Large providers route traffic over their own fiber to avoid congested public paths.
Common mistakes
- Assuming the network is reliable and instant.
Fix: Design for timeouts, retries, packet loss, and variable latency (the fallacies of distributed computing).
- Ignoring round trips.
Fix: Every extra round trip (DNS, TCP, TLS, redirects) costs a full RTT, which dominates on mobile and long distances.
Interview questions
Why can't we make cross-continent latency near zero?
Data cannot travel faster than light in fiber, about 200 km per millisecond. The only fixes are moving data closer to users and reducing the number of round trips.
What is the difference between the IP and TCP layers?
IP delivers packets between machines using addresses, with no guarantees. TCP runs on top and delivers an ordered, reliable byte stream between specific programs identified by ports.
Practice problems
| Problem | Difficulty | What it trains |
|---|---|---|
| Run traceroute to three distant sites and explain the hops | Easy | Routing and latency. |
| Estimate page load time for a user in Sydney hitting a US server | Medium | RTT math and round trips. |
| Design a global static website with low latency | Medium | CDN and edge placement. |