← Knowledge Base

How the Internet Works

A deep dive into networking internals — from physical signals to application protocols, security, infrastructure, and the modern web.

↓
Section 01

The Big Picture

The Internet is a network of networks — billions of devices connected through a hierarchy of local, regional, and global networks, all speaking the same set of protocols. Every interaction you have online, from loading a webpage to streaming video, involves data being broken into packets, routed across multiple networks, and reassembled at the destination.

The Internet is not a single technology but an architecture — a set of agreements about how to communicate, layered on top of each other. — Vint Cerf, co-designer of TCP/IP

Layered Architecture

Networking is organized into layers, each responsible for a specific concern. This separation lets you swap out physical media (copper to fiber) without changing how TCP works, or deploy a new application protocol without touching routing.

APPLICATION HTTP, DNS, FTP, SMTP, WebSocket ← What users interact with ↓ TRANSPORT TCP, UDP, QUIC ← Reliable/unreliable delivery ↓ NETWORK IP, ICMP, BGP ← Addressing & routing ↓ LINK Ethernet, Wi-Fi, PPP ← Local network frames ↓ PHYSICAL Copper, Fiber, Radio ← Raw bits on the wire

Packet Switching vs Circuit Switching

The Internet uses packet switching, not circuit switching (like traditional phone networks). This fundamental design choice is what makes the Internet resilient and efficient.

Circuit Switching

  • Dedicated path for entire call
  • Guaranteed bandwidth
  • Wasteful when idle
  • Connection setup delay
  • Single point of failure
  • Used by: PSTN (phone network)

Packet Switching

  • Data split into independent packets
  • Packets share links (statistical multiplexing)
  • Efficient — bandwidth used only when needed
  • Packets can take different routes
  • Resilient to link failures
  • Used by: The Internet
⚡

End-to-End Principle

The core design philosophy of the Internet: keep the network core simple and dumb (just forward packets), and put intelligence at the endpoints (devices). This is why innovation happens at the edges — you don't need to upgrade the entire Internet to deploy a new protocol.

🔗

Best-Effort Delivery

IP provides no guarantees — packets can be lost, duplicated, reordered, or delayed. Reliability is built on top (TCP), not into the network itself. This keeps routers fast and simple.

Section 02

Physical Layer

Everything on the Internet ultimately comes down to physical signals — electrical voltages, pulses of light, or radio waves. The physical layer is responsible for converting bits (0s and 1s) into signals that can travel across a medium.

Transmission Media

Copper (Twisted Pair)

Electrical signals over copper wires. Cat5e supports up to 1 Gbps, Cat6a up to 10 Gbps. Maximum distance ~100m. Susceptible to electromagnetic interference (EMI). Used in Ethernet LANs.

Fiber Optics

Pulses of light through glass/plastic fibers. Single-mode fiber carries signals 100+ km. Multi-mode for shorter runs (~550m). Immune to EMI. Bandwidth up to 100+ Tbps per fiber with DWDM.

Radio Waves (Wireless)

Wi-Fi (2.4/5/6 GHz), cellular (4G/5G), Bluetooth. Subject to interference, attenuation, and multipath. Wi-Fi 6E reaches ~9.6 Gbps theoretical. 5G mmWave: up to 10+ Gbps but very short range.

Submarine Cables

Fiber optic cables on the ocean floor connecting continents. ~550+ active cables carrying 99% of intercontinental data. Each cable contains multiple fiber pairs, each carrying terabits/s. Repeaters every ~60-100 km amplify the signal.

How Bits Become Signals

Digital Data: 0 1 1 0 1 0 0 1 Copper (NRZ): __ ____ __ ____ | | | | | | | | |__| | |__| |__| |__ 0 1 1 0 1 0 0 1 Fiber (On/Off): . * * . * . . * (dark=0, light pulse=1) Radio (FSK): ~~~ ^^^ ^^^ ~~~ ^^^ ~~~ ~~~ ^^^ (low freq=0, high freq=1)

Real networks use more sophisticated encoding schemes like Manchester encoding (clock embedded in signal), 4B/5B (Ethernet), QAM (Wi-Fi/cellular, encodes multiple bits per symbol), and PAM-4 (modern Ethernet, 2 bits per symbol using 4 voltage levels).

MediumSignal TypeMax SpeedMax DistanceLatency
Cat6a CopperElectrical10 Gbps100m~5 ns/m
Single-mode FiberLight100+ Tbps (DWDM)100+ km~5 ns/m
Wi-Fi 6ERadio (6 GHz)9.6 Gbps~30m (indoor)~1-5 ms
5G mmWaveRadio (24-100 GHz)10+ Gbps~200m~1 ms
Submarine FiberLightTbps per pairTrans-oceanic~5 ns/m + repeaters
Section 03

OSI vs TCP/IP Model

There are two major models for understanding network communication: the OSI model (7 layers, theoretical reference) and the TCP/IP model (4 layers, what the Internet actually uses). The OSI model is more granular; the TCP/IP model is more practical.

OSI Model TCP/IP Model ┌─────────────────┐ │ 7. Application │ ┌─────────────────┐ ├─────────────────┤ │ │ │ 6. Presentation │ ───► │ Application │ ├─────────────────┤ │ │ │ 5. Session │ └────────┬────────┘ └────────┬────────┘ │ ┌─────────────────┐ ┌────────┴────────┐ │ 4. Transport │ ───► │ Transport │ └────────┬────────┘ └────────┬────────┘ ┌─────────────────┐ ┌────────┴────────┐ │ 3. Network │ ───► │ Internet │ └────────┬────────┘ └────────┬────────┘ ┌─────────────────┐ ┌────────┴────────┐ │ 2. Data Link │ │ Network │ ├─────────────────┤ ───► │ Access │ │ 1. Physical │ │ │ └─────────────────┘ └─────────────────┘

All 7 OSI Layers Explained

LayerNameFunctionData UnitProtocols/Examples
7 Application User-facing services and interfaces Data HTTP, FTP, SMTP, DNS, SSH
6 Presentation Data formatting, encryption, compression Data SSL/TLS, JPEG, ASCII, MPEG
5 Session Session management, dialog control Data NetBIOS, RPC, PPTP
4 Transport End-to-end delivery, flow control Segment TCP, UDP, QUIC, SCTP
3 Network Logical addressing, routing Packet IP, ICMP, ARP, BGP, OSPF
2 Data Link Framing, MAC addressing, error detection Frame Ethernet, Wi-Fi (802.11), PPP
1 Physical Raw bit transmission over physical medium Bits Copper, Fiber, Radio, USB

Encapsulation

As data moves down the stack, each layer wraps it with its own header (and sometimes trailer). This is called encapsulation. At the receiving end, each layer strips its header in reverse order.

Data
→
TCP Header + Data
→
IP Header + Segment
→
Frame Header + Packet + FCS
→
Bits on wire
┌──────────────────────────────────────────────────────────────┐ │ Ethernet Header │ IP Header │ TCP Header │ HTTP Data │ FCS │ │ 14 bytes │ 20 bytes │ 20 bytes │ variable │ 4B │ └──────────────────────────────────────────────────────────────┘ Layer 2 Layer 3 Layer 4 Layer 7
Section 04

IP (Internet Protocol)

IP is the network layer protocol that handles addressing and routing. Every device on the Internet has an IP address, and every packet contains source and destination IP addresses so routers know where to forward it. IP is connectionless and provides best-effort delivery.

IPv4 vs IPv6

FeatureIPv4IPv6
Address Size32 bits (4 bytes)128 bits (16 bytes)
Address Space~4.3 billion~3.4 × 1038
Notation192.168.1.12001:0db8::1
Header Size20-60 bytes (variable)40 bytes (fixed)
ChecksumYes (header only)No (removed for speed)
FragmentationRouters + senderSender only
NAT Required?Yes (address exhaustion)No (enough addresses)
IPsecOptionalBuilt-in (mandatory support)
BroadcastYesNo (uses multicast)

IPv4 Packet Structure

IPv4 Header (20 bytes minimum) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ┌───────┬───────┬───────────────┬───────────────────────────────────┐ │Version│ IHL │ DSCP/ECN │ Total Length │ │ (4) │ (4b) │ (8 bits) │ (16 bits) │ ├───────┴───────┼───────┬───────┼───────────────────────────────────┤ │ Identification (16b) │ Flags │ Fragment Offset (13b) │ ├───────────────────────┼───────┴───────────────────────────────────┤ │ TTL (8b) │ Protocol (8b) │ Header Checksum (16b) │ ├───────────────────────┴─────────────────┴─────────────────────────┤ │ Source IP Address (32 bits) │ ├───────────────────────────────────────────────────────────────────┤ │ Destination IP Address (32 bits) │ ├───────────────────────────────────────────────────────────────────┤ │ Options (if IHL > 5) │ └───────────────────────────────────────────────────────────────────┘

Subnetting & CIDR

CIDR (Classless Inter-Domain Routing) replaced the old classful addressing (Class A/B/C). The notation /24 means the first 24 bits are the network portion, leaving 8 bits for hosts.

Example: 192.168.1.0/24 IP Address: 192.168.1.100 11000000.10101000.00000001.01100100 Subnet Mask: 255.255.255.0 (/24) 11111111.11111111.11111111.00000000 ├──── Network (24 bits) ────┤├Host┤ Network ID: 192.168.1.0 (first address) Broadcast: 192.168.1.255 (last address) Usable Hosts: 192.168.1.1 - 192.168.1.254 (254 hosts) Common Subnet Sizes: /32 = 1 host /28 = 16 addresses (14 usable) /30 = 4 addresses /24 = 256 addresses (254 usable) /16 = 65,536 /8 = 16,777,216

NAT (Network Address Translation)

Since IPv4 addresses are exhausted, NAT allows many devices to share one public IP. Your home router has one public IP but assigns private IPs (like 192.168.x.x) to your devices internally.

Your Devices (Private IPs) Router (NAT) Internet Phone 192.168.1.10 ──┐ ├──► 203.0.113.5 ──────► Server Laptop 192.168.1.11 ──┤ (Public IP) │ PC 192.168.1.12 ──┘ NAT Table: ┌──────────────────────────────────┐ │ 192.168.1.10:5000 ↔ 203.0.113.5:40001 │ │ 192.168.1.11:5001 ↔ 203.0.113.5:40002 │ │ 192.168.1.12:5002 ↔ 203.0.113.5:40003 │ └──────────────────────────────────┘
Section 05

TCP (Transmission Control Protocol)

TCP provides reliable, ordered, error-checked delivery of a byte stream between applications. It is the workhorse of the Internet — HTTP, SSH, SMTP, FTP, and most application protocols run on top of TCP. TCP turns IP's unreliable packet delivery into a reliable communication channel.

TCP Segment Structure

TCP Header (20 bytes minimum) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ┌───────────────────────────────┬───────────────────────────────────┐ │ Source Port (16b) │ Destination Port (16b) │ ├───────────────────────────────┴───────────────────────────────────┤ │ Sequence Number (32 bits) │ ├───────────────────────────────────────────────────────────────────┤ │ Acknowledgment Number (32 bits) │ ├──────────┬──────┬─┬─┬─┬─┬─┬─┬───────────────────────────────────┤ │ Data Off │Reserv│U│A│P│R│S│F│ Window Size (16b) │ │ (4b) │ (4b) │R│C│S│S│Y│I│ │ │ │ │G│K│H│T│N│N│ │ ├──────────┴──────┴─┴─┴─┴─┴─┴─┼───────────────────────────────────┤ │ Checksum (16b) │ Urgent Pointer (16b) │ ├──────────────────────────────┴───────────────────────────────────┤ │ Options (variable) │ └───────────────────────────────────────────────────────────────────┘

Three-Way Handshake

Before any data flows, TCP establishes a connection using a three-way handshake. This synchronizes sequence numbers and confirms both sides are ready.

Client Server │ │ │──── SYN (seq=x) ───────────────────────────►│ Step 1: Client sends SYN │ │ │◄─── SYN-ACK (seq=y, ack=x+1) ────────────│ Step 2: Server sends SYN-ACK │ │ │──── ACK (ack=y+1) ─────────────────────────►│ Step 3: Client sends ACK │ │ │◄═══════════ CONNECTION ESTABLISHED ══════════►│ │ │ │──── Data (seq=x+1) ──────────────────────────►│ │◄─── ACK (ack=x+1+len) ───────────────────────│

Connection Teardown (Four-Way)

Client Server │ │ │──── FIN ──────────────────────────────────►│ Client done sending │◄─── ACK ──────────────────────────────────────│ Server acknowledges │ │ │◄─── FIN ──────────────────────────────────│ Server done sending │──── ACK ──────────────────────────────────────►│ Client acknowledges │ │ │ (TIME_WAIT: 2 × MSL) │ │ CONNECTION CLOSED │

Flow Control: Sliding Window

The receiver advertises a window size — how much data it can buffer. The sender never sends more than the window allows. This prevents a fast sender from overwhelming a slow receiver.

Sender's view of the byte stream: ┌─────┬─────┬─────┬─────┬─────┬─────┬─────┬─────┬─────┬─────┐ │ Sent │ Sent │ Sent │ Send │ Send │ Send │ Wait │ Wait │ Wait │ Wait │ │ ACKd │ ACKd │ ACKd │ OK │ OK │ OK │ │ │ │ │ └─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┘ ◄─── confirmed ──►◄── window (can send) ──►◄── cannot send ──►

Congestion Control

TCP also limits sending rate to avoid congesting the network itself. The sender maintains a congestion window (cwnd) separate from the receive window.

Slow Start

Start with cwnd = 1 MSS. Double cwnd every RTT (exponential growth). Continue until hitting ssthresh (slow start threshold) or detecting loss.

Congestion Avoidance (AIMD)

After reaching ssthresh: Additive Increase — add 1 MSS per RTT (linear growth). On packet loss: Multiplicative Decrease — halve cwnd. This sawtooth pattern probes for available bandwidth.

Fast Retransmit

If sender receives 3 duplicate ACKs for the same sequence number, retransmit the lost segment immediately without waiting for the retransmission timeout (RTO).

Fast Recovery

After fast retransmit, set ssthresh = cwnd/2, set cwnd = ssthresh + 3 MSS, then enter congestion avoidance. Avoids going back to slow start for a single lost packet.

TCP Congestion Window Over Time cwnd │ /\ (MSS)│ / \ /\ │ / \ / \ /\ 32 │ / \ / \ / \ │ / \ / \ / \ 16 │ / Slow \ / AIMD \ / \ │ / Start X X \ 8 │ / loss loss \ │ / (exponential) (halve cwnd) \ 1 │/────────────────────────────────────── Time └──────────────────────────────────────► ssthresh hit here ──┘
Section 06

UDP (User Datagram Protocol)

UDP is TCP's minimalist sibling. It provides a connectionless, unreliable transport — just source/destination ports and a checksum. No handshakes, no acknowledgments, no ordering, no flow control. This makes it fast and lightweight, ideal for applications that prefer speed over reliability.

UDP Header (8 bytes only!) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ┌───────────────────────────────┬───────────────────────────────────┐ │ Source Port (16b) │ Destination Port (16b) │ ├───────────────────────────────┼───────────────────────────────────┤ │ Length (16b) │ Checksum (16b) │ └───────────────────────────────┴───────────────────────────────────┘ │ Payload │ └───────────────────────────────────────────────────────────────────┘ Compare: TCP header = 20+ bytes, UDP header = 8 bytes

TCP vs UDP

FeatureTCPUDP
ConnectionConnection-orientedConnectionless
ReliabilityGuaranteed delivery (retransmits)Best-effort (fire and forget)
OrderingIn-order deliveryNo ordering guarantee
Flow ControlYes (sliding window)No
Congestion ControlYes (slow start, AIMD)No
Header Size20-60 bytes8 bytes
SpeedSlower (overhead)Faster (minimal overhead)
Use CasesHTTP, SSH, email, file transferDNS, gaming, video, VoIP, QUIC

When to Use UDP

Real-Time Gaming

A late packet is worse than a lost packet. Player positions update 30-60 times/sec — by the time a retransmit arrives, the data is stale.

DNS Queries

Small request/response (usually fits in one packet). Connection setup overhead of TCP would double latency. UDP keeps DNS lookups fast.

Video/Audio Streaming

VoIP, video conferencing. Missing a frame is acceptable — the codec can interpolate. Retransmitting would cause jitter and delay.

QUIC (HTTP/3)

Builds its own reliability on top of UDP. Gets the benefits of connection management + encryption while avoiding TCP's head-of-line blocking at the OS level.

Section 07

DNS (Domain Name System)

DNS is the phone book of the Internet. It translates human-readable domain names (like example.com) into IP addresses (like 93.184.216.34). Without DNS, you'd have to memorize IP addresses for every website. DNS is a hierarchical, distributed database — no single server has all the answers.

DNS Hierarchy

Root (".") / | \ / | \ .com .org .net (TLDs) / \ | / \ | google amazon wikipedia (Second-level) | | | www mail aws s3 en fr (Subdomains) 13 Root Server clusters (a.root-servers.net through m.root-servers.net) Each is anycast — hundreds of physical instances worldwide

DNS Resolution Flow

You type: www.example.com Your Computer Recursive Resolver Authoritative Servers │ (e.g. 8.8.8.8) │ │ │─1─► "www.example.com?" │ │ (check local cache) │ │ │─2─► Root Server "." │ │◄─── "Ask .com TLD" │ │ │ │─3─► .com TLD Server │ │◄─── "Ask example.com NS" │ │ │ │─4─► example.com Authoritative NS │ │◄─── "93.184.216.34 (TTL=3600)" │ │ │◄──5── "93.184.216.34" │ │ (cached for TTL) │ │ │ ├── Connect to 93.184.216.34 ──────────────────────────────►

Recursive vs Iterative Resolution

Recursive

  • Client asks resolver, resolver does all the work
  • Resolver queries root, TLD, authoritative on your behalf
  • Client gets the final answer
  • Used by: your ISP's resolver, 8.8.8.8, 1.1.1.1
  • Resolver caches results for future queries

Iterative

  • Each server returns a referral, not the answer
  • "I don't know, but ask this server"
  • Client/resolver follows the chain itself
  • Used between: recursive resolvers and authoritative servers
  • Root and TLD servers always respond iteratively

DNS Record Types

TypeNamePurposeExample
AAddressMaps domain to IPv4 addressexample.com → 93.184.216.34
AAAAIPv6 AddressMaps domain to IPv6 addressexample.com → 2606:2800:220:1::248
CNAMECanonical NameAlias one domain to anotherwww.example.com → example.com
MXMail ExchangeMail server for the domainexample.com → mail.example.com (pri 10)
TXTTextArbitrary text (SPF, DKIM, verification)"v=spf1 include:_spf.google.com ~all"
NSName ServerDelegates zone to authoritative NSexample.com → ns1.example.com
SOAStart of AuthorityZone metadata (serial, refresh, TTL)Primary NS, admin email, serial number
PTRPointerReverse DNS (IP to domain)34.216.184.93 → example.com
SRVServiceService location (port + host)_sip._tcp.example.com → sip.example.com:5060

TTL and Caching

Every DNS record has a TTL (Time to Live) in seconds. Resolvers cache the record for that duration. Low TTL (60s) means faster propagation of changes but more DNS queries. High TTL (86400s = 1 day) reduces load but changes propagate slowly.

dig example.com
;; QUESTION SECTION: ;example.com. IN A ;; ANSWER SECTION: example.com. 3600 IN A 93.184.216.34 ↑ TTL = 3600 seconds (1 hour) ;; Query time: 24 msec ;; SERVER: 8.8.8.8#53(8.8.8.8)
Section 08

HTTP/1.1, HTTP/2, HTTP/3

HTTP (HyperText Transfer Protocol) is the application layer protocol that powers the web. It has evolved significantly over three major versions, each solving problems introduced by the previous one.

Evolution at a Glance

HTTP/1.0 (1996)
→
HTTP/1.1 (1997)
→
HTTP/2 (2015)
→
HTTP/3 (2022)
FeatureHTTP/1.1HTTP/2HTTP/3
TransportTCPTCPQUIC (UDP)
FormatText-basedBinary framingBinary framing
MultiplexingNo (1 req/conn)Yes (streams)Yes (streams)
Head-of-Line BlockingTCP + HTTPTCP onlyNone
Header CompressionNoneHPACKQPACK
Server PushNoYesYes
Connection SetupTCP + TLS (3 RTTs)TCP + TLS (2-3 RTTs)1 RTT (0-RTT resumption)
EncryptionOptional (HTTPS)Practically requiredBuilt-in (always TLS 1.3)

The Head-of-Line Blocking Problem

HTTP/1.1 — HTTP-level HOL blocking Connection: │ Req1 │ Resp1 │ Req2 │ Resp2 │ Req3 │ Resp3 │ (sequential — each request waits for previous response) Workaround: open 6 parallel TCP connections HTTP/2 — TCP-level HOL blocking TCP Stream: │ Frame A1 │ LOST │ Frame B1 │ Frame C1 │ ... ▲ TCP retransmits, ALL streams blocked Streams A, B, C multiplexed but share one TCP connection HTTP/3 (QUIC) — No HOL blocking Stream A: │ Frame A1 │ Frame A2 │ (independent) Stream B: │ Frame B1 │ LOST │ ... (only B waits) Stream C: │ Frame C1 │ Frame C2 │ (unaffected) Each stream is independent at the transport level

QUIC: The Foundation of HTTP/3

QUIC is a transport protocol built on UDP that provides connection-oriented, encrypted, multiplexed communication. It was developed by Google and standardized as RFC 9000.

0-RTT Connection Resumption

If you've connected to a server before, QUIC can send data in the very first packet using cached session keys. TCP + TLS needs at least 1-3 round trips before any data flows.

Connection Migration

TCP connections are identified by (src IP, src port, dst IP, dst port). Switch from Wi-Fi to cellular? Connection dies. QUIC uses a Connection ID, so it survives network changes.

Built-in Encryption

QUIC integrates TLS 1.3 directly into its handshake. There is no unencrypted QUIC — even the transport headers are partially encrypted, preventing middlebox interference.

Independent Streams

Each QUIC stream has its own flow control and loss recovery. A lost packet on stream A does not block streams B, C, D — solving TCP's fundamental HOL blocking problem.

HTTP/3 Connection Setup (1-RTT, or 0-RTT with resumption) Client Server │ │ │── QUIC Initial (TLS ClientHello) ────►│ Combined transport │ │ + crypto handshake │◄── QUIC Handshake (TLS ServerHello) ──│ │ + Server's TLS certificate │ │ + QUIC transport params │ │ │ │── QUIC 1-RTT (HTTP/3 request) ───────►│ Data flows after │◄── QUIC 1-RTT (HTTP/3 response) ─────│ just 1 round trip! Compare TCP + TLS 1.3: SYN → SYN-ACK → ACK + ClientHello → ServerHello → Data (2 round trips minimum)
Section 09

TLS/SSL

TLS (Transport Layer Security) provides encryption, authentication, and integrity for communications over TCP. It's what makes HTTPS possible. SSL (Secure Sockets Layer) is the deprecated predecessor — modern "SSL" really means TLS.

What TLS Provides

Confidentiality

Data is encrypted so eavesdroppers see gibberish. Uses symmetric encryption (AES-GCM, ChaCha20-Poly1305) for bulk data, negotiated via the handshake.

Authentication

The server proves its identity using a digital certificate signed by a trusted Certificate Authority (CA). Optionally, the client can also authenticate (mutual TLS / mTLS).

Integrity

Each message includes a MAC (Message Authentication Code) that detects any tampering. If a single bit is modified in transit, the recipient knows and rejects the message.

Symmetric vs Asymmetric Encryption

Symmetric (AES, ChaCha20)

  • Same key to encrypt and decrypt
  • Very fast (~10 Gbps)
  • Problem: how to share the key securely?
  • Used for: bulk data encryption
  • Key sizes: 128 or 256 bits

Asymmetric (RSA, ECDHE)

  • Public key encrypts, private key decrypts
  • Slow (~1000x slower than symmetric)
  • Solves key exchange problem
  • Used for: key exchange, signatures
  • Key sizes: 2048+ bits (RSA), 256 bits (ECDH)

TLS uses both: asymmetric crypto to securely exchange a symmetric key, then symmetric crypto for the actual data. This is called a hybrid cryptosystem.

TLS 1.3 Handshake (Full)

Client Server │ │ │─── ClientHello ─────────────────────────────────►│ │ • Supported TLS versions │ │ • Supported cipher suites │ │ • Client random │ │ • Key share (ECDHE public key) │ │ • Supported groups (curves) │ │ │ │◄── ServerHello ─────────────────────────────────│ │ • Chosen TLS version (1.3) │ │ • Chosen cipher suite │ │ • Server random │ │ • Key share (ECDHE public key) │ │ │ │ ═══ Handshake keys derived (ECDHE) ═══ │ │ (all following messages encrypted) │ │ │ │◄── EncryptedExtensions ────────────────────────│ │◄── Certificate ───────────────────────────────│ │ • Server's X.509 certificate chain │ │◄── CertificateVerify ─────────────────────────│ │ • Signature proving server owns the cert │ │◄── Finished ──────────────────────────────────│ │ • MAC over entire handshake │ │ │ │ Client verifies certificate chain + signature │ │ │ │─── Finished ──────────────────────────────────►│ │ • MAC over entire handshake │ │ │ │ ═══ Application keys derived ═══ │ │ │ │◄══════════ ENCRYPTED APPLICATION DATA ═══════════►│ TLS 1.3: 1 round trip. TLS 1.2: 2 round trips. 0-RTT resumption also possible.

Certificate Authority Chain of Trust

Root CA (self-signed, pre-installed in your browser/OS) │ signs ▼ Intermediate CA (signed by Root CA) │ signs ▼ Server Certificate (signed by Intermediate CA) │ └── Contains: domain name, public key, validity dates, issuer Verification chain: Browser receives: [Server Cert] → [Intermediate CA] → [Root CA] Browser checks: 1. Is Root CA in my trusted store? ✓ 2. Did Root CA sign Intermediate? ✓ (verify signature) 3. Did Intermediate sign Server cert? ✓ (verify signature) 4. Is Server cert for this domain? ✓ 5. Is it expired? No ✓ 6. Is it revoked? (check CRL/OCSP) No ✓

TLS 1.3 vs 1.2 Improvements

FeatureTLS 1.2TLS 1.3
Handshake RTTs2 round trips1 round trip (0-RTT resumption)
Key ExchangeRSA or ECDHEECDHE only (forward secrecy mandatory)
Cipher Suites37 suites (many weak)5 suites (all strong)
Encryption AfterAfter 2nd round tripAfter 1st round trip
Removed—RSA key exchange, RC4, DES, SHA-1, compression, renegotiation
Handshake Encrypted?Partially (certs in clear)Yes (after ServerHello)

Common Cipher Suites (TLS 1.3)

TLS 1.3 Cipher Suites
TLS_AES_256_GCM_SHA384 // AES-256 encryption, GCM mode, SHA-384 for HKDF TLS_AES_128_GCM_SHA256 // AES-128 encryption, GCM mode, SHA-256 for HKDF TLS_CHACHA20_POLY1305_SHA256 // ChaCha20 encryption, Poly1305 MAC (fast on mobile) // Key exchange is always ECDHE (not part of cipher suite name in 1.3) // Common curves: X25519, P-256, P-384
Section 10

HTTPS

HTTPS is simply HTTP over TLS. It wraps every HTTP request and response in a TLS encrypted tunnel. The URL scheme changes from http:// to https://, and the default port changes from 80 to 443.

What HTTPS Protects

Protected

Request/response body (form data, JSON, HTML), headers (cookies, auth tokens), URL path and query string (encrypted after TLS handshake), HTTP method.

NOT Protected

Domain name (visible via DNS query + TLS SNI), IP addresses (visible in packet headers), connection metadata (timing, packet sizes), that you're communicating at all.

Man-in-the-Middle (MITM) Attack

Without HTTPS: Client ──── HTTP (plaintext) ────► Attacker ────► Server reads/modifies everything With HTTPS: Client ──── TLS (encrypted) ─────► Attacker ────► Server sees encrypted gibberish only (can't forge cert) HTTPS MITM (compromised CA / installed root cert): Client ── TLS ──► Attacker ── TLS ──► Server (has valid cert (separate connection) from compromised CA or corporate proxy)

Certificate Pinning

Certificate pinning defends against compromised CAs by hardcoding the expected certificate (or its public key hash) in the client application. If the presented cert doesn't match the pin, the connection is rejected — even if a CA signed it. Used in mobile apps and high-security scenarios.

🔒

HSTS (HTTP Strict Transport Security)

A server can send Strict-Transport-Security: max-age=31536000; includeSubDomains to tell browsers: never connect to this domain over plain HTTP again. Prevents SSL stripping attacks where an attacker downgrades HTTPS to HTTP.

Section 11

Routing

Routing is how packets find their way from source to destination across the Internet — traversing dozens of routers across multiple networks. Each router makes a local forwarding decision based on the destination IP address and its routing table.

How Routing Works

  • Packet arrives at router with destination IP 151.101.1.140
  • Router looks up destination in its routing table (longest prefix match)
  • Routing table says "151.101.0.0/16 → forward to next-hop 10.0.0.1 via interface eth0"
  • Router decrements TTL (Time to Live), recalculates checksum, forwards packet
  • Next router repeats the process until the packet reaches the destination network
  • Autonomous Systems (AS) and BGP

    The Internet is divided into Autonomous Systems (ASes) — networks under a single administrative entity (an ISP, a company, a university). Each AS has a unique ASN (Autonomous System Number). BGP (Border Gateway Protocol) is how ASes exchange routing information with each other.

    ┌──────────────┐ ┌──────────────┐ │ AS 15169 │ │ AS 13335 │ │ (Google) │◄─BGP──►│ (Cloudflare) │ └──────┬───────┘ └──────┬───────┘ │ │ BGP│ BGP│ │ │ ┌──────┴───────┐ ┌──────┴───────┐ │ AS 3356 │◄─BGP──►│ AS 6939 │ │ (Lumen/L3) │ │ (Hurricane) │ └──────┬───────┘ └──────┬───────┘ │ │ BGP│ BGP│ │ │ ┌──────┴───────────────────────┴───────┐ │ Your ISP (AS xxxxx) │ │ Advertises your IP prefixes │ └──────────────────────────────────────┘ BGP is a path-vector protocol. Each AS announces which IP prefixes it can reach and through which AS path. Routers select the shortest AS path (plus policy preferences).

    Traceroute Explained

    traceroute works by sending packets with incrementing TTL values. Each router that decrements TTL to 0 sends back an ICMP "Time Exceeded" message, revealing its IP address.

    traceroute to example.com (93.184.216.34)
    1 192.168.1.1 1.2ms ← Your router (TTL=1, router responds) 2 10.0.0.1 5.1ms ← ISP's first router (TTL=2) 3 72.14.215.85 12.3ms ← ISP backbone 4 108.170.248.33 14.7ms ← Peering point 5 142.251.49.24 18.2ms ← Transit network 6 * * * ← Router doesn't respond (filtered) 7 152.195.76.141 24.1ms ← Getting closer 8 93.184.216.34 26.5ms ← Destination reached!
    How traceroute discovers the path: Packet with TTL=1: Client ──► Router 1 (TTL→0, sends ICMP back) Packet with TTL=2: Client ──► Router 1 ──► Router 2 (TTL→0) Packet with TTL=3: Client ──► Router 1 ──► Router 2 ──► Router 3 (TTL→0) ... Packet with TTL=N: Client ──► ... ──► Destination (port unreachable / echo reply)
    Section 12

    Load Balancing

    Load balancers distribute incoming traffic across multiple backend servers to improve availability, reliability, and performance. They sit between clients and servers, acting as a reverse proxy.

    L4 vs L7 Load Balancing

    Layer 4 (Transport)

    • Routes based on IP + port
    • Doesn't inspect packet contents
    • Very fast (hardware/kernel-level)
    • Works with any protocol (TCP, UDP)
    • No URL-based routing possible
    • Examples: AWS NLB, IPVS, LVS

    Layer 7 (Application)

    • Routes based on HTTP headers, URL, cookies
    • Can terminate TLS, add headers, rewrite URLs
    • Slower (must parse HTTP)
    • URL path routing: /api → backend A, /static → CDN
    • Can do content-based health checks
    • Examples: AWS ALB, NGINX, HAProxy, Envoy

    Load Balancing Algorithms

    AlgorithmHow It WorksBest For
    Round Robin Send to each server in turn: A, B, C, A, B, C... Equal-capacity servers, uniform requests
    Weighted Round Robin Like round robin but servers with higher weight get more traffic Mixed-capacity servers
    Least Connections Send to the server with fewest active connections Varying request durations
    IP Hash Hash client IP to pick server (same client → same server) Session affinity without cookies
    Consistent Hashing Hash ring: minimal redistribution when servers added/removed Caching layers, distributed systems
    Random Pick a server at random Simple, stateless, surprisingly effective
    Least Response Time Send to server with fastest recent response + fewest connections Latency-sensitive applications

    Health Checks

    Load Balancer periodically checks each backend: LB ─── GET /health ──► Server A → 200 OK ✓ healthy LB ─── GET /health ──► Server B → 200 OK ✓ healthy LB ─── GET /health ──► Server C → timeout ✗ unhealthy LB ─── GET /health ──► Server D → 200 OK ✓ healthy Server C removed from rotation after N consecutive failures. Re-added after M consecutive successes. Types: • TCP check: Can we connect to port 80? (L4) • HTTP check: Does GET /health return 2xx? (L7) • Custom: Does the app return {"status":"ok"}? (Deep)
    Section 13

    CDNs (Content Delivery Networks)

    A CDN is a globally distributed network of edge servers that cache content close to users. Instead of every request traveling to the origin server (possibly across the world), it is served from the nearest edge location. This reduces latency, offloads the origin, and improves reliability.

    How CDNs Work

    Without CDN: User (Tokyo) ─── 15,000 km ───► Origin (New York) Latency: ~200ms RTT With CDN: User (Tokyo) ─── 50 km ───► CDN Edge (Tokyo) Latency: ~5ms RTT If edge has cached content → serve immediately (cache HIT) If edge doesn't have it → fetch from origin, cache, then serve (cache MISS) ┌─────────────────┐ │ Origin Server │ │ (New York) │ └────────┬────────┘ │ ┌────────────────┼────────────────┐ │ │ │ ┌──────┴──────┐ ┌─────┴──────┐ ┌──────┴──────┐ │ Edge Tokyo │ │ Edge London │ │ Edge São P. │ └──────┬──────┘ └─────┬──────┘ └──────┬──────┘ │ │ │ Users in Asia Users in EU Users in S. America

    Anycast

    Anycast is how CDNs route users to the nearest edge. Multiple servers in different locations all advertise the same IP address. BGP routing naturally sends packets to the closest one (in terms of network hops).

    Anycast: Same IP, multiple locations DNS: cdn.example.com → 104.16.132.229 (single IP) But that IP exists in all edge locations: Tokyo POP: 104.16.132.229 ← BGP advertises from AS in Tokyo London POP: 104.16.132.229 ← BGP advertises from AS in London New York POP: 104.16.132.229 ← BGP advertises from AS in New York User in Tokyo → BGP routes to Tokyo POP (shortest AS path) User in Paris → BGP routes to London POP (shortest AS path)

    Cache Invalidation

    TTL-Based Expiry

    Origin sets Cache-Control: max-age=3600. Edge serves cached content for 1 hour, then revalidates with origin. Simple but delayed updates.

    Purge / Invalidation API

    Origin tells CDN "invalidate /images/logo.png across all edges." Immediate but requires API integration. Cloudflare, Fastly, Akamai all offer purge APIs.

    Cache Tags / Surrogate Keys

    Tag cached objects (e.g., "product-123"). When the product changes, purge all content tagged "product-123" across all edges. Fine-grained invalidation.

    Versioned URLs

    Use /style.v2.css or /style.css?v=abc123. New version = new URL = guaranteed cache miss. Never need to invalidate — the old URL is simply never requested again.

    Section 14

    What Happens When You Type a URL

    The complete journey from pressing Enter on https://www.example.com/page to seeing a rendered page involves dozens of steps across every layer of the networking stack.

    URL Parse
    →
    DNS Lookup
    →
    TCP Connect
    →
    TLS Handshake
    →
    HTTP Request
    →
    Server Processing
    →
    HTTP Response
    →
    Rendering

    The Full Journey

  • URL Parsing: Browser parses the URL into scheme (https), host (www.example.com), port (443), and path (/page). Checks HSTS list — if the domain is listed, force HTTPS.
  • DNS Resolution: Browser checks its DNS cache, then OS cache, then /etc/hosts, then sends a query to the configured DNS resolver (e.g., 8.8.8.8). The resolver queries root → TLD → authoritative servers and returns the IP (e.g., 93.184.216.34).
  • TCP Three-Way Handshake: Browser initiates SYN → SYN-ACK → ACK with the server's IP on port 443. This takes 1 RTT (~20-200ms depending on distance). The OS picks an ephemeral source port (e.g., 52431).
  • TLS 1.3 Handshake: Browser sends ClientHello with supported ciphers and ECDHE key share. Server responds with ServerHello, certificate, and its key share. Keys are derived, and the connection is encrypted. 1 additional RTT.
  • HTTP Request: Browser sends GET /page HTTP/2 with headers: Host, User-Agent, Accept, Accept-Encoding (gzip, br), Cookie, etc. With HTTP/2, this uses binary framing and header compression (HPACK).
  • Server Processing: The request hits a load balancer, then an application server. The server may query databases, call microservices, compile a template, and generate an HTML response. CDN may serve cached static assets.
  • HTTP Response: Server sends back 200 OK with headers (Content-Type, Cache-Control, Set-Cookie) and the HTML body, compressed with Brotli or gzip. Response streams back in TCP segments.
  • HTML Parsing & Rendering: Browser parses HTML into a DOM tree. Encounters CSS → fetches and builds CSSOM. Encounters JS → fetches and executes (may block parsing). Combines DOM + CSSOM into render tree. Layout (geometry) → Paint (pixels) → Composite (GPU layers) → Display.
  • Timeline: What Happens When (typical) 0ms URL parsed, HSTS check 1ms DNS cache check (miss) 1ms DNS query sent to resolver 25ms DNS response received (IP address) 25ms TCP SYN sent 50ms TCP SYN-ACK received, ACK sent 50ms TLS ClientHello sent 75ms TLS ServerHello received, keys derived 75ms HTTP GET /page sent (encrypted) 76ms Request travels to server 100ms Server processes request 150ms First byte of response received (TTFB = 150ms) 155ms HTML parsing begins 180ms Critical CSS fetched and parsed 200ms First paint (FCP = 200ms) 350ms JavaScript loaded and executed 400ms Largest content rendered (LCP = 400ms) 500ms Page fully interactive (TTI = 500ms)
    Section 15

    WebSockets

    WebSockets provide full-duplex, bidirectional communication over a single TCP connection. Unlike HTTP's request-response model, either side can send messages at any time without the other having to ask. This makes WebSockets ideal for real-time applications.

    The Upgrade Handshake

    A WebSocket connection starts as a regular HTTP request with an Upgrade header. The server agrees by responding with 101 Switching Protocols, and from that point forward the connection is a WebSocket.

    WebSocket Upgrade Request
    GET /chat HTTP/1.1 Host: example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Version: 13
    WebSocket Upgrade Response
    HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    Client Server │ │ │─── HTTP GET /chat (Upgrade: websocket) ──────►│ │◄── HTTP 101 Switching Protocols ──────────────│ │ │ │◄══════ Full-duplex WebSocket connection ═════►│ │ │ │─── Message: "hello" ─────────────────────────►│ │◄── Message: "world" ─────────────────────────│ │◄── Message: "new data available" ────────────│ (server-initiated!) │─── Message: "subscribe: stocks" ─────────────►│ │◄── Message: "AAPL: 185.23" ──────────────────│ │◄── Message: "AAPL: 185.45" ──────────────────│ │ │ │─── Close frame ──────────────────────────────►│ │◄── Close frame ──────────────────────────────│

    WebSockets vs Alternatives

    TechniqueDirectionConnectionLatencyUse Case
    Short Polling Client → Server New HTTP request every N seconds High (up to N seconds delay) Simple, infrequent updates
    Long Polling Client → Server HTTP request held open until data available Medium Near-real-time, broad compatibility
    SSE (Server-Sent Events) Server → Client only Single HTTP connection, text/event-stream Low Live feeds, notifications (one-way)
    WebSocket Bidirectional Single TCP connection, full-duplex Lowest Chat, gaming, collaborative editing
    JavaScript WebSocket Example
    const ws = new WebSocket('wss://example.com/chat'); ws.onopen = () => { console.log('Connected!'); ws.send('Hello server'); }; ws.onmessage = (event) => { console.log('Received:', event.data); }; ws.onclose = (event) => { console.log(`Closed: ${event.code} ${event.reason}`); }; ws.onerror = (error) => { console.error('WebSocket error:', error); };
    Section 16

    gRPC & Protocol Buffers

    gRPC is a high-performance RPC (Remote Procedure Call) framework built by Google. It uses Protocol Buffers (protobuf) for serialization and runs on top of HTTP/2. It's widely used for microservice communication where performance matters.

    Protocol Buffers vs JSON

    JSON (Text-Based)

    • Human-readable text
    • Self-describing (keys in every message)
    • Larger payload size
    • Slower to parse
    • Schema optional (often absent)
    • Universal browser support

    Protobuf (Binary)

    • Binary, not human-readable
    • Schema required (.proto files)
    • 3-10x smaller than JSON
    • Up to 100x faster parsing
    • Strong typing, code generation
    • Backward/forward compatible
    user.proto — Protocol Buffer Schema
    syntax = "proto3"; service UserService { rpc GetUser(GetUserRequest) returns (User); rpc ListUsers(ListUsersRequest) returns (stream User); // server streaming rpc Chat(stream Message) returns (stream Message); // bidirectional streaming } message GetUserRequest { int32 id = 1; } message User { int32 id = 1; string name = 2; string email = 3; int32 age = 4; }

    gRPC Communication Patterns

    PatternClientServerUse Case
    Unary 1 request 1 response Standard request/response (like REST)
    Server Streaming 1 request Stream of responses Live feeds, large result sets
    Client Streaming Stream of requests 1 response File upload, sensor data
    Bidirectional Streaming Stream Stream Chat, real-time collaboration

    gRPC Over HTTP/2

    gRPC uses HTTP/2 features under the hood: HTTP/2 Stream 1: ──► GetUser(id=42) ◄── User{name:"Alice"} HTTP/2 Stream 3: ──► ListUsers() ◄── User{...} User{...} User{...} HTTP/2 Stream 5: ──► (multiplexed on same TCP connection) Wire format: POST /UserService/GetUser HTTP/2 Content-Type: application/grpc Body: [compressed-flag (1B)] [message-length (4B)] [protobuf-bytes] Key advantage: multiple RPCs share one TCP connection via HTTP/2 multiplexing
    ⚙

    When to Use gRPC vs REST

    gRPC: microservice-to-microservice, low latency requirements, streaming data, polyglot services (code generation for 10+ languages). REST: browser clients (gRPC-Web exists but is limited), public APIs, when human readability matters, simpler tooling requirements.

    Section 17

    Internet Backbone

    The physical structure of the Internet is a hierarchy of interconnected networks, from the massive global backbone carriers down to your local ISP. Understanding this hierarchy explains why some paths are faster than others and how the Internet stays connected.

    ISP Tiers

    ┌─────────────────────────────────┐ │ TIER 1 ISPs │ │ (Global backbone, peer freely) │ │ Lumen, NTT, Telia, GTT, Zayo │ │ ◄═══ settlement-free peering ══►│ └───────────────┬─────────────────┘ │ (transit: Tier 2 pays Tier 1) ┌───────────────┴─────────────────┐ │ TIER 2 ISPs │ │ (Regional, peer + buy transit) │ │ Cogent, Comcast, Deutsche Tel. │ │ ◄═══ peer with each other ═════►│ └───────────────┬─────────────────┘ │ (transit: Tier 3 pays Tier 2) ┌───────────────┴─────────────────┐ │ TIER 3 ISPs │ │ (Local, buy transit from above) │ │ Your local ISP, small regional │ │ carriers, municipal networks │ └───────────────┬─────────────────┘ │ End Users
    TierDefinitionRevenue ModelExamples
    Tier 1 Can reach entire Internet through settlement-free peering (no payment to any other network) Sells transit to Tier 2/3 Lumen (Level 3), NTT, Telia, GTT, Zayo
    Tier 2 Peers with some networks, buys transit from Tier 1 for the rest Sells transit to Tier 3, sells to consumers Cogent, Comcast, Orange, Deutsche Telekom
    Tier 3 Buys transit from Tier 2 or Tier 1 for all connectivity Sells Internet to end users Local/regional ISPs, municipal networks

    Peering & Transit

    Peering

    Two networks agree to exchange traffic directly with each other, usually for free (settlement-free). Benefits both: shorter paths, lower latency, no transit costs. Can be private (direct fiber) or public (at an IXP).

    Transit

    A network pays another (larger) network to carry its traffic to the rest of the Internet. The transit provider announces the customer's IP prefixes via BGP, providing global reachability.

    Internet Exchange Points (IXPs)

    An IXP is a physical building where hundreds of networks plug into a shared switch fabric and peer with each other. This keeps local traffic local — instead of packets from ISP-A to ISP-B in the same city going through a Tier 1 backbone across the country and back.

    Inside an IXP (e.g., DE-CIX Frankfurt, ~1000 members): ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Google │ │ Netflix │ │ Amazon │ │ ISP-A │ │ AS15169 │ │ AS2906 │ │ AS16509 │ │ AS1234 │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ │ ═════╪═════════════╪═════════════╪═════════════╪═══════ IXP Switch Fabric ═════╪═════════════╪═════════════╪═════════════╪═══════ │ │ │ │ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ │ ISP-B │ │ ISP-C │ │ CDN-X │ │ ISP-D │ │ AS5678 │ │ AS9012 │ │ AS3456 │ │ AS7890 │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ Major IXPs: DE-CIX (Frankfurt) — 1000+ members, 14+ Tbps peak AMS-IX (Amsterdam) — 900+ members, 12+ Tbps peak LINX (London) — 950+ members, 8+ Tbps peak Equinix IX (global) — multiple locations

    Physical Internet Structure

    The physical Internet, simplified: Submarine Cables Terrestrial Fiber Last Mile North America ═══════ Europe Backbone ─── Metro ── Fiber/Cable/DSL ║ ║ fiber rings fiber to your home ║ ║ │ South America Asia/Pacific Data Centers ║ ║ (where servers live) ╚════ Africa ══════╝ ~550+ submarine cables crisscross the oceans Landing stations connect submarine to terrestrial fiber Each cable has multiple fiber pairs (typically 8-24) DWDM multiplexes 100+ wavelengths per fiber = massive bandwidth Single points of failure: • Submarine cable cuts (anchors, earthquakes, sharks) • Landing stations (few per region) • IXP outages (rare but impactful) • BGP misconfigurations (route leaks, hijacks)
    🌎

    The Internet Is Surprisingly Physical

    Despite being "the cloud," the Internet is made of real things: fiber optic cables running along railroad tracks and ocean floors, routers in climate-controlled buildings, and servers in massive data centers consuming megawatts of power. A ship dragging its anchor across a submarine cable has caused nationwide outages. The Internet's resilience comes from redundancy, not invulnerability.