Chapter 42: The Internet and Data Centers

Era span: 1969 ARPANET → cloud era · Difficulty: extreme
Requires: Ch 26, Ch 36, Ch 41
Unlocks: Ch 43, Ch 46
Data snapshot: volatile figures in this chapter (prices, capacities, deployment counts, regulation, and capability claims) reflect published sources through 2024 unless dated otherwise; check current data before planning.

The internet is not a thing but an AGREEMENT: protocols everyone adopts so any network can talk to any network. Its genius is architectural humility — a dumb, reliable packet core with intelligence pushed to the edges. This chapter builds that agreement and the physical plant underneath it.

42.1 Packet Switching Insight

Circuit switching (telephony) reserves paths even during silence; Baran/Licklider-era insight: chop messages into PACKETS, each finding its way independently, links shared statistically. Survivability (no single node kills the net) and efficiency (multiplexing bursty traffic) fall out of the same design. ARPANET demonstrated it across four nodes (1969); email arrived 1971 and within two years dominated traffic (people, it turns out, want to talk).

42.2 TCP/IP and Layering

January 1, 1983: ARPANET flag-day to TCP/IP. The stack:

End-to-end principle: keep core functions simple where possible and place application intelligence at endpoints. This enables new applications over older networks, but it does not confine all control or censorship to edges: routes, resolvers, transit, platforms, endpoints, and local law all shape what users can do. Ethernet (Metcalfe, 1973) won local networking on cost; switches later replaced shared-medium collision domains.

TCP in four mechanisms: sequence numbers (order) → acknowledgments (receipt) → retransmit on loss → window (flow control) + slow-start (congestion politeness). UDP skips all four for speed (voice, video, games — loss tolerated, delay not).

42.3 Naming and Trust

• DNS (1983): hierarchical names mapping to addresses — human-readable, globally delegated, cached aggressively. DNS is critical infrastructure disguised as phone book.
• Public-key cryptography (Diffie-Hellman 1976, RSA 1977): two parties derive shared secrets over open channels; sign documents without sharing signing keys. TLS wraps connections in authenticated encryption; certificate authorities anchor identity chains (their compromises are systemic risks — pinning/transparency logs mitigate).
• Threat arc: Morris worm (1988, first internet-scale wake-up) → botnets/ransomware era. Defense doctrine: patch velocity, least privilege, defense-in-depth, assume-breach monitoring. Security is process, not product (Ch 47).

DNS: root → TLD → domain → host (cache at every level; TTL balances fresh vs fast)
TLS: hello → certificate (CA-signed) → key exchange (DH/RSA) → encrypted session
THREAT: worm → botnet → ransomware — patch fast, privilege least, monitor always

DNS outage drill: cache + secondary resolvers + anycast + TTL hygiene (lower before migrations). When names fail, numbers still route — keep critical IPs documented offline.

42.4 The Web Layer

Berners-Lee (1989–91): URL + HTTP + HTML — hypertext with global addressing, deliberately royalty-free. Mosaic (1993) made it visual; browsers became universal clients. Then: search (PageRank mapped academic citation logic onto links — authority as algorithm), platforms/recommendation engines (engagement economics — note honestly: attention markets have pathologies regulation is still chasing), and API ecosystems where services compose services.

42.5 Data Centers

The internet's body is warehouses of computation:

• Scale anatomy: megawatt campuses; racks at 10–100+ kW; leaf-spine network fabrics; storage tiers (NVMe → disk → tape archive).
• Efficiency metric PUE (total data-centre power ÷ IT power): older facilities often approach or exceed 2, while the best hyperscale sites can approach 1.1. Climate, cooling design, redundancy, IT load, and measurement boundary determine the result.
• Virtualization → cloud (AWS 2006): hardware abstracted into rentable primitives (compute instances, object storage, managed databases). CapEx became OpEx; startups scaled without buying data centers; utilization economics favored giants who could fill machines.
• CDNs: cache content at edges near users — latency and backbone load both drop; the network effectively grows content outward.
• Reliability engineering: redundancy (N+1 power/cooling), availability zones, chaos testing, SRE error budgets (Ch 47's statistical governance applied to uptime).

42.6 Governance and Doctrine

• IETF culture: open standards via "rough consensus and running code" — RFCs anyone can read/implement. Protocol neutrality kept the network generative.
• Net-neutrality debates, jurisdiction conflicts, platform-governance arguments remain live politics; the ENGINEERING stance stays constant: interoperability, openness, graceful degradation.
• Capacity planning lesson from history's fiber glut: build ahead of demand when marginal cost is low — bandwidth follows demand with lag, and lag costs decades (Ch 41).

SRE error budget (import from Ch 47): 99.9 % uptime = 43 min/month of allowed breakage; spend it on releases, freeze on exhaustion. Chaos-test in daylight with rollback ready — failures rehearsed hurt less than failures surprised.

42.7 Why This Is the Book's Keystone

Earlier information systems depended on physical archives vulnerable to fire, neglect, distance, and selective access. Networks can index, copy, distribute, and update information at global scale, accelerating research, commerce, education, and public coordination while creating new failures in concentration, surveillance, misinformation, and infrastructure dependence. The capability is synchronisation—provided access, governance, and resilience keep it broadly useful.

Capability gate: report availability, adoption, affordability, device access, latency, reliability, safety, and meaningful participation. A majority-subscription threshold alone hides exclusion and does not define information resilience.

42.8 The Internet Papers

• Lineage documents stack up neatly: Licklider's "Intergalactic Computer Network" memo (1962); Baran's RAND On Distributed Communications (1964, eleven volumes); Donald Davies independently coined "packet" at NPL (1965–66). Three origins, one idea — simultaneous invention again (Ch 35's pattern).
• The first ARPANET message (October 29, 1969) crashed after two characters: UCLA's system sent "LO" of "LOGIN" before SRI's host fell. The network's first word was an abbreviation nobody chose.
• RFC culture started casually: Steve Crocker drafted RFC 1 (April 1969) as graduate students documenting protocols without authority — the "Requests for Comments" ethos IS the internet's constitution, and it began as deliberately tentative notes from people with no formal authority.
• Tomlinson chose "@" for email (1971) because it occupied unused Model 33 Teletype keycap real estate and read naturally ("user AT host") — interface archaeology deciding syntax forever.
• Cerf & Kahn's May 1974 paper specified TCP; the January 1, 1983 flag-day cutover happened because coexistence stopped scaling — protocol migration is possible, and this is the canonical proof.
• Berners-Lee's manager Mike Sendall annotated the 1989 proposal "Vague but exciting…" (margin note preserved) and approved it; CERN released the web royalty-free on April 30, 1993 — a licensing decision, more than any code, explains why the web won where proprietary hypertext died.
• Morris worm (November 2, 1988) infected roughly a tenth of the connected internet and produced the first CFAA conviction; DARPA founded the CERT Coordination Center within weeks in response — security institutions form around incidents.
• Netscape's August 9, 1995 IPO (priced $28, closed $58.25) opened the browser era's capital flood; NASDAQ peaked March 10, 2000 (5,048) and the crash burned capital that had, incidentally, laid dark fiber and built data centers — the bust subsidized the broadband decade (Ch 41's overbuild lesson restated financially).

42.9 Build Order (Greenfield Network)

Fiber backbone with spare ducts (Ch 41) → exchange + DNS + anycast resolvers → data hall (N+1 power/cool) → core services (mail, web, backups) → CDN/cache edges → monitoring + incident rota → public access points. Log everything, publish status, drill outages — the network that admits faults keeps users; the one that hides them loses them.

Before the backbone exists: store-and-forward networks deliver useful messaging on very little infrastructure. UUCP (from the late 1970s) and FidoNet (from 1984) relayed mail, news, and files between computers over intermittent dial-up calls; packet radio does the same over shared radio channels (Ch 34). Design early applications to tolerate hours of delay and broken links — queue, retry, and confirm — and the same software keeps working when fibre arrives.