📘 Application Protocols and Infrastructure

This page covers the protocols applications build on top of TCP and UDP, and the boxes that sit between a client and a server. For architecture-level choices see Building Blocks; this page stays close to the wire.

Table of Contents

1. Real-Time Options Compared
2. WebSockets
3. Server-Sent Events
4. WebRTC
5. gRPC and RPC Protocols
6. Other Everyday Protocols
7. Proxies
8. Load Balancers
9. CDNs
10. API Gateways and Service Meshes
11. Connection Management
12. Questions

1. Real-Time Options Compared

2. WebSockets

A WebSocket starts as an HTTP request and upgrades to a raw bidirectional message channel.

GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

The Accept value is a hash of the key plus a fixed GUID, proving the server really speaks WebSocket. After that, both sides exchange frames (text, binary, ping, pong, close). Client-to-server frames are masked to protect against cache poisoning in intermediaries.

Operating WebSockets at scale:

3. Server-Sent Events

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache

id: 41
event: price
data: {"symbol":"ACME","price":101.2}

id: 42
data: plain message

• Browser API: new EventSource("/stream").
• Automatic reconnect with the Last-Event-ID header, so the server can resume.
• Works through most proxies because it is plain HTTP, and over HTTP/2 many streams share one connection.
• Under HTTP/1.1 browsers limit each origin to 6 connections, which SSE tabs can exhaust.
• Proxies that buffer responses (for example nginx with proxy_buffering on) break streaming; disable buffering for these routes.

SSE is the standard way LLM APIs stream tokens.

4. WebRTC

WebRTC gives browsers direct peer-to-peer audio, video, and data channels.

Peer to peer works for two people; group calls use an SFU because full mesh upload bandwidth grows with every participant.

5. gRPC and RPC Protocols

gRPC is an RPC framework using Protocol Buffers over HTTP/2.

Load balancing gotcha: gRPC multiplexes all calls over one long-lived HTTP/2 connection, so an L4 load balancer pins every call from a client to one backend. Use L7 (per-request) balancing: Envoy, a service mesh, or client-side balancing with a lookaside resolver.

6. Other Everyday Protocols

Email authentication in one line each: SPF lists which servers may send for a domain, DKIM signs messages with a key published in DNS, and DMARC tells receivers what to do when both fail and where to send reports. Gmail and Yahoo require all three for bulk senders since 2024.

7. Proxies

A transparent proxy intercepts traffic without client configuration. Proxies pass the original client IP in X-Forwarded-For (or Forwarded) headers, or at L4 with the PROXY protocol. Only trust these headers when they come from your own proxy, otherwise clients can spoof their IP.

8. Load Balancers

Algorithms

Health checks and failure handling

• Active checks probe /healthz; passive checks (outlier detection) eject a backend after errors in real traffic.
• Connection draining stops new requests to a backend being removed but lets in-flight ones finish.
• Sticky sessions keep a client on one backend using a cookie; they hurt balance and failover, so prefer stateless backends.
• Direct server return: the LB handles inbound packets only and backends reply straight to the client, useful when responses are much larger than requests.
• High availability for the LB itself: a pair with a floating virtual IP (VRRP / keepalived), ECMP across many LB nodes, or anycast.

9. CDNs

A CDN is a global network of edge servers (points of presence) that serve content close to users.

How users reach the nearest edge: anycast IPs (Cloudflare) or DNS-based steering (Akamai, CloudFront).

Caching controls:

Invalidation strategies: versioned URLs (best), purge by URL or by tag (surrogate keys), or short TTLs. Origin shield: one mid-tier cache in front of the origin so a cache miss storm across hundreds of edges becomes one origin request.

Classic bug: caching a personalized response publicly because Cache-Control was missing or a Set-Cookie was cached, leaking one user's page to others.

10. API Gateways and Service Meshes

Meshes deploy a proxy next to each service (the sidecar, usually Envoy) or, increasingly, a per-node proxy plus lightweight per-service components (Istio ambient mode, Cilium with eBPF) to cut overhead.

11. Connection Management

Connections are expensive: TCP handshake, TLS handshake, slow start, kernel memory. Most network performance work is really about reusing them.

12. Questions

Next: Network Security.