Load Balancing & Reverse Proxies

Load Balancing Strategies

When a single server can't handle millions of requests, we distribute traffic across a fleet. A Load Balancer sits between clients and backend servers, routing requests based on algorithms.

Layer 4 vs Layer 7

Layer 4 (Transport) load balancers operate on TCP/UDP. They're extremely fast because they don't inspect HTTP headers or payloads — they just forward raw packets. Layer 7 (Application) balancers inspect HTTP requests and can route based on URL paths, cookies, or headers.

# NGINX L7 Load Balancer Configuration
upstream backend {
    # Weighted round-robin
    server api-1.internal:8080 weight=5;
    server api-2.internal:8080 weight=3;
    server api-3.internal:8080 weight=2;

    # Health checks
    server api-4.internal:8080 backup;
}

server {
    listen 443 ssl;
    location /api/ {
        proxy_pass http://backend;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        
        # Connection pooling
        keepalive 32;
    }
}

Consistent Hashing

Simple round-robin fails when you need session affinity or cache locality. Consistent hashing maps both servers and requests onto a virtual ring. When a server is added or removed, only 1/N of requests are remapped.

import hashlib

class ConsistentHash:
    def __init__(self, nodes, replicas=150):
        self.ring = {}
        self.sorted_keys = []
        for node in nodes:
            for i in range(replicas):
                key = self._hash(f"{node}:{i}")
                self.ring[key] = node
                self.sorted_keys.append(key)
        self.sorted_keys.sort()

    def _hash(self, key):
        return int(hashlib.md5(key.encode()).hexdigest(), 16)

    def get_node(self, item):
        h = self._hash(item)
        for key in self.sorted_keys:
            if h <= key:
                return self.ring[key]
        return self.ring[self.sorted_keys[0]]