Fundamentals and Core Concepts

Implementation Example: HAProxy Weighted Least Connections

backend servers
    balance leastconn
    server server1 192.168.1.10:80 weight 5 check
    server server2 192.168.1.11:80 weight 3 check
    server server3 192.168.1.12:80 weight 2 check

When to Use Weighted Least Connections

• In heterogeneous environments with varying server capacities
• For workloads with varying connection durations
• When servers process requests at different rates

Limitations

• Still relies on static weights that require manual adjustment
• Connection count is an imperfect proxy for server load

5. Least Response Time

The Least Response Time algorithm routes requests to the server with the lowest average response time and fewest active connections.

Implementation Example: NGINX Plus Least Time

http {
    upstream backend {
        least_time header;  # Use response time for routing decisions
        server backend1.example.com;
        server backend2.example.com;
        server backend3.example.com;
    }
    
    server {
        listen 80;
        
        location / {
            proxy_pass http://backend;
        }
    }
}

When to Use Least Response Time

• When minimizing response time is critical
• For performance-sensitive applications
• When servers have varying processing capabilities or loads

Limitations

• Requires monitoring response times, which adds overhead
• May lead to oscillation if response times fluctuate rapidly
• Available only in commercial load balancer offerings

6. IP Hash

IP Hash uses the client’s IP address to determine which server receives the request, ensuring that the same client always reaches the same server.

Implementation Example: Nginx IP Hash

http {
    upstream backend {
        ip_hash;
        server backend1.example.com;
        server backend2.example.com;
        server backend3.example.com;
    }
    
    server {
        listen 80;
        
        location / {
            proxy_pass http://backend;
        }
    }
}

When to Use IP Hash

• When session persistence is required and you can’t use cookies
• For applications that don’t have built-in session management
• When client IP addresses are stable and diverse

Limitations

• Uneven distribution if client IP distribution is skewed
• Breaks with NAT or proxy servers (many clients share the same IP)
• Doesn’t adapt to changing server capacities

7. Consistent Hashing

Consistent hashing minimizes redistribution of requests when the server pool changes, making it ideal for dynamic environments.

Implementation Example: Custom Consistent Hashing in Go

package main

import (
    "fmt"
    "hash/crc32"
    "sort"
    "strconv"
)

type Hash uint32

type Ring struct {
    nodes map[uint32]string
    keys  []int
}

func NewRing() *Ring {
    return &Ring{
        nodes: make(map[uint32]string),
        keys:  []int{},
    }
}

func (r *Ring) AddNode(node string, weight int) {
    for i := 0; i < weight; i++ {
        key := hashKey(fmt.Sprintf("%s-%d", node, i))
        r.nodes[key] = node
        r.keys = append(r.keys, int(key))
    }
    sort.Ints(r.keys)
}

func (r *Ring) RemoveNode(node string, weight int) {
    for i := 0; i < weight; i++ {
        key := hashKey(fmt.Sprintf("%s-%d", node, i))
        delete(r.nodes, key)
        for i, k := range r.keys {
            if k == int(key) {
                r.keys = append(r.keys[:i], r.keys[i+1:]...)
                break
            }
        }
    }
}

func (r *Ring) GetNode(key string) string {
    if len(r.keys) == 0 {
        return ""
    }
    
    hash := hashKey(key)
    idx := sort.Search(len(r.keys), func(i int) bool {
        return uint32(r.keys[i]) >= hash
    })
    
    if idx == len(r.keys) {
        idx = 0
    }
    
    return r.nodes[uint32(r.keys[idx])]
}

func hashKey(key string) uint32 {
    return crc32.ChecksumIEEE([]byte(key))
}

func main() {
    ring := NewRing()
    
    // Add servers with weights
    ring.AddNode("server1", 3)
    ring.AddNode("server2", 3)
    ring.AddNode("server3", 3)
    
    // Distribute some keys
    keys := []string{"user1", "user2", "user3", "user4", "user5"}
    for _, key := range keys {
        fmt.Printf("Key %s maps to %s\n", key, ring.GetNode(key))
    }
    
    fmt.Println("\nRemoving server2...")
    ring.RemoveNode("server2", 3)
    
    // Check redistribution
    for _, key := range keys {
        fmt.Printf("Key %s maps to %s\n", key, ring.GetNode(key))
    }
}

When to Use Consistent Hashing

• In dynamic environments where servers are frequently added or removed
• For distributed caching systems
• When minimizing redistribution during scaling is important

Limitations

• More complex to implement than simpler algorithms
• May still lead to uneven distribution without virtual nodes
• Doesn’t account for server load or capacity

Advanced Load Balancing Patterns

Beyond basic algorithms, several advanced patterns can enhance load balancing in distributed systems.

1. Layer 7 (Application) Load Balancing

Layer 7 load balancing operates at the application layer, making routing decisions based on the content of the request (URL, headers, cookies, etc.).