2026

Load Balancer: Distributing Traffic Across Servers

A load balancer sits in front of a pool of servers and distributes incoming requests across them. Here is how distribution algorithms work, what health checks are doing and why session persistence is a tradeoff worth understanding.

A single server can only handle so much traffic before requests start queuing. At some point the response times climb. At a higher point the server stops responding altogether. The obvious fix is to run more servers. The problem that follows immediately is deciding which one each request should reach.

That is the job of a load balancer. It sits in front of a pool of servers and distributes incoming requests across them. Clients connect to the load balancer's address. They have no visibility into how many servers exist behind it. From their perspective the service is a single thing at a single address.

What problem it actually solves

Distributing traffic is only part of what a load balancer does. The deeper value is that it makes your service continue working when individual servers stop. A server that crashes gets removed from the pool. Traffic keeps flowing to the servers that are still up. The client never sees the failure.

This changes the way you think about individual servers. When one server's failure means the service is down, that server becomes precious. You patch it carefully. You restart it slowly. You worry about it. When a load balancer distributes traffic across ten servers, losing one server means the service is running at ninety percent capacity while you replace it. The urgency is different.

Scaling up also becomes a matter of adding a server to the pool rather than migrating to a larger machine. Scaling down is removing one. Neither operation requires downtime. The load balancer adjusts as the pool changes.

How a request moves through it

Request path through a load balancer

Client

↓ connects to the load balancer's public address

Load balancer

↓ checks which backends are currently healthy

↓ applies the distribution algorithm

↓ forwards request to selected backend

Backend server

↓ processes the request

↓ returns response to the load balancer

Load balancer

↓ returns response to the client

Client receives response

In most configurations the response travels back through the load balancer rather than directly from the backend to the client. This keeps the architecture clean. The client's connection is with the load balancer's address from start to finish. The backend's internal address is never exposed.

Distribution algorithms

How the load balancer picks a backend for each request depends on the algorithm it is configured to use. Different algorithms make different tradeoffs. The right choice depends on what your requests look like.

Round-robin

Requests are distributed across the pool in order. The first request goes to server one, the second to server two, the third to server three, then back to server one. It is the simplest possible strategy and works well when requests are roughly uniform in cost and servers are roughly equal in capacity.

Least connections

Each new request goes to whichever server currently has the fewest active connections. This adapts naturally to situations where some requests take much longer than others. A slow request on one server does not accumulate load indefinitely because the load balancer will stop preferring that server until it catches up.

IP hash

The client's IP address is hashed to produce a consistent server selection. The same client always reaches the same backend as long as the pool does not change. This is the straightforward way to implement session persistence without storing any state in the load balancer itself.

Weighted round-robin

Each server is assigned a weight that reflects its relative capacity. A server with weight four receives four times as many requests as a server with weight one. This is useful when the pool contains machines of different sizes and you want to use the larger ones more aggressively.

Random with two choices

Two servers are picked at random. The one with fewer active connections handles the request. This sounds simpler than least connections but performs surprisingly well at scale because it avoids the coordination overhead of tracking exact connection counts across many load balancers.

Layer 4 versus layer 7

Load balancers operate at different levels of the networking stack. Where in the stack they work determines what information they can use when making routing decisions.

Layer 4 (transport)

Routing decisions are based on IP address and port. The load balancer does not read the request body or HTTP headers. It operates purely on the TCP connection. This is fast because the inspection work is minimal. The tradeoff is that it cannot make routing decisions based on application-level information like URL paths or request headers.

Layer 7 (application)

The full HTTP request is read before routing happens. The load balancer can make decisions based on paths, headers, cookies, query parameters or request body content. You can route traffic for one path to one set of servers while traffic for a different path goes elsewhere. This flexibility is why most modern load balancers operate at layer 7.

Most general-purpose web traffic uses layer 7 load balancing because the ability to inspect the HTTP request is worth the extra overhead. High-throughput systems that need to move enormous volumes of traffic at low latency sometimes prefer layer 4 because the inspection cost adds up at scale.

Health checks

Distributing traffic across unhealthy servers defeats the purpose. Health checks are how the load balancer knows which backends are worth sending requests to.

A passive health check watches the responses that come back from real requests. Too many errors in a short window and the server gets marked unhealthy. An active health check sends its own probe requests regardless of what real traffic is doing. The probe is typically a GET request to a lightweight endpoint that returns a fast response when the server is ready to handle work.

When a server fails its health check it is removed from the rotation. Traffic shifts to the remaining healthy servers automatically. When the server starts passing its health check again it is added back. The whole cycle happens without any manual intervention.

Session persistence

Some applications store session state on the server. A user logs in on server one. Their session data lives on server one. If the next request from that user lands on server two, server two has no record of their session. They appear to be logged out.

The clean solution is to store session data somewhere external that all servers can reach, like a shared Redis instance. Then it does not matter which server handles each request. When that is not an option, sticky sessions tell the load balancer to route requests from a given client consistently to the same backend.

Cookie-based

The load balancer injects a cookie into the first response. Subsequent requests from that client carry the cookie, which the load balancer reads to select the same backend. This works at layer 7 and does not require the client's IP to stay the same.

IP-based

The client's source IP is hashed to select a backend. Simple to configure. Unreliable when clients sit behind NAT because many users share one IP. Also breaks if the client's IP changes between requests, which happens with mobile connections.

Session token

The application includes a session identifier in the request, either as a header or a URL parameter. The load balancer inspects this value and routes consistently based on it. Requires the load balancer to understand the application's session format.

Sticky sessions reintroduce the problem that load balancing was meant to solve. If a server goes down, the sessions pinned to it are gone regardless of how the persistence is configured. Externalising session state is the more resilient approach whenever the application allows it.

HAProxy as a load balancer

HAProxy is a dedicated load balancer that has been in production environments for a long time. Its configuration separates frontends from backends clearly. Frontends define how traffic arrives. Backends define where it goes.

frontend

frontend http_front
    bind *:80
    default_backend web_servers

The frontend accepts incoming connections on port 80. All traffic falls through to the backend named web_servers.

backend

backend web_servers
    balance roundrobin
    option httpchk GET /health
    server web1 10.0.1.10:3000 check
    server web2 10.0.1.11:3000 check
    server web3 10.0.1.12:3000 check

Round-robin across three servers. HAProxy sends a GET /health request to each server before marking it available. If the check fails, the server is removed from rotation.

The check keyword on each server line tells HAProxy to probe that server using the method defined by option httpchk. Servers that fail their checks stop receiving traffic. The statistics dashboard that HAProxy exposes shows the current health of every server in the pool in real time.

Where it fits

A load balancer changes the relationship between your service and the machines running it. Servers become interchangeable. Losing one is a capacity reduction rather than an outage. Adding one is straightforward. Neither event requires touching the application or changing what clients connect to.

Most services start with a single server and add a load balancer before they add a second server, not after. The reason is that the load balancer makes adding the second server a configuration change rather than an infrastructure project. By the time you need it, you want the machinery already in place.

The algorithm choice matters less than you might expect early on. Round-robin is the right default. Least connections becomes worth thinking about when request duration varies significantly. Everything else follows from understanding what your traffic actually looks like rather than from picking the most sophisticated option up front.