Beyond Load Balancing Algorithms: Architecting for efficiency
By Noa on
Load balancing isn't just about choosing the right algorithm. Whilst we've explored algorithms like round robin and least connections before, the real performance gains often come from how you architect your entire environment. Poor architecture can lead to wasteful spending on resources that never reach their potential. Smart design, on the other hand, ensures every euro spent translates to actual capacity.
In this article, we'll take a step back from the mechanics of individual load balancers and examine the bigger picture: how to structure your infrastructure to get the most out of the resources you pay for. We'll start with a network-level overview before diving into what happens after traffic arrives. This guidance applies to most cloud platforms, so you can implement these principles regardless of your provider.
Understanding Load Balancers and the OSI Model
To understand where load balancers fit into your architecture, we need to examine the OSI model. This framework appears in countless tech discussions for good reason.
Layer | Layer Name | How It Works | Used by/for/in |
7 | Application | Routes traffic based on application-level data (HTTP headers, cookies, URL paths, content type) | AWS ALB, NGINX, HAProxy, Envoy, Traefik |
6 | Presentation | Can route based on SSL/TLS session data or content encoding | SSL offloading, content compression decisions |
5 | Session | Maintains session affinity (sticky sessions) using session identifiers | Session persistence modules in various LBs |
4 | Transport | Routes based on IP address and TCP/UDP port numbers | AWS NLB, LVS (Linux Virtual Server), HAProxy (TCP mode) |
3 | Network | Routes based on IP address and TCP/UDP port numbers without inspecting packet contents | IPVS, router-based load balancing |
2 | Data-link | Switch-level load balancing, link aggregation | Switch-level load balancing, link aggregation |
1 | Physical | Bonding, NIC teaming | Network infrastructure redundancy, bandwidth aggregation |
The diagram above shows the different OSI layers where load balancers typically operate. A software or hardware load balancer can process traffic through all these layers as it arrives at the virtual or physical load balancer, moving down the layers before forwarding to your service. Individual layers can also work independently to provide load balancing functionality. This is often achieved through network equipment and configuration such as switches or routers.
Explore our Cloud Load-balancing optionsNetwork Load Balancers vs Application Load Balancers
Load balancers generally fall into two categories based on which OSI layers they prioritise when routing traffic. These are called "application load balancers" and "network load balancers", focusing primarily on layer 7 and layer 3 respectively.
Application load balancers make routing decisions based on data within the protocol being received. This includes routes, cookies, or content types. Network load balancers work with source and destination IP addresses and ports.
Both types often consider layer 4 (transport) and layer 5 (session) to ensure traffic reaches the same processor. This matters for use cases requiring session affinity or "stickiness". This is typically achieved by tracking whether source and destination ports match (tabled per IP) or through session identifiers in the transport layer. Having a stateless application brings more flexibility and requires less compute when processing these layers.
Regardless of which layer your load balancer primarily operates on, some algorithm will process the information present in that layer to determine where traffic is forwarded. You can explore the algorithms available on our load balancers here.
Combining Load Balancers in your stack
You can also combine load balancers at different points in the traffic journey and across different platforms. Here's a common example within the Cyso Cloud platform:
You use a Cloud Load Balancer (CLB) with a floating public IP address. This is where your service domain record points. This is a layer 3 network-based load balancer.
You use an Ingress (such as Traefik) in Kubernetes or a reverse proxy (such as NGINX on your compute instances). This is a layer 7 application load balancer.
In this setup, the CLB does basic distribution of requests using an algorithm, whilst your ingress handles more complex distribution on your compute (either in a pod on a Kubernetes cluster or in the compute instance when not using Kubernetes) to route traffic to services based on application-level information.
Scaling your Load Balancer
Once you've got a clear idea of your expected traffic and which load balancers will distribute it, you need to consider scale. What capacity do you need to process the traffic you're expecting?
This depends on both the type of traffic you're expecting and the sheer quantity. In our pricing guidelines for CLBs, we offer three sizes with guidance tiered to expected traffic:
Small: For static traffic such as simple websites
Medium: For transactional traffic. This includes websites with dynamic forms and content. A web shop is a good example
Large: Designated for enterprise-level traffic, such as a GraphQL server for a website receiving thousands of visitors per hour
Scaling down the stack
This determines your layer 3 scale, but scaling is something you need to consider all the way down your stack. If you run your own layer 7 load balancer in Kubernetes or through a service on your compute node, this too will need to scale at some point.
Scaling the pods, containers, or compute instances running your applications is also crucial. The idea that horizontal scaling is "the way to go" (as is often said) is very dependent on your use case.
Wastage and observability
If your service is complex and uses databases and queues, it's important to identify where bottlenecks occur. One of the largest drivers of cloud costs is untargeted scaling. Observability into your service components is therefore essential. Making your load balancers larger has no effect if your database can't process CRUD commands any faster.
Understanding where your actual constraints lie prevents wasteful spending on resources that don't address your real performance bottlenecks.
Want to optimise your cloud infrastructure?
Explore Cyso Cloud's load balancing solutions or contact our team to discuss your specific architecture needs.
