Load balancers

QuickCart needs one address that never changes, in front of however many servers it runs, and something that stops sending customers to a server that has broken.

A load balancer is one fixed address in front of many servers. Customers only ever talk to it. It passes each request on to one of the servers behind it, and it keeps checking which of those servers are healthy.

Servers can come and go, change size and change address. The address customers use never changes.

The pieces

AWS’s load balancer for websites and APIs is the Application Load Balancer, or ALB. It’s made of a few parts that you configure separately:

A request’s path: customer → listener → rule → target group → one healthy target.

Three kinds of load balancer

Health checks: who gets traffic

Every few seconds, the load balancer sends each target a small request, like GET /health. If the answer is right, the check passes. If it’s wrong, slow or missing, it fails.

How long before a broken server stops getting traffic? Interval × unhealthy threshold. For QuickCart that’s 10 × 2 = about 20 seconds. Requests that reach the broken server in that window fail. Checking every second would catch it sooner, but a single slow answer would then pull a healthy server out.

Shallow or deep?

QuickCart tested this. They removed the security group rule that let the servers reach the database. /health kept answering 200, and / answered 500. The site was degraded, not down: the servers stayed in, showed a clear error, and recovered the moment the rule was put back. An alarm on the error rate told a person what was wrong. A deep check would have failed all the servers together and turned a database problem into a total outage.

So database problems are a job for alarms, which tell a person. Health checks are for one broken server, which a machine can fix by replacing it.

Taking a server out gently: deregistration delay

When a target is removed on purpose — during a release, or when auto scaling shrinks the group — some requests to it may still be in progress. The deregistration delay stops new requests going to the target, but waits for those in progress to finish, up to a limit.

The default is 300 seconds. That’s safe, but it makes every release wait five minutes per batch. QuickCart’s API answers in well under a second, so 30 seconds is plenty. This is also called connection draining.

Status codes that come from the load balancer

When something goes wrong, the code tells you where to look:

HTTPS, and sending HTTP to it

The ALB can do the HTTPS work itself, so the servers behind it don’t have to. You add a listener on port 443, attach a certificate from AWS Certificate Manager (free for use with a load balancer), and forward to the target group.

Then the listener on port 80 changes from forward to redirect: send the browser to the same address with https://, with status 301, which means “moved for good”. Nobody who types the plain address gets an error, and nothing travels unencrypted. Domain names and HTTPS walks through getting the certificate.

Only the load balancer may talk to the servers

The servers shouldn’t accept traffic from the internet directly. Their security group allows port 80 only from the load balancer’s security group, not from an IP range. Security groups explains this chain.

An ALB always spreads requests across targets in every zone it’s in, so if one zone loses servers, the others take all the traffic. By default it takes turns, called round robin. The least outstanding requests setting instead sends each request to the target with the fewest in progress, which helps when some requests are much slower than others.

Three servers sit behind QuickCart’s load balancer, in an auto scaling group that replaces unhealthy ones. Send some requests and watch them spread. Then crash the app on one server and send again. Finally, break the database, and compare a shallow health check with a deep one.

What to notice: with the database broken and a shallow check, every request failed — but every server stayed in and answered. Switch to a deep check, and the same problem took out every server at once. Nothing about the servers changed. Only the question the health check asked.

Costs money — about $1.75 a day while it runs — delete it after. The load balancer is about $0.0225 an hour plus a small usage charge (LCUs), each t3.micro is about $0.0104 an hour, and every public IPv4 address is about $0.005 an hour. These are us-east-1 prices; they change, so check the pricing pages. An hour of this lab costs well under 10 cents.

You’ll put two servers in two zones behind an Application Load Balancer, watch it take turns between them, then break one and watch the health check take it out. About 40 minutes.

You need the quickcart-ec2-ssm role from Servers on EC2. If you deleted it, create it again as in that lesson’s first step. Check the region selector says N. Virginia (us-east-1).

1. Create the load balancer's security group

   AWSIn the AWS console: EC2→Security Groups→Create security group

   Field	Enter
   Security group name	quickcart-alb-sg
   Description	QuickCart load balancer - HTTP from anywhere
   VPC	The default VPC

   Under Inbound rules, choose Add rule:

   Type	Source
   HTTP	Anywhere-IPv4 (0.0.0.0/0)

   Leave the outbound rule as it is, and choose Create security group.

   This lab stays on plain HTTP, because a trusted HTTPS certificate needs a domain name. Domain names and HTTPS adds the 443 listener and the redirect.

2. Create the servers' security group

   AWSIn the AWS console: EC2→Security Groups→Create security group

   Field	Enter
   Security group name	quickcart-app-sg
   Description	QuickCart servers - HTTP from the load balancer only
   VPC	The default VPC

   Under Inbound rules, choose Add rule:

   Type	Source
   HTTP	Custom, then start typing quickcart-alb-sg and pick it

   Leave the outbound rule, and choose Create security group.

   You should see

   One inbound rule on port 80 whose source is a security group ID starting sg-, not an IP range. Only things in quickcart-alb-sg can reach these servers.

3. Launch two servers, one in each zone

   AWSIn the AWS console: EC2→Instances→Launch instances

   Field	Enter
   Name	quickcart-web-a
   Application and OS Images	Amazon Linux 2023 AMI, 64-bit (x86)
   Instance type	t3.micro
   Key pair (login)	Proceed without a key pair

   Under Network settings, choose Edit:

   Field	Enter
   VPC	The default VPC
   Subnet	The default subnet in us-east-1a
   Auto-assign public IP	Enable — so the server can reach Systems Manager and download nginx
   Firewall (security groups)	Select existing security group, then quickcart-app-sg

   Under Advanced details:

   Field	Enter
   IAM instance profile	quickcart-ec2-ssm
   Metadata version	V2 only (token required)
   User data	The script below

   user-data.shCopy

   #!/bin/bash
   dnf install -y nginx
   hostname > /usr/share/nginx/html/index.html
   echo ok > /usr/share/nginx/html/health
   systemctl enable --now nginx

   Line	What it does
   #!/bin/bash	Run this as a bash script
   dnf install -y nginx	Install the nginx web server, answering yes automatically
   hostname > /usr/share/nginx/html/index.html	Make the home page the server’s own name, so you can see which server answered
   echo ok > /usr/share/nginx/html/health	Create a page at /health that just says ok. This is the shallow health check: if nginx can answer, it passes
   systemctl enable --now nginx	Start nginx now, and on every boot

   Choose Launch instance. Then launch a second one the same way, with only two differences:

   Field	Enter
   Name	quickcart-web-b
   Subnet	The default subnet in us-east-1b

   You should see

   Two instances, quickcart-web-a in us-east-1a and quickcart-web-b in us-east-1b, both Running.

4. Create the target group and its health check

   AWSIn the AWS console: EC2→Target groups→Create target group

   Field	Enter
   Choose a target type	Instances
   Target group name	quickcart-web-tg
   Protocol : Port	HTTP : 80
   IP address type	IPv4
   VPC	The default VPC
   Protocol version	HTTP1
   Health check protocol	HTTP
   Health check path	/health

   Open Advanced health check settings:

   Field	Enter
   Health check port	Traffic port
   Healthy threshold	2
   Unhealthy threshold	2
   Timeout	5 seconds
   Interval	10 seconds
   Success codes	200

   Choose Next. Tick both quickcart-web-a and quickcart-web-b, check the port says 80, and choose Include as pending below. Then choose Create target group.

   Now shorten the deregistration delay. Open quickcart-web-tg, go to the Attributes tab, and choose Edit:

   Field	Enter
   Deregistration delay	30 seconds

   Choose Save changes.

   You should see

   On the Targets tab, both instances with health status Unused, and a note that the target group isn’t used by any load balancer yet. Health checks only start once a load balancer uses the group.

5. Create the load balancer

   AWSIn the AWS console: EC2→Load balancers→Create load balancer

   Under Application Load Balancer, choose Create.

   Field	Enter
   Load balancer name	quickcart-alb
   Scheme	Internet-facing
   Load balancer IP address type	IPv4
   VPC	The default VPC
   Availability Zones and subnets	Tick us-east-1a and us-east-1b, each with its default subnet
   Security groups	quickcart-alb-sg. Remove the default group

   Under Listeners and routing:

   Protocol	Port	Default action
   HTTP	80	Forward to quickcart-web-tg

   Choose Create load balancer.

   You should see

   quickcart-alb with state Provisioning. After a few minutes it becomes Active. On its Details tab is a DNS name ending in elb.amazonaws.com. Copy it.

6. Check both targets are healthy

   AWSIn the AWS console: EC2→Target groups→quickcart-web-tg→Targets

   You should see

   Both targets go from Initial to Healthy within about half a minute.

7. Watch it take turns

   In a terminal on your computer, ask the load balancer for its home page. Replace ALB_DNS_NAME with the DNS name you copied:

   TerminalCopy

   curl --connect-timeout 10 http://ALB_DNS_NAME/

   Part	What it means
   curl	make a web request and print the answer
   --connect-timeout 10	give up after 10 seconds if nothing answers, instead of waiting over a minute
   http://ALB_DNS_NAME/	the load balancer’s address. Paste yours in place of ALB_DNS_NAME

   You should see

   ip-172-31-20-15.ec2.internal

   That’s the name of the server that answered. Run the same command four or five more times.

   You should see

   The answer switches between two names: one for each server. Your names will differ, but each is the server’s private address written with dashes. Match them to your instances in the console’s Private IPv4 addresses column.

8. Break one server and watch it drop out

   AWSIn the AWS console: EC2→Instances→quickcart-web-a→Connect→Session Manager

   Choose Connect, and in the terminal stop the web server:

   TerminalCopy

   sudo systemctl stop nginx

   Part	What it means
   sudo	run as root
   systemctl stop	stop a running service
   nginx	the web server, and with it the /health page

   Now go back to Target groups → quickcart-web-tg → Targets and refresh.

   You should see

   Within about 20 seconds — two failed checks, 10 seconds apart — quickcart-web-a shows Unhealthy, with a reason like Health checks failed.

   Run your curl command a few more times from your computer.

   You should see

   Every answer now comes from the same server: the one in us-east-1b. The load balancer only sends traffic to targets that pass their checks. If you were quick enough to run curl during those first 20 seconds, you may also have seen a 502 Bad Gateway: a request that went to the broken server before the checks caught up.

   Bring it back:

   TerminalCopy

   sudo systemctl start nginx

   Part	What it means
   systemctl start	start the service again
   nginx	the web server

   You should see

   After two passing checks, about 20 seconds, quickcart-web-a is Healthy again, and curl switches between both names once more.

9. Clean-up step: Clean up

   Delete things in this order. The load balancer first, because it’s the most expensive, and because the security groups can’t be deleted while it uses them.

   AWSIn the AWS console: EC2→Load balancers→quickcart-alb→Actions→Delete load balancer

   Type confirm and choose Delete.

   AWSIn the AWS console: EC2→Target groups→quickcart-web-tg→Actions→Delete

   Choose Yes, delete.

   AWSIn the AWS console: EC2→Instances→quickcart-web-a and quickcart-web-b→Instance state→Terminate instance

   Tick both instances first. Newer consoles call it Terminate (delete) instance. Confirm it.

   AWSIn the AWS console: EC2→Security Groups→quickcart-app-sg→Actions→Delete security groups

   Delete quickcart-app-sg first, then quickcart-alb-sg: the first one refers to the second.

   Keep the quickcart-ec2-ssm role for the next lesson; roles are free.

   You should see

   No load balancers, no target groups, both instances Terminated, and neither security group left. Nothing is billing.