Laptop to Production

Cheat sheets

Each lesson ends with its essentials on one card. They're all here, in course order. Print one card with its button, or print this page for all of them.

The roadmap

00 – Start here

On one card
IdeaIn one line
DevOpsGetting changes to users safely and often, and keeping them running
This courseOne app, QuickCart, from a laptop to millions of users
Every lessonProblem, idea, see it, do it, check yourself, cheat sheet
Every toolThe answer to one problem at one size. Meet the problem first
Jumping inRead the lessons listed under Read first, then start anywhere
uname -smYour system and chip: x86_64 for Intel or AMD, arm64 for ARM
df -h /Free disk space. Look at Avail. You want 20 GB or more
CostEvery lab has a badge. The paid ones have a clean-up step

How this course works

00 – Start here

On one card
IdeaIn one line
The six partsProblem, idea, see it, do it, check yourself, cheat sheet
Root userThe sign-up email. MFA on, no access keys, almost never used
IAM Identity CenterWhere your everyday admin login lives. Credentials expire after hours
Access portalThe awsapps.com/start address you sign in from. Bookmark it
Permission setA named bundle of permissions, like AdministratorAccess
BudgetEmails you at 85% and 100%. It does not stop anything
Free badgeNothing costs money
Pennies badgeA few cents at most
Costs money badgeCharges every hour until the clean-up step
RegionAlways us-east-1 in this course
Free tierDepends on when the account was made. Check Billing → Free Tier

Set up your computer

00 – Start here

On one card
CommandWhat it does
brew install NAMEInstall a terminal tool
brew install --cask NAMEInstall a desktop app (macOS only)
brew upgradeUpdate everything Homebrew installed
aws configure ssoSet up a profile that signs in through IAM Identity Center
aws sso login --profile quickcartSign in again when the session expires
aws sso logoutSign out now
aws sts get-caller-identityWho am I, and in which account?
aws configure listWhich profile, keys and region are in use, and where from
export AWS_PROFILE=quickcartUse this profile in this terminal
~/.bashrc or ~/.zprofileSettings your terminal reads when it opens: Linux and WSL, or macOS
wsl --installWindows: install WSL 2 with Ubuntu, from an administrator PowerShell
Credential order--profile, then key variables, then AWS_PROFILE, then default

The DevOps lifecycle

01 – How software ships

On one card
StageIn one line
PlanDecide what to build next, using what monitoring showed
CodeWrite it, and have it reviewed
BuildTurn code into something that runs
TestCheck it works and broke nothing else
ReleaseDecide it may go to users
DeployPut it on the servers
OperateKeep it running
MonitorWatch errors, speed and usage. Feeds the next plan
Batch sizeHow many changes travel together. Smaller is safer
Value stream mapWorking time vs waiting time, stage by stage
git log --reverse --grep=WORDWhen the work on a change started

Measuring delivery

01 – How software ships

On one card
Metric or commandIn one line
Deployment frequencyHow often you deploy to production. Speed
Lead time for changesCommit to running in production. Speed
Change failure rateShare of deployments that needed a fix. Stability
Time to restoreFrom a failure starting to it being fixed. Stability
The ruleWatch all four together. Smaller batches improve all of them
NeverUse them to rank individual people
git rev-list --count --first-parent --since="30 days ago" mainHow many changes reached main
git log --reverse "ID^1..ID^2"The commits a merge brought in, oldest first
git log -i --grep="revert"Changes that undid earlier ones

Reliability targets

01 – How software ships

On one card
Term or commandIn one line
SLIWhat you measure: good events ÷ all events
SLOThe target for the SLI over a window, like 99.9% over 30 days
Error budget100% minus the SLO. The failure you can afford
SLAA promise to customers with a penalty. Looser than your SLO
99.9%About 43 minutes of failure a month
99.99%About 4 minutes a month
Good vs bad5xx and too slow are bad. 4xx usually aren’t
Error budget policyAgreed in advance: what happens as the budget runs out
grep -c "TEXT" FILECount matching lines
grep -E "PATTERN" FILEMatch a pattern, like 5[0-9][0-9]

Ways to release

01 – How software ships

On one card
Strategy or ideaIn one line
RecreateStop everything, start the new version. Downtime built in
RollingReplace servers a few at a time. No extra cost; bad versions spread until noticed
Blue/greenFull second copy, then switch all at once. Instant undo; everyone hit if bad
CanarySmall share first, grow while watching. Fewest customers hit
Abort criteriaThe rule for stopping a canary, written before you start
Expand, then contractDatabase changes that work with old and new versions at once
Rolling backUndoes code, not what the bad version already did
Lambda aliasA name pointing at one version, or two with weights
aws lambda invoke --function-name NAME:ALIAS out.jsonCall through an alias; ExecutedVersion says who answered

Feature flags

01 – How software ships

On one card
IdeaIn one line
Feature flagAn if whose answer comes from a setting, not the code
Deploy vs releaseDeploy puts code on servers; the flag decides who gets the feature
Percentage rolloutA canary done with a flag. Buckets keep customers in as it grows
BucketA hash of the customer’s ID, from 0 to 99. Same customer, same bucket
Kill switchOff for everyone, on the next request, with no deploy
Safe defaultCan’t read the settings? Everything is off
Flag debtOld flags left in the code. Remove them soon after full rollout
NeverReuse an old flag’s name
Real flag servicesSuch as AWS AppConfig: change a flag with a click, every server follows

What the cloud is

02 – The cloud, simply

On one card
TermIn one line
The cloudRenting computers by the second instead of buying them
Pay for what existsA forgotten resource keeps costing money until you delete it
RegionA separate group of data centres, like us-east-1. Most things live in one
Global serviceNot tied to a region. IAM is the main one
Region selectorTop right of the console. Empty list? Check it first
Account ID12 digits, in the account menu. Shown in this course as 111122223333
Shared responsibilityAWS secures the cloud; you secure what you put in it
BudgetEmails you when spending crosses a limit. Never stops anything
Every labEnds with a clean-up step. Always do it

Who can do what: IAM

02 – The cloud, simply

On one card
TermIn one line
Root userThe account’s first login. No IAM policy can limit it. MFA on, no access keys, almost never used
IAM userA named identity with long-lived credentials
IAM groupUsers that share the same policies
IAM roleNo password, no keys. Assumed, and hands out temporary credentials
PolicyJSON: Effect + Action + Resource in each statement
VersionAlways 2012-10-17
Implicit denyNothing allowed it, so it’s denied. The default
Explicit denyA matching Deny. Beats every Allow
Least privilegeExactly the actions and resources needed, nothing more
Trust policyWho may assume a role
Permissions policyWhat the role can do
S3 ARNsBucket: arn:aws:s3:::name. Files: arn:aws:s3:::name/*
AccessDenied?Look for the missing Allow, or a Deny, in the policies. Never switch to root

Regions and zones

02 – The cloud, simply

On one card
TermIn one line
RegionA separate part of the world with its own copy of AWS, such as us-east-1
Availability zoneOne or more buildings in a region, with their own power and network
Choosing a regionDistance to customers, the law, price, and which services it has
GlobalIAM, Route 53: no region
RegionalVPC, S3 bucket, load balancer: spread across zones for you
ZonalSubnet, EC2 server, EBS disk, NAT gateway: you spread them yourself
Two zonesProtects against one building failing. Each zone must carry the full load alone
Two regionsProtects against a whole region failing. Much more work
Zone name vs IDus-east-1a can differ between accounts; use1-az2 never does
Resources vanished?Check the region selector first

What is a VPC?

03 – Networking

On one card
TermIn one line
VPCYour private network on AWS. Closed by default
SubnetA slice of the VPC, in one availability zone
Route tableThe rules for where a subnet’s traffic goes
Public subnetRoute table sends 0.0.0.0/0 to the internet gateway
Private subnetRoute table sends 0.0.0.0/0 to a NAT gateway — out only
Isolated subnetNo 0.0.0.0/0 route at all
local routeAlways there. Everything in the VPC can reach everything else
/16 and /2465,536 addresses and 256 addresses
Can’t reach something?Check the route table first

Subnets and IP ranges

03 – Networking

On one card
TermIn one line
IPv4 address32 bits, written as four numbers from 0 to 255
/nThe first n bits are fixed. 32 − n bits are free
Addresses in a range2 to the power of the free bits
/16 · /20 · /24 · /2865,536 · 4,096 · 256 · 16
Boundary ruleA block starts on a multiple of its own size
Reserved in every subnetFirst four and the last: 5 addresses. A /24 has 251 usable
Subnet size limitsFrom /16 to /28
Private ranges10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
Growth/16 per VPC, leave most of it unused. You can’t resize a subnet
Several VPCsNever overlap: 10.0.0.0/16, 10.1.0.0/16, 10.2.0.0/16

Routes, gateways and NAT

03 – Networking

On one card
TermIn one line
Route tableDestination → target rules for a subnet. One table per subnet
localThe VPC’s own range. Always there, can’t be removed
Most specific winsOf all matching routes, the one with the biggest number after the slash
Main route tableUsed by any subnet not associated with another. Keep it isolated
Public subnet0.0.0.0/0 → igw. Servers also need a public IP
Private subnet0.0.0.0/0 → nat. Out only
Isolated subnetNo 0.0.0.0/0 route
Internet gatewayOne per VPC, two-way, free
NAT gatewayIn a public subnet, one zone, needs an Elastic IP, bills hourly and per GB
High availabilityOne NAT gateway per zone, each zone routing to its own
Gateway endpointS3 and DynamoDB. A route, free. Cuts NAT charges
Interface endpointOther services. A private address in your subnet, bills hourly
Clean-up orderNAT gateway → wait for Deleted → release Elastic IP → VPC

Security groups

03 – Networking

On one card
TermIn one line
Security groupA firewall on one resource. Allow rules only
StatefulReplies to an allowed connection are allowed automatically
New groupNo inbound rules; outbound allows all traffic
Source = another groupAllows whatever carries that group, at any address
The chainALB from 0.0.0.0/0 on 443 → app from ALB group on 8080 → database from app group on 5432
Times outDropped by the network: security group, route table or network ACL
Refused, fastReached the server; nothing listening on that port
HTTP 500, fastReached the app; the code failed. Read its logs
504 from the load balancerIt couldn’t reach a server within 10 seconds. Check the app’s security group
Can’t log in to a broken server?Actions → Monitor and troubleshoot → Get system log
Network ACLSubnet firewall. Stateless, numbered, allow and deny. Usually left at the default

Domain names and HTTPS

03 – Networking

On one card
TermIn one line
DNSTurns names into IP addresses, by a chain of servers
ResolverAsks on your behalf, and caches answers
TTLHow many seconds an answer may be cached
RegistrarWhere you bought the domain. Sets its name servers
Hosted zoneWhere a domain’s records live in Route 53. $0.50 a month
NS recordThe four name servers in charge. Must match at the registrar
A / AAAAName → IPv4 / IPv6 address
CNAMEName → another name. Never at the apex
AliasRoute 53 only. Name → AWS resource, allowed at the apex
ACMFree trusted certificates for load balancers. DNS validation renews itself
TLS terminationThe load balancer decrypts; servers don’t handle certificates
Port 80 listener301 redirect to HTTPS
Self-signedEncrypted, but untrusted. For learning only
DNS not working?dig +short NS example.com — does it match the hosted zone?

Servers on EC2

04 – Running code

On one card
TermIn one line
InstanceA computer rented by the second
AMIThe starting disk every instance is copied from
t3.microFamily t, generation 3, size micro
User dataA script run once, as root, on first boot. Not secret
/var/log/cloud-init-output.logWhat your user data printed. Look here first
RebootSame ID, same disk, same IP
Stop, then startSame ID, same disk, new public IP
TerminateGone, disk and all
Session ManagerA shell with no open port and no key. Needs the agent, a role, and a way out on 443
Instance profileHow a role is attached to an instance
AmazonSSMManagedInstanceCoreThe policy Session Manager needs
IMDSv2Metadata needs a token first. Set V2 only
Golden ruleNever fix a server by hand. Change the AMI or user data

Load balancers

04 – Running code

On one card
TermIn one line
Load balancerOne fixed address in front of many servers
ALBFor HTTP: can route by path and hostname, and do HTTPS
ListenerA port and protocol, and what to do with requests on it
Target groupThe servers, plus the health check that picks who gets traffic
Health checkA small request every few seconds. Fail enough in a row: no traffic
Time to take a broken server outInterval × unhealthy threshold. 10 s × 2 = about 20 s
Shallow check“Is this server broken?” — use this
Deep check“Is everything behind it working?” — one shared failure takes out every server
Deregistration delayLets in-progress requests finish. Default 300 s; QuickCart uses 30 s
502 / 503 / 504Broken answer / no target / too slow
Port 80 listenerRedirect to HTTPS, 301
Servers’ security groupPort 80 from the load balancer’s security group only
Timed out?Check the security groups first

Auto scaling

04 – Running code

On one card
TermIn one line
Auto Scaling groupKeeps the number of running servers equal to desired
Min / desired / maxNever fewer / right now / never more. Scaling only changes desired
Launch templateThe recipe for each server. Versioned; no subnet
ZonesGive the group subnets in two or more; it balances between them
EC2 health checkIs the machine running? Always on
ELB health checkDoes the app answer? Turn on when behind a load balancer
Grace periodTime before health checks count. Longer than real start-up time
Target trackingKeep a metric near a target, like average CPU 50%. Out fast, in slowly
Terminate gracefullyThrough the group, so it drains first
Instance refreshReplace every server in batches, for a new template version
Clean upDelete the group first. Deleting servers just makes it launch more

Managed databases: RDS

05 – Data

On one card
TermIn one line
RDSAWS runs the database server; you still own the tables and the SQL
EndpointThe DNS name you connect to. Never use an IP address
DB subnet groupThe private subnets RDS may use. At least two zones
Public accessAlways No
Security groupPort 3306 only from the app servers’ group
Multi-AZSynchronous standby in another zone. Failover in about 60–120 s. You can’t query it
Read replicaAsynchronous copy you can query. Its own endpoint. Can lag
Automated backupsDaily snapshot plus change logs. Restore to any second, 1–35 days back
Manual snapshotKept until you delete it. Take one before anything risky
RestoreAlways creates a new database with a new endpoint
Parameter groupEngine settings. The default can’t be edited; make your own
EncryptionChoose at creation. Can’t be switched on later in place
Clean upDelete the database, then its snapshots and subnet group

Running SQL safely

05 – Data

On one card
Step or statementWhat it’s for
CREATE TABLE x_before_fix AS SELECT id, col FROM t WHERE …Save the rows you’ll change. Before the transaction
Undo scriptWritten before the change: UPDATE t JOIN x_before_fix ON … SET …
SELECT … WHERE (same WHERE)Count the rows first
START TRANSACTION or BEGINNothing after this is permanent until you commit
Rows matched: NMust equal your count. If not, ROLLBACK
COMMITMakes it permanent and visible to everyone
ROLLBACKThrows the whole transaction away
AutocommitOn by default. A statement outside a transaction is permanent at once
DDL: CREATE, DROP, TRUNCATE, ALTERCommits at once, and commits your earlier changes. Can’t be rolled back
ERROR 1205 … Lock wait timeoutSomeone left a transaction open. Keep yours short
Private databaseaws ssm start-session with AWS-StartPortForwardingSessionToRemoteHost, then mysql -h 127.0.0.1

Files on S3

05 – Data

On one card
Term or settingIn one line
BucketA container for objects, in one region. Its name is unique worldwide
Object and keyA file, and its full name. The slashes in a key aren’t real folders
Block Public AccessOn by default. Leave it on
AccessIAM roles for servers, bucket policies for rules, presigned URLs for one file at a time
Presigned URLA link to one object that expires. Keep it short: minutes, not days
VersioningTurn it on when you create the bucket. It only protects later changes
Delete markerWhat a delete adds when versioning is on. Delete the marker to undo
SSE-S3Default encryption on every new object. No setup, no charge
HTTPS onlyBucket policy: Deny when aws:SecureTransport is false
Storage classesStandard for often, Standard-IA for rarely, Glacier for archives
Lifecycle ruleMove to cheaper classes and delete old versions on a schedule
Delete a bucketEmpty first, then Delete

Secrets

05 – Data

On one card
Rule or commandWhat it means
One secret, one placeCode and settings hold the secret’s name, never its value
Secrets ManagerPasswords and anything that should rotate. About $0.40 a secret a month
Parameter Store SecureStringSettings and rarely changing secrets. Standard parameters cost nothing to store
Policy resourcearn:aws:secretsmanager:REGION:ACCOUNT:secret:NAME-* — the -* is required
aws secretsmanager get-secret-value --secret-id NAMERead a secret. Add --version-stage AWSPREVIOUS for the one before
aws ssm get-parameter --name NAME --with-decryptionRead a SecureString parameter
AmazonSSMManagedInstanceCoreLets Session Manager in, and can read every parameter in the account
RotationChanges the secret on a schedule. The app must re-fetch when the database rejects it
A hostnameNot a secret. Keep it in plain config
Leaked?Change it first. Clean up the code and history after

Logs and metrics

06 – Watching it run

On one card
Term or queryIn one line
Log groupA set of log streams for one app, where retention is set
Log streamThe lines from one source, such as one server
RetentionDefaults to Never expire. Always set it
CloudWatch agentShips log files and memory and disk metrics from a server
Structured logOne JSON object per line, so every detail is a field
MetricNamespace, name, dimensions; read with a statistic over a period
Sum / Average / p95Counts / levels like CPU / response times
Metric filter{ $.level = "ERROR" } → a metric. Only counts lines after it exists
Default value 0Lets “no errors” look different from “no data”
filter level = "ERROR"Only error lines
stats count() by error_typeCount per type
stats count() by bin(1m)Count per minute: when did it start and stop?
@timestamp, @messageCloudWatch’s own fields: the time, and the whole raw line

Alarms that actually fire

06 – Watching it run

On one card
Term or settingIn one line
OK / ALARM / INSUFFICIENT_DATAFine / crossed the threshold long enough / can’t tell
ThresholdWhere “fine” ends. Write down why you chose it
M out of NM bad datapoints in the last N periods fires it
FlappingSwitching between OK and ALARM. Needs more datapoints or a better threshold
Missing data: goodError counts. No data means no errors
Missing data: badHeartbeats. Silence means something died
SNS topicWhere alarms send notifications. Subscriptions deliver them
Pending confirmationNothing is delivered until the email link is clicked
set-alarm-state --state-value ALARMTests the notification path. CLI only
Real triggerThe only proof it fires in an incident
Composite alarmSeveral alarms, one rule, one notification
Alarm fatigueEvery alarm must mean “do something now”

Who changed what

06 – Watching it run

On one card
Term or actionIn one line
CloudTrailA record of every AWS API call: who, what, when, from where
Management eventA change to, or a look at, a resource. In event history
Data eventReading or writing inside a resource, such as an S3 file. Only in a trail, at extra cost
Event historyFree, always on, 90 days, one region at a time
Read only = falseOnly changes. The first filter in any incident
userIdentity.typeIAMUser, AssumedRole (session name at the end of the ARN), Root, AWSService
errorCode: AccessDeniedA refused call — how to debug permissions
TrailCopies events to S3, all regions, kept as long as you like
Log file validationProves the log files weren’t edited
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=RevokeSecurityGroupIngressThe same lookup from the CLI

Images and containers

07 – Containers

On one card
Command or ideaWhat it means
ImageThe package: code, libraries, system files. Read-only
ContainerA running copy of an image. Its written files go when it’s removed
LayerOne set of file changes in an image. Shared between images
docker pull nginx:1.27Download an image
docker run -d -p 8080:80 --name web nginx:1.27Start a container in the background, with a published port
docker ps / docker ps -aRunning containers / all containers
docker logs webWhat a container has printed
docker exec web COMMANDRun a command inside a running container
docker stop web then docker rm webStop it, then delete it
Tag (:1.27)A movable label. Never release with latest
Digest (@sha256:)The exact content. Never changes
exec format errorBuilt for the wrong processor. Use --platform linux/amd64

Writing a Dockerfile

07 – Containers

On one card
Line or commandWhat it does
FROM python:3.12-slimStart from a pinned, small base image
COPY requirements.txt . then RUN pip install then COPY app.py .Libraries first, code last: fast rebuilds
RUNRuns once, at build time
CMD ["a", "b"]Runs at container start. Brackets, so the app hears stop signals
ENTRYPOINTFixed start command; docker run words are added to it
USER 1001Don’t run as root. Use the number
.dockerignoreKeep .git, .env and keys out of the build
COPY --from=buildMulti-stage: ship the result, not the build tools
docker build -t name:tag .Build from the Dockerfile here
docker build -f File -t name:tag .Build from another Dockerfile
CACHED in the outputThat step was reused
Secret copied in, then rmStill in the image. Never copy it in

Data and networking

07 – Containers

On one card
Command or settingWhat it does
Writable layerThe container’s own files. Deleted by docker rm
docker volume create NAMEMake a named volume
-v NAME:/pathMount a named volume inside the container
--mount type=bind,source=FULL_PATH,target=/pathShow a folder from your computer inside the container
-p 8081:8080Port 8081 on your computer leads to 8080 in the container
-p 127.0.0.1:8080:8080Only reachable from your own computer
docker network create NAMEA network where containers find each other by name
localhost in a containerThat container itself. Use the other container’s name
docker compose up -d --waitStart everything in compose.yaml and wait until healthy
docker compose exec SERVICE COMMANDRun a command inside a service
docker compose downRemove containers and network. Keeps volumes
docker compose down -vAlso deletes the volumes and their data

Storing images: ECR

07 – Containers

On one card
Command or settingWhat it does
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin REGISTRYLog Docker in to ECR for 12 hours
Registry addressACCOUNT_ID.dkr.ecr.REGION.amazonaws.com
docker build --platform linux/amd64 --provenance=false -t name:v1 .Build one plain image for AWS’s processors
docker tag name:v1 REGISTRY/repo:v1Give the image its ECR name
docker push REGISTRY/repo:v1Upload it
repo@sha256:…Pull one exact image by its digest
Immutable tagsA tag can never be moved to another image
Scan on pushCheck every image for known vulnerabilities when it arrives
Lifecycle policyExpire old or untagged images automatically
no basic auth credentialsNot logged in. Run the login again
name unknownThe repository doesn’t exist, or the name is misspelled

ECS on Fargate

08 – Running containers

On one card
Term or settingIn one line
ClusterA named group for services. Holds no servers on Fargate
Task definitionThe recipe: image, CPU, memory, port, logs, roles. Every change is a new revision
TaskOne running copy of a task definition
ServiceKeeps the desired count of tasks running and registered with the load balancer
FargateAWS runs the servers. You pay per task, for what it asks for
awsvpcEach task gets its own network card, private IP and security group
Target typeIP addresses for Fargate
cpu 256 / memory 512A quarter of a vCPU and 512 MB
Execution roleECS’s role: pull the image, send logs, before the app starts
Task roleYour code’s role, while it runs
Task stops with no logsRead the stopped reason
Targets say “Request timed out”Security groups or port, not code
exec format errorImage built for the wrong processor. Build with --platform linux/amd64

Releases without downtime

08 – Running containers

On one card
Setting or ideaWhat to remember
Rolling deploymentReplace copies a few at a time while the service keeps running
Min 100% / Max 200%Start new tasks before removing old ones. Capacity never drops
Health checkMust only pass when the app can really serve a request
Health check grace periodIgnore failed checks while a task starts. Longer than your real start-up time
Deregistration delayLet in-progress requests finish. Longer than your slowest request; QuickCart uses 30 seconds, the default is 300
SIGTERM, then SIGKILL“Please stop”, then after the stop timeout “stop now”
stopTimeout30 seconds by default on Fargate, 120 at most. Longer than the app’s own graceful timeout
Exec form CMD ["gunicorn", …]The app itself receives SIGTERM
Circuit breaker + rollbackA release that keeps failing goes back to the last good one by itself
Stopped reasonThe first place to look when a task won’t start
Database changesAdd, then use, then remove: three releases, never one
ProofCount failed requests during a release. Target: 0

Kubernetes and EKS

08 – Running containers

On one card
Command or ideaWhat it does
Desired stateYou write what you want; controllers keep making it true
Reconcile loopObserve, compare, act. Over and over
Pod / Deployment / ReplicaSetOne copy / “run N copies” / one version’s set of copies
Service / IngressA stable name inside the cluster / a load balancer from outside
Readiness vs liveness probeTake out of traffic / restart the container
kubectl apply -f FILEMake the cluster match a file
kubectl get podsList pods. describe pod NAME explains problems in its Events
kubectl delete pod NAMEDelete one pod. The Deployment replaces it
kubectl set image deployment/NAME app=IMAGERelease a new image
kubectl rollout status / undoWatch a release / go back one version
EKS control planeAbout $0.10 an hour on standard support. Upgrade before extended support
t3.smallAbout 11 pods, however much CPU is free
Zero downtime on EKSReadiness gate label, preStop sleep, terminationGracePeriodSeconds
Clean upkubectl delete -f first, then eksctl delete cluster

ECS or EKS?

08 – Running containers

On one card
Question or factWhat to remember
Same job?Yes. Both run the same image, behind the same load balancers, in the same VPC
ECS control plane costNone. You pay for tasks
EKS control plane costAbout $0.10 an hour on standard support, about six times more on extended support. Prices change
EKS upkeepRegular version upgrades for the cluster, nodes and add-ons
Broken releaseECS with the circuit breaker rolls back itself. Kubernetes waits for kubectl rollout undo
Biggest factorWho will run the platform
PortabilityWorth something only if leaving AWS is really planned
“We need Kubernetes for X”Name X. If you can’t, you don’t
Already on one platform?A second one doubles how you release, debug and secure
A good memoAnswer first, evidence, the strongest counter-argument, and what would change your mind

Why Terraform

09 – Infrastructure as code

On one card
Idea or commandWhat it means
ClickOpsBuilding by clicking in the console. Fine for learning; unrepeatable for real environments
Infrastructure as codeInfrastructure described in text files, kept in git, built by a tool
DeclarativeDescribe the end result; the tool works out the steps
IdempotentRunning it again with no changes does nothing
terraform initSet up a folder and download plugins. Once per folder
terraform planShow what would change. Changes nothing
terraform applyShow the plan, ask for yes, make the changes
terraform destroyRemove everything this code built
+ ~ -Create, change in place, destroy
The ruleEvery fix goes in the code. A console-only fix gets undone

Your first resource

09 – Infrastructure as code

On one card
Command or symbolWhat it means
terraform initDownload providers, create .terraform.lock.hcl. Again after adding a provider or module
terraform fmtTidy the formatting
terraform validateCheck the code without contacting AWS
terraform planShow what would change. Changes nothing
terraform applyShow the plan, ask for yes, make the changes
terraform destroyDelete everything this folder manages
+ / ~ / -Create / change in place / destroy
-/+ and # forces replacementDestroy then create. Stop and read
(known after apply)AWS decides the value when it creates the resource
version = "~> 6.0"Any 6.x provider, never 7.0
Commit.tf files and .terraform.lock.hcl
Never commit.terraform/ and *.tfstate

Variables, locals and outputs

09 – Infrastructure as code

On one card
ThingWhat it does
variable "x" { type, default, description }Declare an input. No default means required
validation { condition, error_message }Reject bad values before anything is planned
sensitive = trueHide the value in output. It’s still in state
ephemeral = trueKeep a variable out of state and plan files (1.10+)
locals { name_prefix = "${var.project}-${var.environment}" }Work a value out once; read it as local.name_prefix
default_tags { tags = local.common_tags }Tag every resource the provider creates
output "x" { value, description }Print a value after apply; how modules hand values back
-var-file=dev.tfvarsUse one environment’s values
-var name=valueSet one value on the command line
TF_VAR_nameSet a value from the environment — weaker than any tfvars file
Precedence, weakest firstdefault → TF_VAR_ → terraform.tfvars → *.auto.tfvars → -var / -var-file (later wins)
terraform output -raw xPrint one output, bare

Loops and lifecycle

09 – Infrastructure as code

On one card
ThingWhat it does
count = length(var.list)Copies by position: x[0], x[1]. Removing a middle item shifts the rest
count = var.enabled ? 1 : 0On/off switch. Refer to it as x[0]
for_each = var.map or toset(var.list)Copies by key: x["orders"]. Removing one touches nothing else
each.key, each.valueThe current key and value inside a for_each
dynamic "ingress" { for_each, content }Repeat a block inside one resource; ingress.value is the item
create_before_destroy = trueReplace by creating first. Pair with name_prefix. Plan shows +/-
prevent_destroy = trueRefuse any plan that destroys or replaces it. Literal true only
ignore_changes = [field]Stop managing a field something else owns. You go blind to it
replace_triggered_by = [resource]Replace when another resource changes. Rare
-/+ in a planDestroy, then create. Stop and ask why
“because index [2] is out of range for count”The count trap. Switch to for_each

Modules

09 – Infrastructure as code

On one card
ThingWhat to remember
Root moduleThe folder you run terraform in. Has the provider and the state
Child moduleA folder called with module "name" { source = … }. No provider, no state of its own
variables.tf / main.tf / outputs.tfInputs / resources / outputs. A habit, not a rule
Inputs with no defaultMust be set by every caller. Use for things that must be decided, like environment
module.NAME.OUTPUTThe only way to read a value from a module
source = "./modules/x"A local folder: versioned with the repository
source = "org/name/aws" + version = "~> 6.0"The public registry, pinned to a major version
source = "git::https://…//x?ref=v1.4.0"Git, pinned to a tag. Never a branch
terraform initAgain after adding a module or changing its source or version
terraform init -upgradeTake newer versions allowed by your constraints
Compose, don’t bloatSmall modules wired together in the root beat one module with 40 inputs

Terraform state

09 – Infrastructure as code

On one card
Command or settingWhat it does
terraform state listList everything Terraform is tracking
terraform state show aws_ecr_repository.ordersEvery detail of one resource
terraform init -migrate-stateMove state into a new backend
terraform state pullPrint the current state, to save a copy
terraform state push FILEReplace the state. Refused if the file is older, unless you add -force
terraform force-unlock LOCK_IDClear a stuck lock, after checking nobody holds it
keyOne per environment: dev/…, stg/…
use_lockfile = trueOne command at a time. Terraform 1.10 or later
Bucket versioningTurn it on before you need it
.gitignore*.tfstate and *.tfstate.* — never commit state

Environments and regions

09 – Infrastructure as code

On one card
ThingWhat to remember
One folder per environmentenvs/dev, envs/stg: you can see where you are
main.tf, variables.tfIdentical in every folder. diff should print nothing
backend.tf → keyDifferent per environment. Never copy it unchanged
backend.tf → regionWhere the state bucket is, not where you build
terraform.tfvarsRead automatically. Holds everything that differs
No defaults on per-environment variablesSo a forgotten value is an error, not dev’s value in prod
terraform init -reconfigureSet a folder up again after changing its backend, without copying state
terraform workspace showWhich workspace you’re in. Fine for short-lived copies, risky for real environments
provider "aws" { alias = "virginia" … }A second configuration, for a second region
configuration_aliases = [aws.us_east_1]In a module: “I need a second AWS provider”
providers = { aws.us_east_1 = aws.virginia }In the module block: which one to use
data "aws_availability_zones", data "aws_ami"Never type in zone names or image IDs

Operating Terraform

09 – Infrastructure as code

On one card
Command or blockWhat it does
terraform planAlways shows drift: any ~ you didn’t write is a question
terraform plan -refresh-onlyShow only what changed outside Terraform. Undo nothing
terraform plan -detailed-exitcodeExit 0 = no changes, 1 = error, 2 = changes. For nightly drift checks
import { to = ADDRESS, id = "ID" }Adopt something made by hand, through a reviewed plan
terraform plan -generate-config-out=generated.tfDraft resource blocks for your import blocks
moved { from = OLD, to = NEW }Rename or move a resource without destroying it
removed { from = ADDRESS } with destroy = falseStop managing a whole resource, and leave it in AWS
terraform state rm 'ADDRESS'Forget one instance. Changes state at once, so add -dry-run first
terraform state pull > state-backup.jsonSave a copy before any repair
-target=ADDRESSEmergencies only. Routine use means the state is too big
terraform init -upgradeNewest provider within your range. Plan, then commit .terraform.lock.hcl

Jenkins from zero

10 – Automation

On one card
ThingRemember
ControllerSchedules, stores, serves the UI. 0 executors
AgentRuns builds. Disposable. Picked by label
Agents on demandAmazon EC2 plugin or Kubernetes plugin
Reaching the controllerSession Manager: aws ssm start-session --target INSTANCE_ID --document-name AWS-StartPortForwardingSession --parameters portNumber=8080,localPortNumber=8080
AWS permissionsInstance role, or withAWS(role: …). Never stored keys
SecretsCredentials store, used with withCredentials, in single quotes
Credential scopeGlobal for pipelines, System for Jenkins itself, folders to fence them off
JENKINS_HOMEEverything. secrets/ decrypts credentials.xml
BackupsNightly EBS snapshots with AWS Backup; ThinBackup for settings; restore one every quarter
Setup as codejenkins.yaml with the Configuration as Code plugin; plugins.txt in your own image
SecurityNot on the internet, no anonymous read, update monthly, short admin list

The Jenkinsfile

10 – Automation

On one card
DirectiveRemember
pipeline { }Declarative. Checked before it runs
agentany, none, { label 'x' }, { docker { image 'x' } }. Per stage too
stages / stage / stepsTop to bottom. The first failure skips the rest
environmentVariables for every step. Read them in sh with single quotes
parameterschoice, booleanParam, string. Read as params.NAME. Active after one run
optionstimeout, timestamps(), disableConcurrentBuilds(), buildDiscarder(logRotator(numToKeepStr: '20'))
whenbranch, changeRequest(), tag, expression { }. beforeAgent, beforeInput
inputmessage, ok, submitter. Put a timeout on the stage. Use agent none at the top
parallelSide by side. failFast true
postalways, success, failure, unstable, aborted, changed, fixed, then cleanup last
Quotes'…' the shell fills in $VAR. "…" Groovy fills in ${…}. Secrets: single quotes only
ReplayTry an edited pipeline once, without saving it

Continuous integration

10 – Automation

On one card
ThingRemember
CIEvery change checked the same way, on a clean agent, before it joins main
OrderCheapest first: lint, tests, build, scan, plan
Fail fastFirst failure stops the rest. disableConcurrentBuilds(abortPrevious: true)
MultibranchA job per branch and pull request. PR builds test the merge with main
Webhookhttps://jenkins.example.com/github-webhook/, with a secret. Polling is the slow fallback
Required checkGitHub ruleset on main: continuous-integration/jenkins/pr-merge must pass
Linthadolint Dockerfile, terraform fmt -check -recursive
TestsWrite JUnit XML; junit in post → always
Scantrivy image --exit-code 1 --severity CRITICAL --ignore-unfixed
Plan on PRRead-only role, shared state, gh pr comment "$CHANGE_ID" --body-file plan.txt
CacheDownloads (layers, packages, providers). Never results
Shared libraryvars/name.groovy → step name(). Load with @Library('lib@v1') _
Flaky testQuarantine with an owner. Never retry a test

Continuous delivery

10 – Automation

On one card
Idea or codeIn one line
Build onceOne image per change. Every environment gets that same image
Deploy by digestrepo@sha256:… never changes. A tag can
Promotiondev → smoke test → stg → smoke test → approval → prod → smoke test
input message: '…', ok: '…'Pause for a person. Add submitter to limit who can approve
timeout around inputNo answer means aborted, never approved
Smoke testA few quick checks that the release really runs where it was deployed
RollbackDeploy the previous digest. No rebuild
retry(3)Only around steps that are safe to repeat
Idempotent“Set to X” can run twice safely. “Add one more” can’t
AWS credentialsAn instance profile, or withAWS(role: …) per environment. Never stored keys

Canary from a pipeline

10 – Automation

On one card
Idea or codeIn one line
CanaryA small share of real traffic tries the new version first
Abort criteriaError rate, latency, minimum sample and bake time, written down before the release
Too little dataNot a pass. Wait for more, or fail the step
Judge per versionALB: one target group per version. Lambda: the ExecutedVersion dimension
ALB weighted forwardTwo target groups with weights. Roll back by setting the canary’s weight to 0
CodeDeployCodeDeployDefault.ECSCanary10Percent5Minutes, with alarms for automatic rollback
Lambda alias--routing-config '{"AdditionalVersionWeights":{"2":0.1}}'
post { failure { … } }The rollback runs whatever made the build fail
Promote100% to the new version. Keep the old one briefly

Where to look first

11 – When things break

On one card
Symptom or placeWhat it means, or where it is
Timed out while connectingThe network dropped it: security groups, routes, network ACLs
Connection refused, instantlyReached the machine; nothing listening on that port
Connected, then timed out with 0 bytesThe app, or something it waits on, is stuck
502 / 503 / 504 from the load balancerBad answer / no healthy targets / target too slow
500The app failed. Read its logs
4xxThe request was wrong. Don’t restart anything
Target.TimeoutHealth check never answered. Check the app’s security group
What changed?Deploys, and CloudTrail with Read only = false
Won’t boot?Actions → Monitor and troubleshoot → Get system log
127.0.0.1 vs 0.0.0.0This machine only vs every address
curl -vShows each step of a connection, and why it failed

Runbooks

11 – When things break

On one card
Part or practiceIn one line
RunbookExact steps for one situation, followable at 3am by anyone
TriggerThe alarm that sends you here
ImpactWhat customers see
StepsWhere, what, and what to do with the answer
VerificationA measurable “fixed”, with a duration
RollbackHow to undo each step that changes something
EscalationWho to call, and after how long
OwnerWho keeps it up to date
Last testedThe date and time of the last dry run. Trust it accordingly
Dry runFollow it with a stopwatch; fix every gap
Link from the alarmPut the runbook link in the alarm description
Automate the fixed partsA rollback is a job, not five commands to type

Real incidents

11 – When things break

On one card
Pattern or practiceIn one line
Most outagesSmall settings — a rule, a port, a route, a typo — not code
No outbound ruleReplies are allowed; connections the server starts are not
Same message, different causesLog error_type and latency_ms as fields
Shallow health checkKeeps servers in service when a dependency fails
Two stacked causesAfter each fix, test again
Numbers look wrongCheck the test before the system
PostmortemSummary, impact, timeline, detection, causes, went well, went badly, lucky, actions
BlamelessWhat made the mistake possible — not who made it
Five whysKeep asking why until you reach something you can change
Action itemSpecific, changes the system, has an owner and a date