Laptop to Production

Jenkins from zero

Controllers, agents, credentials and backups.

Time
75 minutes
Level
Intermediate

Part 1: The problem

QuickCart, the first build server

Theo set up QuickCart’s first Jenkins on a single server. He clicked through the setup, installed a handful of plugins and made the first jobs in the web page. Builds ran on the server itself, two at a time.

It worked for months. Then the team added a job that built container images. Whenever it ran, the Jenkins page took a minute to load, and other builds sat in the queue even though nothing seemed busy.

A few weeks later, someone cleaning up old servers terminated the wrong one. Every job, every build log, and every stored password went with it. Nobody had written down how Jenkins was set up. It took Theo two days to rebuild it from memory, and it was never quite the same.

Both problems come from how Jenkins was set up, not from Jenkins itself: builds ran on the machine that runs everything else, and the only copy of its configuration was on one disk.

This lesson sets Jenkins up the way QuickCart should have from the start: builds on agents, the controller built by Terraform, reachable only through AWS, with its secrets in one place and a backup you’ve tested.

Part 2: The idea

Jenkins is a server that runs your pipelines. Something happens (a push to git, a timer, someone pressing a button) and Jenkins runs the steps you wrote down: test the code, build an image, deploy it. It’s free, open source, and has been around since 2011, so you’ll meet it in a lot of companies.

A Jenkins setup has two kinds of machine.

PartWhat it doesQuickCart has
ControllerThe brain. Serves the web page, decides which build runs where, stores every job, build log and credentialOne, on EC2
AgentThe hands. A machine that runs builds when the controller tells it toSeveral, added and removed as needed
ExecutorA slot on a machine for one build at a time. An agent with 2 executors runs 2 builds at once2 per agent, 0 on the controller
LabelA tag on an agent, like linux or docker. A pipeline asks for a label, and any agent with it can run the buildlinux
QueueBuilds waiting for a free executor with the right labelHopefully short

The controller used to be called the master, and you’ll still see that word in old guides. Today the controller’s own executors are called the built-in node.

Key point: The controller decides and remembers. Agents do the work. Give the controller zero executors.

Why builds must not run on the controller

ReasonWhat goes wrong
PerformanceA heavy build uses the CPU and memory that the scheduler and the web page need. Every other build and every user slows down
SecurityA build on the controller runs as the same user as Jenkins. It can read JENKINS_HOME, where every credential and the key that decrypts them are kept. Anyone who can change a Jenkinsfile could read them all
StabilityA build that fills the disk fills the controller’s disk. Jenkins stops saving anything

A fresh install gives the controller 2 executors so you can try things out. The first thing to do on a real Jenkins is set that to 0.

Where agents come from

KindHow it worksWhen it fits
Permanent agentA server you run all the time, which connects to the controllerA small team with steady work. What you’ll set up in the lab
On demand, on EC2The Amazon EC2 plugin starts an EC2 instance when builds are waiting, and stops it when it has been idle for a whileWork that comes in bursts; you pay only while builds run
On demand, on KubernetesThe Kubernetes plugin starts a pod for each build and deletes it afterwardsTeams that already run a Kubernetes cluster
A container per buildWith the Docker Pipeline plugin, a pipeline says agent { docker { image 'node:22' } } and the agent runs that build inside the containerEach project needs different tools; the agent only needs Docker

Whichever you choose, the idea is the same: agents are cattle, not pets. Any agent can be thrown away and replaced. Nothing important lives on one.

Running the controller on AWS

The controller is the one machine that isn’t disposable, so it gets the most care. QuickCart’s is built by Terraform, so it can be rebuilt the same way every time.

DecisionQuickCart’s choiceWhy
Where it runsOne EC2 instanceJenkins runs one controller at a time; it doesn’t spread across servers
How Jenkins is installedThe official container image, jenkins/jenkins, pinned to an LTS versionUpgrading is changing one version number. LTS (long-term support) versions get fixes for longer
How people reach itSession Manager, which is part of AWS Systems Manager. No SSH, no open portsAccess is controlled by IAM and logged. Nothing on the internet can reach the login page
How it gets AWS permissionsAn instance role, not access keysThe permissions come with the machine and rotate automatically. There’s no key to leak
Where its data livesJENKINS_HOME on an encrypted EBS volumeIt survives a restart, and it can be snapshotted

Later, when GitHub needs to reach Jenkins to announce a push, you put a load balancer with HTTPS in front of it. Continuous integration covers that.

Plugins

Almost everything Jenkins does comes from plugins. These are the ones QuickCart uses:

PluginWhat it adds
PipelineJenkinsfiles: pipelines written as code. Installs a group of plugins together
GitChecking code out of a git repository
GitHub Branch SourceA job per branch and pull request, found automatically
Credentials and Credentials BindingThe credentials store, and withCredentials to use a secret in a build
Pipeline: AWS StepswithAWS, to run steps with an AWS role
Amazon EC2 or KubernetesAgents started on demand
Docker Pipelineagent { docker { image 'node:22' } }
TimestamperA time on every line of a build log
AnsiColorColoured output from tools that print colour codes
Configuration as CodeThe whole Jenkins setup in one YAML file

Every plugin is code running inside the controller, with full access to it. Install only what you need, and keep them updated: most Jenkins security advisories are about plugins, not Jenkins itself.

Pin them. In the web page, Manage Jenkins → Plugins installs the newest version of each plugin. For a controller you can rebuild, list them in a file and bake them into your own image instead:

plugins.txt
configuration-as-code
workflow-aggregator
git
github-branch-source
credentials-binding
pipeline-aws
timestamper
ansicolor
ec2
thinBackup
Dockerfile
FROM jenkins/jenkins:2.568.3-lts-jdk21
COPY --chown=jenkins:jenkins plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt
LineWhat it does
FROM jenkins/jenkins:2.568.3-lts-jdk21Start from the official image: this LTS version of Jenkins, running on Java 21
COPY … plugins.txt …Put the list in the place the official image expects. --chown makes the jenkins user own it
RUN jenkins-plugin-cli …A tool in the official image that downloads every plugin in the list, and the plugins they depend on

Each name in plugins.txt is a plugin’s ID, which is shown on its page in Manage Jenkins → Plugins. A plain ID takes the newest version. Once you have a set that works, write versions too, like git:5.7.0, so a rebuild gives exactly the same plugins. That version number is only an example; use the ones your working Jenkins shows.

The credentials store

Pipelines need secrets: a token for GitHub, a password for a registry. They go in the credentials store, never in a Jenkinsfile. A pipeline refers to a secret by its ID, and Jenkins hands over the value only while that step runs, hiding it in the log as ****.

KindHoldsExample
Secret textOne secret valueA GitHub token for posting comments
Username with passwordA pairA registry login
SSH Username with private keyA private keyChecking out a repository over SSH
Secret fileA whole fileA kubeconfig for a cluster

Every credential also has a scope. Global means pipelines can use it. System means only Jenkins itself can use it, for example to connect agents. You can also keep credentials inside a folder, so only the jobs in that folder can use them. That’s how you stop a test job from using the production deploy key.

Jenkins stores credentials encrypted in JENKINS_HOME. The key that decrypts them is in JENKINS_HOME/secrets/. That’s why builds must stay off the controller, and why a backup of JENKINS_HOME must be protected like a password.

For AWS, store nothing. An agent on EC2 gets permissions from its instance role. A pipeline that needs different permissions assumes a role with withAWS. Continuous delivery shows both. A long-lived AWS access key in Jenkins is the first thing an attacker looks for.

What to back up

Everything Jenkins knows lives in one folder, JENKINS_HOME. In the lab it’s /var/lib/jenkins-home on the instance, and /var/jenkins_home inside the container.

In JENKINS_HOMEWhat it isLose it and
config.xml and other *.xmlJenkins’ own settingsYou rebuild the setup by hand
jobs/Every job’s settings and its build historyBuild history and logs are gone
credentials.xmlStored credentials, encryptedEvery secret must be entered again
secrets/The keys that decrypt themThe credentials above can never be decrypted
plugins/Installed pluginsReinstall them from your list
workspace/Checked-out code for recent buildsNothing. It’s rebuilt on the next build

There are three ways to protect it. Most teams use the first two together.

WayWhat it doesGood for
EBS snapshots, scheduled with AWS BackupA copy of the whole disk, every night. AWS Backup can pick volumes by tag and keep 30 days of copiesGetting everything back, fast
The ThinBackup pluginCopies the settings, jobs and credentials to another folder on a schedule. That folder can then be copied to S3Going back to yesterday’s version of one job
Configuration as codeJenkins’ settings in YAML, and every pipeline in a Jenkinsfile, all in gitRebuilding a clean Jenkins from nothing, and seeing who changed what

A backup you’ve never restored is a hope, not a backup. Try a restore before you need one.

Jenkins Configuration as Code

Clicking through Manage Jenkins leaves no record of what was changed or why. The Configuration as Code plugin, usually called JCasC, reads the whole setup from a YAML file when Jenkins starts:

jenkins.yaml
jenkins:
  systemMessage: "QuickCart CI. Set up by jenkins.yaml: changes made here are lost on restart."
  numExecutors: 0
  securityRealm:
    local:
      allowsSignup: false
      users:
        - id: "admin"
          password: "${JENKINS_ADMIN_PASSWORD}"
  authorizationStrategy:
    loggedInUsersCanDoAnything:
      allowAnonymousRead: false
unclassified:
  location:
    url: "https://jenkins.example.com/"
SettingWhat it does
systemMessageA banner at the top of every page. Here it warns people that clicking changes won’t last
numExecutors: 0No builds on the controller, as code
securityRealm → localJenkins keeps its own list of users. Larger teams connect a company login instead
allowsSignup: falseNobody can create their own account
password: "${JENKINS_ADMIN_PASSWORD}"JCasC fills this in from an environment variable of that name, so the password isn’t in the file
loggedInUsersCanDoAnythingAnyone logged in can do anything. Fine for a small team; larger teams use Matrix Authorization Strategy to give each person only what they need
allowAnonymousRead: falseVisitors who aren’t logged in see nothing
unclassified → location → urlThe address Jenkins uses in links and in messages to GitHub

Point Jenkins at the file with an environment variable, CASC_JENKINS_CONFIG=/path/to/jenkins.yaml. Manage Jenkins → Configuration as Code also has View Configuration, which shows the YAML for the Jenkins you already have. That’s the easiest way to start.

Security basics

DoWhy
Keep the controller off the internet. Use Session Manager, a VPN, or a load balancer that allows only the sources you expectExposed Jenkins servers are scanned for and attacked constantly
Turn off sign-up and anonymous readA Jenkins page can reveal code, logs and the names of your servers
Update Jenkins and plugins every monthSecurity fixes arrive constantly, mostly for plugins
Zero executors on the controllerBuilds can’t read its secrets
Limit who can change jobs and JenkinsfilesChanging a pipeline means you can use every credential it can use
Treat Manage Jenkins → Script Console as root accessIt runs any code on the controller. Only administrators can open it; keep that list short
Use roles for AWS, not stored keysThere’s nothing to steal

Part 3: See it

Below is a Jenkins controller with one agent. Queue a few builds and watch who picks them up. Add an agent. Then give the controller 2 executors, like a fresh install, and queue two heavy builds.

Executors on the controller

Build queue

waiting for a free executor

  1. Empty

Controller

the brain: schedules, stores, serves the UI

  • Web UI and APIresponsive
  • Schedulerrunning
  • JENKINS_HOMEjobs, history, credentials
CPU and memory
15%

No executors. Builds never run here.

Agents

the hands: each executor runs one build

Waiting 0 · Running 0 · Finished 0

ReadyThe controller has 0 executors, so every build goes to an agent. Choose Queue a build a few times.

What to notice: with 0 executors on the controller, a busy Jenkins only means a longer queue, and the fix is another agent. With executors on the controller, one heavy build can stall the scheduler, so free agents sit idle while everyone waits.

Part 4: Do it

You’ll build a Jenkins controller on EC2 with Terraform, reach it without opening a single port, connect an agent, run a build that uses a stored secret, and take a backup. About 60 minutes.

You need Terraform 1.10 or later and a working AWS login in your terminal, as set up in Set up your computer. You also need the Session Manager plugin for the AWS CLI, which lets your computer open a tunnel to the instance. Install it the way that matches your system.

On macOS:

Terminal
brew install --cask session-manager-plugin
PartWhat it means
brew installinstall software with Homebrew
--caskthis one is a packaged app rather than a command-line formula. Casks only exist on macOS
session-manager-pluginthe plugin the AWS CLI uses for Session Manager

On Linux, and on Windows inside Ubuntu (WSL), download AWS’s Ubuntu package:

Terminal
curl -fsSL -o session-manager-plugin.deb https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb
PartWhat it means
curl -fsSLdownload, failing on errors and following redirects
-o session-manager-plugin.debsave it to this file
…/ubuntu_64bit/session-manager-plugin.debAWS’s package for Intel and AMD computers. On an ARM computer, use ubuntu_arm64 in place of ubuntu_64bit

Then install it:

Terminal
sudo dpkg -i session-manager-plugin.deb
PartWhat it means
sudo dpkg -iinstall a downloaded Ubuntu package, as the administrator
session-manager-plugin.debthe file you just downloaded
Terminal
session-manager-plugin
PartWhat it means
session-manager-pluginrun the plugin on its own, just to check it’s installed
You should see
The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.
  1. Write the Terraform for the controller

    Make a folder called quickcart-jenkins and move into it:

    Terminal
    mkdir quickcart-jenkins
    PartWhat it means
    mkdirmake a directory
    quickcart-jenkinsits name
    Terminal
    cd quickcart-jenkins
    PartWhat it means
    cdchange directory, so every later command runs in here
    quickcart-jenkinsthe folder you just made

    Create main.tf:

    main.tf
    terraform {
      required_version = ">= 1.10.0"
    
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "~> 6.0"
        }
      }
    }
    
    provider "aws" {
      region = "us-east-1"
    }
    
    data "aws_vpc" "default" {
      default = true
    }
    
    data "aws_ami" "al2023" {
      most_recent = true
      owners      = ["amazon"]
    
      filter {
        name   = "name"
        values = ["al2023-ami-2023.*-x86_64"]
      }
    }
    
    resource "aws_security_group" "jenkins" {
      name        = "quickcart-jenkins"
      description = "Jenkins controller: nothing in, everything out"
      vpc_id      = data.aws_vpc.default.id
    }
    
    resource "aws_vpc_security_group_egress_rule" "all_out" {
      security_group_id = aws_security_group.jenkins.id
      description       = "Pull images and plugins, reach Session Manager"
      ip_protocol       = "-1"
      cidr_ipv4         = "0.0.0.0/0"
    }
    
    resource "aws_iam_role" "jenkins" {
      name = "quickcart-jenkins-controller"
    
      assume_role_policy = jsonencode({
        Version = "2012-10-17"
        Statement = [{
          Effect    = "Allow"
          Principal = { Service = "ec2.amazonaws.com" }
          Action    = "sts:AssumeRole"
        }]
      })
    }
    
    resource "aws_iam_role_policy_attachment" "ssm" {
      role       = aws_iam_role.jenkins.name
      policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
    
    resource "aws_iam_instance_profile" "jenkins" {
      name = "quickcart-jenkins-controller"
      role = aws_iam_role.jenkins.name
    }
    
    resource "aws_instance" "jenkins" {
      ami                         = data.aws_ami.al2023.id
      instance_type               = "t3.medium"
      vpc_security_group_ids      = [aws_security_group.jenkins.id]
      iam_instance_profile        = aws_iam_instance_profile.jenkins.name
      associate_public_ip_address = true
      user_data_replace_on_change = true
    
      metadata_options {
        http_tokens                 = "required"
        http_put_response_hop_limit = 2
      }
    
      root_block_device {
        volume_size = 30
        volume_type = "gp3"
        encrypted   = true
      }
    
      user_data = <<-EOT
        #!/bin/bash
        set -euo pipefail
        dnf install -y docker
        systemctl enable --now docker
        mkdir -p /var/lib/jenkins-home
        chown 1000:1000 /var/lib/jenkins-home
        docker network create jenkins
        docker run -d --name jenkins --restart unless-stopped --network jenkins -p 127.0.0.1:8080:8080 -v /var/lib/jenkins-home:/var/jenkins_home jenkins/jenkins:2.568.3-lts-jdk21
      EOT
    
      tags = {
        Name    = "quickcart-jenkins"
        Project = "quickcart"
      }
    }
    
    output "instance_id" {
      description = "Use this with Session Manager"
      value       = aws_instance.jenkins.id
    }

    The settings, one block at a time

    SettingWhat it doesIf you leave it outWhen you’d change it
    required_versionRefuse to run on Terraform older than 1.10An old Terraform might fail later with a confusing errorWhen you rely on a newer feature
    required_providers → awsUse the hashicorp/aws plugin, any 6.x versionTerraform picks the newest, which may one day bring breaking changesWhen you deliberately upgrade
    provider "aws" → regionBuild everything in us-east-1Terraform uses the region from your AWS login settingsTo build somewhere else
    data "aws_vpc" "default" → default = trueLook up your account’s default VPC, which every account has in every region unless someone deleted itRequired, to find the VPCIn a real setup, use your own VPC with a private subnet
    data "aws_ami" "al2023"Look up a machine image to start from——
    most_recent = trueIf several images match, take the newestTerraform fails when more than one matchesNever. You want the latest security fixes
    owners = ["amazon"]Only accept images published by AWS itselfRequired. Without it, anyone’s image with a matching name could be pickedNever for a server like this
    filter → name, valuesMatch Amazon Linux 2023 images for 64-bit Intel and AMD processors. * stands for the date part of the nameEvery image from Amazon would matchUse arm64 in the pattern for Graviton instances, which cost less
    SettingWhat it doesIf you leave it outWhen you’d change it
    aws_security_group → nameThe firewall’s name in AWSTerraform makes up a nameNever after creation; it forces a new one
    descriptionA note for people reading the consoleAWS writes Managed by TerraformNever after creation; it forces a new one
    vpc_idWhich VPC the firewall belongs to. It must match the instance’sIt goes in the default VPC anyway, but saying so is clearerWhen you use your own VPC
    No ingress rulesNothing may connect in. Not SSH, not port 8080, nothingThis is leaving it out, on purposeWhen a load balancer needs to reach Jenkins for webhooks. Then allow only the load balancer’s security group
    aws_vpc_security_group_egress_rule → security_group_idWhich firewall this rule belongs toRequired—
    descriptionA note on the ruleNo note—
    ip_protocol = "-1"Every protocol and portRequiredNarrow it to port 443 if your network team asks
    cidr_ipv4 = "0.0.0.0/0"To anywhere. The instance must reach Docker Hub, the Jenkins plugin site and Session ManagerRequiredUse VPC endpoints and a proxy in a locked-down network
    SettingWhat it doesIf you leave it outWhen you’d change it
    aws_iam_role → nameThe role the instance will useTerraform makes up a name—
    assume_role_policyWho may use the role: only the EC2 service, on behalf of an instance. jsonencode turns the Terraform map into the JSON that IAM expectsRequiredNever for an instance role
    aws_iam_role_policy_attachment → role, policy_arnGive the role AWS’s managed policy AmazonSSMManagedInstanceCore: the permissions the SSM agent needs so Session Manager worksSession Manager can’t reach the instance, and you have no way inAdd more policies here, such as pushing to ECR, when builds need them
    aws_iam_instance_profile → name, roleThe container that attaches a role to an EC2 instance. EC2 can’t use a role directlyThe instance can’t get the roleNever
    SettingWhat it doesIf you leave it outWhen you’d change it
    amiStart from the Amazon Linux 2023 image found above. It comes with the SSM agent installedRequired—
    instance_type = "t3.medium"2 vCPUs and 4 GiB of memory. Jenkins alone wants about 2 GiBRequiredBigger if the web page gets slow with many jobs. Never to make room for builds
    vpc_security_group_idsUse the no-way-in firewallThe VPC’s default security group, which may allow more—
    iam_instance_profileAttach the roleThe instance has no AWS permissions and Session Manager can’t reach it—
    associate_public_ip_address = trueGive it a public address, so it can reach the internet from the default VPC. The security group still lets nothing inIn the default VPC it usually gets one anyway, but that depends on a subnet settingIn a private subnet with a NAT gateway, set this to false
    user_data_replace_on_change = trueIf the start-up script changes, build a new instance, because the script only ever runs on the first bootChanging the script would do nothing to the running instanceNever for a server built from its script
    metadata_options → http_tokens = "required"Only accept the newer, safer way of asking the instance for its role’s credentials, called IMDSv2Older requests are also accepted, which some attacks rely onNever
    http_put_response_hop_limit = 2Let containers on the instance reach the instance’s credentials too. Each network hop uses one, and a container adds oneWith the default of 1, a container, such as an agent, can’t use the instance roleSet it to 1 if no container should get the instance’s permissions
    root_block_device → volume_size = 30A 30 GiB disk. Build history and plugins grow over time8 GiB, which fills up quicklyBigger if you keep many builds
    volume_type = "gp3"The current general-purpose SSD type, cheaper than the older gp2Depends on the image; may be gp2Rarely
    encrypted = trueEncrypt the disk and its snapshotsDepends on your account’s defaultNever. JENKINS_HOME holds secrets
    tags → Name, ProjectName is what the console shows in the instance listA nameless instanceAdd Backup = daily when AWS Backup picks instances by tag
    output "instance_id"Print the instance ID after apply. You’ll need it to open a tunnelYou’d look it up in the console—

    The start-up script, line by line

    user_data runs once, as root, the first time the instance boots. <<-EOT and EOT mark the start and end of a block of text in Terraform.

    LineWhat it does
    #!/bin/bashRun this script with bash
    set -euo pipefailStop at the first command that fails, instead of carrying on half-set-up
    dnf install -y dockerInstall Docker with Amazon Linux’s package manager. -y answers yes
    systemctl enable --now dockerStart Docker now, and on every boot
    mkdir -p /var/lib/jenkins-homeThe folder that will be JENKINS_HOME, on the instance’s disk
    chown 1000:1000 /var/lib/jenkins-homeThe official image runs Jenkins as user number 1000. It must own the folder to write to it
    docker network create jenkinsA private Docker network, so an agent container can reach the controller by the name jenkins
    docker run -d --name jenkinsStart the controller in the background, named jenkins
    --restart unless-stoppedStart it again after a crash or a reboot
    --network jenkinsJoin the private network
    -p 127.0.0.1:8080:8080Publish Jenkins’ web port only on the instance’s own loopback address. Even with a firewall mistake, nothing outside the instance could reach it. Session Manager connects from inside
    -v /var/lib/jenkins-home:/var/jenkins_homeKeep JENKINS_HOME on the instance’s disk, so it survives the container being replaced
    jenkins/jenkins:2.568.3-lts-jdk21The official image, pinned to one LTS version. To upgrade, change this, and back up first
  2. Build it

    Terminal
    terraform init
    PartWhat it means
    terraform initdownload the AWS provider and prepare the folder
    You should see
    Terraform has been successfully initialized!
    Terminal
    terraform apply
    PartWhat it means
    terraform applyplan, show the plan, and build it once you type yes

    Type yes when asked.

    You should see
    Plan: 6 to add, 0 to change, 0 to destroy.
    
    Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
    
    Outputs:
    
    instance_id = "i-0a1b2c3d4e5f6789"

    Your instance ID will be different. Keep it; the next steps call it INSTANCE_ID. The instance needs about three minutes to install Docker and start Jenkins.

  3. Get the one-time admin password

    Open a shell on the instance in your browser. Session Manager does this through AWS, with no SSH key and no open port.

    In the AWS console: EC2Instancesquickcart-jenkinsConnect

    FieldEnter
    TabSession Manager

    Choose Connect. A terminal opens in a new browser tab.

    In the browser terminal, check Jenkins is running:

    Terminal
    sudo docker ps
    PartWhat it means
    sudorun as the administrator; Session Manager logs you in as a normal user
    docker pslist running containers
    You should see
    CONTAINER ID   IMAGE                               COMMAND                  CREATED         STATUS         PORTS                                NAMES
    3f2a9c1d7b4e   jenkins/jenkins:2.568.3-lts-jdk21   "/usr/bin/tini -- /u…"   2 minutes ago   Up 2 minutes   127.0.0.1:8080->8080/tcp, 50000/tcp   jenkins

    Your ID and times will differ. Then print the password Jenkins created on its first start:

    Terminal
    sudo cat /var/lib/jenkins-home/secrets/initialAdminPassword
    PartWhat it means
    sudo catprint a file, as the administrator
    /var/lib/jenkins-home/…JENKINS_HOME on the instance’s disk
    secrets/initialAdminPassworda one-time password, used only to unlock the setup page
    You should see
    3b9e5c07a1d24f6e8c2b0d9a7f41e635

    Yours is different. Copy it. Leave this browser tab open; you’ll use it again.

  4. Open a tunnel to Jenkins from your computer

    The console can give you a shell, but it has no button for forwarding a port to your own browser, so this step uses the AWS CLI. In a terminal on your computer:

    Terminal
    aws ssm start-session --target INSTANCE_ID --document-name AWS-StartPortForwardingSession --parameters portNumber=8080,localPortNumber=8080
    PartWhat it means
    aws ssm start-sessionstart a Session Manager session
    --target INSTANCE_IDwhich instance. Replace it with the instance_id Terraform printed
    --document-name AWS-StartPortForwardingSessionthe kind of session: forward a port, rather than open a shell
    --parameters portNumber=8080the port on the instance, where Jenkins listens
    localPortNumber=8080the port on your computer
    You should see
    Starting session with SessionId: quickcart-admin-0f1e2d3c4b5a69788
    Port 8080 opened for sessionId quickcart-admin-0f1e2d3c4b5a69788.
    Waiting for connections...

    Leave it running. Open http://localhost:8080 in your browser. Your computer passes it through the tunnel, AWS passes it to the SSM agent on the instance, and the agent connects to Jenkins on the instance’s loopback address.

  5. Finish the setup in the browser

    Go through the setup pages:

    PageDo this
    Unlock JenkinsPaste the one-time password. Choose Continue
    Customize JenkinsInstall suggested plugins. It takes a few minutes
    Create First Admin UserChoose a username and a long password. Store them in your password manager
    Instance ConfigurationLeave Jenkins URL as http://localhost:8080/. Choose Save and Finish

    Choose Start using Jenkins.

    You should see

    The Jenkins dashboard, with Welcome to Jenkins! in the middle.

  6. Take the builds off the controller

    Go to Manage Jenkins → Nodes → Built-In Node → Configure.

    FieldEnter
    Number of executors0

    Choose Save.

    You should see

    Back on the dashboard, the Build Executor Status panel on the left no longer lists any executors for the built-in node.

  7. Store a secret

    Go to Manage Jenkins → Credentials → System → Global credentials (unrestricted) → Add Credentials. Choose the kind Secret text, then:

    FieldEnter
    ScopeGlobal
    Secretnot-a-real-token-123
    IDquickcart-demo-token
    DescriptionDemo token for the Jenkins lesson

    Choose Create.

    You should see

    quickcart-demo-token in the list. The value itself is never shown again. Anyone with permission can replace it, but nobody can read it back in the web page.

  8. Create a job and watch it wait

    From the dashboard, choose New Item.

    FieldEnter
    Enter an item namehello-agent
    TypePipeline

    Choose OK. Scroll to Pipeline, leave Definition as Pipeline script, and paste this into Script:

    Pipeline script
    pipeline {
        agent { label 'linux' }
        stages {
            stage('Where am I?') {
                steps {
                    sh 'hostname'
                    sh 'whoami'
                }
            }
            stage('Use a secret') {
                steps {
                    withCredentials([string(credentialsId: 'quickcart-demo-token', variable: 'DEMO_TOKEN')]) {
                        sh 'echo "The token is $DEMO_TOKEN"'
                    }
                }
            }
        }
    }
    LineWhat it means
    agent { label 'linux' }run on any agent labelled linux
    stage('Where am I?')a named step of the pipeline. hostname and whoami show which machine and which user ran it
    withCredentials([...])fetch a credential from the store, only for the steps inside the braces
    string(credentialsId: 'quickcart-demo-token', variable: 'DEMO_TOKEN')a Secret text credential, put into an environment variable called DEMO_TOKEN
    sh 'echo "The token is $DEMO_TOKEN"'print it, to see what Jenkins does with it. The single quotes matter; see the mistake below

    The Jenkinsfile explains every part of a pipeline. Choose Save, then Build Now.

    You should see

    Under Builds, build #1 waits with a clock icon. Hold the pointer over it, or open it: Jenkins says there are no nodes with the label linux. There is no agent yet, and the controller now refuses to run builds. That’s correct.

  9. Connect an agent

    Go to Manage Jenkins → Nodes → New Node.

    FieldEnter
    Node nameagent-1
    TypePermanent Agent

    Choose Create, then:

    FieldEnter
    Number of executors2
    Remote root directory/home/jenkins/agent
    Labelslinux
    UsageUse this node as much as possible
    Launch methodLaunch agent by connecting it to the controller

    Choose Save, then open agent-1 in the list. Jenkins shows the commands to start the agent. Copy the long secret from them: the 64-character value after -secret. The next command calls it AGENT_SECRET.

    In a real setup, the agent is a separate EC2 instance. To keep this lab to one server, you’ll run it as a second container on the same instance. Go back to the Session Manager browser tab and run:

    Terminal
    sudo docker run -d --name agent-1 --init --network jenkins -e JENKINS_URL=http://jenkins:8080/ -e JENKINS_AGENT_NAME=agent-1 -e JENKINS_SECRET=AGENT_SECRET -e JENKINS_WEB_SOCKET=true -e JENKINS_AGENT_WORKDIR=/home/jenkins/agent jenkins/inbound-agent
    PartWhat it means
    sudo docker run -dstart a container in the background
    --name agent-1call the container agent-1
    --initadd a tiny process that tidies up finished processes. Recommended for agent containers
    --network jenkinsjoin the same private network as the controller
    -e JENKINS_URL=http://jenkins:8080/where the controller is. On this network, jenkins is the controller container’s name
    -e JENKINS_AGENT_NAME=agent-1which node in Jenkins this agent is
    -e JENKINS_SECRET=AGENT_SECRETproves it’s allowed to be that node. Replace it with the secret you copied
    -e JENKINS_WEB_SOCKET=trueconnect over the same port as the web page, so no extra port is needed
    -e JENKINS_AGENT_WORKDIR=/home/jenkins/agentwhere the agent keeps its files
    jenkins/inbound-agentthe official image for an agent that connects to its controller
    You should see

    Within a few seconds, build #1 starts by itself. Its Console Output says Running on agent-1, then prints the agent container’s hostname, the user jenkins, and then:

    The console output includes
    The token is ****

    Jenkins replaced the secret with **** in the log, because it knows the value it handed over.

  10. Back up JENKINS_HOME

    JENKINS_HOME is on the instance’s disk, so a snapshot of that disk is a complete backup.

    In the AWS console: EC2SnapshotsCreate snapshot

    FieldEnter
    Resource typeInstance
    Instance IDthe quickcart-jenkins instance
    Descriptionquickcart-jenkins backup

    Choose Create snapshot.

    You should see

    A snapshot in the list with Snapshot status: Pending, then Completed after a few minutes.

    To restore it, you’d create a volume from this snapshot and start a new instance with it. For a real controller, don’t take snapshots by hand: in AWS Backup, create a backup plan with a daily rule that keeps 30 days, and assign it to resources with a tag such as Backup = daily. And once a quarter, restore one and check Jenkins starts from it.

  11. Clean-up step: Clean up

    In the terminal running the tunnel, press Ctrl+C to close it.

    Delete the snapshot first. Terraform didn’t create it, so destroy won’t remove it.

    In the AWS console: EC2Snapshotsquickcart-jenkins backupActionsDelete snapshot

    Then, in the quickcart-jenkins folder:

    Terminal
    terraform destroy
    PartWhat it means
    terraform destroyremove everything in the state, once you type yes
    You should see
    Destroy complete! Resources: 6 destroyed.
    You should see

    In EC2 → Instances, quickcart-jenkins shows Terminated. It disappears from the list after about an hour.

Part 5: Check yourself

QuickCart's Jenkins has 2 executors on the controller and no agents. A large image build makes the web UI freeze for everyone. What is the right fix?

Why does a build that runs on the controller put secrets at risk?

Why does the Terraform in this lesson give the Jenkins instance no inbound security group rules at all?

The Jenkins instance is terminated by accident. Which backup gets QuickCart back fastest, with its jobs, build history and credentials?

A pipeline prints a secret with sh "echo ${TOKEN}" in double quotes. What's wrong?

Part 6: Cheat sheet

On one card
ThingRemember
ControllerSchedules, stores, serves the UI. 0 executors
AgentRuns builds. Disposable. Picked by label
Agents on demandAmazon EC2 plugin or Kubernetes plugin
Reaching the controllerSession Manager: aws ssm start-session --target INSTANCE_ID --document-name AWS-StartPortForwardingSession --parameters portNumber=8080,localPortNumber=8080
AWS permissionsInstance role, or withAWS(role: …). Never stored keys
SecretsCredentials store, used with withCredentials, in single quotes
Credential scopeGlobal for pipelines, System for Jenkins itself, folders to fence them off
JENKINS_HOMEEverything. secrets/ decrypts credentials.xml
BackupsNightly EBS snapshots with AWS Backup; ThinBackup for settings; restore one every quarter
Setup as codejenkins.yaml with the Configuration as Code plugin; plugins.txt in your own image
SecurityNot on the internet, no anonymous read, update monthly, short admin list

Finished the build and the quiz? Mark it done to fill in its stop on the route map.