The Jenkinsfile

A Jenkinsfile fixes this by putting the pipeline in the repository, next to the code.

A Jenkinsfile is a text file called Jenkinsfile, at the root of the repository. It describes the pipeline: what runs, in what order, on which agent, and what happens when something fails. Jenkins reads it from the same commit it’s building.

That one change gives the pipeline everything code already has:

There are two ways to write one. Declarative starts with pipeline { and has a fixed structure that Jenkins checks before it runs anything. Scripted starts with node { and is plain Groovy code. This course uses declarative: it’s easier to read, and mistakes are caught before a build starts. When you need a little real code, a script { } block inside a step lets you write some.

The smallest useful Jenkinsfile

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                sh './test.sh'
            }
        }
    }
}

agent: where it runs

agent can also go inside a single stage. Then that stage runs there, whatever the top says.

stages, stage and steps

stages holds one or more stage blocks. Each stage has steps. Common steps:

The folder a build runs in on the agent is called the workspace. Jenkins checks your code out into it before the first stage.

environment: variables for every step

pipeline {
    agent any
    environment {
        APP_NAME = 'quickcart-orders'
        REGION   = 'us-east-1'
    }
    stages {
        stage('Show') {
            steps {
                sh 'echo "Building $APP_NAME for $REGION"'
            }
        }
    }
}

Every name in environment becomes an environment variable for every step. Put it inside a stage instead, and it applies only there. Jenkins adds its own too:

Quotes: who fills in the $

This trips everyone up once. A Jenkinsfile is Groovy, and Groovy and the shell both use $ for variables.

parameters: questions asked when the build starts

pipeline {
    agent any
    parameters {
        choice(name: 'TARGET', choices: ['dev', 'stg', 'prod'], description: 'Where to deploy')
        booleanParam(name: 'RUN_SLOW_TESTS', defaultValue: false, description: 'Also run the slow tests')
        string(name: 'RELEASE_NOTE', defaultValue: 'Routine release', description: 'One line about this release')
    }
    stages {
        stage('Show') {
            steps {
                echo "Target: ${params.TARGET}, slow tests: ${params.RUN_SLOW_TESTS}"
            }
        }
    }
}

The job’s button changes from Build Now to Build with Parameters, but only after Jenkins has read the Jenkinsfile once. You’ll see that in the lab.

options: rules for the whole run

options can also go inside a stage. A timeout there limits only that stage, and if the stage has an input, it limits the wait too.

when: should this stage run?

A stage with a when block runs only if its condition is true. If it’s false, the stage is skipped, which isn’t a failure.

branch, changeRequest and tag need a multibranch job, because that’s what tells Jenkins which branch or pull request it’s building. In a plain Pipeline job they’re always false. Continuous integration sets up a multibranch job.

Two settings change when the check happens. beforeAgent true checks before an agent is found, so a skipped stage never waits for one. beforeInput true checks before asking for approval, so nobody is asked about a stage that would be skipped anyway.

input: wait for a person

pipeline {
    agent none
    stages {
        stage('Build') {
            agent any
            steps {
                echo 'Building'
            }
        }
        stage('Deploy to prod') {
            options {
                timeout(time: 30, unit: 'MINUTES')
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
                submitter 'release-managers'
            }
            agent any
            steps {
                echo 'Deploying'
            }
        }
    }
}

Written this way, a build waiting for approval over a weekend holds no executor. With agent any at the top, it would hold one the whole time.

parallel: stages side by side

pipeline {
    agent any
    stages {
        stage('Checks') {
            failFast true
            parallel {
                stage('Unit tests') {
                    steps {
                        sh 'echo "Unit tests passed"'
                    }
                }
                stage('Lint') {
                    steps {
                        sh 'echo "Lint passed"'
                    }
                }
            }
        }
    }
}

parallel runs the stages inside it at the same time, each with its own executor if it has its own agent. failFast true stops the others as soon as one fails, so you don’t wait for work that no longer matters.

post: what happens afterwards

post runs after the stages, whatever happened. Each block inside runs only if its condition matches.

They run in a fixed order: always first, then the ones that match the result, then cleanup. post can also go inside a stage, for things that belong to that stage.

Below is a short Jenkinsfile. Pick what the build is for, then step through it. Try a pull request, then main, then make the tests fail.

pipeline {
  agent any
  stages {
    stage('Test') {
      steps { sh './test.sh' }
    }
    stage('Plan') {
      when { changeRequest() }
      steps { sh 'terraform plan' }
    }
    stage('Build image') {
      when { branch 'main' }
      steps { sh './build-image.sh' }
    }
    stage('Deploy to prod') {
      when { branch 'main' }
      input { message 'Deploy to prod?' }
      steps { sh './deploy.sh prod' }
    }
  }
  post {
    always  { junit 'reports/*.xml' }
    success { echo 'Shipped' }
    failure { echo 'Tell the team' }
    cleanup { deleteDir() }
  }
}

What to notice: the same file gives three different pipelines. A feature branch only tests. A pull request also plans. Only main builds and deploys. And when a stage fails, the stages after it are skipped, but post still runs.

Free — Jenkins runs on your computer. Nothing in AWS is used.

You’ll run Jenkins on your computer, create a pipeline, and see parameters, when, parallel, input, a timeout and post do their jobs. About 40 minutes. The next three lessons use this same Jenkins, so keep it when you finish.

On your computer, Jenkins’ controller and its only agent are the same machine, and builds run on the controller. That’s fine for learning on your own. Jenkins from zero explains why a shared Jenkins must never work this way.

1. Install and start Jenkins

   TerminalCopy

   brew install jenkins-lts

   Part	What it means
   brew install	install software with Homebrew
   jenkins-lts	the long-term support version of Jenkins. Homebrew installs the Java it needs too

   Start it in a terminal window of its own:

   TerminalCopy

   jenkins-lts --httpListenAddress=127.0.0.1 --httpPort=8080

   Part	What it means
   jenkins-lts	start Jenkins in this terminal. It keeps running until you press Ctrl+C
   --httpListenAddress=127.0.0.1	only accept connections from this computer, not from other machines on your network
   --httpPort=8080	the port for the web page

   Among many lines of log, the first start prints:

   You should see

   *************************************************************
   *************************************************************
   *************************************************************
   
   Jenkins initial setup is required. An admin user has been created and a password generated.
   Please use the following password to proceed to installation:
   
   3b9e5c07a1d24f6e8c2b0d9a7f41e635
   
   This may also be found at: /home/you/.jenkins/secrets/initialAdminPassword
   
   *************************************************************
   *************************************************************
   *************************************************************

   Then, when it’s ready:

   You should see

   Jenkins is fully up and running

   Your password and home folder are different. Copy the password. Leave this terminal open; closing it stops Jenkins. Jenkins keeps everything in ~/.jenkins, the JENKINS_HOME on your computer.

2. Finish the setup

   Open http://localhost:8080 and go through the setup pages:

   Page	Do this
   Unlock Jenkins	Paste the password. Choose Continue
   Customize Jenkins	Install suggested plugins. They include Pipeline, Git, Credentials Binding and Timestamper
   Create First Admin User	A username and a password you’ll remember
   Instance Configuration	Leave http://localhost:8080/. Choose Save and Finish, then Start using Jenkins

3. Create the pipeline

   Choose New Item.

   Field	Enter
   Enter an item name	quickcart-pipeline
   Type	Pipeline

   Choose OK, scroll to Pipeline, leave Definition as Pipeline script, and paste this into Script:

   JenkinsfileCopy

   pipeline {
       agent any
   
       parameters {
           choice(name: 'TARGET', choices: ['dev', 'stg', 'prod'], description: 'Where to deploy')
           booleanParam(name: 'RUN_SLOW_TESTS', defaultValue: false, description: 'Also run the slow tests')
           string(name: 'RELEASE_NOTE', defaultValue: 'Routine release', description: 'One line about this release')
       }
   
       environment {
           APP_NAME = 'quickcart-orders'
           VERSION  = "1.0.${env.BUILD_NUMBER}"
       }
   
       options {
           timeout(time: 15, unit: 'MINUTES')
           timestamps()
           disableConcurrentBuilds()
           buildDiscarder(logRotator(numToKeepStr: '20'))
       }
   
       stages {
           stage('Build') {
               steps {
                   echo "Building ${env.APP_NAME} version ${env.VERSION}"
                   writeFile file: 'out/version.txt', text: "${env.APP_NAME} ${env.VERSION}\n"
               }
           }
   
           stage('Tests') {
               parallel {
                   stage('Unit tests') {
                       steps {
                           sh 'echo "Unit tests passed for $APP_NAME"'
                       }
                   }
                   stage('Slow tests') {
                       when {
                           expression { params.RUN_SLOW_TESTS }
                       }
                       steps {
                           sh 'sleep 5'
                           echo 'Slow tests passed'
                       }
                   }
               }
           }
   
           stage('Only on main') {
               when {
                   branch 'main'
               }
               steps {
                   echo 'This only runs in a multibranch job, on main'
               }
           }
   
           stage('Deploy') {
               when {
                   expression { params.TARGET != 'prod' }
               }
               steps {
                   echo "Deploying ${env.VERSION} to ${params.TARGET}"
               }
           }
   
           stage('Deploy to prod') {
               when {
                   beforeInput true
                   expression { params.TARGET == 'prod' }
               }
               options {
                   timeout(time: 5, unit: 'MINUTES')
               }
               input {
                   message 'Deploy to production?'
                   ok 'Deploy'
               }
               steps {
                   echo "Deploying ${env.VERSION} to prod: ${params.RELEASE_NOTE}"
               }
           }
       }
   
       post {
           always {
               archiveArtifacts artifacts: 'out/version.txt'
           }
           success {
               echo 'The pipeline passed'
           }
           failure {
               echo 'The pipeline failed: tell the team'
           }
           cleanup {
               deleteDir()
           }
       }
   }

   Every directive is explained in the idea. The parts that are new here:

   Line	What it means
   VERSION = "1.0.${env.BUILD_NUMBER}"	Double quotes, so Groovy fills in the build number: build 3 is version 1.0.3
   writeFile file: 'out/version.txt', …	Make a small file, standing in for what a real build produces
   stage('Tests') → parallel	Unit tests and slow tests run side by side
   expression { params.RUN_SLOW_TESTS }	The slow tests run only if the box was ticked
   stage('Only on main')	Here to show that branch is always false in a plain Pipeline job
   expression { params.TARGET != 'prod' }	dev and stg deploy straight away
   beforeInput true	Check the condition before asking, so dev builds never ask for approval
   Stage options → timeout(time: 5, …)	Nobody answers in 5 minutes, the build is aborted
   archiveArtifacts in post → always	Keep version.txt, even if a later stage failed
   deleteDir() in post → cleanup	Empty the workspace at the very end

   Choose Save.

4. Run it once, and meet the parameters

   Choose Build Now. When it finishes, open build #1 and choose Console Output. Among the lines you’ll see:

   The console output includes

   Building quickcart-orders version 1.0.1
   Unit tests passed for quickcart-orders
   Stage "Slow tests" skipped due to when conditional
   Stage "Only on main" skipped due to when conditional
   Deploying 1.0.1 to dev
   The pipeline passed
   Finished: SUCCESS

   Every line also starts with a time, from timestamps(). Go back to the job page.

   You should see

   The button now says Build with Parameters. Before build #1, Jenkins didn’t know the Jenkinsfile had parameters, because it hadn’t read it yet.

5. Deploy to prod, with approval

   Choose Build with Parameters:

   Field	Enter
   TARGET	prod
   RUN_SLOW_TESTS	ticked
   RELEASE_NOTE	Coupons at checkout

   Choose Build. Open the running build’s Console Output.

   You should see

   The slow tests run next to the unit tests this time, and Deploy is skipped. Then the build stops and waits, with the question Deploy to production? and two links: Deploy and Abort.

   Choose Deploy.

   The console output includes

   Deploying 1.0.2 to prod: Coupons at checkout
   The pipeline passed
   Finished: SUCCESS

   Run it once more with prod, and this time don’t answer. After 5 minutes:

   The console output ends with

   Cancelling nested steps due to timeout
   Finished: ABORTED

   Nobody forgot a build that waits forever: the timeout ended it for you.

6. Make a stage fail

   Choose Configure, and change the Unit tests step so it fails:

   JenkinsfileCopy

   pipeline {
       agent any
       stages {
           stage('Unit tests') {
               steps {
                   sh 'echo "1 test failed" && exit 1'
               }
           }
           stage('Deploy') {
               steps {
                   echo 'Deploying'
               }
           }
       }
       post {
           success {
               echo 'The pipeline passed'
           }
           failure {
               echo 'The pipeline failed: tell the team'
           }
       }
   }

   This shorter pipeline replaces the long one, so the failure is easy to see.

   Part	What it means
   echo "1 test failed"	print a line, as a test tool would
   &&	run the next command only if this one succeeded
   exit 1	end the shell with code 1, which means failure. Jenkins fails the stage

   Choose Save, then Build Now.

   The console output includes

   1 test failed
   Stage "Deploy" skipped due to earlier failure(s)
   The pipeline failed: tell the team
   ERROR: script returned exit code 1
   Finished: FAILURE

   Deploy never ran. failure did; success didn’t.

7. Clean-up step: Clean up

   Keep Jenkins and ~/.jenkins: the next three lessons use them. To stop Jenkins for now, press Ctrl+C in its terminal. Start it again later with the same jenkins-lts command; your jobs and settings are still there.

   When you’ve finished the course and want it gone:

   TerminalCopy

   brew uninstall jenkins-lts

   Part	What it means
   brew uninstall	remove software Homebrew installed
   jenkins-lts	Jenkins. Your ~/.jenkins folder stays until you delete it

   Then delete the .jenkins folder in your home folder, which holds every job and setting:

   TerminalCopy

   rm -rf ~/.jenkins

   Part	What it means
   rm	remove files
   -rf	the whole folder and everything inside it, without asking about each file. There’s no undo, so check the name before pressing Enter
   ~/.jenkins	the .jenkins folder in your home folder. The dot at the start makes it hidden