Build a CI/CD Pipeline for a Serverless Application

In the previous tutorial, we successfully deployed a Jenkins cluster using the powerful combination of Terraform and Packer. In this 2nd part, we’re building a CI/CD pipeline using the 'Pipeline as Code' approach, with our sights set on Lambda-based applications.

Before we get our hands dirty with code, let's take a moment to understand the bigger picture. The diagram below is a summary of the architecture we're aiming to build by the end of this tutorial:

Whenever you update your Lambda function's source code, a pipeline will be automatically triggered The Jenkins master, in turn, will coordinate a build on one of the accessible Jenkins workers. This worker is responsible for carrying out the steps outlined in the Jenkinsfile, which is located in the root directory of the function's Git repository. Subsequently, a deployment package will be built and saved in a remote S3 bucket. In the final stage of this pipeline, the Lambda function will be updated with the most recent updates.

You can find all the source code used in this tutorial on GitHub.

Deploying a Lambda-based Application

Before delving into the details of the CI/CD pipeline, we'll first write a basic Lambda function in Go. This function will return the active runtime environment as determined by environment variable. The code for the function main.go handler is provided as follows:

Next, create a GitHub repository to host the source code of the Lambda function. Following that, with your favorite Infrastructure as Code (IaC) tool provision a new AWS Lambda function, using the above code. Complete the setup by creating three distinct aliases: "sandbox", "staging", and "production".

Configuring a Multi-Branch Pipeline

Navigate to the Jenkins dashboard and proceed to create a new multi-branch pipeline job. Configure it with the details of your Git repository and set the discovery behavior to recognize all branches within the repository.

Jenkins will scan the GitHub repository, looking for branches with a "Jenkinsfile" in the root repository. So far, there are none, and we can check that by clicking the Scan Repository Log button from the left sidebar.

Create three main branches: develop, preprod, and master branches to help organize the code and isolate the under-development code from the one running in production. The GitHub repository should look like follows:

Create a Jenkinsfile in the root repository and copy the following scripted pipeline code and paste it into your empty Jenkinsfile:

1. Checkout: This step fetches the latest source code changes from your GitHub repository.
2. Build: This stage builds a Docker image, which compiles the Go code and builds a binary. Subsequently, the "docker cp" command is used to copy the binary to the local worker machine, where a deployment package (in the form of a zip file) is created.

Instead of hardcoding the zip file name, we can use the Git commit ID for the deployment package to give a meaningful and significant name for each release and be able to roll back to a specific commit if things go wrong.

The "Dockerfile" used to build the Go binary is provided in Dockerfile.build file. It uses the official "golang:1:20" as a base image, installs any missing Go dependencies, and builds a binary compiled specifically for the Linux operating system running on x64 processor architecture.

Following these steps, push the changes to the develop branch of the application repository. Afterward, return to the Jenkins dashboard and initiate the build. At this point, the develop pipeline will be recognized and the pipeline will be triggered as detailed below:

Up until now, we've built the pipeline manually by clicking the Build Now button. Although this method is functional, it isn't the most convenient because it requires you to remember to navigate to the Jenkins dashboard and start the build after committing to the repository. A more streamlined solution is to trigger the jobs via a push event by setting up a webhook on the GitHub repository.

To do this, go to the GitHub repository and select the Settings tab. In the left-hand menu, click on Webhooks, then hit ton, bringing up the dialog shown below:

• The payload URL should follow this template: JENKINS_URL/multibranch-webhook-trigger/invoke?token=NAME_OF_PROJECT
• Set the content type to application/json.
• Choose push event as the trigger and leave the Secret field blank.

Leave the rest of the options at their default. A test payload should be sent to Jenkins. With these Github updates done, if you push some changes to the Git repository, a new event should get kicked off automatically.

Updating Lambda function code

To complete the CI/CD pipeline, we'll add the following stages to the Jenkinsfile:

1. Push: This stage takes the generated zip file and uploads it to an S3 bucket.
2. Deploy: This stage utilizes the AWS Lambda CLI to execute the update-function-code command. This updates the function code with the zip file stored in the S3 bucket from the previous step. Additionally, it publishes a new version from the "$latest" version and then associates the alias corresponding to the current Git branch (with master branch linked to production alias, preprod branch to staging alias, and develop branch to sandbox alias) to the newly deployed version.

Push the changes to the develop branch. The function code will be updated, a new version will be created, and the sandbox alias will point to the newest published version, as you can see:

To deploy the application to the staging environment, you'll need to create a pull request to merge the develop branch into the preprod branch. Jenkins will post the build status of the develop job on this pull request. Once you've verified everything is in order, proceed to merge develop into preprod:

Once the PR is merged, a new build will be triggered on the preprod branch. At the end of the CI/CD pipeline, the staging alias will point to the newly deployed version:

Follow the same workflow to deploy to the production environment. To enhance control over deployment, consider incorporating a measure that requests developer authorization prior to the actual production deployment. This can be achieved by using the Jenkins Input Step plugin.

Regardless if you are a Developer, DevOps, or Cloud engineer. Dealing with the cloud can be tough at times, especially on your own. If you are using Tailwarden or Komiser and want to share your thoughts doubts and insights with other cloud practitioners feel free to join our Tailwarden discord server. Where you will find tips, community calls, and much more.