Utilizing Terraform with GitHub Actions: A Step-by-Step Guide
Summary
The article provides a comprehensive guide on utilizing Terraform with GitHub Actions to streamline development processes. It commences with an introduction to Terraform and GitHub Actions, followed by a detailed explanation of the necessary steps to set up the integration. The guide covers topics such as creating a GitHub repository, developing a Terraform configuration file, configuring Github Actions, and setting up the workflow to use Terraform. Each step is further illustrated with relevant code snippets. Additionally, the article contains valuable practical tips for working with Terraform and GitHub Actions, including managing Terraform state. By following this guide, readers can efficiently utilize Terraform in GitHub Actions, thereby enhancing their workflow and productivity.
What is Terraform?
Terraform is an open-source infrastructure as code (IaC) tool developed by HashiCorp that enables users to define and provision their infrastructure through code. Terraform uses a declarative language to describe infrastructure resources and their dependencies, allowing for the creation, modification, and destruction of resources in a safe and predictable manner. With Terraform, users can manage infrastructure resources across various cloud providers and on-premises data centers using a unified workflow. Terraform’s ability to manage infrastructure as code can greatly simplify the process of infrastructure provisioning and management, allowing for more efficient and consistent deployments.
What is GitHub Action?
GitHub Actions is a platform that enables developers to automate software workflows and streamline their software development processes. It provides a framework for automating tasks such as building, testing, and deploying software, and can integrate with various tools and services. GitHub Actions is based on a YAML file format, and allows developers to define workflows that specify a series of steps to be executed in response to various events, such as code commits or pull requests. Workflows can be triggered automatically or manually, and can be run on various platforms and environments. With GitHub Actions, developers can create custom workflows that fit their specific needs and automate many of the repetitive tasks involved in software development, ultimately improving their productivity and efficiency.
Let’s Begin
Let us assume that you have a Git repository that contains your service code, and you intend to develop an infrastructure for it and deploy it to a particular environment. Therefore, the first step is to prepare your infrastructure code using Terraform.
First, we create a directory within our Git repository to contain our Terraform code.
mkdir terraformNext, you need to write your Terraform code and place it inside the terraform folder. Here, we use a simple Terraform code for the purpose of learning, but you can place any infrastructure code that you desire.
So, let’s create a simple Terraform file called main.tf.
touch terraform/main.tfNext, place the following simple infrastructure code inside the main.tf file.
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
}This code sets up an AWS provider in the US East 1 region and creates an EC2 instance with the specified Amazon Machine Image (AMI) and instance type. This is just an example and not meant for production use.
Now that you have written your infrastructure code, it is time to use automation to apply your code each time you make changes and make it easy to use. For this automation, we can use GitHub Actions.
Let’s create the GitHub workflow file for applying our Terraform code using the following commands inside the root folder of our Git repository.
mkdir -p .github/workflows
touch .github/workflows/terraform.ymlNext, place the following content inside your terraform.yml file.
name: Terraform
on:
pull_request:
branches:
- master
push:
branches:
- master
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
jobs:
terraform:
name: Plan / Apply
runs-on: ubuntu-20.04
defaults:
run:
working-directory: terraform
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Use Terraform 1.3.7
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.3.7
- name: Terraform Format
id: fmt
run: terraform fmt -check
continue-on-error: true
- name: Terraform Init
id: init
run: terraform init
continue-on-error: true
- name: Terraform Validate
id: validate
run: terraform validate -no-color
continue-on-error: true
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: terraform plan -no-color -input=false
continue-on-error: true
- name: Pull Request Comment
uses: actions/github-script@v6
if: github.event_name == 'pull_request'
env:
PLAN: "${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
})
const botComment = comments.find(comment => {
return comment.user.type === 'Bot' && comment.body.includes('Terraform Format and Style')
})
const output = `#### Terraform Format and Style 🖌 \`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️ \`${{ steps.init.outcome }}\`
#### Terraform Validation 🤖 \`${{ steps.validate.outcome }}\`
<details><summary>Validation Output</summary>
\`\`\`\n
${{ steps.validate.outputs.stdout }}
\`\`\`
</details>
#### Terraform Plan 📖 \`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`terraform\n
${process.env.PLAN}
\`\`\`
</details>`;
if (botComment) {
github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: output
})
} else {
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
}
- name: Terraform Status
if: steps.plan.outcome == 'failure' || steps.validate.outcome == 'failure' || steps.init.outcome == 'failure' || steps.fmt.outcome == 'failure'
run: exit 1
- name: Terraform Apply
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
run: terraform apply -auto-approve -input=falseNow, let’s explain each part of the workflow file.
name: Terraform
on:
pull_request:
branches:
- master
push:
branches:
- masterThis code defines the name of the workflow as “Terraform”. The on section specifies when the workflow should be triggered. In this case, it is triggered when there is a pull request or a push on the master branch of the repository.
The pull_request and push events are specified under on. This means that the workflow will be triggered when either a pull request or push event occurs. The branches under each event specify which branches the workflow should run on. In this case, the workflow will only run on the master branch.
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}The code snippet is defining environment variables for the GitHub Actions workflow. In this case, the environment variables are named AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. The values for these variables are being accessed from secrets stored in the GitHub repository. The secrets object is a secure way of storing sensitive information, such as access keys, so that it can be used in the workflow without being exposed in the code.
The values for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are being accessed using the ${{ secrets.<SECRET_NAME> }} syntax, where <SECRET_NAME> is the name of the secret in the repository. These environment variables are commonly used to authenticate and access resources in Amazon Web Services (AWS).
The Terraform AWS provider requires the AWS access key and secret access key to authenticate with the AWS account. These credentials are used to authorize the Terraform AWS provider to manage the resources on the user’s behalf. The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are environment variables that store the access key and secret access key respectively. When Terraform runs the AWS provider, it reads these variables to authenticate with the AWS account.
The environment variables can be set either through the system environment variables or through Terraform’s input variables. However, it is not recommended to store the access key and secret access key directly in the Terraform code, as this can be a security risk. Instead, it is recommended to store them as encrypted secrets in a secure location and retrieve them using Terraform’s input variables or as environment variables during runtime.
jobs:
terraform:
name: Plan / Apply
runs-on: ubuntu-20.04
defaults:
run:
working-directory: terraform
permissions:
contents: read
pull-requests: writeThe code block represents the jobs section of the GitHub Actions workflow file and defines a job named terraform that will run on an Ubuntu 20.04 machine.
The defaults section specifies that all subsequent run commands should be executed within the terraform directory.
The permissions section grants read access to the contents of the repository and write access to pull requests to the job.
The job itself is named “Plan / Apply”, indicating that it will perform both a Terraform plan and apply operation. The specific commands that will be executed to perform these operations are not included in this code block.
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Use Terraform 1.3.7
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.3.7
- name: Terraform Format
id: fmt
run: terraform fmt -check
continue-on-error: true
- name: Terraform Init
id: init
run: terraform init
continue-on-error: true
- name: Terraform Validate
id: validate
run: terraform validate -no-color
continue-on-error: true
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: terraform plan -no-color -input=false
continue-on-error: trueThis code specifies the sequence of steps to be executed in the Terraform job. Each step is executed sequentially and has a name and unique identifier assigned to it.
The first step is to check out the code from the repository using the actions/checkout@v2 action.
The second step sets up Terraform version 1.3.7 using the hashicorp/setup-terraform@v2 action.
The third step formats the Terraform code and checks for any errors using the terraform fmt command. The continue-on-error option is set to true to ensure that the workflow continues even if there are errors in the formatting.
The fourth step initializes the Terraform backend and downloads any required providers using the terraform init command. The continue-on-error option is set to true to ensure that the workflow continues even if there are errors in the initialization.
The fifth step validates the Terraform code using the terraform validate command. The continue-on-error option is set to true to ensure that the workflow continues even if there are errors in the validation.
The sixth step creates a Terraform plan using the terraform plan command. This step is only executed if the event name is a pull request. The continue-on-error option is set to true to ensure that the workflow continues even if there are errors in creating the plan.
- name: Pull Request Comment
uses: actions/github-script@v6
if: github.event_name == 'pull_request'
env:
PLAN: "${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
})
const botComment = comments.find(comment => {
return comment.user.type === 'Bot' && comment.body.includes('Terraform Format and Style')
})
const output = `#### Terraform Format and Style 🖌 \`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️ \`${{ steps.init.outcome }}\`
#### Terraform Validation 🤖 \`${{ steps.validate.outcome }}\`
<details><summary>Validation Output</summary>
\`\`\`\n
${{ steps.validate.outputs.stdout }}
\`\`\`
</details>
#### Terraform Plan 📖 \`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`terraform\n
${process.env.PLAN}
\`\`\`
</details>`;
if (botComment) {
github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: output
})
} else {
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
}This code block is responsible for creating a comment on a pull request when the workflow is triggered by a pull request event.
The if attribute specifies that this step should only run if the GitHub event is a pull request.
The “env” attribute sets an environment variable called PLAN to the value of ${{ steps.plan.outputs.stdout }}, which is the output of the “Terraform Plan” step.
The script attribute contains JavaScript code that creates a comment on the pull request.
The code first uses the GitHub REST API to list all comments on the pull request and finds the comment made by the Terraform bot. It then constructs a message containing the results of various Terraform commands, including format, initialization, validation, and plan.
Finally, the code checks whether a comment by the bot already exists. If so, it updates the existing comment with a new message. Otherwise, it creates a new comment.
- name: Terraform Status
if: steps.plan.outcome == 'failure' || steps.validate.outcome == 'failure' || steps.init.outcome == 'failure' || steps.fmt.outcome == 'failure'
run: exit 1
- name: Terraform Apply
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
run: terraform apply -auto-approve -input=falseThe above code consists of two steps in the Terraform job of the GitHub Actions workflow.
The first step is named “Terraform Status” and has a conditional statement that checks if any of the previous steps (Terraform Plan, Terraform Validation, Terraform Init, Terraform Format) have failed. If any of the previous steps have failed, this step will also fail and exit with a status code of 1, indicating a failure.
The second step is named “Terraform Apply” and has a conditional statement that checks if the current Git branch is the master branch and if the event that triggered the workflow is a push event. If both conditions are met, this step will run the command terraform apply -auto-approve -input=false, which will automatically apply any changes to the infrastructure.
Together, these two steps ensure that any failed steps are caught and that changes are only automatically applied to the infrastructure when pushed to the master branch.
What’s Next?
Great job! Now that you have both the Terraform and workflow code, you can push your changes to GitHub and observe the results.
What is Terraform State?
Terraform state is a snapshot of the resources and their configurations that Terraform manages in a particular infrastructure. It represents the current state of the infrastructure that Terraform is aware of, and it is stored locally on the Terraform user’s machine or remotely in a backend. Terraform uses the state to plan and apply changes to the infrastructure. The state is critical to Terraform’s functionality, as it helps Terraform determine the actions required to achieve the desired infrastructure configuration and ensure that the infrastructure is in the desired state.
Now that you are using GitHub Actions to apply your Terraform code, you may have observed that after each commit to the master branch, Terraform applies the code and creates a new AWS EC2 instance. This happens because you have not stored your Terraform state file in a proper manner, which would allow Terraform to use it each time it tries to apply your code.
When you run terraform apply or terraform destroy, Terraform creates or modifies infrastructure resources in your cloud provider based on the code you've written in your Terraform configuration files. It needs to keep track of what it has created, what changes have been made, and what needs to be updated or destroyed if changes are made to the configuration.
Terraform keeps track of all this information in a file called the “state file”. The state file is a JSON file that records the attributes and metadata of each resource that Terraform has created. It’s very important to keep this file up-to-date, as it’s the source of truth for Terraform’s knowledge of the state of your infrastructure.
By default, Terraform stores the state file locally in a file called terraform.tfstate. This is not an ideal solution because you might work with a team or multiple machines, and having the state file locally can create conflicts between different contributors.
To solve this problem, Terraform supports “remote state” backends, which store the state file remotely on a central storage system that is accessible to all members of the team. This can be achieved by using remote backends like Amazon S3, Azure Blob Storage, Google Cloud Storage, or HashiCorp’s own Terraform Cloud.
By using remote state, you can ensure that all members of the team are working with the same version of the state file, reducing the risk of conflicts and errors. It also enables better collaboration between team members, who can easily see what changes have been made and who made them.
The following article can help you set up a remote backend for your Terraform using AWS S3.
Conclusion
Combining Terraform with GitHub Actions can streamline the infrastructure deployment process. By following the steps outlined in this guide, you can leverage the power of both tools to automate infrastructure deployment and management, increasing efficiency and reducing the potential for human error.
