Variable traceability using Terraform Cloud

Recently I have had the opportunity to implement environment based Terrafrom workspaces for a real world client. Whenever you have to expand a basic solution to the real world it highlights unforeseen issues. The main issue is one many larger companies will have, and that is traceability. In this article I will cover the issue of traceability, along with a couple of solutions to resolve it when using this deployment method.

Traceability

When developing code maintaining traceability is integral to tracking changes that were made. By tracking these changes we can quickly troubleshoot and revert anything that caused issue. Using the Terraform Cloud environment based workspaces method described in my previous article we are granted this traceability for the code itself, however using the Terraform Cloud UI to handle our variables means we cannot trace when these have been changed, as well as having to grant permissions to users to access them. This method works well for small teams and projects, but once we start defining and deploying a large amount of resources with a large amount of variables it starts to get out of hand.

Whilst implementing this method for a client it became apparent the UI variables approach was not sufficient and another solution was required. This lead to the development of the following solution.

Solution

.tfvars file

Using per environment .tfvars files to define variables is not a new approach, however implementing it in Terraform cloud requires some adjustments. The first thing to look at when using this approach is the directory structure of the terraform dir:

terraform/
|- env_vars/
|  |- dev.tfvars
|  |- stg.tfvars
|  |- prd.tfvars
|- main.tf
|- variables.tf

As you can see we have separated out each set of environment variables to their own .tfvars file. Each set should contain the same variables, but with different values e.g.:

dev.tfvars

env          = "dev"
machine_type = "e2-micro"

stg.tfvars

env          = "stg"
machine_type = "e2-small"

prd.tfvars

env          = "prd"
machine_type = "e2-medium"

and our variables are being defined in the variables.tf file:

variable "env" {
  description = "Environment name"
  type        = string
}
variable "machine_type" {
  description = "Machine type to be used for the VM"
  type        = string
}

Using this structure we can easily see when a variable has been added or changed during a git merge or push. Operating under a standard git development method these changes can be made and traced without having to access the variables via the UI.

NOTE: It is important that since these files are stored in git, no sensitive values should be kept in them. All sensitive variables should still be added via the UI

Specifying file

When applied locally the -var-file flag can be used to select which .tfvars file to inject variables into the deployment. When we apply via Terraform Cloud however it is not as straightforward to add this flag to the command. Since we cannot change the apply command directly we must make use of the TF_CLI_ARGS environment variable. This variable functions as the command line arguments, and when defined is added onto the end of the terraform apply command. To have Terraform Cloud make use of this variable we need to define it in the UI. This reduces having to define all our variables in the UI separately down to just one. Adding this variable simply means going to your workspace in the UI and under Variables adding:

Make sure to add this as an Environment variable rather than as a Terraform variable. This is done as we want to pass it in during runtime. Also ensure the path location is correct and matches the workspace environment i.e. env_vars/{environment}.tfvars

NOTE: Due to the way Terraform Cloud works, it maybe necessary to change the variable name to TF_CLI_ARGS_plan to avoid error. This is because Terraform Cloud injects the variables into the plan stage, and then saves that plan for further stages. This means it is not possible to inject the variables at later stages.

Now when we initiate our run, terraform cloud will select the designated .tfvars file and use the variables from within it to inject into the plan.

Output

Using a simple output in the main.tf file we can check our environments are reacting as expected:

output "vars" {
  value = "${var.env} - ${var.machine_type}"
}

When we apply the environments we get

dev

stg

prd

Summary

Using our .tfvars files we have now separated out our variables based on environment. This is what we were doing previously with the UI storage approach. However, this time around, since the files are stored in git, we can trace when a variable is updated. Also, all devs that have access to the repo have access to the vars. This avoids having to manage permissions to the Terraform Cloud UI workspace variables on top of the repo permissions.

Then, using our Terraform Cloud environment variable, each workspace will select its related .tfvars file, and use it in the deployment.