Blog

Module Parameter Defaults with the Terraform Object Type

02 Jan, 2020
Xebia Background Header Wave

In this post, I will walk you through the challenges I’ve faced when adopting the object type in my Terraform 0.12 modules and the solutions I came up with to work around the caveats. As an example, I will use the google_storage_bucket resource, as this is part of one of the modules I’ve built.

The Terraform Object Type

With the launch of Terraform 0.12 back in May 2019, many new cool features are introduced. Compared with Terraform 0.11, where you would find yourself repeating a lot of code, you can now utilise the new for_each functionality and object type to write cleaner code. If you haven’t discovered the new object type yet, you may be surprised by its potential. It is one of the two complex types Terraform provides and gives you the possibility to describe object structures. While this new type has many advantages, such as mixing types and defining multi-layered structures, it also has a few caveats that I will explain further down.

The content below is taken from the Terraform docs itself:
object(...): a collection of named attributes that each have their own type.
The schema for object types is { <key> = <type>, <key> = <type>, ... }</type></key></type></key> — a pair of curly braces containing a comma-separated series of <key> = <type></type></key> pairs.
Values that match the object type must contain all of the specified keys, and the value for each key must match its specified type. (Values with additional keys can still match an object type, but the extra attributes are discarded during type conversion.)

Example object structure:

my_object = {
  a_string  = example,
  a_number  = 1,
  a_boolean = true,
  a_map = {
    type1 = 1,
    type2 = 2,
    type3 = 3
  }
}

Provide Module Default Params with Object

As promised, in this blog post I will explain how I used the object type in my custom Terraform module. In the example below I’ve used the object type to define the supported settings of the GCP storage bucket.

variable bucket_settings {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })
}

Previously, when writing your Terraform module, you would need to create a variable for each setting you want to pass to your resource. Sure, you would have maps and lists, but a map could only contain values of the same type, limiting its use significantly. When using the object type, we can combine these settings in a complex structure. I can use this to populate my bucket, providing all these settings in a variable (bucket_settings in the listing below).

bucket_settings = {
  location           = europe-west4
  storage_class      = REGIONAL
  versioning_enabled = true
  bucket_policy_only = true
  lifecycle_rules    = {}
}

Since I have certain settings that I want to have applied to all of my buckets, I want to have most of these settings to be set for me by default. To do this, we can provide a default value for the bucket_settings object in the variable definition itself:

variable bucket_settings {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })

  default = {
    location           = europe-west4
    storage_class      = REGIONAL
    versioning_enabled = true
    bucket_policy_only = true
    lifecycle_rules    = {}
  }
}

Now, if I create my bucket, I can just import the module like this:

module my_bucket {
  source = path.to/my/module
  name   = bucket-with-defaults
}

And since I’ve set the defaults, I don’t need to provide the bucket_settings variable at all. ## Change One Setting, and The Defaults Disappear

But what if I want to overwrite just one of the defaults and keep the rest? In the following example, I try to set a lifecycle policy for the bucket using my module. Notice how I am passing the bucket_settings as a parameter.

module my_bucket {
  source = ./modules/bucket
  name   = bucket-with-defaults

  bucket_settings = {
    lifecycle_rules = {
      delete rule = {
        action = { type = Delete }
        condition = {
          age        = 30
          with_state = ANY
        }
      }
    }
  }
}

Unfortunately, this will result in multiple errors: Missing attributes

The given value is not suitable for child module variable bucket_settings
defined at modules/bucket/main.tf:6,1-27: attributes bucket_policy_only,
location, storage_class, and versioning_enabled are required.

Since I provided the bucket_settings for this module, it will overwrite the entire defaults I’ve set for the variable. I will need to provide the keys like location, storage_class, versioning_enabled and bucket_policy_only. But even after providing these settings, I’m getting a new error:

The given value is not suitable for child module variable bucket_settings
defined at modules/bucket/main.tf:6,1-27: attribute lifecycle_rules: element
delete rule: attribute condition: attributes created_before,
matches_storage_class, and num_newer_versions are required.

For the lifecycle_rule condition, I’ve only provided the age and with_state since I don’t care about created_before, matches_storage_class or num_newer_versions. But since I’ve defined them in the object definition, Terraform will complain and tell me I need to provide them as well. So to make this work with the module setup described above, I will need to provide all of the settings again. As you can see, especially when you want to create multiple buckets with only a few differences in the bucket configuration, this will become quite cumbersome. # Solution, using a defaults variable and merge

Because I only want to provide the settings that differ from the defaults, I’ve developed a solution for this. When I create a separate variable for the defaults, I can merge the provided bucket_settings with this bucket_defaults variable before calling the actual bucket provider. The result looks like this:

Module code snippet

variable name {
  type = string
  description = "The name of the bucket"
}

variable bucket_defaults {
  type = object({
    location           = string
    storage_class      = string
    versioning_enabled = bool
    bucket_policy_only = bool
    lifecycle_rules = map(object({
      action = map(string)
      condition = object({
        age                   = number
        with_state            = string
        created_before        = string
        matches_storage_class = list(string)
        num_newer_versions    = number
      })
    }))
  })

  default = {
    location           = europe-west4
    storage_class      = REGIONAL
    versioning_enabled = true
    bucket_policy_only = true
    lifecycle_rules    = {}
  }
}

variable bucket_settings {
  description = "Map of bucket settings to be applied, which will be merged with the bucket_defaults. Allowed keys are the same as defined for bucket_defaults."
}

locals {
  merged_bucket_settings = merge(var.bucket_defaults, var.bucket_settings)
}

I can now use the module by providing only the lifecycle rule that I want to set for my bucket, and all other defaults will still be applied. Using the module

module my_bucket {
  source = path.to/my/module
  name   = bucket-with-defaults

  bucket_settings = {
    lifecycle_rules = {
    delete rule = {
        action = { type = Delete }
        condition = {
          age          = 30
          with_state = ANY
        }
      }
    }
  }
}

Conclusion

I like the fact that I can now define objects with mixed types to create more complex structures, but there are a couple of things I think that could be improved:

  • The way variable defaults are handled. It would be great if these would be merged with the provided value, or at least have a setting to allow for this behaviour.
  • You should be able to define optional keys for your objects. However, the need for this will become less urgent when the first point is taken care of.

Hopefully, this blog post will help you understand the object type and its limitations while giving you an idea of utilising it to its best potential. In my next blog post, I will tell you more about the for_each functionality and how to use this in your Terraform modules.

Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts