Cloud | DevOps | Google Cloud Platform | Technology | Terraform | Topics
Simplifying Google Cloud IP Management with Terraform 
Bruno Schaatsbergen 29 Jan, 2023
Share 
Adding new resources or features to the Hashicorp Terraform provider for Google, is normally done by updating Magic Modules resource definitions. In this blog I will show you how you can quickly generate and update these resource definitions using a simple utility.
Somewhere in September of this year, Google released mTLS support on the Google Load Balancer. I wanted to write a blog about it, and showcase the configuration with terraform templates. However, it turned out that the resources I needed where not supported, while other resources where missing essential properties to make it work.
adding new terraform resources
To add a new terraform resource, you have to create new Magic Modules resource definitions. These resource definitions are 90-95% cut-and-paste of REST API resource definition from Google’s REST API discovery documents.
This is a very labourious, slow, boring and error prone task, which is best automated. So I created a small utility: the magic-module-scaffolder!
updating a resource definition
With the utility, adding missing properties to existing resources becomes a breeze. I will showcase this with the resource google_cloud_run_v2_service. To see which properties are missing, just type:
$ pip install magic-module-scaffolder
$ git checkout git@github.com:GoogleCloudPlatform/magic-modules.git
$ cd magic-modules
$ mm-scaffolder --inplace --resource-file mmv1/products/cloudrunv2/Service.yaml
INFO: schema type name in create method is GoogleCloudRunV2Service, not Service
WARNING: the field 'name' is missing from the API definition, but keeping it in the as it is a special name
INFO: adding satisfiesPzs as ga field to definition of Service
INFO: adding scaling as ga field to definition of Service
WARNING: mismatch of type on field Service.generation, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Integer
WARNING: mismatch of type on field Service.template.labels, existing tag !ruby/object:Api::Type::KeyValuePairs and defined !ruby/object:Api::Type::KeyValueLabels
WARNING: mismatch of type on field Service.template.annotations, existing tag !ruby/object:Api::Type::KeyValuePairs and defined !ruby/object:Api::Type::KeyValueAnnotations
INFO: adding tcpSocket as ga field to definition of Service.template.containers..livenessProbe
INFO: adding nfs as ga field to definition of Service.template.volumes.
INFO: adding gcs as ga field to definition of Service.template.volumes.
WARNING: mismatch of type on field Service.observedGeneration, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Integer
WARNING: mismatch of type on field Service.terminalCondition.state, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.terminalCondition.severity, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.terminalCondition.reason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.terminalCondition.revisionReason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.terminalCondition.executionReason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.conditions..state, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.conditions..severity, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.conditions..reason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.conditions..revisionReason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.conditions..executionReason, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.trafficStatuses..type, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Enum
WARNING: mismatch of type on field Service.etag, existing tag !ruby/object:Api::Type::String and defined !ruby/object:Api::Type::Fingerprint
The output shows that the current definition is missing the fields satisfiesPzs, scaling, tcpSocket, nfs and gcs with respect to the API discovery document.
In addition, it gives a bunch of warnings where the current resource definition has chosen a different type than was expected on the basis of the API discovery document. When you look at the git diff, you will notice that it looks pretty neat!
@@ -554,6 +537,17 @@ properties:
The name of the service to place in the gRPC HealthCheckRequest
(see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
If this is not specified, the default behavior is defined by gRPC.
+ - !ruby/object:Api::Type::NestedObject
+ name: tcpSocket
+ description: TCPSocketAction describes an action based on opening a socket
+ properties:
+ - !ruby/object:Api::Type::Integer
+ name: port
+ description: |-
+ Port number to access on the container. Must be in the range 1 to 65535.
+ If not specified, defaults to the exposed port of the container, which
+ is the value of container.ports[0].containerPort.
+ required: true
- !ruby/object:Api::Type::NestedObject
name: 'startupProbe'
description: |-
@@ -749,7 +741,30 @@ properties:
- !ruby/object:Api::Type::String
name: 'sizeLimit'
description: |-
- Limit on the storage usable by this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. This field's values are of the 'Quantity' k8s type: https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/quantity/. The default is nil which means that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir.
+ Limit on the storage usable by this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. This field's values are of the 'Quantity' k8s type: https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/quantity/. The default is nil which means that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir.
+ - !ruby/object:Api::Type::NestedObject
+ name: nfs
+ description: Represents an NFS mount.
+ properties:
+ - !ruby/object:Api::Type::String
+ name: server
+ description: Hostname or IP address of the NFS server
+ - !ruby/object:Api::Type::String
+ name: path
+ description: Path that is exported by the NFS server.
+ - !ruby/object:Api::Type::Boolean
+ name: readOnly
+ description: If true, mount the NFS volume as read only
+ - !ruby/object:Api::Type::NestedObject
+ name: gcs
+ description: Represents a GCS Bucket mounted as a volume.
+ properties:
+ - !ruby/object:Api::Type::String
+ name: bucket
+ description: GCS Bucket name
+ - !ruby/object:Api::Type::Boolean
+ name: readOnly
+ description: If true, mount the GCS bucket as read-only
- !ruby/object:Api::Type::Enum
name: 'executionEnvironment'
description: |-
@@ -960,3 +975,20 @@ properties:
output: true
description: |
A system-generated fingerprint for this version of the resource. May be used to detect modification conflict during updates.
+ - !ruby/object:Api::Type::Boolean
+ name: satisfiesPzs
+ description: Output only. Reserved for future use.
+ output: true
+ - !ruby/object:Api::Type::NestedObject
+ name: scaling
+ description: |-
+ Scaling settings applied at the service level rather than at the
+ revision level.
+ properties:
+ - !ruby/object:Api::Type::Integer
+ name: minInstanceCount
+ description: |-
+ total min instances for the service. This number of instances is divided
+ among all revisions with specified traffic based on the percent of
+ traffic they are receiving. (ALPHA)
+ required: true
From this diff, I omitted some formatting differences. The ruamel.yaml library used does not preserve all of the whitespace layout in this yaml.
With the updated definition, I could generate a new version of the Google Provider and define a Cloud Run service which mounts a GCS bucket and a NFS share into the container without any special tricks!
resource "google_cloud_run_v2_service" "default" {
name = "service"
location = "us-central1"
ingress = "INGRESS_TRAFFIC_ALL"
launch_stage = "BETA"
template {
execution_environment = "EXECUTION_ENVIRONMENT_GEN2"
containers {
image = "us-docker.pkg.dev/cloudrun/container/hello:latest"
volume_mounts {
name = "nfs"
mount_path = "/mnt/nfs"
}
volume_mounts {
name = "gcs"
mount_path = "/mnt/bucket"
}
}
vpc_access {
network_interfaces {
network = "default"
subnetwork = "default"
}
}
volumes {
name = "nfs"
nfs {
server = google_filestore_instance.default.networks[0].ip_addresses[0]
path = "/share1"
}
}
volumes {
name = "bucket"
gcs {
bucket = google_storage_bucket.default.name
}
}
}
}
resource "google_storage_bucket" "default" {
name = "my-bucket"
location = "US"
}
resource "google_filestore_instance" "default" {
name = "my-filestore"
location = "us-central1-b"
file_shares {
capacity_gb = 1024
name = "share1"
}
networks {
network = "default"
modes = ["MODE_IPV4"]
}
}
These changes resulted in pull request #9728 on the magic modules!
generating new resource definitions
As it all started with the need to add new resources to the Google provider, I will show you how that is done. The ones I needed were serverTlsPolicies and addressGroups. They are now present in the provider, but let’s see what happens if I generate them from the discovery document:
$ mm-scaffolder generate --product-directory mmv1/products/networksecurity serverTlsPolicies addressGroups
INFO: Writing to definition of ServerTlsPolicy to mmv1/products/networksecurity/ServerTlsPolicy.yaml
INFO: Writing to definition of AddressGroup to mmv1/products/networksecurity/AddressGroup.yaml
If you run a git diff, you will see that the generated resource definitions match the defined resource pretty well, and would be a really good start!
A word of caution
Do not use the scaffolder to blindly submit PR on the magic-modules without careful inspection and testing of the generated changes. The changes are generated based on what I can derive from the field description in the discovery document, and it may not always be correct.
Use it as a way to bootstrap updates and determine whether new features have been added with respect to the existing resource definition.
Conclusion
The Magic Modules resource definitions are largely transformations from the Google API discovery document. The utility is not perfect, as I had to guess a lot of the mappings. But, I am confident it will help speed up the maintenance of the terraform resources supported by the Google provider.
image generated by DALL·E 2.
Mark van Holsteijn is a senior software systems architect at Xebia Cloud-native solutions. He is passionate about removing waste in the software delivery process and keeping things clear and simple.
