Deploy Golang AWS CloudFormation Custom Provider in 5 Min

In this blog I will show you how to create and deploy a Golang AWS CloudFormation custom provider in less than 5 minutes using a copier template.

Creating a custom resource in CloudFormation is really simple. You just implement a create, update and delete method in a Lambda and you are done. But that is the easy part: you still have to create zip files, unit tests, documentation, demo’s, CI/CD deployment pipelines and more. This copier template has it all!

it creates the source code for a Golang custom resource provider
it provides all the build commands you need to build, test and deploy your provider
it deploys lambdas to buckets in all AWS regions in the world
it provides a ready-to-deploy CI/CD pipeline on AWS Codebuild
it supports semantic versioning using git-release-tag

getting started!

Let’s say you want to create a custom resource for a ECR Container Image, so that you an clone public images into an ECR repository. Although there is a Python library to implement this, it is unfortunately no longer supported. Fortunately, there is a Golang library go-containerregistry to do the trick. To skaffold a project to implement a custom provider based on Golang, type:

$ pip install 'copier>=8.0.0'
$ copier copy --trust \
   https://github.com/binxio/cloudformation-custom-provider-golang-template \
   /tmp/cfn-container-image-provider

🎤 the name of your custom resource type?
   ContainerImage
🎤 The name of your resource provider?
   cfn-container-image-provider
🎤 a short description for the custom provider?
   manages container images
🎤 golang version to use
   1.20
🎤 Your full name?
   Mark van Holsteijn
🎤 Your email address?
   mark.vanholsteijn@xebia.com
🎤 the go module name
   github.com/binxio/cfn-container-image-provider
🎤 the URL to git source repository?
   https://github.com/binxio/cfn-container-image-provider.git
🎤 the AWS region name
   eu-central-1
🎤 prefix for the S3 bucket name to store the lambda zipfiles?
   binxio-public
🎤 Access to lambda zip files?
   public
  
Copying from template version 0.1.0
 ...
 > Running task 1 of 1: [ ! -f go.sum ] &&  (go mod download || echo "WARNING: failed to run go mod">&2); [ ! -d .git ] && ( git init && git add . && git commit -m 'initial import' && git tag 0.0.0) || exit 0
Initialized empty Git repository in ...
[main (root-commit) c97b9e2] initial import
 15 files changed, 529 insertions(+)

source code directory

The copier generates the following source code directory:

├── Dockerfile.lambda                   # creates the zip file
├── Makefile.mk                         # generic build steps for the provider
├── Makefile                            # customization of build steps
├── cloudformation
│   ├── cfn-custom-image-provider.yaml  # to deploy the provider
│   ├── cicd-pipeline.yaml              # to deploy the Codebuild CI/CD pipeline
│   └── demo.yaml                       # to deploy the demo
├── doc
│   └── ContainerImage.md               # start documentation of resource
├── go.mod
├── go.sum
├── main.go                             # main entrypoint
└── pkg
    └── resources
        └── container_image
            └── handler.go              # sample code

That is all that is needed to create a project with a working custom provider for the resource ContainerImage. When you change to the directory and type:

$ make deploy-provider
$ make deploy-demo

Your provider will be up-and-running in less than 5 minutes! Now it is up to you refactor the implementation to provide the required functionality.

Available build commands

to help you in the development process, the following build commands are available:

$ make help

build                -  build the lambda zip file
fmt                  -  formats the source code

test                 -  run unit tests
test-templates       -  validate CloudFormation templates

deploy               -  AWS lambda zipfile to bucket
deploy-all-regions   -  AWS lambda zipfiles to all regional buckets
undeploy-all-regions -  deletes AWS lambda zipfile of this release from all
buckets in all regions

deploy-provider      -  deploys the custom provider
delete-provider      -  deletes the custom provider

deploy-pipeline      -  deploys the CI/CD deployment pipeline
delete-pipeline      -  deletes the CI/CD deployment pipeline

deploy-demo          -  deploys the demo stack
delete-demo          -  deletes the demo stack

tag-patch-release    -  create a tag for a new patch release
tag-minor-release    -  create a tag for a new minor release
tag-major-release    -  create a tag for new major release

show-version         -  shows the current version of the workspace
help                 -  Show this help.

Deploy the zip file to the bucket

To copy the zip file with the source code of the AWS Lambda of the custom resource provider, the buckets must already exist. If they do not, type:

$ BUCKET=<bucket-prefix>-<bucket-region>
$ aws s3 mb s3://$BUCKET
$ aws s3api put-bucket-ownership-controls \
    --bucket $BUCKET --ownership-controls \
    'Rules=[{ObjectOwnership=BucketOwnerPreferred}]'

The build system expects the bucket name to consists of the prefix and the region name. This allows the provider to be made available for use in all regions.

Deploy the custom resource provider into the account

To configure the run-time parameters and permissions of the your provider, change the CloudFormation template in the directory ./cloudformation. Once that is done, type:

$ make deploy-provider

Deploy the custom resource demo

To deploy a CloudFormation stack with an example use of the custom resource, type:

$ make deploy-demo

Version your custom resource provider

Semantic versioning of the provider is implemented using the utility git-release-tag. If you have not installed git-release-tag, type:

$ pip install git-release-tag

To version your custom resource provider, you can use the following commands:

make tag-patch-release    -  create a tag for a new patch release
make tag-minor-release    -  create a tag for a new minor release
make tag-major-release    -  create a tag for new major release

This will:

run the pre-tag command in the file ./release, to update all files with references to the semantic version
commit all outstanding changes in the workspace
tag the commit with the new version

To show the current version of the workspace, type:

make show-version

Deploy provider to all regions

To deploy the current version of your provider to all regions, type:

make deploy-all-regions

This assumes you have buckets in all regions with the defined prefix.

Deploy CI/CD pipeline

To deploy the CI/CD pipeline based on AWS Codebuild, make sure that the AWS account can access the source repository. If that is the case, type:

make deploy-pipeline

Now, every time you tag a new release, it will automatically be deployed to all regions.

Conclusion

This copier template provides everything you need to quickly build, deploy and maintain a new Golang custom AWS CloudFormation Provider! If you have any questions, problems or feedback feel free to contact me or add issues at https://github.com/binxio/cloudformation-custom-provider-golang-template.

If you want to create your resource provider in Python, we also have a solution for that!

Image by Michal Jarmoluk from Pixabay

How to create and deploy a golang AWS CloudFormation custom provider in less than 5 minutes