How to create and deploy an AWS CloudFormation custom provider in less than 5 minutes

In this blog I will show you how to create and deploy an AWS CloudFormation custom provider
in less than 5 minutes using a Python copier template.

It is not hard to create a custom resource in CloudFormation, it just requires you
to implement three method: create, update and delete in a Lambda. However,
the entire setup to create and maintain a custom resource is quite laborious. I used to copy
and edit existing setups, but this became to error prone and laborious. I contemplated
a cookie-cutter setup for a while, but the fact that you cannot update backed cookies held me back.
The copier does allow you to update previously created projects, so I wanted to give it a try. And
I must say, I really like it.

The template has the following features:
– create the source code for a CloudFormation resource provider quickly
– support for semantic versioning using git-release-tag
– distribute lambdas to buckets in all AWS regions in the world
– re-recordable unit tests using botocore stubber recorder
– deployable AWS Codebuild pipeline

getting started!

Let’s say you want to create a custom resource for a Custom Domain of an AWS AppRunner service,
because it does not yet exist. To get started, type:

pip install copier
copier  https://github.com/binxio/cloudformation-custom-provider-template /tmp/cfn-app-runner-custom-domain-provider

🎤 the name of your custom resource type?
   AppRunnerCustomDomain
🎤 The name of your resource provider project?
   cfn-app-runner-custom-domain-provider
🎤 The name of your Python module?
   cfn_app_runner_custom_domain_provider
🎤 a short description for the custom provider?
   manages app runner custom domains 
🎤 Python version to use
   3.9
🎤 Your full name?
   Mark van Holsteijn
🎤 Your email address?
   mark.vanholsteijn@xebia.com
🎤 the URL to git source repository?
   https://github.com/binxio/cfn-app-runner-custom-domain-provider
🎤 the AWS profile name
   integration-test
🎤 the AWS region name
   eu-central-1
🎤 prefix for the S3 bucket name to store the lambda zipfiles?
   binxio-public
🎤 Access to lambda zip files?
   public

> Running task 1 of 1: [[ ! -d .git ]] &&  ( git init &&  git add . &&  git commit -m 'initial import'  && git tag 0.0.0) || exit 0
Initialized empty Git repository in /tmp/cfn-app-runner-custom-domain-provider/.git/
[main (root-commit) b2ce863] initial import
 21 files changed, 619 insertions(+)
...
````
This creates a project with a working custom provider for the resource <code>AppRunnerCustomDomain. Change to 
the directory and type make deploy-provider and make demo. Your provider will be up-and-running
in less than 5 minutes!

## what is in the box
When you type make help, you will get a list of all of available actions.

```text
build                -  build the lambda zip file
fmt                  -  formats the source code
test                 -  run python unit tests
test-record          -  run python unit tests, while recording the boto3 calls
test-templates       -  validate CloudFormation templates

deploy               -  AWS lambda zipfile to bucket
deploy-all-regions   -  AWS lambda zipfiles to all regional buckets
undeploy-all-regions -  deletes AWS lambda zipfile of this release from all buckets in all regions

deploy-provider      -  deploys the custom provider
delete-provider      -  deletes the custom provider

deploy-pipeline      -  deploys the CI/CD deployment pipeline
delete-pipeline      -  deletes the CI/CD deployment pipeline

deploy-demo          -  deploys the demo stack
delete-demo          -  deletes the demo stack

tag-patch-release    -  create a tag for a new patch release
tag-minor-release    -  create a tag for a new minor release
tag-major-release    -  create a tag for new major release

run the unit tests

To run the unit tests, type:

make test

The unit test will test the scaffold implementation generated by the botocore stubber recorder.
To create unit tests for your resource, edit the source code in ./tests/. To implement your custom
resource, edit the source code under ./src/.

re-recordable unit tests

Once you have your custom resource provider, it undoubtedly does some AWS API calls.
The botocore stubber recorder will
allow you to create unit test by running the test against a real account. The tests
will record the actual calls and generate the stubs. To run your unit tests, type:

make test-record

This will run the unit tests and record the AWS calls. To run the unit tests with the
newly created stubs, type:

make test

The integration tests are run against the AWS profile and region you specified.

Deploy the zip file to the bucket

To copy the zip file with the source code of the AWS Lambda of the custom resource provider, type:

aws s3 mb s3://<bucket-prefix>-<bucket-region>

As you can see, the zipfile will be copied to a bucket name which consists of the prefix
and the region name. This allows the zipfile to be made available for use in
all regions.

Deploy the custom resource provider into the account

Now the zip file is available, you deploy the custom resource provider, by typing:

make deploy-provider

This deploys the provider as an AWS Lambda function. To configure
the run-time parameters and permissions of the Lambda change the CloudFormation
template in the directory ./cloudformation.

Deploy the custom resource demo

To deploy the demo CloudFormation stack using the custom resource provider, type:

make deploy-demo

This deploys an CloudFormation stack with an example custom resource as a CloudFormation stack.
the run-time parameters and permissions of the Lambda change the CloudFormation
template in the file ./cloudformation/demo.yaml. Change the configuration of the custom
resource to match your implementation.

Version your custom resource provider

To version your custom resource provider, you can use the following commands:

make tag-patch-release    -  create a tag for a new patch release
make tag-minor-release    -  create a tag for a new minor release
make tag-major-release    -  create a tag for new major release

This will:
– run the pre-tag command in the file ./release
– commit all outstanding changes in the workspace
– tag the commit with the new version.

To show the current version of the workspace, type:

shell
make show-version
The utility git-release-tag
implements this functionality.

Deploy provider to all regions

To deploy the current version of your provider to all regions, type:

make deploy-all-regions

This assumes you have buckets in all regions with the defined prefix.

Deploy CI/CD pipeline

To deploy the CI/CD pipeline based on AWS Codebuild, make sure that the AWS account can
access the source repository. If that is the case, type:

make deploy-pipeline

Now every time you tag a new release, it will automatically be deployed to all regions.

conclusion

This copier template provides everything you need to quickly
build, deploy and maintain a new custom AWS CloudFormation Provider!
Photo by Saffu on Unsplash