One of the great advantages of using the public cloud is the ability to deploy applications and services at scale. But scale has a flip side, as working with dozens or hundreds of servers imposes new constraints on systems administration. Where we could manage one or two devices using a CLI or a GUI, or 10 or 20 devices using our own scripts, managing a massive fleet of devices requires a very different approach. We need infrastructure as code and automation.
This approach is the basis for Microsoft’s Azure Automation, a collection of tools for managing virtual infrastructures using a mix of declarative deployments and PowerShell-based Desired State Configuration (DSC). Azure Automation brings together familiar technologies like Azure Resource Manager and the Bicep infrastructure definition language, reducing the learning curve and extending their capabilities.
At the same time, having a testable framework for deployment reduces the risks associated with scaling and with application updates. Once your infrastructure is in place, Azure Automation goes beyond deployments, as support for DSC gives you a hands-off approach to keeping servers up-to-date with security patches.
Azure Automation takes a programmatic approach to process automation, using runbooks to manage operations. Runbooks let you wrap all of your configuration and deployment rules in a single executable file, written in PowerShell or Python. For simpler automations, Microsoft provides a graphical authoring environment to simplify creating new runbooks.
To get the benefit of tools like this, you need to integrate them into both your build system and your event framework. Microsoft uses webhooks in Azure Automation to trigger operations. For example, you could have an Azure Function triggered by a spike in demand that will run an automation to provision more instances of the application. The same webhooks could be triggered by GitHub Actions or Azure DevOps to deploy a fresh infrastructure as part of a build.
Building runbooks in Azure Automation
Runbooks are created in Azure Automation’s built-in text editor or through a graphical editor. The text editor has a similar look and feel to Visual Studio Code, with support for IntelliSense and a library of code snippets to simplify authoring.
If you’re using PowerShell, your runbook code can include pre-written cmdlets, as well as support for child runbooks. This last option lets you build a library of runbooks that embed key functions (for creating a new server, for example) and assemble them into a more complex infrastructure deployment.
You’re not limited to creating and deploying infrastructure. Azure Automation also provides tools for working with Azure platform services, using the same Azure CLI APIs to configure services and make the resulting resources available to your code. You can use the managed identity tools in Azure Automation to give it an account and role that are used to configure and manage your platform service, for example to create new tables in Azure SQL Database for your application.
If you add support for Desired State Configuration in your Bicep or Azure Resource Manager templates for managed VMs, you can keep drift to a minimum, ensuring that servers in different regions and availability zones keep the same configuration. Enabling DSC amounts to one line of PowerShell, or an entry in an Azure Resource Manager template. It’s possible to automate configuration too, using Azure Automate to set up configuration manager settings, and then compile them for deployment across a fleet of servers.
Alternatively, you can configure your VMs to use the new Azure Automanage service, which is an updated version of DSC aimed at managing machine properties through Azure Resource Manager. This approach provides tools for both server configuration and for compliance. In practice it’s best to use compliance management, as this ensures machines are updated as soon as they’re deployed and that all instances stay in sync, reducing risk as new security updates roll out.
Managing automation runtimes in Azure Automation
A key tool for managing and supporting runbooks is Azure Automation’s Runtime environment. Runtime environment lets you define and control the environments used to host and run automation code. By locking this down you reduce the risks associated with using an unknown runtime, avoiding any potential mismatches.
By defining a runtime environment, you can control three key factors: the language used for your runbooks, the version targeted, and any required software packages. Locking down the language version avoids differences between releases, so that you’re always running the version you used to build and test your code. New releases with new functionality can then increment runtime versions as needed.
There are good reasons to use a recent PowerShell runtime (such as PowerShell 7.2), because these allow you to include Azure CLI commands in your scripts. Azure Automation will always use the latest Azure CLI version supported by the underlying language, so keep this in mind if you’re relying on commands that may have been deprecated even though the rest of your runbook is unchanged.
Runtime environments are created and managed from the Azure Portal or via its REST API. Configuration is simple enough, as you select language and version from the supported releases. Once that’s done you choose the packages and modules you want to use and upload them to your Azure account. These can come from your local development environment or from the Azure Automation gallery.
If you want to update the language version for a new release of a runbook, all you need to do is select the latest version and save your updated environment. You may need to update packages as a result of any changes.
Azure Automation for systems management
It’s all very well being able to create complex new infrastructures on demand, but you still need to manage those new servers and networks. Azure Automation offers change tracking tools that give you a real-time inventory of your servers. When you link them to system management tools, you’re able to check resources and monitor operations.
Support for modern management tools includes Azure Automation State Configuration, a cloud-hosted version of the on-premises PowerShell DSC. By running a DSC server in Azure, you’re able to manage server state and automate updates as needed. Azure Automation State Configuration helps you control what’s updated and how, ensuring that unwanted updates are never deployed to a server.
Note that you can use Azure Automation outside of Azure, as part of the Azure Arc hybrid cloud management service. This adds a Hybrid Runbook role for non-Azure servers, including physical hardware. Azure Arc allows you to manage both edge devices and Azure-hosted systems as part of an IoT platform, or to manage secure services that are require to be on premises for regulatory compliance.
Remember when we began talking about servers as cattle? As we added more and more servers, we eventually discovered we could no longer manage them the old fashioned way, like pets. The cloud has brought us to a new level, where we don’t even know how many servers we have. That’s why we need to automate cloud infrastructure, to make our lifted-and-shifted applications behave as closely as possible to the cloud-native services they’re using.