DSC with Infrastructure-As-Code and Azure Automation is a potent combination

Source Code Download

Introduction

This article shows how DSC, infrastructure-as-code and Azure Automation play together by presenting a fully automated sample of a vm that is provisioned  using an ARM template and  a DSC extension, deployed using an Azure Automation Runbook, whose state is managed using DSC. This is a powerful combination indeed, and I have attempted to build up to this automated sample by showing how one would provision the vm and add it to DSC manually. I have also tried to show the business value that DSC, infrastructure-as-code and automation brings to the traditional ways of provisioning and managing the state of servers.

The Problems we are trying to solve

When we provision a new virtual machine, we often require  software preinstalled on it, or  have some required network settings, some required features , some configuration settings, or some registry setting  present on it. We would like to avoid manually installing and configuring these pieces of software and settings.

Once we provision these machines with these set of “desired” features, something like a thermostat should exists on these machines that maintains them in this golden state, and prevent what we call “configuration drift”.

Our server documentation should be “auto-documenting”. There should be a magical document that keeps itself up to date with any changes we make on our servers. We should be able to version our servers, and be able to go back to a prior version if necessary.

Infrastructure as code, Desired State Configuration (DSC), and automation,used together, fulfills these needs for state management, versioning and auto-documenting for virtual machines.

In the past we had our “golden image”, which was a fully patched image that had all our needed software, registry settings, and configurations installed. However, keeping the machines cloned from these golden images up-to-date with latest versions of software and patches was a non-trivial task. This is where infrastructure-as-code plays its part. With script, be it an ARM template, a Powershell script, or a Terraform script, we can create our servers in code, doing away with these “golden images”.

And, to keep these scripted machines in a state of continual deployment readiness, up-to-date with the required installs, patches and configuration settings, we use a process called Desired State Configuration (DSC).

The underlying idea of a DSC Pull model (there is a Push model also, which is beyond the scope of this post), is this: that there is a server somewhere that holds a magical document called a “DSC Configuration Document”, in which we list out the  state we want our servers to continually be in.

Then, on the virtual machine that requires its state to be maintained, there exists a agent called a Local Configuration Manager (LCM) that constantly polls the DSC server and “pulls” down this magic DSC Configuration document , reads it, and applies the instructions in this document to the machine it manages. Since it is constantly polling the pull server, it reapplies any fresh instructions, or reverts back the machine to the “golden state”, if someone inadvertently or maliciously attempts to change the machine’s utopia.

Auto-Documentation is an attribute of this magical DSC Configuration document. In the past, when we were standing up hard physical servers from our “gold images”, the knowledge of building these servers was often codified in elaborate forms, and updating these documents was never optimal. The person who owned that knowledge was like a key man. Instead of being a static build form, the DSC Configuration  document is a functional document that first describes  how the server is to be built , and then  makes the server take on  the state it describes. 

The DSC Configuration document makes versioning of servers easy. Versions of the configuration documents can be source controlled and we can always go back to earlier state if something fails. We can easily look at history of our server changes in source control.

And, Automation makes the setting up of a DSC server trivial. There is no setting up to do as an automation account in Azure automatically provisions a DSC pull server.

DSC With Automation Quick Start

Create an Azure Automation Account

  1. Create an Azure Automation account from portal.azure.com
  2. A DSC pull server will also be created. You will see a section called “CONFIGURATION MANAGEMENT” with three navigation links. These are:
    1. DSC nodes
    2. DSC configurations
    3. DSC node configurationspic : dsc-1

I will call these links “DSC Sections” , and since they are so similar in their names, will refer to them in full.

Create a DSC Configuration file

A DSC Config file is a simple text file that has instructions as in the examples below.

A few sample DSC Config files have been provided in the source code download, in the Configuration folder.

CreateFileDemo.ps1

This file shows how to refer to a Automation Variable called “DownloadPackagesPath” and then to write it out to a text file on the server at the path specified in the “DestinationPath.”

Automation variables allow us to pass in input variables to the DSC Configuration files.

You create a Variable from the “Variables” link on the Automation account and enter the Name, Type & Value. In this case we created a variable called “DownloadPackagesPath” with the value of some file share path.

Configuration CreateFileDemo
{

   $samplestr = Get-AutomationVariable –Name 'DownloadPackagesPath'

    Node "localhost"
    {
       
        File CreateFile {
            DestinationPath = 'C:\myTest.txt'
            Ensure = "Present"
            Contents = $samplestr
        }
       
    }
}

 

Here are the instructions to import & compile this sample file:

  1. From this code download, import the file called CreateFileDemo.ps1 to the DSC Configurations section in your-automation-account (or paste the above into a text file and import).
  2. When Imported, click on it, compile it by choosing Compile on the Toolbar. Let it default for the ComputerName (“Default will be used”).

Instead of repeating the Microsoft documentation, I point you to the relevant quick-start document on the MS site. It is a simple click-thru guide and it is here:

https://docs.microsoft.com/en-us/azure/automation/automation-dsc-getting-started

Creating and Onboarding a machine manually

The basic steps are: you create a virtual machine and add the machine to the DSC Node of the automation account. Once added, it will be automatically managed by the DSC Server, based on state specified in the DSC Configuration file.

Later, I will show how to automatically provision a vm and add it to the DSC Server using an ARM template with the DSC extension. However to get a quick feel of how the DSC stuff works, you can provision a vm manually from the Azure portal and manually add it to the DSC Nodes section of the automation account.

Instead of repeating the Microsoft documentation, I point you to the relevant quick-start document on the MS site. It is a simple click-thru guide and it is here:

https://docs.microsoft.com/en-us/azure/automation/automation-dsc-getting-started

Seeing the DSC maintain state

Once the machine is onboarded, and a valid and compiled DSC Configuration exists, the LSM on the virtual machine will pull the Configuration file and apply it.

The machine you onboarded should appear in the DSC Nodes section with a status of “Compliant”. If you remote into the virtual machine, you should see the file c:\myTest.txt.

A more complex DSC Configuration

Suppose you want a software called FireEye to be installed on all your DSC managed servers. You would create a Azure file share. You will upload the FireEye installer files to this file share. You will create a Automation Credential Asset ( similar to how we created the variable asset DownloadPackagesPath) by clicking the Credentials Link on the automation account called (say) DSCPackageStorage as follows:

Credential Asset

Name: DSCPackageStorage

Username: AZURE\file-storage-name (this is the name of the storage account, prefixed by “AZURE\”)

Password: Key of the file storage

Variable Asset

We created the Variable Asset DownloadPackagePath earlier. Now in the value, enter the path to your folder where you uploaded your install files for FireEye

The following DSC Configuration copies the install files from the file share to a local c:\packages folder, using the specified credentials and the sourcePath specified in the DownloadPackagePath automation variable.

Once copied locally, the software FireEye gets installed.

Configuration DownloadInstallPackages
{
   $storageCredential = Get-AutomationPSCredential -Name "DSCPackageStorage"
   $sourcePath = Get-AutomationVariable –Name 'DownloadPackagesPath'

    Node "localhost"
    {
       
        File DirectoryCopy
        {
            Ensure = "Present"  # You can also set Ensure to "Absent"
            Type = "Directory" # Default is "File".
            Checksum = "ModifiedDate"
            MatchSource = $true
            Force = $true
            Recurse = $true # Ensure presence of subdirectories, too
            Credential = $storageCredential
            SourcePath = $sourcePath
            DestinationPath = "C:\Packages"    
        }
        Log AfterDirectoryCopy
        {
            # The message below gets written to the Microsoft-Windows-Desired State         Configuration/Analytic log
            Message = "Finished running the file resource with ID DirectoryCopy"
            DependsOn = "[File]DirectoryCopy" # This means run "DirectoryCopy" first.
        }
       
        Package Install_FireEye
        {
            Ensure = "Present"
            Name = "xagt"
            DependsOn = "[File]DirectoryCopy"
            Path = "C:\Packages\FireEye\xagtSetup_21.33.7_universal.msi"
            Arguments = "/q"
            ProductId = "55E1EF02-DA68-46D3-8659-6A29822F65C1"
        }
    }
}

 

Assigning the new Configuration to Virtual Machines

Upload and compile this configuration file as explained before,

Once the DSC configurations node says “Completed” for the configuration:

  1. Go to “DSC nodes” menu
  2. Click on a virtual machine
  3. Click on “Assign node configurations”
  4. Select & apply the new configuration in the “Assign Node Configuration” blade

To jump start the Configuration

The DSC will normally wait for the time specified for the poll frequency. To immediately apply the state, on the virtual machine, run the following powershell.

Update-DSCConfiguration

 

Use an ARM template with a DSC extension to automatically provision a VM and register it with DSC

In the source code download, in the deployment folder, I have provided a powershell script file called CreateStorage.ps1 and an arm template called vmautomation_dsc.json. This arm template will create a virtual machine in the vnet and subnet provided as parameters. The ARM template also includes a DSC extension resource that will automatically add the created VM to the automation DSC pull server so that it can be managed by it. To point it to the automation DSC server, we will provide the arm template with parameters that identify the DSC Server (the server key and url. We will come to it in a minute).We will wrap all this logic in a powershell RunBook.

We want to create a blob storage and upload this template to this blog. If you run the CreateStorage.ps1 powershell, it will automatically create a blob storage account and upload the arm template to it. Here are the steps to run it.

Run the deployment\CreateStorage.ps1 file in the project.

It expects Parameters

  1. ResourceGroup : The resource group the automation account was created under (Step1)
  2. Location: [Enter your region – example: SouthCentralUS]

This will:

  1. Creates a blog storage account,
  2. create a container
  3. Give public access to the container
  4. upload the ARM template “vmautomation_dsc.json” to the blob storage. This is the ARM template for creating the VM + associated resources. It also has the DSC extension & will register the VM as a node in the DSC server, when run later by the Run Book “DeployVm”

Make note of the path of the template file

You will need to provide the path of the template file as a parameter when running Run Book DeployVM . Path will be something like: https://dccteststorage.blob.core.windows.net/abcarmtemplate/vmautomation_dsc.json

Import Run Books in Automation Account

In the “RunBooks” tab of the automation account, click “Add a runbook” and import the following runbooks provided in the Runbooks folder of the source code provided.

    1. RunDeploy.ps1 : a helper file to pass in the parameters to DeployVm.ps1

There a number of variables passed in but the important ones are

$RegistrationKey : Key of the DSC automation account (found on the Keys section of the automation account)

$RegistrationUrl: URL of the automation account, found also in the Keys section of the automation account

$NodeConfigurationName:  The name of the DSC configuration under the "DSC nodes configuration" section. For example: CreateFileDemo.localhost if using the simple  configuration used above in the quickstart

 

  1. DeployVm.ps1: the main file that creates the VM by calling the ARM template
  2. Publish the run book DeployVm.ps1
  3. Run the runbook called RunDeploy.ps1. Once the VM is provisioned, you should see it appear under the DSC nodes section of the automation account and reported as “Compliant”

Conclusion

Using Infrastructure as code to script out virtual machine creation, DSC to maintain its state, and automation to instantiate the vm provisioning is a potent combination , one that can give great power to your server deployments and management.