Implementing DTAP / OTAP Solution in Terraform via Azure DevOps

What is DTAP?

One of the tasks in the infrastructure environment of Azure is to deploy DTAP environments. This is a phased approach for software testing and deployment according to Wikipedia and can be defined here below:

Implementing DTAP pipeline code via Terraform

In the previous blog article, we’ve set up Terraform for Azure via Azure DevOps.

If we go back to the pipeline code of terraform-plan.yaml, we can see that there are some parameters and variables made for one specific environment already:

parameters:
- name: environment
  displayName: Environment OneManITArmy
  type: string
  values:
  - dev

# Create if statement to decide which environment Terraform needs to deploy.
variables:
  ${{ if eq(parameters.environment, 'dev') }}:
    serviceConnectionName: "<your service connection name>"                                      # Specify service connection name in Azure DevOps project settings.
    workingDirectory: '$(System.DefaultWorkingDirectory)'                                        # Specify working directory of your Terraform files.
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "<your resource group name>"                                       # Specify RG-name that is created via Terraform where Storage account is held.
    backendStorageAccountName: "<your new storage account name>"                                 # Specify name of the Storage account is that created.
    backendContainerName: "<your tfstate blob container name>"                                   # Specify name of blob container in Storage account.
    backendKey: "<your tfstate backendkey name>.terraform.tfstate"                               # Specify filename mentioned in blob container above.

From the code we can see here above, we see that the only parameter value we have given is the Dev environment (line 6). If we add the other TAP environments in the values, we would have an option to choose which environment the pipeline needs to run at:

parameters:
- name: environment
  displayName: Environment OneManITArmy
  type: string
  values:
  - dev
  - test
  - acc
  - prod

If we check the variables code, we see that there is an if statement that if we have chosen our parameter value (based on the screenshot here above), then it will be using the defined variables to run the pipeline.

If we copy-and-paste that for the other environments, then the only thing that needs to be changed are:

serviceConnectionName – each environment has its own subscription (make sure you created the service connections before-hand!)
varFile – each environment has its own unique specific configuration (in Terraform, we call this our .tfvars file) that needs to be used (for the ARM/Bicep users, this will be your JSON PARAM or BICEP PARAM file)
backendKey – Terraform works based on its tfstate file (store configuration file) and since each environment is unique, each environment will be using their own tfstate file. With this reference, Terraform will create a new tfstate file to store its configuration file.

The other variables can stay the same since each tfstate file can be stored in the same location centrally.

The code will now look like this:

# Create if statement to decide which environment Terraform needs to deploy.
variables:
  ${{ if eq(parameters.environment, 'dev') }}:
    serviceConnectionName: "onemanitarmy dev service connection"         # Specify service connection name in Azure DevOps project settings.
    workingDirectory: '$(System.DefaultWorkingDirectory)'            # Specify working directory of your Terraform files.
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              # Specify RG-name that is created via Terraform where Storage account is held.
    backendStorageAccountName: "<your existing storage account name>"                      # Specify name of the Storage account is that created.
    backendContainerName: "tfstate"                                  # Specify name of blob container in Storage account.
    backendKey: "stonemanitarmydevai.terraform.tfstate"                 # Specify filename mentioned in blob container above.
  ${{ if eq(parameters.environment, 'test') }}:
    serviceConnectionName: "onemanitarmy test service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmytestai.terraform.tfstate"  
  ${{ if eq(parameters.environment, 'acc') }}:
    serviceConnectionName: "onemanitarmy acc service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmyaccai.terraform.tfstate"
  ${{ if eq(parameters.environment, 'prod') }}:
    serviceConnectionName: "onemanitarmy prod service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmyprodai.terraform.tfstate"

Your pipeline code that is modified should now be this:

parameters:
- name: environment
  displayName: Environment OneManITArmy
  type: string
  values:
  - dev
  - test
  - acc
  - prod

# Create if statement to decide which environment Terraform needs to deploy.
variables:
  ${{ if eq(parameters.environment, 'dev') }}:
    serviceConnectionName: "onemanitarmy dev service connection"         # Specify service connection name in Azure DevOps project settings.
    workingDirectory: '$(System.DefaultWorkingDirectory)'            # Specify working directory of your Terraform files.
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              # Specify RG-name that is created via Terraform where Storage account is held.
    backendStorageAccountName: "<your existing storage account name>"                      # Specify name of the Storage account is that created.
    backendContainerName: "tfstate"                                  # Specify name of blob container in Storage account.
    backendKey: "stonemanitarmydevai.terraform.tfstate"                 # Specify filename mentioned in blob container above.
  ${{ if eq(parameters.environment, 'test') }}:
    serviceConnectionName: "onemanitarmy test service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmytestai.terraform.tfstate"  
  ${{ if eq(parameters.environment, 'acc') }}:
    serviceConnectionName: "onemanitarmy acc service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmyaccai.terraform.tfstate"
  ${{ if eq(parameters.environment, 'prod') }}:
    serviceConnectionName: "onemanitarmy prod service connection"         
    workingDirectory: '$(System.DefaultWorkingDirectory)'            
    varFile: "$(System.DefaultWorkingDirectory)/env-tfvars/${{ parameters.environment }}.tfvars"
    backendResourceGroupName: "rg-onemanitarmy-tfstate"              
    backendStorageAccountName: "<your existing storage account name>"                      
    backendContainerName: "tfstate"                                  
    backendKey: "stonemanitarmyprodai.terraform.tfstate"

Copy and paste this code also for your terraform-apply.yaml file.

Implementing DTAP Terraform code

In the previous blog, we have also created a tfvars-file (dev.tfvars):

Tfvars file is your unique configuration file that can be applied to your specific tfstate file. If you want to have a DTAP environment, it means you need to have 4 unique configuration files to work it. Once again, the Tfvars’ files can be compared with your JSON PARAM and BICEP PARAM files if you are working with ARM/Bicep.

For this reason, copy the dev.tfvars file and paste it in the same location and rename the file as follows:

Adjust the resource-group-name value to your specific name. For test environment, this would be:

backendResourceGroupName                 = "<your tfstate resource group name>"
backendStorageAccountName                = "<your tfstate storage account name>"
backendContainerName                     = "<your tfstate blob container name>"
backendKey                               = "<your tfstate backendkey name>.terraform.tfstate"

resource-group-name = "test-resource-group"

If we check back at the main.tf file, we see that we are creating a resource group in which the name is defined in the variables var.resource-group-name:

resource "azurerm_resource_group" "rg-helloworld" {
name = var.resource-group-name
location = "West Europe"
}

Terraform will check in its variables.tf file to see if this variables is already defined with a value.
In this case, it is defined, but it does not have a value in it:

variable "backendResourceGroupName" {
  description = "Terraform Backend Resource Group"
  type        = string
}

variable "backendStorageAccountName" {
  description = "Terraform Backend Storage Account"
  type        = string
}

variable "backendContainerName" {
  description = "Terraform Backend Container Name"
  type        = string
}

variable "backendKey" {
  description = "Terraform Backend Key"
  type        = string
}

variable "resource-group-name" {
  description = "Creating a Resource Group Name"
  type = string
}

If Terraform does not find a value in the variables.tf file (even though it is defined), it will check in the tfvars.file ([dtap].tfvars) to see if the variable name exists and if there is a parameter value assigned to it (in which case it does as we did earlier).

If we do a Terraform Plan and Apply, then the resource group (specifically in the test environment subscription) will be created: