importer-labs/azure_devops/4-custom-transformers.md

Using custom transformers to customize Valet's behavior

In this lab we will build upon the dry-run command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:

1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.

Prerequisites

1. Followed the steps here to set up your Codespace environment and bootstrap an Azure DevOps project.
2. Completed the configure lab.
3. Completed the audit lab.
4. Completed the dry-run lab.

Perform a dry run

We will be performing a dry-run for a pipeline in the bootstrapped Azure DevOps project. We will need to answer the following questions before running this command:

1. What is the id of the pipeline to convert?

   • :pipeline_id. This id can be found by:
     • Navigating to the build pipelines in the bootstrapped Azure DevOps project https://dev.azure.com/:organization/:project/_build
     • Selecting the pipeline with the name "valet-custom-transformer-example"
     • Inspecting the URL to locate the pipeline id https://dev.azure.com/:organization/:project/_build?definitionId=:pipeline_id

2. Where do we want to store the result?

   • ./tmp/dry-run-lab. This can be any path within the working directory that Valet commands are executed from.

Steps

1. Navigate to the codespace terminal

2. Run the following command from the root directory:

   gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab
   

3. The command will list all the files written to disk when the command succeeds.

4. View the converted workflow:

   • Find ./tmp/dry-run-lab in the file explorer pane in codespaces.
   • Click valet-custom-transformer-example.yml to open.

The converted workflow that is generated can be seen below:

Converted workflow 👇

name: valet-bootstrap/pipelines/valet-custom-transformer-example
on:
  push:
    branches:
    - "*"
env:
  BUILDCONFIGURATION: Release
  BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
jobs:
  Job_1:
    name: Agent job 1
    runs-on: windows-latest
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - uses: actions/checkout@v2
    - name: Use Node 10.16.3
      uses: actions/setup-node@v2
      with:
        node-version: 10.16.3
    - name: Restore
      run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
    - name: Build
      run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}

Note: You can refer to the previous lab to learn about the fundamentals of the dry-run command.

Custom transformers for build steps

We can use custom transformers override Valet's default behavior. In this scenario, we may want to override the behavior for converting DotnetCoreCLI@2 tasks to support parameters that are glob patterns. We will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?

   • DotnetCoreCLI@2

2. What is the desired Actions syntax to use instead?

   • After some research, we have determined that the uploading test results as an artifact will be suitable:

       - run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
         shell: bash
     

Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the .rb file extension. You can create this file by running the following command in your codespace terminal:

code transformers.rb

Next, we will define a transform method for the DotnetCoreCLI@2 identifier by adding the following code to transformers.rb:

transform "DotNetCoreCLI@2" do |item|
  projects = item["projects"]
  command = item["command"]
  run_command = []

  if projects.include?("$")
    command = "build" if command.nil?
    run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
  else
    run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
  end

  {
    run:   run_command.join("\n"),
    shell: "bash",
  }
end

This method can use any valid ruby syntax and should return a Hash that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the item parameter for the original values configured in Azure DevOps.

Now, we can perform another dry-run command and use the --custom-transformers CLI option to provide this custom transformer. Run the following command within your codespace terminal:

gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab --custom-transformers transformers.rb

Open the workflow that is generated and inspect the contents. Now, the DotnetCoreCLI@2 steps are converted using the customized behavior!

-    - name: Restore
-      run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
-    - name: Build
-      run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
+    - name: Restore
+      run: shopt -s globstar; for f in ./**/*.csproj; do dotnet restore $f  ; done
+      shell: bash
+    - name: Build
+      run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
+      shell: bash

Custom transformers for environment variables

We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the BUILDCONFIGURATION environment variable to be Debug instead of Release.

To do this, add the following code to the transformers.rb file.

env "BUILDCONFIGURATION", "Debug"

In this example, the first parameter to the env method is the environment variable name and the second is the updated value.

Now, we can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the DB_ENGINE environment variable will be set to mongodb:

env:
-  BUILDCONFIGURATION: Release
+  BUILDCONFIGURATION: Debug
  BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"

Custom transformers for runners

Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:

1. What is label of the runner in Azure DevOps to update?

   • windows-latest

2. What is the label of the runner in Actions to use instead?

   • ubuntu-latest

With these questions answered, we can add the following code to the transformers.rb file:

runner "windows-latest", "ubuntu-latest"

In this example, the first parameter to the runner method is the Azure DevOps label and the second is the Actions runner label.

Now, we can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the runs-on statement will use the customized runner label:

runs-on:
-  - windows-latest
+  - ubuntu-latest

At this point of the lab the file contents of transformers.rb should match this:

Custom transformers 👇

transform "DotNetCoreCLI@2" do |item|
  projects = item["projects"]
  command = item["command"]
  run_command = []

  if projects.include?("$")
    command = "build" if command.nil?
    run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
  else
    run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
  end

  {
    shell: "bash",
    run:   run_command.join("\n")
  }
end

env "BUILDCONFIGURATION", "Debug"

runner "windows-latest", "ubuntu-latest"

Thats it! At this point you have overridden Valet's default behavior by customizing the conversion of:

• Build steps
• Environment variables
• Runners

Next lab

Perform a production migration of an Azure DevOps pipeline