importer-labs/azure_devops/4-custom-transformers.md

# Using custom transformers to customize Valet's behavior

In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:

1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.

## Prerequisites

1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and bootstrap an Azure DevOps project.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [audit lab](./2-audit.md).
4. Completed the [dry-run lab](./3-dry-run.md).

## Perform a dry run

We will be performing a dry-run for a pipeline in the bootstrapped Azure DevOps project. We will need to answer the following questions before running this command:

1. What is the id of the pipeline to convert?
    - __:pipeline_id__. This id can be found by:
      - Navigating to the build pipelines in the bootstrapped Azure DevOps project <https://dev.azure.com/:organization/:project/_build>
      - Selecting the pipeline with the name "valet-custom-transformer-example"
      - Inspecting the URL to locate the pipeline id <https://dev.azure.com/:organization/:project/_build?definitionId=:pipeline_id>

2. Where do we want to store the result?
    - __./tmp/dry-run-lab__. This can be any path within the working directory that Valet commands are executed from.

### Steps

1. Navigate to the codespace terminal
2. Run the following command from the root directory:

    ```bash
    gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab
    ```

3. The command will list all the files written to disk when the command succeeds.
4. View the converted workflow:
    - Find `./tmp/dry-run-lab` in the file explorer pane in codespaces.
    - Click `valet-custom-transformer-example.yml` to open.

The converted workflow that is generated can be seen below:

<details>
  <summary><em>Converted workflow 👇</em></summary>

```yaml
name: valet-bootstrap/pipelines/valet-custom-transformer-example
on:
  push:
    branches:
    - "*"
env:
  BUILDCONFIGURATION: Release
  BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
jobs:
  Job_1:
    name: Agent job 1
    runs-on: windows-latest
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - uses: actions/checkout@v2
    - name: Use Node 10.16.3
      uses: actions/setup-node@v2
      with:
        node-version: 10.16.3
    - name: Restore
      run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
    - name: Build
      run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
```

</details>

_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.

## Custom transformers for build steps

We can use custom transformers override Valet's default behavior. In this scenario, we may want to override the behavior for converting `DotnetCoreCLI@2` tasks to support parameters that are glob patterns. We will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?
    - __DotnetCoreCLI@2__

2. What is the desired Actions syntax to use instead?
    - After some research, we have determined that the uploading test results as an artifact will be suitable:

      ```yaml
        - run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
          shell: bash
      ```

Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:

```bash
code transformers.rb
```

Next, we will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`:

```ruby
transform "DotNetCoreCLI@2" do |item|
  projects = item["projects"]
  command = item["command"]
  run_command = []

  if projects.include?("$")
    command = "build" if command.nil?
    run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
  else
    run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
  end

  {
    run:   run_command.join("\n"),
    shell: "bash",
  }
end
```

This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.

Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:

```bash
gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab --custom-transformers transformers.rb
```

Open the workflow that is generated and inspect the contents. Now, the `DotnetCoreCLI@2` steps are converted using the customized behavior!

```diff
-    - name: Restore
-      run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
-    - name: Build
-      run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
+    - name: Restore
+      run: shopt -s globstar; for f in ./**/*.csproj; do dotnet restore $f  ; done
+      shell: bash
+    - name: Build
+      run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
+      shell: bash
```

## Custom transformers for environment variables

We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `BUILDCONFIGURATION` environment variable to be `Debug` instead of `Release`.

To do this, add the following code to the `transformers.rb` file.

```ruby
env "BUILDCONFIGURATION", "Debug"
```

In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option.  When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:

```diff
env:
-  BUILDCONFIGURATION: Release
+  BUILDCONFIGURATION: Debug
  BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
```

## Custom transformers for runners

Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:

1. What is label of the runner in Azure DevOps to update?
    - __windows-latest__

2. What is the label of the runner in Actions to use instead?
    - __ubuntu-latest__

With these questions answered, we can add the following code to the `transformers.rb` file:

```ruby
runner "windows-latest", "ubuntu-latest"
```

In this example, the first parameter to the `runner` method is the Azure DevOps label and the second is the Actions runner label.

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option.  When you open the converted workflow the `runs-on` statement will use the customized runner label:

```diff
runs-on:
-  - windows-latest
+  - ubuntu-latest
```

At this point of the lab the file contents of `transformers.rb` should match this:

<details>
  <summary><em>Custom transformers 👇</em></summary>

```ruby
transform "DotNetCoreCLI@2" do |item|
  projects = item["projects"]
  command = item["command"]
  run_command = []

  if projects.include?("$")
    command = "build" if command.nil?
    run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
  else
    run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
  end

  {
    shell: "bash",
    run:   run_command.join("\n")
  }
end

env "BUILDCONFIGURATION", "Debug"

runner "windows-latest", "ubuntu-latest"
```

</details>

Thats it! At this point you have overridden Valet's default behavior by customizing the conversion of:

- Build steps
- Environment variables
- Runners

## Next lab

[Perform a production migration of an Azure DevOps pipeline](5-migrate.md)