importer-labs/jenkins/4-custom-transformers.md

# Use custom transformers to customize Valet's behavior

In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:

1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.

## Prerequisites

1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [dry-run lab](./3-dry-run.md).

## Perform a dry-run

We will be performing a `dry-run` command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:

    ```bash
    gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run
    ```

The converted workflow that is generated by the above command can be seen below

<details>
  <summary><em>Converted workflow 👇</em></summary>

```yaml
name: test_pipeline
on:
  push:
    paths: "*"
  schedule:
  - cron: 0-29/10 * * * *
env:
  DISABLE_AUTH: 'true'
  DB_ENGINE: sqlite
jobs:
  build:
    runs-on:
      - self-hosted
      - TeamARunner
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: echo message
      run: echo "Database engine is ${{ env.DB_ENGINE }}"
#     # This item has no matching transformer
#     - sleep:
#       - key: time
#         value:
#           isLiteral: true
#           value: 80
    - name: echo message
      run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
  test:
    runs-on:
      - self-hosted
      - TeamARunner
    needs: build
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: Publish test results
      uses: EnricoMi/[email protected]
      if: always()
      with:
        files: "**/target/*.xml"
```

</details>

_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.

## Custom transformers for an unknown step

The converted workflow above contains a `sleep` step was not automatically converted. We will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?
    - __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile.

2. What is the desired Actions syntax to use instead?
    - After some research, we have determined that the following bash script will provide similar functionality:

      ```yaml
      - name: Sleep for 80 seconds
        run: sleep 80s
        shell: bash
      ```

Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:

```bash
code transformers.rb
```

Next, we will define a `transform` method for the `sleep` identifier by adding the following code to `transformers.rb`:

```ruby
transform "sleep" do |item|
  wait_time = item["arguments"][0]["value"]["value"]

  {
    "name": "Sleep for #{wait_time} seconds",
    "run": "sleep #{wait_time}s",
    "shell": "bash"
  }
end
```

This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Jenkins.

Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:

```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run --custom-transformers transformers.rb
```

Open the workflow that is generated and inspect the contents. Now, the `sleep` step is converted and uses the customized behavior!

```diff
- #     # This item has no matching transformer
- #     - sleep:
- #       - key: time
- #         value:
- #           isLiteral: true
- #           value: 80
+  - name: Sleep for 80 seconds
+    run: sleep 80s
+    shell: bash
```

## Custom transformers for a known step

We can also override Valet's default behavior. In this scenario, we may not desire to use the third-party action for publishing junit test results that is used by default. Again, we will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?
    - __junit__

2. What is the desired Actions syntax to use instead?
    - After some research, we have determined that the uploading test results as an artifact will be suitable:

      ```yaml
      - uses: actions/upload-artifact@v3
        with:
          name: junit-artifact
          path: path/to/artifact/world.txt
      ```

We will build this custom transformer similar to the previous custom transformer, however, we first need to inspect the `item` keyword to programmatically use the file path to junit's test results in the `actions/upload-artifact@v3` step.

To do this, we will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:

```ruby
transform "junit" do |item|
  puts "This is the item: #{item}"
end
```

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:

![img](https://user-images.githubusercontent.com/19557880/186782050-65ece0c4-52a3-4f88-818f-0f860c50c2b7.png)

Now that we know the data structure of `item` we can access the file path programmatically by editing the custom transformer to following:

```ruby
transform "junit" do |item|
  test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
  file_path = test_results.dig("value", "value")

  {
    "uses" => "actions/upload-artifact@v3",
    "with" => {
      "name" => "junit-artifact",
      "path" => file_path
    }
  }
end
```

_Note_: `transformers.rb` should contain a `transform` method for both `sleep` and `junit`.

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `EnricoMi/publish-unit-test-result-action@v1.7` action will have been replaced with the customized steps.

```diff
-    - name: Publish test results
-      uses: EnricoMi/[email protected]
-      if: always()
-      with:
-        files: "**/target/*.xml"
+    - uses: actions/upload-artifact@v3
+      with:
+        name: junit-artifact
+        path: path/to/artifact/world.txt
```

## Custom transformers for environment variables

We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `DB_ENGINE` environment variable to be `mongodb` instead of `sqlite`.

To do this, add the following code to the `transformers.rb` file.

```ruby
env "DB_ENGINE", "mongodb"
```

In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option.  When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:

```diff
env:
  DISABLE_AUTH: 'true'
-  DB_ENGINE: sqlite
+  DB_ENGINE: mongodb
```

## Custom transformers for runners

Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:

1. What is label of the runner in Jenkins to update?
    - __TeamARunner__

2. What is the label of the runner in Actions to use instead?
    - __ubuntu-latest__

With these questions answered, we can add the following code to the `transformers.rb` file:

```ruby
runner "TeamARunner", "ubuntu-latest"
```

In this example, the first parameter to the `runner` method is the Jenkins label and the second is the Actions runner label.

Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option.  When you open the converted workflow the `runs-on` statement will use the customized runner label:

```diff
runs-on:
-  - self-hosted
-  - TeamARunner
+  - ubuntu-latest
```

At this point of the lab the file contents of `transformers.rb` should match this:

<details>
  <summary><em>Custom transformers 👇</em></summary>

```ruby
  runner "TeamARunner", "ubuntu-latest"

  env "DB_ENGINE", "mongodb"

  transform "sleep" do |item|
      wait_time = item["arguments"][0]["value"]["value"]

      {
        "name": "Sleep for #{wait_time} seconds",
        "run": "sleep #{wait_time}s",
        "shell": "bash"
      }
  end

  transform "junit" do |item|
    test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
    file_path = test_results.dig("value", "value")

    [
      {
        "uses": "actions/upload-artifact@v3",
        "with": {
          "name": "my-artifact",
          "path": file_path
        }
      }
    ]
  end
```

</details>

Thats it! Congratulations you have overridden Valet's default behavior by customizing the conversion of:

- Unknown steps
- Known steps
- Environment variables
- Runners

## Next lab

[Perform a production migration of a Jenkins pipeline](5-migrate.md)