importer-labs/jenkins/4-custom-transformers.md

Use custom transformers to customize Valet's behavior

In this lab we will build upon the dry-run command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:

1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.

Prerequisites

1. Followed the steps here to set up your Codespace environment and start a Jenkins server.
2. Completed the configure lab.
3. Completed the dry-run lab.

Perform a dry-run

We will be performing a dry-run command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:

```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run
```

The converted workflow that is generated by the above command can be seen below

Converted workflow 👇

name: test_pipeline
on:
  push:
    paths: "*"
  schedule:
  - cron: 0-29/10 * * * *
env:
  DISABLE_AUTH: 'true'
  DB_ENGINE: sqlite
jobs:
  build:
    runs-on:
      - self-hosted
      - TeamARunner
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: echo message
      run: echo "Database engine is ${{ env.DB_ENGINE }}"
#     # This item has no matching transformer
#     - sleep:
#       - key: time
#         value:
#           isLiteral: true
#           value: 80
    - name: echo message
      run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
  test:
    runs-on:
      - self-hosted
      - TeamARunner
    needs: build
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: Publish test results
      uses: EnricoMi/publish-unit-test-result-action@v1.7
      if: always()
      with:
        files: "**/target/*.xml"

Note: You can refer to the previous lab to learn about the fundamentals of the dry-run command.

Custom transformers for an unknown step

The converted workflow above contains a sleep step was not automatically converted. We will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?

   • sleep. The identifier will be the key of a key value pair within the step of a Jenkinsfile.

2. What is the desired Actions syntax to use instead?

   • After some research, we have determined that the following bash script will provide similar functionality:

     - name: Sleep for 80 seconds
       run: sleep 80s
       shell: bash
     

Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the .rb file extension. You can create this file by running the following command in your codespace terminal:

code transformers.rb

Next, we will define a transform method for the sleep identifier by adding the following code to transformers.rb:

transform "sleep" do |item|
  wait_time = item["arguments"][0]["value"]["value"]

  {
    "name": "Sleep for #{wait_time} seconds",
    "run": "sleep #{wait_time}s",
    "shell": "bash"
  }
end

This method can use any valid ruby syntax and should return a Hash that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the item parameter for the original values configured in Jenkins.

Now, we can perform another dry-run command and use the --custom-transformers CLI option to provide this custom transformer. Run the following command within your codespace terminal:

gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run --custom-transformers transformers.rb

Open the workflow that is generated and inspect the contents. Now, the sleep step is converted and uses the customized behavior!

- #     # This item has no matching transformer
- #     - sleep:
- #       - key: time
- #         value:
- #           isLiteral: true
- #           value: 80
+  - name: Sleep for 80 seconds
+    run: sleep 80s
+    shell: bash

Custom transformers for a known step

We can also override Valet's default behavior. In this scenario, we may not desire to use the third-party action for publishing junit test results that is used by default. Again, we will need to answer the following questions before writing a custom transformer:

1. What is the "identifier" of the step to customize?

   • junit

2. What is the desired Actions syntax to use instead?

   • After some research, we have determined that the uploading test results as an artifact will be suitable:

     - uses: actions/upload-artifact@v3
       with:
         name: junit-artifact
         path: path/to/artifact/world.txt
     

We will build this custom transformer similar to the previous custom transformer, however, we first need to inspect the item keyword to programmatically use the file path to junit's test results in the actions/upload-artifact@v3 step.

To do this, we will print item to the console. You can achieve this by adding the following custom transformer to transformers.rb:

transform "junit" do |item|
  puts "This is the item: #{item}"
end

Now, we can perform another dry-run command with the --custom-transformers CLI option. The output of the dry-run command should look similar to this:

Now that we know the data structure of item we can access the file path programmatically by editing the custom transformer to following:

transform "junit" do |item|
  test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
  file_path = test_results.dig("value", "value")

  {
    "uses" => "actions/upload-artifact@v3",
    "with" => {
      "name" => "junit-artifact",
      "path" => file_path
    }
  }
end

Note: transformers.rb should contain a transform method for both sleep and junit.

Now, we can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the EnricoMi/publish-unit-test-result-action@v1.7 action will have been replaced with the customized steps.

-    - name: Publish test results
-      uses: EnricoMi/publish-unit-test-result-action@v1.7
-      if: always()
-      with:
-        files: "**/target/*.xml"
+    - uses: actions/upload-artifact@v3
+      with:
+        name: junit-artifact
+        path: path/to/artifact/world.txt

Custom transformers for environment variables

We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the DB_ENGINE environment variable to be mongodb instead of sqlite.

To do this, add the following code to the transformers.rb file.

env "DB_ENGINE", "mongodb"

In this example, the first parameter to the env method is the environment variable name and the second is the updated value.

Now, we can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the DB_ENGINE environment variable will be set to mongodb:

env:
  DISABLE_AUTH: 'true'
-  DB_ENGINE: sqlite
+  DB_ENGINE: mongodb

Custom transformers for runners

Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:

1. What is label of the runner in Jenkins to update?

   • TeamARunner

2. What is the label of the runner in Actions to use instead?

   • ubuntu-latest

With these questions answered, we can add the following code to the transformers.rb file:

runner "TeamARunner", "ubuntu-latest"

In this example, the first parameter to the runner method is the Jenkins label and the second is the Actions runner label.

Now, we can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the runs-on statement will use the customized runner label:

runs-on:
-  - self-hosted
-  - TeamARunner
+  - ubuntu-latest

At this point of the lab the file contents of transformers.rb should match this:

Custom transformers 👇

  runner "TeamARunner", "ubuntu-latest"

  env "DB_ENGINE", "mongodb"

  transform "sleep" do |item|
      wait_time = item["arguments"][0]["value"]["value"]

      {
        "name": "Sleep for #{wait_time} seconds",
        "run": "sleep #{wait_time}s",
        "shell": "bash"
      }
  end

  transform "junit" do |item|
    test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
    file_path = test_results.dig("value", "value")

    [
      {
        "uses": "actions/upload-artifact@v3",
        "with": {
          "name": "my-artifact",
          "path": file_path
        }
      }
    ]
  end

Thats it! Congratulations you have overridden Valet's default behavior by customizing the conversion of:

• Unknown steps
• Known steps
• Environment variables
• Runners

Next lab

Perform a production migration of a Jenkins pipeline