294 lines
9.5 KiB
Markdown
294 lines
9.5 KiB
Markdown
# Use custom transformers to customize Valet's behavior
|
|
|
|
In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:
|
|
|
|
1. Convert items that are not automatically converted.
|
|
2. Convert items that were automatically converted using different actions.
|
|
3. Convert environment variable values differently.
|
|
4. Convert references to runners to use a different runner name in Actions.
|
|
|
|
## Prerequisites
|
|
|
|
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
|
|
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
|
|
3. Completed the [dry-run lab](./3-dry-run.md).
|
|
|
|
## Perform a dry-run
|
|
|
|
We will be performing a `dry-run` command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:
|
|
|
|
```bash
|
|
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run
|
|
```
|
|
|
|
The converted workflow that is generated by the above command can be seen below:
|
|
|
|
<details>
|
|
<summary><em>Converted workflow 👇</em></summary>
|
|
|
|
```yaml
|
|
name: test_pipeline
|
|
on:
|
|
push:
|
|
paths: "*"
|
|
schedule:
|
|
- cron: 0-29/10 * * * *
|
|
env:
|
|
DISABLE_AUTH: 'true'
|
|
DB_ENGINE: sqlite
|
|
jobs:
|
|
build:
|
|
runs-on:
|
|
- self-hosted
|
|
- TeamARunner
|
|
steps:
|
|
- name: checkout
|
|
uses: actions/checkout@v2
|
|
- name: echo message
|
|
run: echo "Database engine is ${{ env.DB_ENGINE }}"
|
|
# # This item has no matching transformer
|
|
# - sleep:
|
|
# - key: time
|
|
# value:
|
|
# isLiteral: true
|
|
# value: 80
|
|
- name: echo message
|
|
run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
|
|
test:
|
|
runs-on:
|
|
- self-hosted
|
|
- TeamARunner
|
|
needs: build
|
|
steps:
|
|
- name: checkout
|
|
uses: actions/checkout@v2
|
|
- name: Publish test results
|
|
uses: EnricoMi/[email protected]
|
|
if: always()
|
|
with:
|
|
files: "**/target/*.xml"
|
|
```
|
|
|
|
</details>
|
|
|
|
_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.
|
|
|
|
## Custom transformers for an unknown step
|
|
|
|
The converted workflow above contains a `sleep` step was not automatically converted. We will need to answer the following questions before writing a custom transformer:
|
|
|
|
1. What is the "identifier" of the step to customize?
|
|
- __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile.
|
|
|
|
2. What is the desired Actions syntax to use instead?
|
|
- After some research, we have determined that the following bash script will provide similar functionality:
|
|
|
|
```yaml
|
|
- name: Sleep for 80 seconds
|
|
run: sleep 80s
|
|
shell: bash
|
|
```
|
|
|
|
Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
|
|
|
```bash
|
|
code transformers.rb
|
|
```
|
|
|
|
Next, we will define a `transform` method for the `sleep` identifier by adding the following code to `transformers.rb`:
|
|
|
|
```ruby
|
|
transform "sleep" do |item|
|
|
wait_time = item["arguments"][0]["value"]["value"]
|
|
|
|
{
|
|
"name": "Sleep for #{wait_time} seconds",
|
|
"run": "sleep #{wait_time}s",
|
|
"shell": "bash"
|
|
}
|
|
end
|
|
```
|
|
|
|
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Jenkins.
|
|
|
|
Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
|
|
|
|
```bash
|
|
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run --custom-transformers transformers.rb
|
|
```
|
|
|
|
Open the workflow that is generated and inspect the contents. Now, the `sleep` step is converted and uses the customized behavior!
|
|
|
|
```diff
|
|
- # # This item has no matching transformer
|
|
- # - sleep:
|
|
- # - key: time
|
|
- # value:
|
|
- # isLiteral: true
|
|
- # value: 80
|
|
+ - name: Sleep for 80 seconds
|
|
+ run: sleep 80s
|
|
+ shell: bash
|
|
```
|
|
|
|
## Custom transformers for a known step
|
|
|
|
We can also override Valet's default behavior. In this scenario, we may not desire to use the third-party action for publishing junit test results that is used by default. Again, we will need to answer the following questions before writing a custom transformer:
|
|
|
|
1. What is the "identifier" of the step to customize?
|
|
- __junit__
|
|
|
|
2. What is the desired Actions syntax to use instead?
|
|
- After some research, we have determined that the uploading test results as an artifact will be suitable:
|
|
|
|
```yaml
|
|
- uses: actions/upload-artifact@v3
|
|
with:
|
|
name: junit-artifact
|
|
path: path/to/artifact/world.txt
|
|
```
|
|
|
|
We will build this custom transformer similar to the previous custom transformer, however, we first need to inspect the `item` keyword to programmatically use the file path to junit's test results in the `actions/upload-artifact@v3` step.
|
|
|
|
To do this, we will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
|
|
|
|
```ruby
|
|
transform "junit" do |item|
|
|
puts "This is the item: #{item}"
|
|
end
|
|
```
|
|
|
|
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
|
|
|
|
|
|
|
|
Now that we know the data structure of `item` we can access the file path programmatically by editing the custom transformer to following:
|
|
|
|
```ruby
|
|
transform "junit" do |item|
|
|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
|
|
file_path = test_results.dig("value", "value")
|
|
|
|
{
|
|
"uses" => "actions/upload-artifact@v3",
|
|
"with" => {
|
|
"name" => "junit-artifact",
|
|
"path" => file_path
|
|
}
|
|
}
|
|
end
|
|
```
|
|
|
|
_Note_: `transformers.rb` should contain a `transform` method for both `sleep` and `junit`.
|
|
|
|
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `EnricoMi/publish-unit-test-result-action@v1.7` action will have been replaced with the customized steps.
|
|
|
|
```diff
|
|
- - name: Publish test results
|
|
- uses: EnricoMi/[email protected]
|
|
- if: always()
|
|
- with:
|
|
- files: "**/target/*.xml"
|
|
+ - uses: actions/upload-artifact@v3
|
|
+ with:
|
|
+ name: junit-artifact
|
|
+ path: path/to/artifact/world.txt
|
|
```
|
|
|
|
## Custom transformers for environment variables
|
|
|
|
We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `DB_ENGINE` environment variable to be `mongodb` instead of `sqlite`.
|
|
|
|
To do this, add the following code to the `transformers.rb` file.
|
|
|
|
```ruby
|
|
env "DB_ENGINE", "mongodb"
|
|
```
|
|
|
|
In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.
|
|
|
|
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:
|
|
|
|
```diff
|
|
env:
|
|
DISABLE_AUTH: 'true'
|
|
- DB_ENGINE: sqlite
|
|
+ DB_ENGINE: mongodb
|
|
```
|
|
|
|
## Custom transformers for runners
|
|
|
|
Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:
|
|
|
|
1. What is label of the runner in Jenkins to update?
|
|
- __TeamARunner__
|
|
|
|
2. What is the label of the runner in Actions to use instead?
|
|
- __ubuntu-latest__
|
|
|
|
With these questions answered, we can add the following code to the `transformers.rb` file:
|
|
|
|
```ruby
|
|
runner "TeamARunner", "ubuntu-latest"
|
|
```
|
|
|
|
In this example, the first parameter to the `runner` method is the Jenkins label and the second is the Actions runner label.
|
|
|
|
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
|
|
|
|
```diff
|
|
runs-on:
|
|
- - self-hosted
|
|
- - TeamARunner
|
|
+ - ubuntu-latest
|
|
```
|
|
|
|
At this point of the lab the file contents of `transformers.rb` should match this:
|
|
|
|
<details>
|
|
<summary><em>Custom transformers 👇</em></summary>
|
|
|
|
```ruby
|
|
runner "TeamARunner", "ubuntu-latest"
|
|
|
|
env "DB_ENGINE", "mongodb"
|
|
|
|
transform "sleep" do |item|
|
|
wait_time = item["arguments"][0]["value"]["value"]
|
|
|
|
{
|
|
"name": "Sleep for #{wait_time} seconds",
|
|
"run": "sleep #{wait_time}s",
|
|
"shell": "bash"
|
|
}
|
|
end
|
|
|
|
transform "junit" do |item|
|
|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
|
|
file_path = test_results.dig("value", "value")
|
|
|
|
[
|
|
{
|
|
"uses": "actions/upload-artifact@v3",
|
|
"with": {
|
|
"name": "my-artifact",
|
|
"path": file_path
|
|
}
|
|
}
|
|
]
|
|
end
|
|
```
|
|
|
|
</details>
|
|
|
|
Thats it! Congratulations you have overridden Valet's default behavior by customizing the conversion of:
|
|
|
|
- Unknown steps
|
|
- Known steps
|
|
- Environment variables
|
|
- Runners
|
|
|
|
## Next lab
|
|
|
|
[Perform a production migration of a Jenkins pipeline](5-migrate.md)
|