actions/importer-labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Edit azdo custom transformers lab

Browse Source
This commit is contained in:
Ethan Dennis
2022-09-07 13:16:25 -07:00
parent 35905b5174
commit ac8d383cab
2 changed files with 200 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
azure_devops/4-custom-transformers.md
+199 -54
View File
@@ -1,54 +1,106 @@
# Migrate an Azure DevOps pipeline to GitHub Actions with a custom transformer
In this lab, you will create a custom plugin that transforms some of the existing migration mapping and replaces it with your own mapping. Then use the `migrate` subcommand to migrate the pipeline. The `migrate` subcommand can be used to convert a pipeline to its GitHub Actions equivalent and then create a pull request with the contents.
# Using custom transformers to customize Valet's behavior
- [Prerequisites](#prerequisites)
- [Identify the Azure DevOps pipeline ID to use](#identify-the-azure-devops-pipeline-id-to-use)
- [Create a custom transformer](#create-a-custom-transformer)
- [Migrate with a custom transformer](#migrate-with-a-custom-transformer)
- [View the pull request](#view-the-pull-request)
In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:
1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.
## Prerequisites
1. Follow all steps [here](../azure_devops#readme) to set up your environment
2. Create or start a codespace in this repository (if not started)
3. Complete the [Valet audit lab](valet-audit-lab.md).
4. Complete the [Valet migrate lab](valet-migrate-lab.md).
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and bootstrap an Azure DevOps project.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [audit lab](./2-audit.md).
4. Completed the [dry-run lab](./3-dry-run.md).
## Identify the Azure DevOps pipeline ID to use
You will need the `valet-custom-transformer-example` Azure DevOps pipeline ID to perform the migration
1. Go to the `valet/ValetBootstrap/pipelines` folder
2. Open the `valet/ValetBootstrap/pipelines/valet-custom-transformer-example.config.json` file
3. Look for the `web - href` link
4. At the end of the link is the pipeline ID. Copy or note the ID.
## Perform a dry run
### Example
![mapperprops](https://user-images.githubusercontent.com/26442605/175090567-525b97a7-60d2-41b7-9dcd-d559ca1c5bd7.png)
We will be performing a dry-run for a pipeline in the bootstrapped Azure DevOps project. We will need to answer the following questions before running this command:
## Create a custom transformer
1. What is the id of the pipeline to convert?
- __:pipeline_id__. This id can be found by:
- Navigating to the build pipelines in the bootstrapped Azure DevOps project <https://dev.azure.com/:organization/:project/_build>
- Selecting the pipeline with the name "valet-custom-transformer-example"
- Inspecting the URL to locate the pipeline id <https://dev.azure.com/:organization/:project/_build?definitionId=:pipeline_id>
To create a transformer, you need to create a Ruby file that looks as follows:
``` ruby
transform "azuredevopstaskname" do |item|
# your ruby code here that produces output
end
```
2. Where do we want to store the result?
- __./tmp/dry-run-lab__. This can be any path within the working directory that Valet commands are executed from.
We start by creating a new folder called `plugin` under the `valet` folder in your repository. In there create a file called `DotNetCoreCLI.rb`.
### Steps
Next change the function name to match the Azure DevOps task name `DotNetCoreCLI@2`.
The way you find this name is by clicking the **view yaml** button at a task in the pipeline:
1. Navigate to the codespace terminal
2. Run the following command from the root directory:
This results in the following code:
``` ruby
transform "DotNetCoreCLI@2" do |item|
# your ruby code here that produces output
end
```
The parameter item is a collection of items than contain the properties of the original task that was retrieved from Azure DevOps.
In this case we can see in the yaml that the properties that are set are `command` and `projects`.
Add the following code to the ruby file:
``` Ruby
```bash
gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab
```
3. The command will list all the files written to disk when the command succeeds.
4. View the converted workflow:
- Find `./tmp/dry-run-lab` in the file explorer pane in codespaces.
- Click `valet-custom-transformer-example.yml` to open.
The converted workflow that is generated can be seen below:
<details>
<summary><em>Converted workflow 👇</em></summary>
```yaml
name: valet-bootstrap/pipelines/valet-custom-transformer-example
on:
push:
branches:
- "*"
env:
BUILDCONFIGURATION: Release
BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
jobs:
Job_1:
name: Agent job 1
runs-on: windows-latest
steps:
- name: checkout
uses: actions/checkout@v2
- uses: actions/checkout@v2
- name: Use Node 10.16.3
uses: actions/setup-node@v2
with:
node-version: 10.16.3
- name: Restore
run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
- name: Build
run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
```
</details>
_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.
## Custom transformers for build steps
We can use custom transformers override Valet's default behavior. In this scenario, we may want to override the behavior for converting `DotnetCoreCLI@2` tasks to support parameters that are glob patterns. We will need to answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __DotnetCoreCLI@2__
2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the uploading test results as an artifact will be suitable:
```yaml
- run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
shell: bash
```
Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
```bash
code transformers.rb
```
Next, we will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`:
```ruby
transform "DotNetCoreCLI@2" do |item|
projects = item["projects"]
command = item["command"]
@@ -60,28 +112,121 @@ transform "DotNetCoreCLI@2" do |item|
else
run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
end
{
run: run_command.join("\n"),
shell: "bash",
}
end
```
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
```bash
gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab --custom-transformers transformers.rb
```
Open the workflow that is generated and inspect the contents. Now, the `DotnetCoreCLI@2` steps are converted using the customized behavior!
```diff
- - name: Restore
- run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
- - name: Build
- run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
+ - name: Restore
+ run: shopt -s globstar; for f in ./**/*.csproj; do dotnet restore $f ; done
+ shell: bash
+ - name: Build
+ run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
+ shell: bash
```
## Custom transformers for environment variables
We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `BUILDCONFIGURATION` environment variable to be `Debug` instead of `Release`.
To do this, add the following code to the `transformers.rb` file.
```ruby
env "BUILDCONFIGURATION", "Debug"
```
In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:
```diff
env:
- BUILDCONFIGURATION: Release
+ BUILDCONFIGURATION: Debug
BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
```
## Custom transformers for runners
Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:
1. What is label of the runner in Azure DevOps to update?
- __windows-latest__
2. What is the label of the runner in Actions to use instead?
- __ubuntu-latest__
With these questions answered, we can add the following code to the `transformers.rb` file:
```ruby
runner "windows-latest", "ubuntu-latest"
```
In this example, the first parameter to the `runner` method is the Azure DevOps label and the second is the Actions runner label.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
```diff
runs-on:
- - windows-latest
+ - ubuntu-latest
```
At this point of the lab the file contents of `transformers.rb` should match this:
<details>
<summary><em>Custom transformers 👇</em></summary>
```ruby
transform "DotNetCoreCLI@2" do |item|
projects = item["projects"]
command = item["command"]
run_command = []
if projects.include?("$")
command = "build" if command.nil?
run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
else
run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
end
{
shell: "bash",
run: run_command.join("\n")
}
end
env "BUILDCONFIGURATION", "Debug"
runner "windows-latest", "ubuntu-latest"
```
## Migrate with a custom transformer
</details>
Run the `gh valet migrate` command from the `valet` directory with the transformer again and pass it the custom plugin. Look at the result and see if it results in a successful build.
Thats it! At this point you have overridden Valet's default behavior by customizing the conversion of:
```
gh valet migrate azure-devops pipeline --target-url https://github.com/GITHUB-ORG/GITHUB-REPO --pipeline-id PIPELINE-ID --custom-transformers plugin/DotNetCoreCLI.rb --output-dir migrate
```
- Build steps
- Environment variables
- Runners
### Example
![valet-cm-1](https://user-images.githubusercontent.com/26442605/169618556-7c79b34b-6d4c-48d5-98e5-7f8d771117a5.png)
## View the pull request
The migrate output will show you the pull request on GitHub. Note here that the checks on the pull request all passed!
### Example
![mapper-ex3](https://user-images.githubusercontent.com/26442605/161117488-93e38847-3034-4f04-a768-e74e16dba4ae.png)
## Next lab
[Perform a production migration of an Azure DevOps pipeline](5-migrate.md)
jenkins/4-custom-transformers.md
+1 -1
View File
@@ -139,7 +139,7 @@ We can also override Valet's default behavior. In this scenario, we may not desi
- __junit__
2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the uploading test results as an artifact will be suitable:
- After some research, we have determined that uploading test results as an artifact will be suitable:
```yaml
- uses: actions/upload-artifact@v3