actions/importer-labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Edit azdo custom transformers lab

Browse Source
This commit is contained in:
Ethan Dennis
2022-09-07 13:16:25 -07:00
parent 35905b5174
commit ac8d383cab
2 changed files with 200 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
azure_devops/4-custom-transformers.md
+192 -47
View File
@@ -1,54 +1,201 @@
# Migrate an Azure DevOps pipeline to GitHub Actions with a custom transformer # Using custom transformers to customize Valet's behavior
In this lab, you will create a custom plugin that transforms some of the existing migration mapping and replaces it with your own mapping. Then use the `migrate` subcommand to migrate the pipeline. The `migrate` subcommand can be used to convert a pipeline to its GitHub Actions equivalent and then create a pull request with the contents.
- [Prerequisites](#prerequisites) In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:
- [Identify the Azure DevOps pipeline ID to use](#identify-the-azure-devops-pipeline-id-to-use)
- [Create a custom transformer](#create-a-custom-transformer) 1. Convert items that are not automatically converted.
- [Migrate with a custom transformer](#migrate-with-a-custom-transformer) 2. Convert items that were automatically converted using different actions.
- [View the pull request](#view-the-pull-request) 3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.
## Prerequisites ## Prerequisites
1. Follow all steps [here](../azure_devops#readme) to set up your environment 1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and bootstrap an Azure DevOps project.
2. Create or start a codespace in this repository (if not started) 2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Complete the [Valet audit lab](valet-audit-lab.md). 3. Completed the [audit lab](./2-audit.md).
4. Complete the [Valet migrate lab](valet-migrate-lab.md). 4. Completed the [dry-run lab](./3-dry-run.md).
## Identify the Azure DevOps pipeline ID to use ## Perform a dry run
You will need the `valet-custom-transformer-example` Azure DevOps pipeline ID to perform the migration
1. Go to the `valet/ValetBootstrap/pipelines` folder
2. Open the `valet/ValetBootstrap/pipelines/valet-custom-transformer-example.config.json` file
3. Look for the `web - href` link
4. At the end of the link is the pipeline ID. Copy or note the ID.
### Example We will be performing a dry-run for a pipeline in the bootstrapped Azure DevOps project. We will need to answer the following questions before running this command:
![mapperprops](https://user-images.githubusercontent.com/26442605/175090567-525b97a7-60d2-41b7-9dcd-d559ca1c5bd7.png)
## Create a custom transformer 1. What is the id of the pipeline to convert?
- __:pipeline_id__. This id can be found by:
- Navigating to the build pipelines in the bootstrapped Azure DevOps project <https://dev.azure.com/:organization/:project/_build>
- Selecting the pipeline with the name "valet-custom-transformer-example"
- Inspecting the URL to locate the pipeline id <https://dev.azure.com/:organization/:project/_build?definitionId=:pipeline_id>
To create a transformer, you need to create a Ruby file that looks as follows: 2. Where do we want to store the result?
``` ruby - __./tmp/dry-run-lab__. This can be any path within the working directory that Valet commands are executed from.
transform "azuredevopstaskname" do |item|
# your ruby code here that produces output ### Steps
end
1. Navigate to the codespace terminal
2. Run the following command from the root directory:
```bash
gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab
``` ```
We start by creating a new folder called `plugin` under the `valet` folder in your repository. In there create a file called `DotNetCoreCLI.rb`. 3. The command will list all the files written to disk when the command succeeds.
4. View the converted workflow:
- Find `./tmp/dry-run-lab` in the file explorer pane in codespaces.
- Click `valet-custom-transformer-example.yml` to open.
Next change the function name to match the Azure DevOps task name `DotNetCoreCLI@2`. The converted workflow that is generated can be seen below:
The way you find this name is by clicking the **view yaml** button at a task in the pipeline:
<details>
<summary><em>Converted workflow 👇</em></summary>
```yaml
name: valet-bootstrap/pipelines/valet-custom-transformer-example
on:
push:
branches:
- "*"
env:
BUILDCONFIGURATION: Release
BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
jobs:
Job_1:
name: Agent job 1
runs-on: windows-latest
steps:
- name: checkout
uses: actions/checkout@v2
- uses: actions/checkout@v2
- name: Use Node 10.16.3
uses: actions/setup-node@v2
with:
node-version: 10.16.3
- name: Restore
run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
- name: Build
run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
```
</details>
_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.
## Custom transformers for build steps
We can use custom transformers override Valet's default behavior. In this scenario, we may want to override the behavior for converting `DotnetCoreCLI@2` tasks to support parameters that are glob patterns. We will need to answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __DotnetCoreCLI@2__
2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the uploading test results as an artifact will be suitable:
```yaml
- run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
shell: bash
```
Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
```bash
code transformers.rb
```
Next, we will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`:
This results in the following code:
```ruby ```ruby
transform "DotNetCoreCLI@2" do |item| transform "DotNetCoreCLI@2" do |item|
# your ruby code here that produces output projects = item["projects"]
command = item["command"]
run_command = []
if projects.include?("$")
command = "build" if command.nil?
run_command << "shopt -s globstar; for f in ./**/*.csproj; do dotnet #{command} $f #{item['arguments']} ; done"
else
run_command << "dotnet #{command} #{item['projects']} #{item['arguments']}"
end
{
run: run_command.join("\n"),
shell: "bash",
}
end end
``` ```
The parameter item is a collection of items than contain the properties of the original task that was retrieved from Azure DevOps.
In this case we can see in the yaml that the properties that are set are `command` and `projects`.
Add the following code to the ruby file: This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
``` Ruby
Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
```bash
gh valet dry-run azure-devops pipeline --pipeline-id :pipeline_id -o tmp/dry-run-lab --custom-transformers transformers.rb
```
Open the workflow that is generated and inspect the contents. Now, the `DotnetCoreCLI@2` steps are converted using the customized behavior!
```diff
- - name: Restore
- run: dotnet restore ${{ env.BuildParameters_RESTOREBUILDPROJECTS }}
- - name: Build
- run: dotnet build ${{ env.BuildParameters_RESTOREBUILDPROJECTS }} --configuration ${{ env.BUILDCONFIGURATION }}
+ - name: Restore
+ run: shopt -s globstar; for f in ./**/*.csproj; do dotnet restore $f ; done
+ shell: bash
+ - name: Build
+ run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
+ shell: bash
```
## Custom transformers for environment variables
We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `BUILDCONFIGURATION` environment variable to be `Debug` instead of `Release`.
To do this, add the following code to the `transformers.rb` file.
```ruby
env "BUILDCONFIGURATION", "Debug"
```
In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:
```diff
env:
- BUILDCONFIGURATION: Release
+ BUILDCONFIGURATION: Debug
BuildParameters_RESTOREBUILDPROJECTS: "**/*.csproj"
```
## Custom transformers for runners
Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:
1. What is label of the runner in Azure DevOps to update?
- __windows-latest__
2. What is the label of the runner in Actions to use instead?
- __ubuntu-latest__
With these questions answered, we can add the following code to the `transformers.rb` file:
```ruby
runner "windows-latest", "ubuntu-latest"
```
In this example, the first parameter to the `runner` method is the Azure DevOps label and the second is the Actions runner label.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
```diff
runs-on:
- - windows-latest
+ - ubuntu-latest
```
At this point of the lab the file contents of `transformers.rb` should match this:
<details>
<summary><em>Custom transformers 👇</em></summary>
```ruby
transform "DotNetCoreCLI@2" do |item| transform "DotNetCoreCLI@2" do |item|
projects = item["projects"] projects = item["projects"]
command = item["command"] command = item["command"]
@@ -66,22 +213,20 @@ transform "DotNetCoreCLI@2" do |item|
run: run_command.join("\n") run: run_command.join("\n")
} }
end end
env "BUILDCONFIGURATION", "Debug"
runner "windows-latest", "ubuntu-latest"
``` ```
## Migrate with a custom transformer </details>
Run the `gh valet migrate` command from the `valet` directory with the transformer again and pass it the custom plugin. Look at the result and see if it results in a successful build. Thats it! At this point you have overridden Valet's default behavior by customizing the conversion of:
``` - Build steps
gh valet migrate azure-devops pipeline --target-url https://github.com/GITHUB-ORG/GITHUB-REPO --pipeline-id PIPELINE-ID --custom-transformers plugin/DotNetCoreCLI.rb --output-dir migrate - Environment variables
``` - Runners
### Example ## Next lab
![valet-cm-1](https://user-images.githubusercontent.com/26442605/169618556-7c79b34b-6d4c-48d5-98e5-7f8d771117a5.png)
## View the pull request
The migrate output will show you the pull request on GitHub. Note here that the checks on the pull request all passed!
### Example
![mapper-ex3](https://user-images.githubusercontent.com/26442605/161117488-93e38847-3034-4f04-a768-e74e16dba4ae.png)
[Perform a production migration of an Azure DevOps pipeline](5-migrate.md)
jenkins/4-custom-transformers.md
+1 -1
View File
@@ -139,7 +139,7 @@ We can also override Valet's default behavior. In this scenario, we may not desi
- __junit__ - __junit__
2. What is the desired Actions syntax to use instead? 2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the uploading test results as an artifact will be suitable: - After some research, we have determined that uploading test results as an artifact will be suitable:
```yaml ```yaml
- uses: actions/upload-artifact@v3 - uses: actions/upload-artifact@v3