actions/importer-labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #68 from valet-customers/luke-edits

Browse Source 
Initial edits to ADO/gitlab/jenkins
This commit is contained in:
Luke Cheung Engle
2022-09-20 13:38:57 -07:00
committed by GitHub
parent d27d8c45a1 285053ac1e
commit a5c4ba9d7a
17 changed files with 113 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
azure_devops/1-configure.md
+6 -4
View File
@@ -13,11 +13,11 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- Click `Personal access tokens`.
- Select `+ New Token`
- Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days.
- Select the following scopes (you may need to `Show more scopes` to reveal all scopes):
- Select the following scopes (you may need to `Show all scopes` to reveal all scopes):
- Agents Pool: `Read`
- Build: `Read & Execute`
- Code: `Read & Write`
- Project and Team: `Read, Write, & Manage`
- Build: `Read & execute`
- Code: `Read & write`
- Project and Team: `Read, write, & manage`
- Release: `Read`
- Service Connections: `Read`
- Task Groups: `Read`
@@ -31,6 +31,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- In the left panel, click `Developer Settings`.
- Click `Personal access tokens` and then `Legacy tokens` (if present).
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
- Name your token in the `Note` field.
- Select the following scopes: `workflow` and `read:packages`.
- Click `Generate token`.
- Copy the generated PAT and save it in a safe location.
@@ -39,6 +40,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- Select the `TERMINAL` tab from within the codespace terminal.
- Run the following command: `gh valet configure`.
- Use the down arrow key to highlight `Azure DevOps`, press the spacebar to select, and then press enter to continue.
- At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter.
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 2 and press enter.
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 2 and press enter.
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
azure_devops/3-forecast.md
+1 -1
View File
@@ -35,7 +35,7 @@ Answer the following questions before running the `forecast` command:
![img](https://user-images.githubusercontent.com/18723510/187690315-6312088d-9888-4c55-9bbf-c6f2687fa547.png)
4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` command using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json).
4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json).
![img](https://user-images.githubusercontent.com/18723510/187692843-623d4bdc-8970-4348-a632-73c8b00a40f8.png)
azure_devops/5-custom-transformers.md
+27 -4
View File
@@ -87,7 +87,7 @@ You can use custom transformers to override Valet's default behavior. In this sc
- __DotnetCoreCLI@2__
2. What is the desired Actions syntax to use instead?
- After some research, you have determined that the uploading test results as an artifact will be suitable:
- After some research, you have determined that the following script will provide the desired functionality:
```yaml
- run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
@@ -100,7 +100,32 @@ Now you can begin to write the custom transformer. Custom transformers use a DSL
touch transformers.rb && code transformers.rb
```
Next, you will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`:
To build this custom transformer, you first need to inspect the `item` keyword to programmatically obtain the projects, command, and arguments to use in the `DotNetCoreCLI@2` step.
To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
```ruby
transform "DotNetCoreCLI@2" do |item|
puts "This is the item: #{item}"
end
```
The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
```console
$ gh valet dry-run azure-devops pipeline --pipeline-id 6 --output-dir tmp/dry-run --custom-transformers transformers.rb
[2022-09-20 18:39:50] Logs: 'tmp/dry-run/log/valet-20220920-183950.log'
This is the item: {"command"=>"restore", "projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)"}
This is the item: {"projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)", "arguments"=>"--configuration $(BUILDCONFIGURATION)"}
[2022-09-20 18:39:51] Output file(s):
[2022-09-20 18:39:51] tmp/dry-run/lab-test/pipelines/valet-custom-transformer-example.yml
```
In the above command you will see two instances of `item` printed to the console. This is because there are two `DotNetCoreCLI@2` steps in the pipeline. Each item listed above represents each `DotNetCoreCLI@2` step in the order that they are defined in the pipeline.
Now that you know the data structure of `item`, you can access the dotnet projects, command, and arguments programmatically by editing the custom transformer to the following:
```ruby
transform "DotNetCoreCLI@2" do |item|
@@ -122,8 +147,6 @@ transform "DotNetCoreCLI@2" do |item|
end
```
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
```bash
azure_devops/6-migrate.md
-1
View File
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert an Azure DevOps pipel
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and bootstrap an Azure DevOps project.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
3. Completed the [dry-run lab](./4-dry-run.md).
4. Completed the [custom transformers lab](./5-custom-transformers.md).
## Performing a migration
azure_devops/readme.md
+4 -4
View File
@@ -45,11 +45,11 @@ These steps **must** be completed prior to starting other labs.
- Click `Personal access tokens`.
- Select `+ New Token`
- Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days.
- Select the following scopes (you may need to `Show more scopes` to reveal all scopes):
- Select the following scopes (you may need to `Show all scopes` at the bottom of the page to reveal all scopes):
- Agents Pool: `Read`
- Build: `Read & Execute`
- Code: `Read & Write`
- Project and Team: `Read, Write, & Manage`
- Build: `Read & execute`
- Code: `Read & write`
- Project and Team: `Read, write, & manage`
- Release: `Read`
- Service Connections: `Read`
- Task Groups: `Read`
circle_ci/1-configure.md
+1 -3
View File
@@ -29,8 +29,6 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- At the CircleCI token prompt, enter the CircleCI access token from step 2 and press enter.
- At the CircleCI base url prompt, hit enter to accept the default value (`https://circleci.com`).
- At the CircleCI organization name prompt, enter `valet-labs`. This is the organization we'll be using throughout these labs.
- At the access token to fetch source code in GitHub prompt, enter the GitHub PAT generated in step 1 and press enter.
- At the GitHub instance url containing source code prompt, press enter to accept the default value (`https://github.com`).
```console
$ gh valet configure
@@ -59,7 +57,7 @@ To verify our environment is configured correctly, we are going to run the `upda
2. You should see a confirmation that you were logged into the GitHub Container Registry and Valet was updated to the latest version.
```console
$ gh valet version
$ gh valet update
Login Succeeded
latest: Pulling from valet-customers/valet-cli
Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb
circle_ci/2-audit.md
+1 -1
View File
@@ -34,7 +34,7 @@ You will be performing an `audit` for the __valet-labs__ CircleCI organization t
1. Find the `audit_summary.md` file in the file explorer.
2. Right-click the `audit_summary.md` file and select `Open Preview`.
3. This file contains details that summarizes what percentage of your pipelines were converted automatically.
3. This file contains details that summarize what percentage of your pipelines were converted automatically.
### Review audit summary
circle_ci/3-forecast.md
+1 -1
View File
@@ -12,7 +12,7 @@ In this lab you will use the `forecast` command to forecast potential GitHub Act
Answer the following questions before running the `forecast` command:
1. What is the date you want to start forecasting from?
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were trigger in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were triggered in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
2. Where do you want to store the results?
- **tmp/forecast_reports**
circle_ci/5-custom-transformers.md
+3 -3
View File
@@ -64,7 +64,7 @@ _Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the f
## Custom transformers for an unknown step
The converted workflow above contains an `codecov_codecov_upload` step that was not automatically converted. Answer the following questions before writing a custom transformer:
The converted workflow above contains a `codecov_codecov_upload` step that was not automatically converted. Answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __codecov_codecov_upload__
@@ -127,7 +127,7 @@ end
## Custom transformers for environment variables
You can use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `COVERAGE_DIR` environment variable to be `$RUNNER_TEMP/cov` instead of `tmp/cov`.
You can use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `COVERAGE_DIR` environment variable to be `$RUNNER_TEMP/cov` instead of `./tmp/cov`.
To do this, add the following code to the `transformers.rb` file.
@@ -191,7 +191,7 @@ That's it! Congratulations, you have overridden Valet's default behavior by cust
- Unknown steps
- Environment variables
- Runner
- Runners
## Next lab
circle_ci/6-migrate.md
-1
View File
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a CircleCI pipeline a
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
3. Completed the [dry-run lab](./4-dry-run.md).
4. Completed the [custom transformers lab](./5-custom-transformers.md).
## Performing a migration
gitlab/1-configure.md
+3 -2
View File
@@ -31,6 +31,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- In the left panel, click `Developer Settings`.
- Click `Personal access tokens` and then `Legacy tokens` (if present).
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
- Name your token in the `Note` field.
- Select the following scopes: `workflow` and `read:packages`.
- Click `Generate token`.
- Copy the generated PAT and save it in a safe location.
@@ -39,7 +40,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- Select the `TERMINAL` tab from within the codespace terminal window.
- Run the following command: `gh valet configure`.
- Use the down arrow key to highlight `GitLab CI`, press the spacebar to select, and then press enter to continue.
- At the prompt, enter your GitHub username and press enter.
- At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter.
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 4 and press enter.
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 4 and press enter.
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
@@ -57,7 +58,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
✔ Private token for GitLab: ***************
✔ Base url of the GitLab instance: https://gitlab.com
Environment variables successfully updated.
```
```
## Verify your environment
gitlab/4-dry-run.md
+8 -2
View File
@@ -211,7 +211,13 @@ include:
- local: /config/test.gitlab-ci.yml
```
The output of the `dry-run` command can be seen below:
Run the following command from the root directory:
```bash
gh valet dry-run gitlab --output-dir tmp/dry-run --namespace valet --project included-files-example
```
The output of the command above can be seen below:
```yaml
name: valet/included-files-example
@@ -246,7 +252,7 @@ jobs:
- run: echo "this is from a local file"
```
It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date.
It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date.
As an added challenge, try constructing and running the `dry-run` command yourself. Hint, you should only have to change the project name.
gitlab/5-custom-transformers.md
+54 -14
View File
@@ -9,7 +9,7 @@ In this lab you will build upon the `dry-run` command to override Valet's defaul
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start you GitLab server.
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start your GitLab server.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
3. Completed the [dry-run lab](./4-dry-run.md).
@@ -63,7 +63,7 @@ The converted workflow above contains an `artifacts.terraform` step that was not
- __artifacts.terraform__
2. What is the desired Actions syntax to use instead?
- After some research, you have determined that the following bash script will provide similar functionality:
- After some research, you have determined that the following action will provide similar functionality:
```yaml
- uses: actions/upload-artifact@v3
@@ -77,12 +77,34 @@ Now you can begin to write the custom transformer. Custom transformers use a DSL
touch transformers.rb && code transformers.rb
```
Next, you will define a `transform` method for the `artifacts.terraform` identifier by adding the following code to `transformers.rb`:
To build this custom transformer, you first need to inspect the `item` keyword to programmatically use the file path of the terraform json in the `actions/upload-artifact@v3` step.
To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
```ruby
transform "artifacts.terraform" do |item|
puts "This is the item: #{item}"
end
```
The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab.
Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
```console
$ gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example --custom-transformers transformers.rb
[2022-09-20 17:47:55] Logs: 'tmp/log/valet-20220920-174755.log'
This is the item: $PLAN_JSON
[2022-09-20 17:47:56] Output file(s):
[2022-09-20 17:47:56] tmp/valet/terraform-example.yml
```
Now that you know the data structure of `item`, you can access the file path programmatically by editing the custom transformer to the following:
```ruby
transform "artifacts.terraform" do |item|
{
uses: "actions/upload-artifact@v2",
uses: "actions/upload-artifact@v3",
with: {
path: item
}
@@ -90,8 +112,6 @@ transform "artifacts.terraform" do |item|
end
```
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab.
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
```bash
@@ -107,14 +127,6 @@ The converted workflow that is generated by the above command will now use the c
+ path: "$PLAN_JSON"
```
_Note_: If you were unsure what the data structure of `item` was, you could use the following code in the custom transformer to print `item` to the console:
```ruby
transform "artifacts.terraform" do |item|
puts item
end
```
## Custom transformers for environment variables
You can also use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `PLAN_JSON` environment variable to be `custom_plan.json` instead of `plan.json`.
@@ -136,12 +148,40 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C
+ PLAN_JSON: custom_plan.json
```
## Custom transformers for runners
Finally, you can use custom transformers to dictate which runners the converted workflows should use. To do this, answer the following questions:
1. What is label of the runner in GitLab to update?
- __:default__. This is a special keyword to define the default runner to use. You can optional target specific `tags` in a job.
2. What is the label of the runner in Actions to use instead?
- __custom-runner__
With these questions answered, you can add the following code to the `transformers.rb` file:
```ruby
runner :default, "custom-runner"
```
In this example, the first parameter to the `runner` method is the GitLab label and the second is the Actions runner label.
Now you can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
```diff
runs-on:
- - ubuntu-latest
+ - custom-runner
```
At this point, the file contents of `transformers.rb` should match this:
<details>
<summary><em>Custom transformers 👇</em></summary>
```ruby
runner :default, "custom-runner"
env "PLAN_JSON", "custom_plan.json"
transform "artifacts.terraform" do |item|
gitlab/6-migrate.md
-1
View File
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a GitLab pipeline and
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a GitLab server.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
3. Completed the [dry-run lab](./4-dry-run.md).
4. Completed the [custom transformers lab](./5-custom-transformers.md).
## Performing a migration
jenkins/1-configure.md
+2 -1
View File
@@ -33,11 +33,12 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
- Select the `TERMINAL` tab from within the codespace terminal window.
- Run the following command: `gh valet configure`.
- Use the down arrow key to highlight `Jenkins`, press the spacebar to select, and then press enter to continue.
- At the prompt, enter your GitHub username and press enter.
- At the GitHub handle prompt, enter the GitHub username used to generate the GitHub PAT in step 3 and press enter.
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 3 and press enter.
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 3 and press enter.
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
- At the Jenkins token prompt, enter the Jenkins access token from step 2 and press enter.
- At the Jenkins username prompt, enter `admin` and press enter.
- At the Jenkins URL prompt, enter `http://localhost:8080/` and press enter.
```console
jenkins/5-custom-transformers.md
+2 -2
View File
@@ -75,7 +75,7 @@ _Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the f
## Custom transformers for an unknown step
The converted workflow above contains a `sleep` step was not automatically converted. Answer the following questions before writing a custom transformer:
The converted workflow above contains a `sleep` step that was not automatically converted. Answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile.
@@ -198,7 +198,7 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C
+ - uses: actions/upload-artifact@v3
+ with:
+ name: junit-artifact
+ path: path/to/artifact/world.txt
+ path: "**/target/*.xml"
```
## Custom transformers for environment variables
jenkins/6-migrate.md
-3
View File
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a Jenkins pipeline an
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
3. Completed the [dry-run lab](./4-dry-run.md).
4. Completed the [custom transformers lab](./5-custom-transformers.md).
## Performing a migration
@@ -44,8 +43,6 @@ The first thing we should notice about the pull request is that there is a list
![img](https://user-images.githubusercontent.com/19557880/186784161-b7882ac4-ac99-4462-b69f-f49b9202527b.png)
Next, review the workflow you are adding by clicking on the `Files changed` tab. This is where you double check that everything looks good. If it didn't, you could push commits with the required changes, prior to merging.
Next, you can inspect the "Files changed" in this pull request and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this pull request.
Finally, you can merge the pull request once your review has completed. You can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub.