From b321b60c85832516208b6207a60ad5f1e8cf63d3 Mon Sep 17 00:00:00 2001 From: Luke Engle Date: Mon, 19 Sep 2022 17:18:05 -0700 Subject: [PATCH 01/10] Initial edits to ADO/gitlab/jenkins --- azure_devops/1-configure.md | 10 ++++++---- azure_devops/3-forecast.md | 2 +- azure_devops/5-custom-transformers.md | 2 +- azure_devops/6-migrate.md | 5 ++--- azure_devops/readme.md | 8 ++++---- gitlab/1-configure.md | 3 ++- gitlab/4-dry-run.md | 10 ++++++++-- gitlab/5-custom-transformers.md | 6 +++--- gitlab/6-migrate.md | 5 ++--- jenkins/1-configure.md | 3 ++- jenkins/5-custom-transformers.md | 4 ++-- jenkins/6-migrate.md | 7 ++----- 12 files changed, 35 insertions(+), 30 deletions(-) diff --git a/azure_devops/1-configure.md b/azure_devops/1-configure.md index 0f0fc00..d7645c7 100644 --- a/azure_devops/1-configure.md +++ b/azure_devops/1-configure.md @@ -14,11 +14,11 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - Click `Personal access tokens`. - Select `+ New Token` - Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days. - - Select the following scopes (you may need to `Show more scopes` to reveal all scopes): + - Select the following scopes (you may need to `Show all scopes` to reveal all scopes): - Agents Pool: `Read` - - Build: `Read & Execute` - - Code: `Read & Write` - - Project and Team: `Read, Write, & Manage` + - Build: `Read & execute` + - Code: `Read & write` + - Project and Team: `Read, write, & manage` - Release: `Read` - Service Connections: `Read` - Task Groups: `Read` @@ -32,6 +32,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - In the left panel, click `Developer Settings`. - Click `Personal access tokens` and then `Legacy tokens` (if present). - Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step. + - Name your token in the `Note` field. - Select the following scopes: `workflow` and `read:packages`. - Click `Generate token`. - Copy the generated PAT and save it in a safe location. @@ -40,6 +41,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - Select the `TERMINAL` tab from within the codespace terminal. - Run the following command: `gh valet configure`. - Use the down arrow key to highlight `Azure DevOps`, press the spacebar to select, and then press enter to continue. + - At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter. - At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 2 and press enter. - At the GitHub PAT prompt, enter the GitHub PAT generated in step 2 and press enter. - At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`). diff --git a/azure_devops/3-forecast.md b/azure_devops/3-forecast.md index 6d0bd7d..8e9ef35 100644 --- a/azure_devops/3-forecast.md +++ b/azure_devops/3-forecast.md @@ -35,7 +35,7 @@ Answer the following questions before running the `forecast` command: ![img](https://user-images.githubusercontent.com/18723510/187690315-6312088d-9888-4c55-9bbf-c6f2687fa547.png) -4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` command using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json). +4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json). ![img](https://user-images.githubusercontent.com/18723510/187692843-623d4bdc-8970-4348-a632-73c8b00a40f8.png) diff --git a/azure_devops/5-custom-transformers.md b/azure_devops/5-custom-transformers.md index 6d2d0d5..24a3c4d 100644 --- a/azure_devops/5-custom-transformers.md +++ b/azure_devops/5-custom-transformers.md @@ -87,7 +87,7 @@ You can use custom transformers to override Valet's default behavior. In this sc - __DotnetCoreCLI@2__ 2. What is the desired Actions syntax to use instead? - - After some research, you have determined that the uploading test results as an artifact will be suitable: + - After some research, you have determined that the following script will provide the desired functionality: ```yaml - run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done diff --git a/azure_devops/6-migrate.md b/azure_devops/6-migrate.md index 551f5af..98e4ed2 100644 --- a/azure_devops/6-migrate.md +++ b/azure_devops/6-migrate.md @@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert an Azure DevOps pipel 1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and bootstrap an Azure DevOps project. 2. Completed the [configure lab](./1-configure.md#configuring-credentials). 3. Completed the [dry-run lab](./4-dry-run.md). -4. Completed the [custom transformers lab](./5-custom-transformers.md). ## Performing a migration @@ -21,14 +20,14 @@ Answer the following questions before running a `migrate` command: 2. Where do you want to store the logs? - __tmp/migrate__ 3. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. + - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. ### Steps 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url https://github.com/:owner/:repo --output-dir tmp/migrate + gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url $(git remote get-url origin) --output-dir ./tmp/migrate ``` 2. The command will write the URL to the pull request that was created when the command succeeds. diff --git a/azure_devops/readme.md b/azure_devops/readme.md index ca3805e..6b30624 100644 --- a/azure_devops/readme.md +++ b/azure_devops/readme.md @@ -45,11 +45,11 @@ These steps **must** be completed prior to starting other labs. - Click `Personal access tokens`. - Select `+ New Token` - Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days. - - Select the following scopes (you may need to `Show more scopes` to reveal all scopes): + - Select the following scopes (you may need to `Show all scopes` at the bottom of the page to reveal all scopes): - Agents Pool: `Read` - - Build: `Read & Execute` - - Code: `Read & Write` - - Project and Team: `Read, Write, & Manage` + - Build: `Read & execute` + - Code: `Read & write` + - Project and Team: `Read, write, & manage` - Release: `Read` - Service Connections: `Read` - Task Groups: `Read` diff --git a/gitlab/1-configure.md b/gitlab/1-configure.md index 6a00435..4df7fd6 100644 --- a/gitlab/1-configure.md +++ b/gitlab/1-configure.md @@ -31,6 +31,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - In the left panel, click `Developer Settings`. - Click `Personal access tokens` and then `Legacy tokens` (if present). - Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step. + - Name your token in the `Note` field. - Select the following scopes: `workflow` and `read:packages`. - Click `Generate token`. - Copy the generated PAT and save it in a safe location. @@ -39,7 +40,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - Select the `TERMINAL` tab from within the codespace terminal window. - Run the following command: `gh valet configure`. - Use the down arrow key to highlight `GitLab CI`, press the spacebar to select, and then press enter to continue. - - At the prompt, enter your GitHub username and press enter. + - At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter. - At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 4 and press enter. - At the GitHub PAT prompt, enter the GitHub PAT generated in step 4 and press enter. - At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`). diff --git a/gitlab/4-dry-run.md b/gitlab/4-dry-run.md index 169e925..597e9a3 100644 --- a/gitlab/4-dry-run.md +++ b/gitlab/4-dry-run.md @@ -211,7 +211,13 @@ include: - local: /config/test.gitlab-ci.yml ``` -The output of the `dry-run` command can be seen below: +Run the following command from the root directory: + +```bash +gh valet dry-run gitlab --output-dir tmp/dry-run --namespace valet --project included-files-example +``` + +The output of the command above can be seen below: ```yaml name: valet/included-files-example @@ -246,7 +252,7 @@ jobs: - run: echo "this is from a local file" ``` -It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date. +It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date. As an added challenge, try constructing and running the `dry-run` command yourself. Hint, you should only have to change the project name. diff --git a/gitlab/5-custom-transformers.md b/gitlab/5-custom-transformers.md index 5413ed4..6d34709 100644 --- a/gitlab/5-custom-transformers.md +++ b/gitlab/5-custom-transformers.md @@ -9,7 +9,7 @@ In this lab you will build upon the `dry-run` command to override Valet's defaul ## Prerequisites -1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start you GitLab server. +1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start your GitLab server. 2. Completed the [configure lab](./1-configure.md#configuring-credentials). 3. Completed the [dry-run lab](./4-dry-run.md). @@ -63,7 +63,7 @@ The converted workflow above contains an `artifacts.terraform` step that was not - __artifacts.terraform__ 2. What is the desired Actions syntax to use instead? - - After some research, you have determined that the following bash script will provide similar functionality: + - After some research, you have determined that the following GitHub Action will provide similar functionality: ```yaml - uses: actions/upload-artifact@v3 @@ -82,7 +82,7 @@ Next, you will define a `transform` method for the `artifacts.terraform` identif ```ruby transform "artifacts.terraform" do |item| { - uses: "actions/upload-artifact@v2", + uses: "actions/upload-artifact@v3", with: { path: item } diff --git a/gitlab/6-migrate.md b/gitlab/6-migrate.md index 3dcd734..3bac8ff 100644 --- a/gitlab/6-migrate.md +++ b/gitlab/6-migrate.md @@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a GitLab pipeline and 1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a GitLab server. 2. Completed the [configure lab](./1-configure.md#configuring-credentials). 3. Completed the [dry-run lab](./4-dry-run.md). -4. Completed the [custom transformers lab](./5-custom-transformers.md). ## Performing a migration @@ -20,14 +19,14 @@ Answer the following questions before running a `migrate` command: 3. Where do you want to store the logs? - __tmp/migrate__ 4. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should should follow the pattern with `:owner` and `:repo` replaced with your values. + - __this repository__. The URL should should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. ### Steps 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate gitlab --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --namespace valet --project rails-example + gh valet migrate gitlab --target-url $(git remote get-url origin) --output-dir ./tmp/migrate --namespace valet --project rails-example ``` 2. The command will write the URL to the pull request that was created when the command succeeds. diff --git a/jenkins/1-configure.md b/jenkins/1-configure.md index 8128140..71ede17 100644 --- a/jenkins/1-configure.md +++ b/jenkins/1-configure.md @@ -32,11 +32,12 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - Select the `TERMINAL` tab from within the codespace terminal window. - Run the following command: `gh valet configure`. - Use the down arrow key to highlight `Jenkins`, press the spacebar to select, and then press enter to continue. - - At the prompt, enter your GitHub username and press enter. + - At the GitHub handle prompt, enter the GitHub username used to generate the GitHub PAT in step 3 and press enter. - At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 3 and press enter. - At the GitHub PAT prompt, enter the GitHub PAT generated in step 3 and press enter. - At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`). - At the Jenkins token prompt, enter the Jenkins access token from step 2 and press enter. + - At the Jenkins username prompt, enter `admin` and press enter. - At the Jenkins URL prompt, enter `http://localhost:8080/` and press enter. - At the Personal access token to fetch source code in GitHub prompt, press enter to accept the default value. diff --git a/jenkins/5-custom-transformers.md b/jenkins/5-custom-transformers.md index 281c4a2..c4628e0 100644 --- a/jenkins/5-custom-transformers.md +++ b/jenkins/5-custom-transformers.md @@ -75,7 +75,7 @@ _Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the f ## Custom transformers for an unknown step -The converted workflow above contains a `sleep` step was not automatically converted. Answer the following questions before writing a custom transformer: +The converted workflow above contains a `sleep` step that was not automatically converted. Answer the following questions before writing a custom transformer: 1. What is the "identifier" of the step to customize? - __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile. @@ -198,7 +198,7 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C + - uses: actions/upload-artifact@v3 + with: + name: junit-artifact -+ path: path/to/artifact/world.txt ++ path: "**/target/*.xml" ``` ## Custom transformers for environment variables diff --git a/jenkins/6-migrate.md b/jenkins/6-migrate.md index 773b929..04b8348 100644 --- a/jenkins/6-migrate.md +++ b/jenkins/6-migrate.md @@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a Jenkins pipeline an 1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a Jenkins server. 2. Completed the [configure lab](./1-configure.md#configuring-credentials). 3. Completed the [dry-run lab](./4-dry-run.md). -4. Completed the [custom transformers lab](./5-custom-transformers.md). ## Performing a migration @@ -18,14 +17,14 @@ Answer the following questions before running a `migrate` command: 2. Where do you want to store the logs? - __tmp/migrate__ 3. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. + - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. ### Steps 1. Run the following `migrate` command in your codespace terminal: ```bash - gh valet migrate jenkins --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle + gh valet migrate jenkins --target-url $(git remote get-url origin) --output-dir ./tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle ``` 2. The command will write the URL to the pull request that was created when the command succeeds. @@ -44,8 +43,6 @@ The first thing we should notice about the pull request is that there is a list ![img](https://user-images.githubusercontent.com/19557880/186784161-b7882ac4-ac99-4462-b69f-f49b9202527b.png) -Next, review the workflow you are adding by clicking on the `Files changed` tab. This is where you double check that everything looks good. If it didn't, you could push commits with the required changes, prior to merging. - Next, you can inspect the "Files changed" in this pull request and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this pull request. Finally, you can merge the pull request once your review has completed. You can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub. From 71786f95274b9a6651049a62eec8f92bf5f8bc97 Mon Sep 17 00:00:00 2001 From: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> Date: Mon, 19 Sep 2022 17:29:02 -0700 Subject: [PATCH 02/10] Update 6-migrate.md --- azure_devops/6-migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/azure_devops/6-migrate.md b/azure_devops/6-migrate.md index 98e4ed2..f64ea2b 100644 --- a/azure_devops/6-migrate.md +++ b/azure_devops/6-migrate.md @@ -27,7 +27,7 @@ Answer the following questions before running a `migrate` command: 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url $(git remote get-url origin) --output-dir ./tmp/migrate + gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url $(git remote get-url origin) --output-dir tmp/migrate ``` 2. The command will write the URL to the pull request that was created when the command succeeds. From ea795e56b470d476f13b795e70d11b4349437807 Mon Sep 17 00:00:00 2001 From: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> Date: Mon, 19 Sep 2022 17:32:01 -0700 Subject: [PATCH 03/10] Update 6-migrate.md --- gitlab/6-migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/6-migrate.md b/gitlab/6-migrate.md index 3bac8ff..cb96850 100644 --- a/gitlab/6-migrate.md +++ b/gitlab/6-migrate.md @@ -26,7 +26,7 @@ Answer the following questions before running a `migrate` command: 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate gitlab --target-url $(git remote get-url origin) --output-dir ./tmp/migrate --namespace valet --project rails-example + gh valet migrate gitlab --target-url $(git remote get-url origin) --output-dir tmp/migrate --namespace valet --project rails-example ``` 2. The command will write the URL to the pull request that was created when the command succeeds. From 50130ff591aa43285d6b2b3e55aff8a67a79c9de Mon Sep 17 00:00:00 2001 From: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> Date: Mon, 19 Sep 2022 17:32:26 -0700 Subject: [PATCH 04/10] Update 6-migrate.md --- jenkins/6-migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jenkins/6-migrate.md b/jenkins/6-migrate.md index 04b8348..76907de 100644 --- a/jenkins/6-migrate.md +++ b/jenkins/6-migrate.md @@ -24,7 +24,7 @@ Answer the following questions before running a `migrate` command: 1. Run the following `migrate` command in your codespace terminal: ```bash - gh valet migrate jenkins --target-url $(git remote get-url origin) --output-dir ./tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle + gh valet migrate jenkins --target-url $(git remote get-url origin) --output-dir tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle ``` 2. The command will write the URL to the pull request that was created when the command succeeds. From dd9609b42c55484ee8ea1c41b637422edd535a98 Mon Sep 17 00:00:00 2001 From: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> Date: Tue, 20 Sep 2022 09:33:10 -0700 Subject: [PATCH 05/10] Change 'GitHub Action' to 'action Co-authored-by: Ethan Dennis --- gitlab/5-custom-transformers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/5-custom-transformers.md b/gitlab/5-custom-transformers.md index 6d34709..98f533a 100644 --- a/gitlab/5-custom-transformers.md +++ b/gitlab/5-custom-transformers.md @@ -63,7 +63,7 @@ The converted workflow above contains an `artifacts.terraform` step that was not - __artifacts.terraform__ 2. What is the desired Actions syntax to use instead? - - After some research, you have determined that the following GitHub Action will provide similar functionality: + - After some research, you have determined that the following action will provide similar functionality: ```yaml - uses: actions/upload-artifact@v3 From ef4b59aea4c6f504a26d253f2a4cb690950a3afb Mon Sep 17 00:00:00 2001 From: Luke Engle Date: Tue, 20 Sep 2022 09:44:41 -0700 Subject: [PATCH 06/10] Remove git remote get-url --- azure_devops/6-migrate.md | 4 ++-- gitlab/6-migrate.md | 4 ++-- jenkins/6-migrate.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/azure_devops/6-migrate.md b/azure_devops/6-migrate.md index f64ea2b..4466b6a 100644 --- a/azure_devops/6-migrate.md +++ b/azure_devops/6-migrate.md @@ -20,14 +20,14 @@ Answer the following questions before running a `migrate` command: 2. Where do you want to store the logs? - __tmp/migrate__ 3. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. + - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. ### Steps 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url $(git remote get-url origin) --output-dir tmp/migrate + gh valet migrate azure-devops pipeline --pipeline-id :pipeline_id --target-url https://github.com/:owner/:repo --output-dir tmp/migrate ``` 2. The command will write the URL to the pull request that was created when the command succeeds. diff --git a/gitlab/6-migrate.md b/gitlab/6-migrate.md index cb96850..ba561fb 100644 --- a/gitlab/6-migrate.md +++ b/gitlab/6-migrate.md @@ -19,14 +19,14 @@ Answer the following questions before running a `migrate` command: 3. Where do you want to store the logs? - __tmp/migrate__ 4. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. + - __this repository__. The URL should should follow the pattern with `:owner` and `:repo` replaced with your values. ### Steps 1. Run the following `migrate` command in the codespace terminal: ```bash - gh valet migrate gitlab --target-url $(git remote get-url origin) --output-dir tmp/migrate --namespace valet --project rails-example + gh valet migrate gitlab --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --namespace valet --project rails-example ``` 2. The command will write the URL to the pull request that was created when the command succeeds. diff --git a/jenkins/6-migrate.md b/jenkins/6-migrate.md index 76907de..85a45b7 100644 --- a/jenkins/6-migrate.md +++ b/jenkins/6-migrate.md @@ -17,14 +17,14 @@ Answer the following questions before running a `migrate` command: 2. Where do you want to store the logs? - __tmp/migrate__ 3. What is the URL for the GitHub repository to add the workflow to? - - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. The command below uses `git remote get-url` to automatically populate these values. + - __this repository__. The URL should follow the pattern with `:owner` and `:repo` replaced with your values. ### Steps 1. Run the following `migrate` command in your codespace terminal: ```bash - gh valet migrate jenkins --target-url $(git remote get-url origin) --output-dir tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle + gh valet migrate jenkins --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle ``` 2. The command will write the URL to the pull request that was created when the command succeeds. From d58cd99cf15137c4ef0085895da03e4584648b38 Mon Sep 17 00:00:00 2001 From: Luke Engle Date: Tue, 20 Sep 2022 11:49:00 -0700 Subject: [PATCH 07/10] Add more thorough custom transformer instructions --- azure_devops/5-custom-transformers.md | 29 ++++++++++++++++++++--- gitlab/5-custom-transformers.md | 34 ++++++++++++++++++--------- 2 files changed, 49 insertions(+), 14 deletions(-) diff --git a/azure_devops/5-custom-transformers.md b/azure_devops/5-custom-transformers.md index 24a3c4d..2a9a3a8 100644 --- a/azure_devops/5-custom-transformers.md +++ b/azure_devops/5-custom-transformers.md @@ -100,7 +100,32 @@ Now you can begin to write the custom transformer. Custom transformers use a DSL touch transformers.rb && code transformers.rb ``` -Next, you will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`: +To build this custom transformer, you first need to inspect the `item` keyword to programmatically obtain the projects, command, and arguments to use in the `DotNetCoreCLI@2` step. + +To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`: + +```ruby +transform "DotNetCoreCLI@2" do |item| + puts "This is the item: #{item}" +end +``` + +The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps. + +Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this: + +```console +$ gh valet dry-run azure-devops pipeline --pipeline-id 6 --output-dir tmp/dry-run --custom-transformers transformers.rb +[2022-09-20 18:39:50] Logs: 'tmp/dry-run/log/valet-20220920-183950.log' +This is the item: {"command"=>"restore", "projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)"} +This is the item: {"projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)", "arguments"=>"--configuration $(BUILDCONFIGURATION)"} +[2022-09-20 18:39:51] Output file(s): +[2022-09-20 18:39:51] tmp/dry-run/lab-test/pipelines/valet-custom-transformer-example.yml +``` + +In the above command you will see two instances of `item` printed to the console. This is because there are two `DotNetCoreCLI@2` steps in the pipeline. Each item listed above represents each `DotNetCoreCLI@2` step in the order that they are defined in the pipeline. + +Now that you know the data structure of `item`, you can access the dotnet projects, command, and arguments programmatically by editing the custom transformer to the following: ```ruby transform "DotNetCoreCLI@2" do |item| @@ -122,8 +147,6 @@ transform "DotNetCoreCLI@2" do |item| end ``` -This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps. - Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal: ```bash diff --git a/gitlab/5-custom-transformers.md b/gitlab/5-custom-transformers.md index 98f533a..4ac3624 100644 --- a/gitlab/5-custom-transformers.md +++ b/gitlab/5-custom-transformers.md @@ -77,7 +77,29 @@ Now you can begin to write the custom transformer. Custom transformers use a DSL touch transformers.rb && code transformers.rb ``` -Next, you will define a `transform` method for the `artifacts.terraform` identifier by adding the following code to `transformers.rb`: +To build this custom transformer, you first need to inspect the `item` keyword to programmatically use the file path of the terraform json in the `actions/upload-artifact@v3` step. + +To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`: + +```ruby +transform "artifacts.terraform" do |item| + puts "This is the item: #{item}" +end +``` + +The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab. + +Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this: + +```console +$ gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example --custom-transformers transformers.rb +[2022-09-20 17:47:55] Logs: 'tmp/log/valet-20220920-174755.log' +This is the item: $PLAN_JSON +[2022-09-20 17:47:56] Output file(s): +[2022-09-20 17:47:56] tmp/valet/terraform-example.yml +``` + +Now that you know the data structure of `item`, you can access the file path programmatically by editing the custom transformer to the following: ```ruby transform "artifacts.terraform" do |item| @@ -90,8 +112,6 @@ transform "artifacts.terraform" do |item| end ``` -This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab. - Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal: ```bash @@ -107,14 +127,6 @@ The converted workflow that is generated by the above command will now use the c + path: "$PLAN_JSON" ``` -_Note_: If you were unsure what the data structure of `item` was, you could use the following code in the custom transformer to print `item` to the console: - -```ruby -transform "artifacts.terraform" do |item| - puts item -end -``` - ## Custom transformers for environment variables You can also use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `PLAN_JSON` environment variable to be `custom_plan.json` instead of `plan.json`. From 640df0c0891698297730fe3fbfc8e8ba3088600b Mon Sep 17 00:00:00 2001 From: Luke Engle Date: Tue, 20 Sep 2022 12:12:05 -0700 Subject: [PATCH 08/10] Add custom runner example to gitlab --- gitlab/5-custom-transformers.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/gitlab/5-custom-transformers.md b/gitlab/5-custom-transformers.md index 4ac3624..215a2cf 100644 --- a/gitlab/5-custom-transformers.md +++ b/gitlab/5-custom-transformers.md @@ -148,12 +148,42 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C + PLAN_JSON: custom_plan.json ``` +## Custom transformers for runners + +Finally, you can use custom transformers to dictate which runners the converted workflows should use. To do this, answer the following questions: + +1. What is label of the runner in GitLab to update? + - Since there is no `tags` stanza defined in the `terraform-example` project in GitLab, you will need to use the Ruby symbol for the default runner: + + __:default__ + +2. What is the label of the runner in Actions to use instead? + - __custom-runner__ + +With these questions answered, you can add the following code to the `transformers.rb` file: + +```ruby +runner :default, "custom-runner" +``` + +In this example, the first parameter to the `runner` method is the GitLab label and the second is the Actions runner label. + +Now you can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label: + +```diff +runs-on: +- - ubuntu-latest ++ - custom-runner +``` + At this point, the file contents of `transformers.rb` should match this:
Custom transformers 👇 ```ruby + runner :default, "custom-runner" + env "PLAN_JSON", "custom_plan.json" transform "artifacts.terraform" do |item| From 40bdb2d5474c3afb5fe725f8ae9f7f9f81bd742e Mon Sep 17 00:00:00 2001 From: Luke Engle Date: Tue, 20 Sep 2022 13:26:17 -0700 Subject: [PATCH 09/10] Circle edits --- circle_ci/1-configure.md | 4 +--- circle_ci/2-audit.md | 2 +- circle_ci/3-forecast.md | 2 +- circle_ci/5-custom-transformers.md | 6 +++--- circle_ci/6-migrate.md | 1 - 5 files changed, 6 insertions(+), 9 deletions(-) diff --git a/circle_ci/1-configure.md b/circle_ci/1-configure.md index 16c05de..45ff5b3 100644 --- a/circle_ci/1-configure.md +++ b/circle_ci/1-configure.md @@ -29,8 +29,6 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - At the CircleCI token prompt, enter the CircleCI access token from step 2 and press enter. - At the CircleCI base url prompt, hit enter to accept the default value (`https://circleci.com`). - At the CircleCI organization name prompt, enter `valet-labs`. This is the organization we'll be using throughout these labs. - - At the access token to fetch source code in GitHub prompt, enter the GitHub PAT generated in step 1 and press enter. - - At the GitHub instance url containing source code prompt, press enter to accept the default value (`https://github.com`). ```console $ gh valet configure @@ -59,7 +57,7 @@ To verify our environment is configured correctly, we are going to run the `upda 2. You should see a confirmation that you were logged into the GitHub Container Registry and Valet was updated to the latest version. ```console - $ gh valet version + $ gh valet update Login Succeeded latest: Pulling from valet-customers/valet-cli Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb diff --git a/circle_ci/2-audit.md b/circle_ci/2-audit.md index 1e954ef..30e842b 100644 --- a/circle_ci/2-audit.md +++ b/circle_ci/2-audit.md @@ -34,7 +34,7 @@ You will be performing an `audit` for the __valet-labs__ CircleCI organization t 1. Find the `audit_summary.md` file in the file explorer. 2. Right-click the `audit_summary.md` file and select `Open Preview`. -3. This file contains details that summarizes what percentage of your pipelines were converted automatically. +3. This file contains details that summarize what percentage of your pipelines were converted automatically. ### Review audit summary diff --git a/circle_ci/3-forecast.md b/circle_ci/3-forecast.md index a798adc..a4f5c53 100644 --- a/circle_ci/3-forecast.md +++ b/circle_ci/3-forecast.md @@ -12,7 +12,7 @@ In this lab you will use the `forecast` command to forecast potential GitHub Act Answer the following questions before running the `forecast` command: 1. What is the date you want to start forecasting from? - - **2022-09-02**. This date is needed as it is prior to when the bulk of builds were trigger in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage. + - **2022-09-02**. This date is needed as it is prior to when the bulk of builds were triggered in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage. 2. Where do you want to store the results? - **tmp/forecast_reports** diff --git a/circle_ci/5-custom-transformers.md b/circle_ci/5-custom-transformers.md index 04a7d05..21523a4 100644 --- a/circle_ci/5-custom-transformers.md +++ b/circle_ci/5-custom-transformers.md @@ -64,7 +64,7 @@ _Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the f ## Custom transformers for an unknown step -The converted workflow above contains an `codecov_codecov_upload` step that was not automatically converted. Answer the following questions before writing a custom transformer: +The converted workflow above contains a `codecov_codecov_upload` step that was not automatically converted. Answer the following questions before writing a custom transformer: 1. What is the "identifier" of the step to customize? - __codecov_codecov_upload__ @@ -127,7 +127,7 @@ end ## Custom transformers for environment variables -You can use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `COVERAGE_DIR` environment variable to be `$RUNNER_TEMP/cov` instead of `tmp/cov`. +You can use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `COVERAGE_DIR` environment variable to be `$RUNNER_TEMP/cov` instead of `./tmp/cov`. To do this, add the following code to the `transformers.rb` file. @@ -191,7 +191,7 @@ That's it! Congratulations, you have overridden Valet's default behavior by cust - Unknown steps - Environment variables -- Runner +- Runners ## Next lab diff --git a/circle_ci/6-migrate.md b/circle_ci/6-migrate.md index ee136a9..915e6bc 100644 --- a/circle_ci/6-migrate.md +++ b/circle_ci/6-migrate.md @@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a CircleCI pipeline a 1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment. 2. Completed the [configure lab](./1-configure.md#configuring-credentials). 3. Completed the [dry-run lab](./4-dry-run.md). -4. Completed the [custom transformers lab](./5-custom-transformers.md). ## Performing a migration From 285053ac1edba680d7cd83ae04bbe160ab80b122 Mon Sep 17 00:00:00 2001 From: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> Date: Tue, 20 Sep 2022 13:32:02 -0700 Subject: [PATCH 10/10] Update gitlab/5-custom-transformers.md Co-authored-by: Ethan Dennis --- gitlab/5-custom-transformers.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/gitlab/5-custom-transformers.md b/gitlab/5-custom-transformers.md index 15b30e5..9b29f96 100644 --- a/gitlab/5-custom-transformers.md +++ b/gitlab/5-custom-transformers.md @@ -153,9 +153,7 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C Finally, you can use custom transformers to dictate which runners the converted workflows should use. To do this, answer the following questions: 1. What is label of the runner in GitLab to update? - - Since there is no `tags` stanza defined in the `terraform-example` project in GitLab, you will need to use the Ruby symbol for the default runner: - - __:default__ + - __:default__. This is a special keyword to define the default runner to use. You can optional target specific `tags` in a job. 2. What is the label of the runner in Actions to use instead? - __custom-runner__