From e3862e5cc8c77e9b9367fc164c293e4f52c42a6e Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:31:41 -0400 Subject: [PATCH 01/24] Rename valet-configure-lab.md to 1-configure.md --- gitlab/{valet-configure-lab.md => 1-configure.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename gitlab/{valet-configure-lab.md => 1-configure.md} (100%) diff --git a/gitlab/valet-configure-lab.md b/gitlab/1-configure.md similarity index 100% rename from gitlab/valet-configure-lab.md rename to gitlab/1-configure.md From 6b870aed73951bf7d98b110c592954f10effbfb2 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:32:23 -0400 Subject: [PATCH 02/24] Update and rename valet-audit-lab.md to 2-audit.md --- gitlab/{valet-audit-lab.md => 2-audit.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename gitlab/{valet-audit-lab.md => 2-audit.md} (99%) diff --git a/gitlab/valet-audit-lab.md b/gitlab/2-audit.md similarity index 99% rename from gitlab/valet-audit-lab.md rename to gitlab/2-audit.md index 77be67a..2c90dbf 100644 --- a/gitlab/valet-audit-lab.md +++ b/gitlab/2-audit.md @@ -336,4 +336,4 @@ The remaining sections `Successful` and `Failed` are groupings of the generated Note: this has files under the `.github` directory. This tells us that this pipeline generated reusable workflows from the `include` statements used in the source pipeline. ### Next Lab -[Dry run the migration of a GitLab pipeline to GitHub Actions](valet-dry-run-lab.md) +[Dry run the migration of a GitLab pipeline to GitHub Actions](3-dry-run.md) From fb041e723f04b9c755145b22dec2d3a4830379c2 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:34:02 -0400 Subject: [PATCH 03/24] Update 1-configure.md --- gitlab/1-configure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/1-configure.md b/gitlab/1-configure.md index 2be1f8f..fe4a59b 100644 --- a/gitlab/1-configure.md +++ b/gitlab/1-configure.md @@ -62,5 +62,5 @@ To verify Valet works we are going to run a `update` and `dry-run` command. We ![configure-dry-run](https://user-images.githubusercontent.com/18723510/183989794-d51fa29d-b4c0-4074-8402-a55ffcea3a6b.png) ### Next Lab -[Dry run the migration of an GitLab pipeline to GitHub Actions](../gitlab/valet-audit-lab.md) +[Perform and a audit of GitLab](../gitlab/2-audit.md) From 2fa84cbcdf3d4f3c662f1b6c6fa2bbfca37860dc Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:36:26 -0400 Subject: [PATCH 04/24] Update and rename valet-dry-run-lab.md to 3-dry-run.md --- gitlab/{valet-dry-run-lab.md => 3-dry-run.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename gitlab/{valet-dry-run-lab.md => 3-dry-run.md} (99%) diff --git a/gitlab/valet-dry-run-lab.md b/gitlab/3-dry-run.md similarity index 99% rename from gitlab/valet-dry-run-lab.md rename to gitlab/3-dry-run.md index 9c743c0..ac0d8f6 100644 --- a/gitlab/valet-dry-run-lab.md +++ b/gitlab/3-dry-run.md @@ -261,6 +261,6 @@ jobs: Try constructing and running the `dry-run` command yourself. Hint, you should just have to change the project name. ## Next Lab -[Audit GitLab Pipelines to GitHub Actions](../gitlab/valet-audit-lab.md) +[Using Custom Transformers in a dry-runs](../gitlab/4-custom-transformers.md) From d954ae9250389a5b7ba2fcc67bdebe9f493b62b6 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:37:49 -0400 Subject: [PATCH 05/24] Update and rename valet-custom-transformers-lab.md to 4-custom-transformers.md --- ...alet-custom-transformers-lab.md => 4-custom-transformers.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename gitlab/{valet-custom-transformers-lab.md => 4-custom-transformers.md} (98%) diff --git a/gitlab/valet-custom-transformers-lab.md b/gitlab/4-custom-transformers.md similarity index 98% rename from gitlab/valet-custom-transformers-lab.md rename to gitlab/4-custom-transformers.md index 802a202..3cebec7 100644 --- a/gitlab/valet-custom-transformers-lab.md +++ b/gitlab/4-custom-transformers.md @@ -108,4 +108,4 @@ Now that we have this custom transformers file we can add additional `transform` Note: The custom transformers will only affect the pipeline being transformed if they contain the matching identifiers. If you believe a custom transformer should have altered the output, double check that the identifier is correct. ## Next Lab -[Migrating a GitLab Pipeline](../gitlab/valet-migrate-lab.md) +[Forecast GitLab Usage](../gitlab/5-forecast.md) From ce58680f08c8a4ab8aa85d268bd0d23e08410bfc Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:39:23 -0400 Subject: [PATCH 06/24] Update and rename valet-forecast-lab.md to 5-forecast.md --- gitlab/{valet-forecast-lab.md => 5-forecast.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename gitlab/{valet-forecast-lab.md => 5-forecast.md} (97%) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/5-forecast.md similarity index 97% rename from gitlab/valet-forecast-lab.md rename to gitlab/5-forecast.md index 09aacf5..a7ec802 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/5-forecast.md @@ -183,4 +183,4 @@ Using `--source-file-path` we can combine data from multiple forecast runs into ![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png) ## Next steps -This concludes the Valet labs for GitLab, if you are interested exploring the power of Valet more. You can leverage the demo GitLab Instance and modify and add new projects that more closely match your needs and try out the commands again! +[Migrating a GitLab Pipeline](../jenkins/6-migrate.md) From c0810ecbe08a3658f5e194e06f0b4e8ba48a82b5 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:40:45 -0400 Subject: [PATCH 07/24] Update and rename valet-migrate-lab.md to 6-migrate.md --- gitlab/{valet-migrate-lab.md => 6-migrate.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename gitlab/{valet-migrate-lab.md => 6-migrate.md} (94%) diff --git a/gitlab/valet-migrate-lab.md b/gitlab/6-migrate.md similarity index 94% rename from gitlab/valet-migrate-lab.md rename to gitlab/6-migrate.md index 23041a6..3321dd0 100644 --- a/gitlab/valet-migrate-lab.md +++ b/gitlab/6-migrate.md @@ -42,4 +42,4 @@ gh valet migrate gitlab --target-url https://github.com/GITHUB-ORG-USERNAME-HERE - The migration is complete and we have successfully transformed a pipeline from GitLab into a GitHub Actions workflow, added it to a GitHub Repository, and run the workflow in Actions. ### Next Lab -[Forecast GitLab Usage](../gitlab/valet-forecast-lab.md) +This concludes the Valet labs for GitLab, if you are interested exploring the power of Valet more. You can leverage the demo GitLab Instance and modify and add new projects that more closely match your needs and try out the commands again! From e4f1e22968f26ff1a1277ac5401c402995c5bd61 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:42:07 -0400 Subject: [PATCH 08/24] Update 5-forecast.md --- gitlab/5-forecast.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/5-forecast.md b/gitlab/5-forecast.md index a7ec802..4d583dc 100644 --- a/gitlab/5-forecast.md +++ b/gitlab/5-forecast.md @@ -183,4 +183,4 @@ Using `--source-file-path` we can combine data from multiple forecast runs into ![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png) ## Next steps -[Migrating a GitLab Pipeline](../jenkins/6-migrate.md) +[Migrating a GitLab Pipeline](../gitlab/6-migrate.md) From a30ddf200819661d841088f925583b4c559b4e07 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 1 Sep 2022 16:47:53 -0400 Subject: [PATCH 09/24] Update readme.md --- gitlab/readme.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/gitlab/readme.md b/gitlab/readme.md index 29812d1..f3a2fa4 100644 --- a/gitlab/readme.md +++ b/gitlab/readme.md @@ -38,12 +38,12 @@ This lab bootstraps a Valet environment using GitHub Codespaces and enables you ## Labs for GitLab Perform the following labs to test-drive Valet -- [Configure Valet to work with GitLab](valet-configure-lab.md) -- [Audit GitLab using the Valet audit command](valet-audit-lab.md) -- [Dry run the migration of an GitLab pipeline to GitHub Actions](valet-dry-run-lab.md) -- [Using Custom Transformers in a dry-run](valet-custom-transformers-lab.md) -- [Migrate an GitLab Project to GitHub Actions](valet-migrate-lab.md) -- [Forecast the usage of a GitLab namespace](valet-forecast-lab.md) +1. [Configure credentials for Valet](1-configure.md) +2. [Perform an audit on GitLab pipelines](2-audit.md) +3. [Perform a dry-run of a GitLab pipeline](3-dry-run.md) +4. [Use custom transformers to customize Valet's behavior](4-custom-transformers.md) +5. [Forecast potential build runner usage](5-forecast.md) +6. [Perform a production migration of a GitLab pipeline](6-migrate.md) ## Troubleshooting - **GitHub CLI Valet extension is not installed**. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet) From 2dfcc8021c72ff9b9a424f876cbde1e96cbac485 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 08:41:10 -0400 Subject: [PATCH 10/24] Update 1-configure.md --- gitlab/1-configure.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/gitlab/1-configure.md b/gitlab/1-configure.md index fe4a59b..86cfeb4 100644 --- a/gitlab/1-configure.md +++ b/gitlab/1-configure.md @@ -2,15 +2,15 @@ In this lab, you will use the Valet `configure` command to set up the required information to communicate with the GitLab and GitHub instances. The `configure` command can be used for all of the supported providers, in this lab we will be focusing on GitLab. - [Prerequisites](#prerequisites) -- [Configuring Valet](#configuring-valet) -- [Verify Valet Works](#verify-valet-works) +- [Configuring credentials](#configuring-credentials) +- [Verify your environment](#verify-your-environment) - [Next Lab](#next-lab) ## Prerequisites 1. Followed [steps](../gitlab#readme) to set up your codespace environment. -## Configuring Valet +## Configuring credentials 1. Run the setup script in the codespace terminal `source ./gitlab/bootstrap/setup.sh` to ensure the GitLab instance is ready. 2. Login to the GitLab instance to generate a personal access token: - Click the `PORTS` tab in the codespace terminal window. @@ -27,7 +27,7 @@ In this lab, you will use the Valet `configure` command to set up the required i - Select at least these scopes: `read packages` and `workflow`. Optionally, provide a text in the **Note** field and change the expiration. - Click `Generate token` - Copy the token somewhere safe and temporary. -3. Run Valet configure commands +4. Run Valet configure commands - In the codespace terminal window click back to the `TERMINAL` tab. - Run `gh valet configure` - Use the down arrow key to highlight `GitLab CI`, press the spacebar to select, then hit enter to accept. @@ -38,12 +38,12 @@ In this lab, you will use the Valet `configure` command to set up the required i - At the GitLab CI token prompt enter the GitLab CI access token from step 2 and press enter. - At the GitLab CI url prompt enter `http://localhost` and press enter. - At the Personal access token to fetch source code in GitHub prompt, if any of your Jenkins pipelines have source code in a GitHub repository enter the GitHub PAT that would have acess to these files. -4. If all went well you should see a similar output in your terminal and a new file (.env.local) should have been created in the root of the project. +5. If all went well you should see a similar output in your terminal and a new file (.env.local) should have been created in the root of the project. ![configure-result](https://user-images.githubusercontent.com/18723510/183990474-d0b2559c-d2bf-40d9-ac43-19af53e45329.png) -## Verify Valet Works -To verify Valet works we are going to run a `update` and `dry-run` command. We will go further into details about the `dry-run` command in a later lab, but for now we want to get the latest version of Valet and confirm that Valet can perform a dry-run with no errors. +## Verify your environment +To verify our environment is configured correctly, we are going to run a `update` and `dry-run` command. The `update` commnad will download the latest version of Valet to your codespace. We will go further into details about the `dry-run` command in a later lab, but for now we will just verify it finishes with no errors. 1. In the codespace terminal update Valet by running `gh valet update` 2. In the terminal you should see a confirmation that it logged into the GitHub Container Registry and pulled the latest version. From 9d81e45940eb430b04b03e6eca5407298b0ccf9d Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 10:44:02 -0400 Subject: [PATCH 11/24] Update readme.md --- gitlab/readme.md | 65 +++++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 29 deletions(-) diff --git a/gitlab/readme.md b/gitlab/readme.md index f3a2fa4..9361318 100644 --- a/gitlab/readme.md +++ b/gitlab/readme.md @@ -1,40 +1,47 @@ -# Valet labs for GitLab +# GitLab to Actions migrations powered by Valet +The instructions below will guide you through configuring a GitHub Codespace environment that will be used in subsequent labs that demonstrate how to use Valet to migrate GitLab pipelines to GitHub Actions. -This lab bootstraps a Valet environment using GitHub Codespaces and enables you to run the Valet CI/CD migration tool against an isolated GitLab instance running in Docker. +These steps **must** be completed prior to starting other labs. -- [Prerequisites](#prerequisites) -- [Setup Codespace for Labs](#setup-codespace-for-labs) -- [Labs for GitLab](#labs-for-gitlab) +## Create your own repository for these labs +1. Ensure that you have created a repository using the [valet-customers/labs](https://github.com/valet-customers/labs) as a template. - -## Prerequisites -1. [Valet Labs](https://github.com/valet-customers/labs) template has been used to generate this repository. - -## Setup Codespace for Labs +## Configure your Codespace 1. Start the codespace - - Click the `Code` button down arrow above the repository on the repository's landing page. - - Click the `Codespaces` tab. - - Click `Create codespaces on main` to create the codespace. If you are in another branch then the `main` branch, the codespace button will have the current branch specified. - - Wait a couple minutes, then verify that the codespace starts up. Once it is fully booted up, the terminal should be present. -2. Verify Valet CLI is installed and working. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet) - - Run `gh valet version` in the codespace terminal and verify it returns results like below, actual versions will vary. - ``` - gh version 2.14.3 (2022-07-26) - gh valet github/gh-valet v0.1.12 - valet-cli unknown - ``` - if results do not match try manually installing the valet extension, using the [troubleshooting](#troubleshooting) section. - + +- Click the `Code` button down arrow above the repository on the repository's landing page. +- Click the `Codespaces` tab. +- Click `Create codespaces on main` to create the codespace. +- After the Codespace has initialized there will be a terminal present. + +2. Verify the Valet CLI is installed and working. More information on the Valet extension for the official GitHub CLI can be found [here](https://github.com/github/gh-valet). + +- Run the following command in the codespace's terminal: + +```bash +gh valet version +``` + +- Verify the output is similar to below, if not try manually installing the valet extension, using the [troubleshooting](#troubleshooting) section. + +```bash +gh version 2.14.3 (2022-07-26) +gh valet github/gh-valet v0.1.12 +valet-cli unknown +``` + 3. Run the GitLab setup script. This script will set up GitLab and ensure it is ready to use. In general, this script should be run first if you are starting a new codespace or restarting an existing one. - - from the codespace terminal run `source gitlab/bootstrap/setup.sh`. + +- from the codespace terminal run `source gitlab/bootstrap/setup.sh`. 4. Verify you can login to the GitLab Server. - - Click the `PORTS` tab in the codespace terminal window. - - In the row that starts with `80` mouse over the address under the column heading `Local Address`, and click the globe icon. - - Verify a new browser tab opened to the GitLab login screen. - - Login with the username: `root` and password: `valet-labs!`. - - Verify that you can see the auto populated projects in GitLab, under the `valet` namespace by clicking the `Menu` icon in GitLab, then `Your projects`. + +- Click the `PORTS` tab in the codespace terminal window. +- In the row that starts with `80` mouse over the address under the column heading `Local Address`, and click the globe icon. +- Verify a new browser tab opened to the GitLab login screen. +- Login with the username: `root` and password: `valet-labs!`. +- Verify that you can see the auto populated projects in GitLab, under the `valet` namespace by clicking the `Menu` icon in GitLab, then `Your projects`. ## Labs for GitLab Perform the following labs to test-drive Valet From 821cb5b3d4a4538743d97a27515e4f7b40d174c0 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 11:02:05 -0400 Subject: [PATCH 12/24] Update readme.md --- gitlab/readme.md | 60 ++++++++++++++++++++++++++++++------------------ 1 file changed, 38 insertions(+), 22 deletions(-) diff --git a/gitlab/readme.md b/gitlab/readme.md index 9361318..ae06aa0 100644 --- a/gitlab/readme.md +++ b/gitlab/readme.md @@ -19,17 +19,18 @@ These steps **must** be completed prior to starting other labs. - Run the following command in the codespace's terminal: -```bash -gh valet version -``` + ```bash + gh valet version + ``` -- Verify the output is similar to below, if not try manually installing the valet extension, using the [troubleshooting](#troubleshooting) section. - -```bash -gh version 2.14.3 (2022-07-26) -gh valet github/gh-valet v0.1.12 -valet-cli unknown -``` +- Verify the output is similar to below. + - If `gh valet version` did not produce similar output then please follow the troubleshooting [guide](#troubleshoot-the-valet-cli). + + ```bash + gh version 2.14.3 (2022-07-26) + gh valet github/gh-valet v0.1.12 + valet-cli unknown + ``` 3. Run the GitLab setup script. This script will set up GitLab and ensure it is ready to use. In general, this script should be run first if you are starting a new codespace or restarting an existing one. @@ -44,7 +45,9 @@ valet-cli unknown - Verify that you can see the auto populated projects in GitLab, under the `valet` namespace by clicking the `Menu` icon in GitLab, then `Your projects`. ## Labs for GitLab + Perform the following labs to test-drive Valet + 1. [Configure credentials for Valet](1-configure.md) 2. [Perform an audit on GitLab pipelines](2-audit.md) 3. [Perform a dry-run of a GitLab pipeline](3-dry-run.md) @@ -52,15 +55,28 @@ Perform the following labs to test-drive Valet 5. [Forecast potential build runner usage](5-forecast.md) 6. [Perform a production migration of a GitLab pipeline](6-migrate.md) -## Troubleshooting -- **GitHub CLI Valet extension is not installed**. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet) - - Verify you are in the codespace terminal - - Run `gh extension install github/gh-valet` - - Verify the result of the command is: `✓ Installed extension github/gh-valet` - - If you get a similar error to the following, click the link to authorize the token - ![linktolcickauth](https://user-images.githubusercontent.com/26442605/169588015-9414404f-82b6-4d0f-89d4-5f0e6941b029.png) - - Restart the codespace after clicking the link -- **Port 80 is not being forwarded for GitLab server**. This should be auto detected by codespaces, but can be manually setup by: - - In the codespace terminal click the PORTS tab - - Click the "Add Port" button - - Enter 80 and hit enter +## Troubleshoot the Valet CLI + +The CLI extension for Valet can be manually installed by following these steps: + +- Verify you are in the codespace terminal +- Run this command from within the codespace's terminal: + + ```bash + gh extension install github/gh-valet + ``` + +- Verify the result of the install contains: + + ```bash + ✓ Installed extension github/gh-valet + ``` + +- If you get an error similar to the image below, then click the link in the terminal output to authorize the token. + - Restart the codespace after clicking the link. + ![img](https://user-images.githubusercontent.com/26442605/169588015-9414404f-82b6-4d0f-89d4-5f0e6941b029.png) +- Verify Valet CLI extension is installed and working by running the following command from the codespace's terminal: + + ```bash + gh valet version + ``` From 962749d4e1b8ec7077f6b80999973421e23d2399 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 11:12:13 -0400 Subject: [PATCH 13/24] Update readme.md --- gitlab/readme.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/gitlab/readme.md b/gitlab/readme.md index ae06aa0..4d9b091 100644 --- a/gitlab/readme.md +++ b/gitlab/readme.md @@ -32,16 +32,24 @@ These steps **must** be completed prior to starting other labs. valet-cli unknown ``` -3. Run the GitLab setup script. This script will set up GitLab and ensure it is ready to use. In general, this script should be run first if you are starting a new codespace or restarting an existing one. +## Bootstrap a GitLab Server -- from the codespace terminal run `source gitlab/bootstrap/setup.sh`. +1. Execute the GitLab setup script that will start a container with a GitLab server running inside of it. The script should be executed when starting a new Codespace or restarting an existing one. -4. Verify you can login to the GitLab Server. +- Run the following command from the codespace's terminal + + ```bash + source gitlab/bootstrap/setup.sh + ``` + +2. Wait for the script to finish and then verify you can login to the GitLab Server. - Click the `PORTS` tab in the codespace terminal window. - In the row that starts with `80` mouse over the address under the column heading `Local Address`, and click the globe icon. - Verify a new browser tab opened to the GitLab login screen. -- Login with the username: `root` and password: `valet-labs!`. +- Login using the following credentials: + - Username: `root` + - Password: `valet-labs!` - Verify that you can see the auto populated projects in GitLab, under the `valet` namespace by clicking the `Menu` icon in GitLab, then `Your projects`. ## Labs for GitLab From 8f8ce0e0ddb9e3293acb0b03e4973ce4437ffa3e Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 11:32:33 -0400 Subject: [PATCH 14/24] Update readme.md --- gitlab/readme.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/gitlab/readme.md b/gitlab/readme.md index 4d9b091..f8f56e1 100644 --- a/gitlab/readme.md +++ b/gitlab/readme.md @@ -24,7 +24,6 @@ These steps **must** be completed prior to starting other labs. ``` - Verify the output is similar to below. - - If `gh valet version` did not produce similar output then please follow the troubleshooting [guide](#troubleshoot-the-valet-cli). ```bash gh version 2.14.3 (2022-07-26) @@ -32,6 +31,8 @@ These steps **must** be completed prior to starting other labs. valet-cli unknown ``` + - If `gh valet version` did not produce similar output then please follow the troubleshooting [guide](#troubleshoot-the-valet-cli). + ## Bootstrap a GitLab Server 1. Execute the GitLab setup script that will start a container with a GitLab server running inside of it. The script should be executed when starting a new Codespace or restarting an existing one. From 04305031d85976f42d141497e58c444d542c17cd Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 12:59:56 -0400 Subject: [PATCH 15/24] Update 2-audit.md --- gitlab/2-audit.md | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/gitlab/2-audit.md b/gitlab/2-audit.md index 2c90dbf..2ea26fe 100644 --- a/gitlab/2-audit.md +++ b/gitlab/2-audit.md @@ -1,4 +1,4 @@ -# Audit GitLab using the Valet audit command +# Perform an audit on GitLab pipelines In this lab, you will use Valet to `audit` a GitLab namespace. The `audit` command will scan the GitLab namespace for all projects with pipelines that have run at least once, perform a `dry-run` on each of those pipelines, and finally perform an aggregation of all of the transformed workflows. This aggregate summary can be used as a planning tool and help understand how complete of a migration is possible with Valet. The goal of this lab is to performed an audit on the demo GitLab instance, and gain a good understanding of the components that make up an audit output. @@ -16,24 +16,27 @@ The goal of this lab is to performed an audit on the demo GitLab instance, and g 2. Follow all steps [here](../gitlab#valet-configure-lab) to configure Valet. ## Perform an audit -Before running the command we need to collect some information: - 1. What namespace or group do we want to audit? __valet__ - 2. Where do we want to store the result? __tmp/audit__ +1. Collect information for constructing `audit` command: +- What namespace or group do we want to audit? __valet__ +- Where do we want to store the result? __tmp/audit__ -### Steps +2. Navigate to the codespace terminal. -1. Navigate to the codespace terminal. -2. Run the following Valet audit command with the provided answers above: +3. Run the Valet audit command using the answers from step 1: ``` gh valet audit gitlab --output-dir tmp/audit --namespace valet ``` -3. Valet will print the locations of the audit results in the terminal when complete - Screen Shot 2022-08-16 at 9 57 36 AM +4. Valet will print the locations of the audit results in the terminal when complete. + +5. Verify the audit results are present in the explorer. + +Screen Shot 2022-08-16 at 9 57 36 AM ## Audit Files + The `audit` command outputs the following files: | File Type | Naming Convention | Description | @@ -44,12 +47,17 @@ The `audit` command outputs the following files: | GitHub Actions workflows | valet/PROJECT_NAME.yml | GitHub Actions workflow that would be migrated to GitHub | | Error file | valet/PROJECT_NAME.error.txt | Created when there is a error transforming the pipeline | | Audit summary | audit_summary.md | Contains a summary of audit results and the main file of interest to understand the how complete of a migration can be done with Valet | -| GitHub Actions Reusable Workflows & Composite Actions | .github/workflows/NAME_HERE.yml | These files will only appear if Valet encountered a pipeline that would generated them. These files would be part of the migration to GitHub if they existed | +| GitHub Actions Reusable Workflows | .github/workflows/NAME_HERE.yml | Will only appear if Valet encountered a pipeline that would generated them. | +| GitHub Composite Actions | .github/actions/NAME_HERE.yml | Will only appear if Valet encountered a pipeline that would generated them. | ## Review audit summary + 1. Under the `audit` folder find the `audit_summary.md`. + 2. Right-click the `audit_summary.md` file and select `Open Preview`. + 3. This file contains details about what can be migrated 100% automatically vs. what will need some manual intervention or aren't supported by GitHub Actions. + 4. Review the file, it should match the `audit_summary` below:
Click to expand audit_summary.md @@ -240,7 +248,9 @@ Secrets: **1**
## Review the Pipelines Section + The audit summary starts by giving a summary of the types of pipelines that were extracted. + ```yaml ## Pipelines @@ -271,7 +281,7 @@ Unsupported: **1 (9%)** - Auto DevOps: **1** ``` -Under the `Build steps` section we can see a breakdown of the build steps that are used in the pipelines and what was `Known` and `Unsupported` by Valet. In a later lab we will address the unsupported step `artifacts.terraform`. +- Under the `Build steps` section we can see a breakdown of the build steps that are used in the pipelines and what was `Known` and `Unsupported` by Valet. In a later lab we will address the unsupported step `artifacts.terraform`. ```yaml ### Build steps From 8c6e41bf7a8ff0aabc46e212a8d5d845a8e49f9a Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:08:54 -0400 Subject: [PATCH 16/24] Update 3-dry-run.md --- gitlab/3-dry-run.md | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/gitlab/3-dry-run.md b/gitlab/3-dry-run.md index ac0d8f6..a30dd1d 100644 --- a/gitlab/3-dry-run.md +++ b/gitlab/3-dry-run.md @@ -1,4 +1,4 @@ -# Dry run the migration of an GitLab pipeline to GitHub Actions +# Perform a Dry-run of a GitLab pipeline In this lab, you will use the Valet `dry-run` command to convert a GitLab pipeline to its equivalent GitHub Actions workflow. The end result of this command will be the actions workflow written to your local filesystem. @@ -15,24 +15,31 @@ The end result of this command will be the actions workflow written to your loca ## Perform a dry run -We will be performing a dry-run against a preconfigured project in the GitLab instance. Before running the command we need to collect some information: - 1. What is the project we want to convert? __basic-pipeline-example__ - 2. What is the namespace for that project? __Valet. In this case the namespace is the same as the group the project is in__ - 3. Where do we want to store the result? __./tmp/dry-run-lab. This can be any valid path on the system. In the case of codespaces it is generally best to use `./tmp/SOME_DIRECTORY_HERE` so the files show in explorer__ -### Steps -1. Navigate to the codespace terminal -2. Run the dry-run command using the values determined above +1) Collect information to construct the `dry-run` command: +- What is the project we want to convert? + - basic-pipeline-example +- What is the namespace for that project? + - Valet. In this case the namespace is the same as the group the project is in. +- Where do we want to store the result? + - ./tmp/dry-run-lab. This can be any valid path on the system. In the case of codespaces it is generally best to use `./tmp/SOME_DIRECTORY_HERE` so the files show in explorer. + +2. Run the dry-run command in the terminal using the values from step 1. + ``` gh valet dry-run gitlab --output-dir ./tmp/dry-run-lab --namespace valet --project basic-pipeline-example ``` -3. When the command finishes the output files should be printed to the terminal. - dry-run-terminal -4. Open generated actions workflow - - Find `./tmp/dry-run-lab/valet` in the file explorer pane in codespaces. - - Click `basic-pipeline-example.yml` to open - dry-run-explorer +3. Verify the command printed the result files to the terminal. + + dry-run-terminal + +4. Open generated actions workflow + + - Find `./tmp/dry-run-lab/valet` in the file explorer pane in codespaces. + - Click `basic-pipeline-example.yml` to open. + + dry-run-explorer ## Review dry-run output The dry-run output will show you the GitHub Actions yaml that would be migrated to GitHub with the `migrate` command. We will now take a quick look at what was generated. From 92f25a8941a58417e7581a4cdf1df5a73ce1c0e9 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:25:01 -0400 Subject: [PATCH 17/24] Update 4-custom-transformers.md --- gitlab/4-custom-transformers.md | 88 +++++++++++++++++++-------------- 1 file changed, 52 insertions(+), 36 deletions(-) diff --git a/gitlab/4-custom-transformers.md b/gitlab/4-custom-transformers.md index 3cebec7..cbf4363 100644 --- a/gitlab/4-custom-transformers.md +++ b/gitlab/4-custom-transformers.md @@ -1,4 +1,4 @@ -# Using Custom Transformers in a `dry-run` +# Using custom transformers to customize Valet's behavior In this lab we want to do a `dry-run` of the `terraform-example` project. We can now perform a `dry-run` of this project, however, we have discovered that the output will need to be customized to transform the Terraform artifact report. After some research, we have determined that the `actions/upload-artifact` action will be an adequate substitute for it. Additionally, we will need to change the environment variable `PLAN_JSON` to use a different value: `custom_plan.json`. This customization will be present in many pipelines and an automated way to apply this would be ideal. In this lab, we will use the `--custom-transformers` flag to change the behavior of Valet using its DSL built on top of the Ruby language. - [Prerequisites](#prerequisites) @@ -13,11 +13,14 @@ In this lab we want to do a `dry-run` of the `terraform-example` project. We ca 3. Completed the [dry-run lab](../gitlab/valet-dry-run-lab.md) ## Write Custom Transformers -- Let’s run the `dry-run` command to see what information we can get from the generated action yaml. - ```bash - gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example - ``` -- Open the resulting GitHub Actions workflow by navigating to `tmp/valet/custom-transformer.yml` from the explorer +1. Run the `dry-run` command to see what information we can get from the generated action yaml. + +```bash +gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example +``` + +2. Open the resulting GitHub Actions workflow by navigating to `tmp/valet/custom-transformer.yml` from the explorer + ```yaml name: valet/custom-transformer on: @@ -42,18 +45,20 @@ jobs: - run: terraform show --json $PLAN | convert_report > $PLAN_JSON # # 'artifacts.terraform' was not transformed because there is no suitable equivalent in GitHub Actions ``` -- We can see from the last line that `artifacts.terraform` was not transformed. In order for us to write a custom transformer for this we need to know the identifier. In general, the identifier will be the value between the backticks, which in this case is `artifacts.terraform`. This is how our custom transformer will target the correct step. -- The custom transformers file can have any name, but it is recommended that you use an `.rb` extension so the codespaces editor knows it is a ruby file and can provide syntax highlighting. -- we have chosen the `actions/upload-artifacts` as our replacement so we should look at the action's [documentation](https://github.com/marketplace/actions/upload-a-build-artifact) to determine the correct final yaml - ```yaml - - uses: actions/upload-artifact@v3 - with: - path: VALUE_FROM_GITLAB - ``` -- Now that we know the final yaml needed for the transformer, we can start to write the ruby file. In the custom transformers file we will call the `transform` method. This is a special method that Valet exposes, that takes the identifier we determined earlier and returns a ruby Hash of the final YAML for the pipeline. The ruby Hash can be thought of as the JSON representation of the YAML we want. Valet will call that method when it encounters the identifier and pass in an `item`. The `item` is the values defined for that step in GitLab. In this case the item is the path of the terraform report. +3. We can see from the last line that `artifacts.terraform` was not transformed. In order for us to write a custom transformer for this we need to know the identifier. In general, the identifier will be the value between the backticks, which in this case is `artifacts.terraform`. This is how our custom transformer will target the correct step. -Note: If you were unsure what `item` represents, you could use some basic ruby to print `item` to the terminal. You can achieve this by adding the following line in the transform method: -`puts "This is the item: #{item}"` +4. The custom transformers file can have any name, but it is recommended that you use an `.rb` extension so the codespaces editor knows it is a ruby file and can provide syntax highlighting. + +5. we have chosen the `actions/upload-artifacts` as our replacement so we should look at the action's [documentation](https://github.com/marketplace/actions/upload-a-build-artifact) to determine the correct final yaml + +```yaml +- uses: actions/upload-artifact@v3 + with: + path: VALUE_FROM_GITLAB +``` + +6. Now that we know the final yaml needed for the transformer, we can start to write the ruby file. In the custom transformers file we will call the `transform` method. This is a special method that Valet exposes, that takes the identifier we determined earlier and returns a ruby Hash of the final YAML for the pipeline. The ruby Hash can be thought of as the JSON representation of the YAML we want. Valet will call that method when it encounters the identifier and pass in an `item`. The `item` is the values defined for that step in GitLab. In this case the item is the path of the terraform report. + ```ruby transform "artifacts.terraform" do |item| { @@ -64,38 +69,49 @@ Note: If you were unsure what `item` represents, you could use some basic ruby t } end ``` + + - Note: If you were unsure what `item` represents, you could use some basic ruby to print `item` to the terminal. You can achieve this by adding the following line in the transform method: + `puts "This is the item: #{item}"` -- The custom transformers file also lets you replace values of `variables` by using the `env` method. Let’s replace the value for `PLAN_JSON` by using the below line. The first value of the `env` method is the target variable name and the second is the new value to be used. - ```ruby - env "PLAN_JSON", "custom_plan.json" - ``` -- create a new file in the root of the workspace called `transformers.rb` with the following contents - ```ruby - env "PLAN_JSON", "custom_plan.json" +7. The custom transformers file also lets you replace values of `variables` by using the `env` method. Let’s replace the value for `PLAN_JSON` by using the below line. The first value of the `env` method is the target variable name and the second is the new value to be used. - transform "artifacts.terraform" do |item| - { - uses: "actions/upload-artifact@v2", - with: { - path: item - } +```ruby +env "PLAN_JSON", "custom_plan.json" +``` + +8. Create a new file in the root of the workspace called `transformers.rb` with the following contents + +```ruby +env "PLAN_JSON", "custom_plan.json" + +transform "artifacts.terraform" do |item| + { + uses: "actions/upload-artifact@v2", + with: { + path: item } - end - ``` + } +end +``` + ## Run Again with Customer Transformers -To run the `dry-run` with our custom transformer we will add the `--custom-transformers` flag followed by the path of the custom transformer ruby file +1. Run the `dry-run` with our custom transformer by adding the `--custom-transformers` option followed by the path of the custom transformer ruby file + ```bash gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example --custom-transformers transformers.rb ``` -The custom transformer worked and now we have the `upload-artifact` in the place of the unsupported result. +2. Verify the custom transformer worked and now you have the `upload-artifact` in the place of the unsupported result. + ```diff - # # 'artifacts.terraform' was not transformed because there is no suitable equivalent in GitHub Actions + uses: actions/upload-artifact@v2 + with: + path: "$PLAN_JSON" ``` -Also we can see the `PLAN_JSON` env has been updated to `custom_plan.json` + +3. Verify that `PLAN_JSON` env has been updated to `custom_plan.json` + ```diff env: PLAN: plan.cache @@ -103,7 +119,7 @@ Also we can see the `PLAN_JSON` env has been updated to `custom_plan.json` + PLAN_JSON: custom_plan.json ``` -Now that we have this custom transformers file we can add additional `transform`and `env` methods as needed and reuse it while running other `dry-run` and `migrate` commands. +Now that you have a custom transformers file you can add additional `transform`and `env` methods as needed and reuse it while running other `dry-run` and `migrate` commands. Note: The custom transformers will only affect the pipeline being transformed if they contain the matching identifiers. If you believe a custom transformer should have altered the output, double check that the identifier is correct. From aa638062f11dd8e1fbdcbf09f3691d238f399f73 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:27:02 -0400 Subject: [PATCH 18/24] Update 3-dry-run.md --- gitlab/3-dry-run.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/3-dry-run.md b/gitlab/3-dry-run.md index a30dd1d..8f56aac 100644 --- a/gitlab/3-dry-run.md +++ b/gitlab/3-dry-run.md @@ -268,6 +268,6 @@ jobs: Try constructing and running the `dry-run` command yourself. Hint, you should just have to change the project name. ## Next Lab -[Using Custom Transformers in a dry-runs](../gitlab/4-custom-transformers.md) +[Use custom transformers to customize Valet's behavior](../gitlab/4-custom-transformers.md) From bebefe0971d4cf88800c04da25424d4a5375297d Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:27:50 -0400 Subject: [PATCH 19/24] Update 2-audit.md --- gitlab/2-audit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/2-audit.md b/gitlab/2-audit.md index 2ea26fe..0fe2817 100644 --- a/gitlab/2-audit.md +++ b/gitlab/2-audit.md @@ -346,4 +346,4 @@ The remaining sections `Successful` and `Failed` are groupings of the generated Note: this has files under the `.github` directory. This tells us that this pipeline generated reusable workflows from the `include` statements used in the source pipeline. ### Next Lab -[Dry run the migration of a GitLab pipeline to GitHub Actions](3-dry-run.md) +[Perform a dry-run of a GitLab pipeline](3-dry-run.md) From a9d89c05531ca9f1942dd0783390733282b028a3 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:46:32 -0400 Subject: [PATCH 20/24] Update 5-forecast.md --- gitlab/5-forecast.md | 60 +++++++++++++++++++++++++++++++------------- 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/gitlab/5-forecast.md b/gitlab/5-forecast.md index 4d583dc..56a8f80 100644 --- a/gitlab/5-forecast.md +++ b/gitlab/5-forecast.md @@ -1,11 +1,11 @@ -# Forecast the usage of a GitLab namespace +# Forecast potential build runner usage + In this lab we will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from the historical pipeline data in our GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. - [Prerequisites](#prerequisites) - [Prepare for forecast](#prepare-for-forecast) - [Perform a forecast](#perform-a-forecast) - [Review forecast report](#review-forecast-report) -- [Review additional files](#review-additional-files) - [Forecasting multiple providers](#forecasting-multiple-providers) - [Next steps](#next-steps) @@ -14,29 +14,40 @@ In this lab we will use the `forecast` command to forecast potential GitHub Acti 1. Followed [steps](../gitlab#readme) to set up your codespace environment. 2. Completed the [configure lab](../gitlab/valet-configure-lab.md). 3. Ran the setup script in the terminal to make sure the GitLab instance is ready. - ``` - source gitlab/bootstrap/setup.sh - ``` + +``` +source gitlab/bootstrap/setup.sh +``` ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command. -1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ -2) What is the date we want to start forecasting from? __2022-08-02. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understanding of the typical usage. Too little data and the metrics might not give an accurate picture__ -3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ + +1) What namespace do we want to run the forecast for? +- **valet**. This is the only group in the demo GitLab instance. +2) What is the date we want to start forecasting from? +- **2022-08-02**. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understanding of the typical usage. Too little data and the metrics might not give an accurate picture +3) Where do we want to store the results? +- **./tmp/forecast_reports**. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace. ## Perform a forecast -- Using the answers above we get the following `forecast` command: + +1. Run `forecast` command in the terminal using the answers above. + ``` gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 2022-08-02 ``` -- Run the command in the codespace terminal. -- Verify that the command output is similar to this. - ![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) + +2. Verify that the command output is similar to this. + +![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) ## Review forecast report Open the forecast report and review the calculated metrics. + - From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and right-click, and select __Open Preview__. + ![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png) + - The file should be similar to this.
example forecast_report.md @@ -116,16 +127,20 @@ Open the forecast report and review the calculated metrics. | Max | The highest value | ### Total Section -- This section shows the metrics for all of the jobs run in projects contained in the `valet` namespace, from 08/02/2022 to the time the command was executed. + +- This section shows the metrics for all of the jobs that ran for projects contained in the `valet` namespace, from 08/02/2022 to the time the command was executed. + ## Total - Job count: **57** - Pipeline count: **15** --- + We can see we ran 15 pipelines that contained 57 jobs. The number of jobs is expected to be larger than pipelines because a pipeline is typically a collection of jobs. For example `basic-pipeline-example` contains 6 jobs ![basic-pipeline-jobs](https://user-images.githubusercontent.com/18723510/185423928-ec1b13b5-01fc-4e48-bbe5-0a77be7cecea.png) - `Execution time` shows the metrics for the time a job __took to run__. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quickest job took less than a minute and was rounded down to 0. + - Execution time - Total: **135 minutes** - Median: **0 minutes** @@ -134,20 +149,27 @@ Open the forecast report and review the calculated metrics. - Max: **10 minutes** - `Queue time` shows the metrics for how long jobs __waited__ for a runner to be available. + - Queue time - Median: **0 minutes** - P90: **5 minutes** - Min: **0 minutes** - Max: **42 minutes** -- `Concurrent jobs` show the metrics for how many jobs were run at the __same time__. + +- `Concurrent jobs` show the metrics for how many jobs started or ended within the same 60 second time slice. The time slice window can be changed using the `--time-slice` option. + - Concurrent jobs - Median: **0** - P90: **0** - Min: **0** - Max: **9** + ### Runner Group Sections -- The preceding section shows the same metrics as the `Total` section, but are grouped by runner group. A runner group is a machine (or group of machines) that each job runs on + +- The preceding section shows the same metrics as the `Total` section, but are grouped by runner group. A runner group is a machine (or group of machines) that jobs can run on + - In this case we only have one runner group `gitlab-runner` so the metrics match the `Total` section. If there were different groups we could possibly identify runner types that needed to be increased or decreased when moving to GitHub Actions + ## gitlab-runner - Job count: **57** @@ -173,14 +195,18 @@ Open the forecast report and review the calculated metrics. - Max: **9** ## Forecasting multiple providers + If we examine the help for the `forecast` command by running `gh valet forecast --help` we can see a new option `--source-file-path` ![forecast-help](https://user-images.githubusercontent.com/18723510/185643813-8a56710a-b5e6-4f62-a073-67d548cade1c.png) Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as GitLab and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a space-delimited list of data file paths or a glob pattern that will match all of the data files we want to include and combine into a single report. We will use a glob pattern, which in general should match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the previous value used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`! + - run `gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast` -- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider, it only operates on the data files it finds. + +- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider, it only operates on the data files it finds. + ![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png) ## Next steps -[Migrating a GitLab Pipeline](../gitlab/6-migrate.md) +[Perform a production migration of a GitLab pipeline](../gitlab/6-migrate.md) From 42fa4e288a901ff1ac1322999540514761254711 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:50:30 -0400 Subject: [PATCH 21/24] Update 6-migrate.md --- gitlab/6-migrate.md | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/gitlab/6-migrate.md b/gitlab/6-migrate.md index 3321dd0..fa42794 100644 --- a/gitlab/6-migrate.md +++ b/gitlab/6-migrate.md @@ -1,4 +1,4 @@ -# Migrate an GitLab Project to GitHub Actions +# Perform a production migration of a GitLab pipeline In this lab, we will use the Valet `migrate` command to migrate a GitLab pipeline to GitHub Actions. The previous commands used in the labs, such as `audit` and `dry-run` have prepared us to run a migration. The `migrate` command will transform the Gitlab pipeline into a GitHub Actions workflow like the `dry-run` command did, but instead of writing these files locally, it will open a pull request with the files. @@ -11,12 +11,14 @@ The pull request will contain a checklist of `Manual Tasks` if required. These t - [Next Lab](#next-lab) ## Prerequisites + 1. Followed [steps](../gitlab#readme) to set up your codespace environment. 2. Completed the [configure lab](../gitlab/valet-configure-lab.md) 3. Completed the [dry-run lab](../gitlab/valet-dry-run-lab.md) 4. Completed the [dry-run lab with custom transformer](../gitlab/valet-custom-transformers-lab.md) ## Preparing for migration + Before running the command we need to collect some information: 1. What is the project we want to migrate? __rails-example__ 2. What is the namespace for that project? __Valet. In this case the namespace is the same as the group the project is in__ @@ -24,22 +26,33 @@ Before running the command we need to collect some information: 4. What is the URL for the GitHub repository we want to add the workflow too? __this repo!__. *When constructing the value for the migrate command it should match this url https://github.com/GITHUB-ORG-USERNAME-HERE/GITHUB-REPO-NAME-HERE with `GITHUB-ORG-USERNAME` and `GITHUB-REPO-NAME` substitued with your values* ## Performing a migration + 1. Run `migrate` command using the information collected above, make sure to update the `--target-url` value with the information from step 4. + ``` gh valet migrate gitlab --target-url https://github.com/GITHUB-ORG-USERNAME-HERE/GITHUB-REPO-NAME-HERE --output-dir ./tmp/migrate --namespace valet --project rails-example ``` + 2. Valet will create a pull request directly in the target GitHub repository. -3. Open the pull request by clicking the green pull request link in the output of the migrate command, if you have trouble clicking it you can always copy and paste the url in your browser. - ![pr-screen-shot](https://user-images.githubusercontent.com/18723510/184953133-9bafd9a1-c3f0-40b3-8414-f23cea698c8e.png) + +4. Open the pull request by clicking the green pull request link in the output of the migrate command, if you have trouble clicking it you can always copy and paste the url in your browser. + +![pr-screen-shot](https://user-images.githubusercontent.com/18723510/184953133-9bafd9a1-c3f0-40b3-8414-f23cea698c8e.png) ## Reviewing the pull request -- Lets first look at the `Conversation` tab of the PR. It tells us we have a manual task to perform before the GitHub Actions workflow is functional. We need to add a secret. We can use the GitHub [documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) for secrets and add a `actions` secret for `PASSWORD` with any value. -- Next, lets review the workflow we are adding by clicking on `Files changed` tab. This is where you would double check everything looks good. If it didn't you could push commits with the required changes, prior to merging. -- Now our review is completed we want to go back to the `Conversation` tab and click `Merge pull request` -- Once the PR is merged the new workflow should start and we can view it by clicking `Actions` in the top navigation - actions-screen-shot -- The migration is complete and we have successfully transformed a pipeline from GitLab into a GitHub Actions workflow, added it to a GitHub Repository, and run the workflow in Actions. +1. Lets first look at the `Conversation` tab of the PR. It tells us we have a manual task to perform before the GitHub Actions workflow is functional. We need to add a secret. We can use the GitHub [documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) for secrets and add a `actions` secret for `PASSWORD` with any value. + +2. Review the workflow we are adding by clicking on `Files changed` tab. This is where you would double check everything looks good. If it didn't you could push commits with the required changes, prior to merging. + +3. Go back to the `Conversation` tab and click `Merge pull request` + +4. Once the PR is merged the new workflow should start and we can view it by clicking `Actions` in the top navigation bar + +actions-screen-shot + +5. The migration is complete and we have successfully transformed a pipeline from GitLab into a GitHub Actions workflow, added it to a GitHub Repository, and run the workflow in Actions. ### Next Lab + This concludes the Valet labs for GitLab, if you are interested exploring the power of Valet more. You can leverage the demo GitLab Instance and modify and add new projects that more closely match your needs and try out the commands again! From 68f489e273f7b6ef140626a0a651411033c549ec Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 2 Sep 2022 13:54:54 -0400 Subject: [PATCH 22/24] Update 1-configure.md --- gitlab/1-configure.md | 1 + 1 file changed, 1 insertion(+) diff --git a/gitlab/1-configure.md b/gitlab/1-configure.md index 86cfeb4..5811812 100644 --- a/gitlab/1-configure.md +++ b/gitlab/1-configure.md @@ -59,6 +59,7 @@ To verify our environment is configured correctly, we are going to run a `update gh valet dry-run gitlab --output-dir ./tmp/dry-run-lab --namespace valet --project basic-pipeline-example ``` 4. In the terminal you should see the command was successful, if not it is a good time to practice the configure command again and make sure the access tokens values are correct and were generated with the correct permissions. + ![configure-dry-run](https://user-images.githubusercontent.com/18723510/183989794-d51fa29d-b4c0-4074-8402-a55ffcea3a6b.png) ### Next Lab From 08f0a353e3ee08b50f1d5e4570507d183b776687 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Tue, 6 Sep 2022 10:22:04 -0400 Subject: [PATCH 23/24] Update 1-configure.md --- gitlab/1-configure.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gitlab/1-configure.md b/gitlab/1-configure.md index 5811812..5a9e5de 100644 --- a/gitlab/1-configure.md +++ b/gitlab/1-configure.md @@ -43,7 +43,7 @@ In this lab, you will use the Valet `configure` command to set up the required i ![configure-result](https://user-images.githubusercontent.com/18723510/183990474-d0b2559c-d2bf-40d9-ac43-19af53e45329.png) ## Verify your environment -To verify our environment is configured correctly, we are going to run a `update` and `dry-run` command. The `update` commnad will download the latest version of Valet to your codespace. We will go further into details about the `dry-run` command in a later lab, but for now we will just verify it finishes with no errors. +To verify our environment is configured correctly, we are going to run a `update` and `dry-run` command. The `update` command will download the latest version of Valet to your codespace. We will go further into details about the `dry-run` command in a later lab, but for now we will just verify it finishes with no errors. 1. In the codespace terminal update Valet by running `gh valet update` 2. In the terminal you should see a confirmation that it logged into the GitHub Container Registry and pulled the latest version. @@ -54,7 +54,7 @@ To verify our environment is configured correctly, we are going to run a `update Status: Image is up to date for ghcr.io/valet-customers/valet-cli:latest ghcr.io/valet-customers/valet-cli:latest ``` - 3. Next, lets run the dry-run command in the codespaces terminal, to verify we can talk to GitLab + 3. Run the dry-run command in the codespaces terminal to verify Valet can communicate with the GitLab server ``` gh valet dry-run gitlab --output-dir ./tmp/dry-run-lab --namespace valet --project basic-pipeline-example ``` From 94b5c6bec3ff8340ef1571ebf421a5a0ea2837bb Mon Sep 17 00:00:00 2001 From: j-dunham Date: Tue, 6 Sep 2022 10:40:26 -0400 Subject: [PATCH 24/24] Update 2-audit.md --- gitlab/2-audit.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gitlab/2-audit.md b/gitlab/2-audit.md index 0fe2817..8a10226 100644 --- a/gitlab/2-audit.md +++ b/gitlab/2-audit.md @@ -304,7 +304,7 @@ Unsupported: **1 (0%)** - artifacts.terraform: **1** ``` -Under the `Actions` section in `Build Steps` we have the list of the Actions that were used in order to implement the transformation of all of these build steps. +Under the `Actions` section in `Build Steps` we have the list of the Actions that were used in order to implement the transformation of the of the `known` build steps. ```yaml Actions: **135** @@ -322,8 +322,8 @@ Actions: **135** Valet is a planning tool that can help in facilitating the migration into GitHub Actions and this list of Actions is a great place to understand what dependencies you would be taking on third-party Actions after this migration. For example, if you are doing things like setting up the allow list of third-party Actions in a GitHub Enterprise server instance this list of Actions is a fantastic place to begin security reviews and audits of what third-party actions to depend on. Valet breaks down the pipeline components further into `Triggers`, `Environment`, `Other`, and `Manual tasks`. -- Triggers are a list of pipeline trigger found -- Environment are a list of project variables found +- Triggers are a list of pipeline trigger found. +- Environment are a list of project variables found. - Manual tasks are a list of user tasks that needs to be done in order for a pipeline to be functional when migrating to GitHub, such as adding `secrets` for a masked project variable, like we see here for the variable `PASSWORD`. In a later lab we will see how these manual tasks appear on a pull request when we do a migration. ``` Secrets: **1**