From c546aca77811f81f1cebd9c0ea73d9025fb49bc5 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 8 Sep 2022 10:04:16 -0400 Subject: [PATCH 1/6] Rename circle-ci/1-configure.md to circle_ci/1-configure.md --- {circle-ci => circle_ci}/1-configure.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {circle-ci => circle_ci}/1-configure.md (100%) diff --git a/circle-ci/1-configure.md b/circle_ci/1-configure.md similarity index 100% rename from circle-ci/1-configure.md rename to circle_ci/1-configure.md From 7e8b3d2d52c8c25726b49bc0ee333d9bd12eccd5 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 8 Sep 2022 10:42:28 -0400 Subject: [PATCH 2/6] Create 2-audit.md --- circle_ci/2-audit.md | 108 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 circle_ci/2-audit.md diff --git a/circle_ci/2-audit.md b/circle_ci/2-audit.md new file mode 100644 index 0000000..3e36a17 --- /dev/null +++ b/circle_ci/2-audit.md @@ -0,0 +1,108 @@ +# Perform an audit of CircleCI + +In this lab, you will use the `audit` command to get a high-level view of all projects in CircleCI. + +The `audit` command operates by fetching all of the projects defined in CircleCI , converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. + +## Prerequisites + +1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment. +2. Completed the [configure lab](./1-configure.md#configure-credentials-for-valet). + +## Perform an audit + +We will be performing an audit against a organization in CircleCI created for the purpose of this lab. The name of this organization was already set during the configure lab. The only question that remains to be answered is: + +1. Where do we want to store the result? + - __./tmp/audit__. This can be any path within the working directory that Valet commands are executed from. + +### Steps + +1. Navigate to the codespace terminal. +2. Run the following command from the root directory: + + ```bash + gh valet audit circle-ci --output-dir tmp/audit + ``` + +3. The command will list all the files written to disk in green when the command succeeds. + +## Inspect the output files + +1. Find the `audit_summary.md` file in the file explorer. +2. Right-click the `audit_summary.md` file and select `Open Preview`. +3. This file contains details that summarizes what percentage of your pipelines were converted automatically. + +### Review audit summary + +#### Pipelines + +The pipeline summary section contains high level statistics regarding the conversion rate done by Valet: + +```md +UPDATE_ME +``` + +Here are some key terms in the “Pipelines” section in the above example: + +- __Successful__ pipelines had 100% of the pipeline constructs and individual items converted automatically to their GitHub Actions equivalent. +- __Partially successful__ pipelines had all of the pipeline constructs converted, however, there were some individual items that were not converted automatically to their GitHub Actions equivalent. +- __Failed pipelines__ encountered a fatal error when being converted. This can occur for one of three reasons: + - The pipeline was misconfigured and not valid in CircleCI. + - Valet encountered an internal error when converting it. + - There was an unsuccessful network response, often due to invalid credentials, that caused the pipeline to be inaccessible. + +The "Job types" section will summarize which types of pipelines are being used and which are supported or unsupported by Valet. + +#### Build steps + +The build steps summary section presents an overview of the individual build steps that are used across all pipelines and how many were automatically converted by Valet. + +```md +UPDATE_ME +``` + +Here are some key terms in the "Build steps" section in the above example: + +- A __known__ build step is a step that was automatically converted to an equivalent action. +- An __unknown__ build step is a step that was not automatically converted to an equivalent action. +- An __unsupported__ build step is a step that is either: + - A step that is fundamentally not supported by GitHub Actions. + - A step that is configured in a way that is incompatible with GitHub Actions. +- An __action__ is a list of the actions that were used in the converted workflows. This is important for the following scenarios: + - Gathering the list of actions to sync to your appliance if you use GitHub Enterprise Server. + - Defining an organization-level allowlist of actions that can be used. This list of actions is a comprehensive list of which actions their security and/or compliance teams will need to review. + +There is an equivalent breakdown of build triggers, environment variables, and other uncategorized items displayed in the audit summary file. + +#### Manual Tasks + +The manual tasks summary section presents an overview of the manual tasks that you will need to perform that Valet is not able to complete automatically. + +```md +UPDATE_ME and update pipeline to add this section +``` + +Here are some key terms in the “Manual tasks” section in the above example: + +- A __secret__ refers to a repository or organization level secret that is used by the converted pipelines. These secrets will need to be created manually in Actions in order for these pipelines to function properly. +- A __self-hosted runner__ refers to a label of a runner that is referenced by a converted pipeline that is not a GitHub-hosted runner. You will need to manually define these runners in order for these pipelines to function properly. + +#### Files + +The final section of the audit report provides a manifest of all of the files that are written to disk during the audit. These files include: + +```md +UPDATE_ME +``` + +Each pipeline will have a variety of files written that include: + +- The original pipeline as it was defined in GitHub. +- Any network responses used to convert a pipeline. +- The converted workflow. +- Stack traces that can used to troubleshoot a failed pipeline conversion + +### Next lab + +[Perform a dry-run of a GitLab pipeline](3-dry-run.md) From d4645af5ae5c19707772fc3e05dbc51951093269 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 8 Sep 2022 11:03:07 -0400 Subject: [PATCH 3/6] Update 1-configure.md --- circle_ci/1-configure.md | 1 + 1 file changed, 1 insertion(+) diff --git a/circle_ci/1-configure.md b/circle_ci/1-configure.md index e2bf147..6f7b768 100644 --- a/circle_ci/1-configure.md +++ b/circle_ci/1-configure.md @@ -28,6 +28,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - At the GitHub url prompt enter the GitHub instance url or hit enter to accept the default value (`https://github.com`). - At the CircleCI token prompt enter the CircleCI access token from step 2 and press enter. - At the CircleCI base url prompt hit enter to accept the default value (`https://circleci.com`). + - At the CircleCI organization name prompt enter `labs-data`. This is a preconfigured organization created just for the purpose of these labs. - At the access token to fetch source code in GitHub prompt enter the GitHub PAT generated in step 1 and press enter. - At the GitHub instance url containing source code prompt press enter to accept the default value (`https://github.com`). From 1cfb227fb25ab80e3bb387c4d47c209b94ff45df Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 8 Sep 2022 16:44:19 -0400 Subject: [PATCH 4/6] Update 2-audit.md --- circle_ci/2-audit.md | 99 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 92 insertions(+), 7 deletions(-) diff --git a/circle_ci/2-audit.md b/circle_ci/2-audit.md index 3e36a17..9e8d7b4 100644 --- a/circle_ci/2-audit.md +++ b/circle_ci/2-audit.md @@ -1,8 +1,8 @@ # Perform an audit of CircleCI -In this lab, you will use the `audit` command to get a high-level view of all projects in CircleCI. +In this lab, you will use the `audit` command to get a high-level view of all projects in a CircleCI organization. -The `audit` command operates by fetching all of the projects defined in CircleCI , converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. +The `audit` command operates by fetching all of the projects defined in CircleCI organization, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. ## Prerequisites @@ -11,7 +11,7 @@ The `audit` command operates by fetching all of the projects defined in CircleCI ## Perform an audit -We will be performing an audit against a organization in CircleCI created for the purpose of this lab. The name of this organization was already set during the configure lab. The only question that remains to be answered is: +We will be performing an audit against a organization in CircleCI that was created for the purposes of this lab *labs-data*. Valet was already set to use this organization during the configure lab. The only remaining information needed for the `audit` command is: 1. Where do we want to store the result? - __./tmp/audit__. This can be any path within the working directory that Valet commands are executed from. @@ -40,7 +40,21 @@ We will be performing an audit against a organization in CircleCI created for th The pipeline summary section contains high level statistics regarding the conversion rate done by Valet: ```md -UPDATE_ME +## Pipelines + +Total: **6** + +- Successful: **6 (100%)** +- Partially successful: **0 (0%)** +- Unsupported: **0 (0%)** +- Failed: **0 (0%)** + +### Job types + +Supported: **6 (100%)** + +- 2.1: **5** +- 2: **1** ``` Here are some key terms in the “Pipelines” section in the above example: @@ -59,7 +73,35 @@ The "Job types" section will summarize which types of pipelines are being used a The build steps summary section presents an overview of the individual build steps that are used across all pipelines and how many were automatically converted by Valet. ```md -UPDATE_ME +### Build steps + +Total: **42** + +Known: **42 (100%)** + +- run: **10** +- checkout: **6** +- circleci_node_install_packages: **6** +- circleci_node_job_test: **5** +- store_artifacts: **3** +- circleci_python_install_packages: **3** +- circleci_ruby_install_deps: **2** +- restore_cache: **2** +- attach_workspace: **1** +- persist_to_workspace: **1** +- store_test_results: **1** +- circleci_ruby_rspec_test: **1** +- circleci_ruby_rubocop_check: **1** + +Actions: **45** + +- run: **23** +- actions/checkout@v2: **7** +- actions/cache@v2: **6** +- actions/upload-artifact@v2: **5** +- ruby/setup-ruby@v1: **2** +- actions/download-artifact@v2: **1** +- ./.github/actions/greeting: **1** ``` Here are some key terms in the "Build steps" section in the above example: @@ -80,7 +122,13 @@ There is an equivalent breakdown of build triggers, environment variables, and o The manual tasks summary section presents an overview of the manual tasks that you will need to perform that Valet is not able to complete automatically. ```md -UPDATE_ME and update pipeline to add this section +### Manual tasks + +Total: **1** + +Self hosted runners: **1** + +- `medium+`: **1** ``` Here are some key terms in the “Manual tasks” section in the above example: @@ -93,7 +141,44 @@ Here are some key terms in the “Manual tasks” section in the above example: The final section of the audit report provides a manifest of all of the files that are written to disk during the audit. These files include: ```md -UPDATE_ME +### Successful + +#### labs-data/circleci-hello-world + +- [labs-data/circleci-hello-world/say-hello-workflow.yml](labs-data/circleci-hello-world/say-hello-workflow.yml) +- [labs-data/circleci-hello-world.config.json](labs-data/circleci-hello-world.config.json) +- [labs-data/circleci-hello-world.source.yml](labs-data/circleci-hello-world.source.yml) + +#### labs-data/circleci-command-example + +- [labs-data/circleci-command-example/my-workflow.yml](labs-data/circleci-command-example/my-workflow.yml) +- [.github/actions/greeting/action.yml](.github/actions/greeting/action.yml) +- [labs-data/circleci-command-example.config.json](labs-data/circleci-command-example.config.json) +- [labs-data/circleci-command-example.source.yml](labs-data/circleci-command-example.source.yml) + +#### labs-data/circleci-node-example + +- [labs-data/circleci-node-example/sample.yml](labs-data/circleci-node-example/sample.yml) +- [labs-data/circleci-node-example.config.json](labs-data/circleci-node-example.config.json) +- [labs-data/circleci-node-example.source.yml](labs-data/circleci-node-example.source.yml) + +#### labs-data/circleci-python-example + +- [labs-data/circleci-python-example/sample.yml](labs-data/circleci-python-example/sample.yml) +- [labs-data/circleci-python-example.config.json](labs-data/circleci-python-example.config.json) +- [labs-data/circleci-python-example.source.yml](labs-data/circleci-python-example.source.yml) + +#### labs-data/circleci-demo-java-spring + +- [labs-data/circleci-demo-java-spring/workflow.yml](labs-data/circleci-demo-java-spring/workflow.yml) +- [labs-data/circleci-demo-java-spring.config.json](labs-data/circleci-demo-java-spring.config.json) +- [labs-data/circleci-demo-java-spring.source.yml](labs-data/circleci-demo-java-spring.source.yml) + +#### labs-data/circleci-demo-ruby-rails + +- [labs-data/circleci-demo-ruby-rails/build_and_test.yml](labs-data/circleci-demo-ruby-rails/build_and_test.yml) +- [labs-data/circleci-demo-ruby-rails.config.json](labs-data/circleci-demo-ruby-rails.config.json) +- [labs-data/circleci-demo-ruby-rails.source.yml](labs-data/circleci-demo-ruby-rails.source.yml) ``` Each pipeline will have a variety of files written that include: From c2e584abead6e10c0e77dd139f852cd6237b0b0b Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 8 Sep 2022 16:56:39 -0400 Subject: [PATCH 5/6] Update 2-audit.md --- circle_ci/2-audit.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/circle_ci/2-audit.md b/circle_ci/2-audit.md index 9e8d7b4..9fb1ee7 100644 --- a/circle_ci/2-audit.md +++ b/circle_ci/2-audit.md @@ -2,7 +2,7 @@ In this lab, you will use the `audit` command to get a high-level view of all projects in a CircleCI organization. -The `audit` command operates by fetching all of the projects defined in CircleCI organization, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. +The `audit` command operates by fetching all of the projects defined in a CircleCI organization, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. ## Prerequisites @@ -11,7 +11,7 @@ The `audit` command operates by fetching all of the projects defined in CircleCI ## Perform an audit -We will be performing an audit against a organization in CircleCI that was created for the purposes of this lab *labs-data*. Valet was already set to use this organization during the configure lab. The only remaining information needed for the `audit` command is: +We will be performing an audit against the **labs-data** organization in CircleCI, which was created for the purposes of this lab. Valet was already been set to use this organization during the configure lab. The only remaining information needed for the `audit` command is: 1. Where do we want to store the result? - __./tmp/audit__. This can be any path within the working directory that Valet commands are executed from. @@ -57,7 +57,7 @@ Supported: **6 (100%)** - 2: **1** ``` -Here are some key terms in the “Pipelines” section in the above example: +Here are some key terms that can appear in the “Pipelines” section: - __Successful__ pipelines had 100% of the pipeline constructs and individual items converted automatically to their GitHub Actions equivalent. - __Partially successful__ pipelines had all of the pipeline constructs converted, however, there were some individual items that were not converted automatically to their GitHub Actions equivalent. @@ -104,7 +104,7 @@ Actions: **45** - ./.github/actions/greeting: **1** ``` -Here are some key terms in the "Build steps" section in the above example: +Here are some key terms that can appear in "Build steps" section: - A __known__ build step is a step that was automatically converted to an equivalent action. - An __unknown__ build step is a step that was not automatically converted to an equivalent action. @@ -131,7 +131,7 @@ Self hosted runners: **1** - `medium+`: **1** ``` -Here are some key terms in the “Manual tasks” section in the above example: +Here are some key terms that can appear in “Manual tasks” section: - A __secret__ refers to a repository or organization level secret that is used by the converted pipelines. These secrets will need to be created manually in Actions in order for these pipelines to function properly. - A __self-hosted runner__ refers to a label of a runner that is referenced by a converted pipeline that is not a GitHub-hosted runner. You will need to manually define these runners in order for these pipelines to function properly. From dfd2b68fe97535aebe145acf3aa929be40307564 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 9 Sep 2022 12:29:56 -0400 Subject: [PATCH 6/6] Update circle_ci/1-configure.md Co-authored-by: Ethan Dennis --- circle_ci/1-configure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/circle_ci/1-configure.md b/circle_ci/1-configure.md index 6f7b768..8cdc30b 100644 --- a/circle_ci/1-configure.md +++ b/circle_ci/1-configure.md @@ -28,7 +28,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi - At the GitHub url prompt enter the GitHub instance url or hit enter to accept the default value (`https://github.com`). - At the CircleCI token prompt enter the CircleCI access token from step 2 and press enter. - At the CircleCI base url prompt hit enter to accept the default value (`https://circleci.com`). - - At the CircleCI organization name prompt enter `labs-data`. This is a preconfigured organization created just for the purpose of these labs. + - At the CircleCI organization name prompt enter `labs-data`. This is the organization we'll be using throughout these labs. - At the access token to fetch source code in GitHub prompt enter the GitHub PAT generated in step 1 and press enter. - At the GitHub instance url containing source code prompt press enter to accept the default value (`https://github.com`).