From 4f34acd39ecf064741501f52215135921a564daf Mon Sep 17 00:00:00 2001 From: Begona Guereca Date: Thu, 15 Sep 2022 15:36:02 -0700 Subject: [PATCH 1/3] Travis CI Audit lab --- travis/2-audit.md | 338 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 338 insertions(+) create mode 100644 travis/2-audit.md diff --git a/travis/2-audit.md b/travis/2-audit.md new file mode 100644 index 0000000..7c10b3b --- /dev/null +++ b/travis/2-audit.md @@ -0,0 +1,338 @@ +# Perform an audit of CircleCI + +In this lab, you will use the `audit` command to get a high-level view of all projects in a Travis CI organization. + +The `audit` command operates by fetching all of the projects defined in a Travis CI organization, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. + +## Prerequisites + +1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment. +2. Completed the [configure lab](./1-configure.md#configure-credentials-for-valet). + +## Perform an audit + +We will be performing an audit against the **labs-data** organization in Travis CI, which was created for the purposes of this lab. Valet was already been set to use this organization during the configure lab. The only remaining information needed for the `audit` command is: + +1. Where do we want to store the result? + - **./tmp/audit**. This can be any path within the working directory that Valet commands are executed from. + +### Steps + +1. Navigate to the codespace terminal. +2. Run the following command from the root directory: + + ```bash + gh valet audit travis-ci --output-dir tmp/audit + ``` + +3. The command will list all the files written to disk in green when the command succeeds. + +## Inspect the output files + +1. Find the `audit_summary.md` file in the file explorer. +2. Right-click the `audit_summary.md` file and select `Open Preview`. +3. This file contains details that summarizes what percentage of your pipelines were converted automatically. + +### Review audit summary + +#### Pipelines + +The pipeline summary section contains high level statistics regarding the conversion rate done by Valet: + +```md +# Audit summary + +Summary for [Travis CI instance](https://travis-ci.com/valet-travis-labs) + +- Valet version: **0.1.0.13539 (5bb6c723d6db3688ea0653e14bbf3c63df445bfc)** +- Performed at: **9/15/22 at 22:24** + +## Pipelines + +Total: **6** + +- Successful: **0 (0%)** +- Partially successful: **6 (100%)** +- Unsupported: **0 (0%)** +- Failed: **0 (0%)** + +### Job types + +Supported: **6 (100%)** + +- ruby: **2** +- c: **1** +- nodejs: **1** +- php: **1** +- python: **1** + +### Build steps + +Total: **29** + +Known: **26 (89%)** + +- script: **7** +- install: **6** +- before_script: **3** +- rvm: **2** +- after_deploy: **2** +- ccache: **1** +- pushover: **1** +- dependencies: **1** +- before_deploy: **1** +- php: **1** +- irc: **1** + +Unknown: **1 (3%)** + +- codedeploy: **1** + +Unsupported: **2 (6%)** + +- sudo: **2** + +Actions: **32** + +- run: **19** +- actions/checkout@v2: **7** +- ruby/setup-ruby@v1: **2** +- actions/cache@v2: **1** +- desiderati/github-action-pushover@v1: **1** +- shivammathur/setup-php@v2: **1** +- rectalogic/notify-irc@v1: **1** + +### Triggers + +Total: **36** + +Known: **36 (100%)** + +- crons: **6** +- branches: **6** +- config_validation: **6** +- build_pull_requests: **6** +- build_pushes: **6** +- builds_only_with_travis_yml: **6** + +Actions: **12** + +- pull_request: **6** +- push: **6** + +### Environment + +Total: **1** + +Known: **1 (100%)** + +- DB: **1** + +Actions: **1** + +- DB: **1** + +### Other + +Total: **18** + +Known: **12 (66%)** + +- auto_cancel_pull_requests: **6** +- auto_cancel_pushes: **6** + +Unknown: **6 (33%)** + +- maximum_number_of_builds: **6** + +Actions: **3** + +- matrix: **2** +- DB: **1** + +### Manual tasks + +Total: **1** + +Secrets: **1** + +- `${{ secrets.PUSHOVER_USER_KEY }}`: **1** + +### Partially successful + +#### valet-travis-labs/deploy-example + +- [valet-travis-labs/deploy-example.yml](valet-travis-labs/deploy-example.yml) +- [valet-travis-labs/deploy-example.config.json](valet-travis-labs/deploy-example.config.json) +- [valet-travis-labs/deploy-example.source.yml](valet-travis-labs/deploy-example.source.yml) + +#### valet-travis-labs/c-sharp-example + +- [valet-travis-labs/c-sharp-example.yml](valet-travis-labs/c-sharp-example.yml) +- [valet-travis-labs/c-sharp-example.config.json](valet-travis-labs/c-sharp-example.config.json) +- [valet-travis-labs/c-sharp-example.source.yml](valet-travis-labs/c-sharp-example.source.yml) + +#### valet-travis-labs/ruby-example + +- [valet-travis-labs/ruby-example.yml](valet-travis-labs/ruby-example.yml) +- [valet-travis-labs/ruby-example.config.json](valet-travis-labs/ruby-example.config.json) +- [valet-travis-labs/ruby-example.source.yml](valet-travis-labs/ruby-example.source.yml) + +#### valet-travis-labs/nodejs-example + +- [valet-travis-labs/nodejs-example.yml](valet-travis-labs/nodejs-example.yml) +- [valet-travis-labs/nodejs-example.config.json](valet-travis-labs/nodejs-example.config.json) +- [valet-travis-labs/nodejs-example.source.yml](valet-travis-labs/nodejs-example.source.yml) + +#### valet-travis-labs/php-example + +- [valet-travis-labs/php-example.yml](valet-travis-labs/php-example.yml) +- [valet-travis-labs/php-example.config.json](valet-travis-labs/php-example.config.json) +- [valet-travis-labs/php-example.source.yml](valet-travis-labs/php-example.source.yml) + +#### valet-travis-labs/python + +- [valet-travis-labs/python.yml](valet-travis-labs/python.yml) +- [valet-travis-labs/python.config.json](valet-travis-labs/python.config.json) +- [valet-travis-labs/python.source.yml](valet-travis-labs/python.source.yml) + +``` + +Here are some key terms that can appear in the “Pipelines” section: + +- **Successful** pipelines had 0% of the pipeline constructs and individual items converted automatically to their GitHub Actions equivalent. +- **Partially successful** pipelines had 100% of all of the pipeline constructs converted, however, there were some individual items that were not converted automatically to their GitHub Actions equivalent. +- **Failed pipelines** encountered a fatal error when being converted. This can occur for one of three reasons: + - The pipeline was misconfigured and not valid in Travis CI. + - Valet encountered an internal error when converting it. + - There was an unsuccessful network response, often due to invalid credentials, that caused the pipeline to be inaccessible. + +The "Job types" section will summarize which types of pipelines are being used and which are supported or unsupported by Valet. + +#### Build steps + +The build steps summary section presents an overview of the individual build steps that are used across all pipelines and how many were automatically converted by Valet. + +```md +Total: **29** + +Known: **26 (89%)** + +- script: **7** +- install: **6** +- before_script: **3** +- rvm: **2** +- after_deploy: **2** +- ccache: **1** +- pushover: **1** +- dependencies: **1** +- before_deploy: **1** +- php: **1** +- irc: **1** + +Unknown: **1 (3%)** + +- codedeploy: **1** + +Unsupported: **2 (6%)** + +- sudo: **2** + +Actions: **32** + +- run: **19** +- actions/checkout@v2: **7** +- ruby/setup-ruby@v1: **2** +- actions/cache@v2: **1** +- desiderati/github-action-pushover@v1: **1** +- shivammathur/setup-php@v2: **1** +- rectalogic/notify-irc@v1: **1** +``` + +Here are some key terms that can appear in "Build steps" section: + +- A **known** build step is a step that was automatically converted to an equivalent action. +- An **unknown** build step is a step that was not automatically converted to an equivalent action. +- An **unsupported** build step is a step that is either: + - A step that is fundamentally not supported by GitHub Actions. + - A step that is configured in a way that is incompatible with GitHub Actions. +- An **action** is a list of the actions that were used in the converted workflows. This is important for the following scenarios: + - Gathering the list of actions to sync to your appliance if you use GitHub Enterprise Server. + - Defining an organization-level allowlist of actions that can be used. This list of actions is a comprehensive list of which actions their security and/or compliance teams will need to review. + +There is an equivalent breakdown of build triggers, environment variables, and other uncategorized items displayed in the audit summary file. + +#### Manual Tasks + +The manual tasks summary section presents an overview of the manual tasks that you will need to perform that Valet is not able to complete automatically. + +```md +### Manual tasks + +Total: **1** + +Secrets: **1** + +- `${{ secrets.PUSHOVER_USER_KEY }}`: **1** +``` + +Here are some key terms that can appear in “Manual tasks” section: + +- A **secret** refers to a repository or organization level secret that is used by the converted pipelines. These secrets will need to be created manually in Actions in order for these pipelines to function properly. +- A **self-hosted runner** refers to a label of a runner that is referenced by a converted pipeline that is not a GitHub-hosted runner. You will need to manually define these runners in order for these pipelines to function properly. + +#### Files + +The final section of the audit report provides a manifest of all of the files that are written to disk during the audit. These files include: + +```md +### Partially successful + +#### valet-travis-labs/deploy-example + +- [valet-travis-labs/deploy-example.yml](valet-travis-labs/deploy-example.yml) +- [valet-travis-labs/deploy-example.config.json](valet-travis-labs/deploy-example.config.json) +- [valet-travis-labs/deploy-example.source.yml](valet-travis-labs/deploy-example.source.yml) + +#### valet-travis-labs/c-sharp-example + +- [valet-travis-labs/c-sharp-example.yml](valet-travis-labs/c-sharp-example.yml) +- [valet-travis-labs/c-sharp-example.config.json](valet-travis-labs/c-sharp-example.config.json) +- [valet-travis-labs/c-sharp-example.source.yml](valet-travis-labs/c-sharp-example.source.yml) + +#### valet-travis-labs/ruby-example + +- [valet-travis-labs/ruby-example.yml](valet-travis-labs/ruby-example.yml) +- [valet-travis-labs/ruby-example.config.json](valet-travis-labs/ruby-example.config.json) +- [valet-travis-labs/ruby-example.source.yml](valet-travis-labs/ruby-example.source.yml) + +#### valet-travis-labs/nodejs-example + +- [valet-travis-labs/nodejs-example.yml](valet-travis-labs/nodejs-example.yml) +- [valet-travis-labs/nodejs-example.config.json](valet-travis-labs/nodejs-example.config.json) +- [valet-travis-labs/nodejs-example.source.yml](valet-travis-labs/nodejs-example.source.yml) + +#### valet-travis-labs/php-example + +- [valet-travis-labs/php-example.yml](valet-travis-labs/php-example.yml) +- [valet-travis-labs/php-example.config.json](valet-travis-labs/php-example.config.json) +- [valet-travis-labs/php-example.source.yml](valet-travis-labs/php-example.source.yml) + +#### valet-travis-labs/python + +- [valet-travis-labs/python.yml](valet-travis-labs/python.yml) +- [valet-travis-labs/python.config.json](valet-travis-labs/python.config.json) +- [valet-travis-labs/python.source.yml](valet-travis-labs/python.source.yml) + +``` + +Each pipeline will have a variety of files written that include: + +- The original pipeline as it was defined in GitHub. +- Any network responses used to convert a pipeline. +- The converted workflow. +- Stack traces that can used to troubleshoot a failed pipeline conversion + +### Next lab + +[Perform a dry-run of a Travis CI pipeline](3-dry-run.md) From 63b163ad5f55c74e1530f8aecfbe283824c8904c Mon Sep 17 00:00:00 2001 From: Begona Guereca Date: Fri, 16 Sep 2022 09:55:49 -0700 Subject: [PATCH 2/3] Update travis/2-audit.md Co-authored-by: Ethan Dennis --- travis/2-audit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/travis/2-audit.md b/travis/2-audit.md index 7c10b3b..57938e5 100644 --- a/travis/2-audit.md +++ b/travis/2-audit.md @@ -11,7 +11,7 @@ The `audit` command operates by fetching all of the projects defined in a Travis ## Perform an audit -We will be performing an audit against the **labs-data** organization in Travis CI, which was created for the purposes of this lab. Valet was already been set to use this organization during the configure lab. The only remaining information needed for the `audit` command is: +You will be performing an audit against the **labs-data** Travis CI organization that was created for the purposes of this lab. Your environment was configured to use this organization during the [configure lab](./1-configure.md). The remaining information needed to perform an `audit` is: 1. Where do we want to store the result? - **./tmp/audit**. This can be any path within the working directory that Valet commands are executed from. From d8b35d040638d03c050caab681750d4571d992b2 Mon Sep 17 00:00:00 2001 From: Begona Guereca Date: Fri, 16 Sep 2022 12:31:27 -0700 Subject: [PATCH 3/3] Update travis/2-audit.md Co-authored-by: Luke Cheung Engle <99493186+luke-engle@users.noreply.github.com> --- travis/2-audit.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/travis/2-audit.md b/travis/2-audit.md index 57938e5..077dc5e 100644 --- a/travis/2-audit.md +++ b/travis/2-audit.md @@ -2,7 +2,10 @@ In this lab, you will use the `audit` command to get a high-level view of all projects in a Travis CI organization. -The `audit` command operates by fetching all of the projects defined in a Travis CI organization, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet. +The `audit` command operates by performing the following: +- Fetching all of the projects defined in a Travis CI organization. +- Converting each to their equivalent GitHub Actions workflow. +- Generating a report that summarizes how complete and complex of a migration is possible with Valet. ## Prerequisites