Merge pull request #56 from valet-customers/feature/circleci-labs
Copy edit CCI labs
This commit is contained in:
Ethan Dennis
+51
-37
@@ -7,48 +7,62 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
|
||||
## Configuring credentials
|
||||
|
||||
1. Create an Azure DevOps personal access token (PAT).
|
||||
|
||||
__Note__: you can skip this step if you still have the PAT created during the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
- Navigate to your existing organization (<https://dev.azure.com/:organization>) in your browser.
|
||||
- In the top right corner of your screen, click `User settings`.
|
||||
- Click `Personal access tokens`.
|
||||
- Select `+ New Token`
|
||||
- Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days.
|
||||
- Select the following scopes (you may need to `Show more scopes` to reveal all scopes):
|
||||
- Agents Pool: `Read`
|
||||
- Build: `Read & Execute`
|
||||
- Code: `Read & Write`
|
||||
- Project and Team: `Read, Write, & Manage`
|
||||
- Release: `Read`
|
||||
- Service Connections: `Read`
|
||||
- Task Groups: `Read`
|
||||
- Variable Groups: `Read`
|
||||
- Click `Create`.
|
||||
- Copy the generated API token and save it in a safe location.
|
||||
__Note__: you can skip this step if you still have the PAT created during the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
- Navigate to your existing organization (<https://dev.azure.com/:organization>) in your browser.
|
||||
- In the top right corner of your screen, click `User settings`.
|
||||
- Click `Personal access tokens`.
|
||||
- Select `+ New Token`
|
||||
- Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days.
|
||||
- Select the following scopes (you may need to `Show all scopes` to reveal all scopes):
|
||||
- Agents Pool: `Read`
|
||||
- Build: `Read & execute`
|
||||
- Code: `Read & write`
|
||||
- Project and Team: `Read, write, & manage`
|
||||
- Release: `Read`
|
||||
- Service Connections: `Read`
|
||||
- Task Groups: `Read`
|
||||
- Variable Groups: `Read`
|
||||
- Click `Create`.
|
||||
- Copy the generated API token and save it in a safe location.
|
||||
|
||||
2. Create a GitHub personal access token (PAT):
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and then click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and then click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Name your token in the `Note` field.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
|
||||
3. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `Azure DevOps`, press the spacebar to select, and then press enter to continue.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the Azure DevOps token prompt, enter the access token from step 1 and press enter.
|
||||
- At the Azure DevOps URL prompt, enter your Azure DevOps URL or press enter to accept the default value (`https://dev.azure.com`).
|
||||
- At the prompt, enter your Azure DevOps organization name. This should be the same organization used in the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
- At the prompt, enter your Azure DevOps project name. This should be the same project name used in the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
- Select the `TERMINAL` tab from within the codespace terminal.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `Azure DevOps`, press the spacebar to select, and then press enter to continue.
|
||||
- At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the Azure DevOps token prompt, enter the access token from step 1 and press enter.
|
||||
- At the Azure DevOps URL prompt, enter your Azure DevOps URL or press enter to accept the default value (`https://dev.azure.com`).
|
||||
- At the prompt, enter your Azure DevOps organization name. This should be the same organization used in the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
- At the prompt, enter your Azure DevOps project name. This should be the same project name used in the setup steps [here](./readme.md#bootstrap-your-azure-devops-organization).
|
||||
|
||||
|
||||
```console
|
||||
$ gh valet configure
|
||||
✔ Which CI providers are you configuring?: Azure DevOps
|
||||
Enter the following values (leave empty to omit):
|
||||
✔ GitHub handle used to authenticate with the GitHub Container Registry: mona
|
||||
✔ Personal access token to authenticate with the GitHub Container Registry: ***************
|
||||
✔ Personal access token for GitHub: ***************
|
||||
✔ Base url of the GitHub instance: https://github.com
|
||||
✔ Personal access token for Azure DevOps: ***************
|
||||
✔ Base url of the Azure DevOps instance: https://dev.azure.com
|
||||
✔ Azure DevOps organization name: :organization
|
||||
✔ Azure DevOps project name: :project
|
||||
Environment variables successfully updated.
|
||||
```
|
||||
|
||||
## Verify your environment
|
||||
|
||||
|
||||
+30
-1
@@ -2,7 +2,10 @@
|
||||
|
||||
In this lab, you will use the `audit` command to get a high-level view of all pipelines in an Azure DevOps organization or project.
|
||||
|
||||
The `audit` command operates by fetching all of the pipelines defined in an Azure DevOps organization or project, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
The `audit` command will perform the following steps:
|
||||
1. Fetch all of the projects defined in an Azure DevOps organization.
|
||||
2. Convert each pipeline to their equivalent GitHub Actions workflow.
|
||||
3. Generate a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -170,6 +173,32 @@ Each pipeline will have a variety of files written that include:
|
||||
- The converted workflow.
|
||||
- Stack traces that can used to troubleshoot a failed pipeline conversion
|
||||
|
||||
## Inspect the workflow usage csv file
|
||||
|
||||
1. Open the `tmp/audit/workflow_usage.csv` file in the file explorer.
|
||||
2. This file contains a comma-separated list of all actions, secrets, and runners that are used by each successfully converted pipeline:
|
||||
|
||||
```csv
|
||||
Pipeline,Action,File path
|
||||
lab-test/pipelines/valet-pipeline2,actions/checkout@v2,/data/tmp/adoa/lab-test/pipelines/valet-pipeline2.yml
|
||||
lab-test/pipelines/valet-pipeline1,actions/checkout@v2,/data/tmp/adoa/lab-test/pipelines/valet-pipeline1.yml
|
||||
lab-test/pipelines/valet-custom-transformer-example,actions/checkout@v2,/data/tmp/adoa/lab-test/pipelines/valet-custom-transformer-example.yml
|
||||
lab-test/pipelines/valet-custom-transformer-example,actions/setup-node@v2,/data/tmp/adoa/lab-test/pipelines/valet-custom-transformer-example.yml
|
||||
|
||||
Pipeline,Secret,File path
|
||||
|
||||
|
||||
Pipeline,Runner,File path
|
||||
lab-test/pipelines/valet-pipeline2,mechamachine,/data/tmp/adoa/lab-test/pipelines/valet-pipeline2.yml
|
||||
lab-test/pipelines/valet-custom-transformer-example,mechamachine,/data/tmp/adoa/lab-test/pipelines/valet-custom-transformer-example.yml
|
||||
```
|
||||
|
||||
The contents of this file can be useful in answering questions similar to the following:
|
||||
- What workflows will depend on which actions?
|
||||
- What workflows use an action that must go through a security review?
|
||||
- What workflows use specific secrets?
|
||||
- What workflows use specific runners?
|
||||
|
||||
### Next lab
|
||||
|
||||
[Forecast potential build runner usage](3-forecast.md)
|
||||
|
||||
@@ -35,7 +35,7 @@ Answer the following questions before running the `forecast` command:
|
||||
|
||||
|
||||
|
||||
4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` command using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json).
|
||||
4. If you inspect the help menu using the `gh valet forecast --help` command, you will see a `--source-file-path` option. You can use this option to perform a `forecast` using json files that are already present on the filesystem. These labs come bundled with sample json files located [here](./bootstrap/jobs.json).
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -87,20 +87,45 @@ You can use custom transformers to override Valet's default behavior. In this sc
|
||||
- __DotnetCoreCLI@2__
|
||||
|
||||
2. What is the desired Actions syntax to use instead?
|
||||
- After some research, you have determined that the uploading test results as an artifact will be suitable:
|
||||
- After some research, you have determined that the following script will provide the desired functionality:
|
||||
|
||||
```yaml
|
||||
- run: shopt -s globstar; for f in ./**/*.csproj; do dotnet build $f --configuration ${{ env.BUILDCONFIGURATION }} ; done
|
||||
shell: bash
|
||||
```
|
||||
|
||||
Now you can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
||||
Now you can begin to write the custom transformer. Custom transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
||||
|
||||
```bash
|
||||
code transformers.rb
|
||||
touch transformers.rb && code transformers.rb
|
||||
```
|
||||
|
||||
Next, you will define a `transform` method for the `DotnetCoreCLI@2` identifier by adding the following code to `transformers.rb`:
|
||||
To build this custom transformer, you first need to inspect the `item` keyword to programmatically obtain the projects, command, and arguments to use in the `DotNetCoreCLI@2` step.
|
||||
|
||||
To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
|
||||
|
||||
```ruby
|
||||
transform "DotNetCoreCLI@2" do |item|
|
||||
puts "This is the item: #{item}"
|
||||
end
|
||||
```
|
||||
|
||||
The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
|
||||
|
||||
Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
|
||||
|
||||
```console
|
||||
$ gh valet dry-run azure-devops pipeline --pipeline-id 6 --output-dir tmp/dry-run --custom-transformers transformers.rb
|
||||
[2022-09-20 18:39:50] Logs: 'tmp/dry-run/log/valet-20220920-183950.log'
|
||||
This is the item: {"command"=>"restore", "projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)"}
|
||||
This is the item: {"projects"=>"$(BuildParameters.RESTOREBUILDPROJECTS)", "arguments"=>"--configuration $(BUILDCONFIGURATION)"}
|
||||
[2022-09-20 18:39:51] Output file(s):
|
||||
[2022-09-20 18:39:51] tmp/dry-run/lab-test/pipelines/valet-custom-transformer-example.yml
|
||||
```
|
||||
|
||||
In the above command you will see two instances of `item` printed to the console. This is because there are two `DotNetCoreCLI@2` steps in the pipeline. Each item listed above represents each `DotNetCoreCLI@2` step in the order that they are defined in the pipeline.
|
||||
|
||||
Now that you know the data structure of `item`, you can access the dotnet projects, command, and arguments programmatically by editing the custom transformer to the following:
|
||||
|
||||
```ruby
|
||||
transform "DotNetCoreCLI@2" do |item|
|
||||
@@ -122,8 +147,6 @@ transform "DotNetCoreCLI@2" do |item|
|
||||
end
|
||||
```
|
||||
|
||||
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Azure DevOps.
|
||||
|
||||
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert an Azure DevOps pipel
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and bootstrap an Azure DevOps project.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
4. Completed the [custom transformers lab](./5-custom-transformers.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
|
||||
@@ -45,11 +45,11 @@ These steps **must** be completed prior to starting other labs.
|
||||
- Click `Personal access tokens`.
|
||||
- Select `+ New Token`
|
||||
- Name your token, select the organization where you want to use the token, and set your token to automatically expire after a set number of days.
|
||||
- Select the following scopes (you may need to `Show more scopes` to reveal all scopes):
|
||||
- Select the following scopes (you may need to `Show all scopes` at the bottom of the page to reveal all scopes):
|
||||
- Agents Pool: `Read`
|
||||
- Build: `Read & Execute`
|
||||
- Code: `Read & Write`
|
||||
- Project and Team: `Read, Write, & Manage`
|
||||
- Build: `Read & execute`
|
||||
- Code: `Read & write`
|
||||
- Project and Team: `Read, write, & manage`
|
||||
- Release: `Read`
|
||||
- Service Connections: `Read`
|
||||
- Task Groups: `Read`
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# Configure credentials for Valet
|
||||
|
||||
In this lab, you will use the `configure` CLI command to set the required credentials and information for Valet to use when working with CircleCI and GitHub.
|
||||
|
||||
You will need to complete all of the setup instructions [here](./readme.md#configure-your-codespace) prior to performing this lab.
|
||||
|
||||
## Configuring credentials
|
||||
|
||||
1. Create a GitHub Personal Access Token (PAT):
|
||||
- Open github.com in a new browser tab.
|
||||
- Click your profile photo in the top right of the UI and click `Settings`.
|
||||
- Click on `Developer Settings` in the left hand panel.
|
||||
- Click `Personal Access Tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Legacy tokens`. You may be required to authenticate with GitHub during this step.
|
||||
- Select the following scopes: `read:packages` and `workflow`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save in a safe location.
|
||||
|
||||
3. Create a CircleCI personal API token using CircleCI's [documentation](https://circleci.com/docs/managing-api-tokens#creating-a-personal-api-token) and store the token in a safe location.
|
||||
|
||||
2. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal window.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Using the down arrow key to highlight `CircleCI`, press the spacebar to select, and then hit enter to continue.
|
||||
- At the GitHub handle prompt, enter the GitHub username used to generate the GitHub PAT in step 2 and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 1 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 1 and press enter.
|
||||
- At the GitHub url prompt, enter the GitHub instance url or hit enter to accept the default value (`https://github.com`).
|
||||
- At the CircleCI token prompt, enter the CircleCI access token from step 2 and press enter.
|
||||
- At the CircleCI base url prompt, hit enter to accept the default value (`https://circleci.com`).
|
||||
- At the CircleCI organization name prompt, enter `valet-labs`. This is the organization we'll be using throughout these labs.
|
||||
|
||||
```console
|
||||
$ gh valet configure
|
||||
✔ Which CI providers are you configuring?: CircleCI
|
||||
Enter the following values (leave empty to omit):
|
||||
✔ GitHub handle used to authenticate with the GitHub Container Registry: mona
|
||||
✔ Personal access token to authenticate with the GitHub Container Registry: ***************
|
||||
✔ Personal access token for GitHub: ***************
|
||||
✔ Base url of the GitHub instance: https://github.com
|
||||
✔ Personal access token for CircleCI: ***************
|
||||
✔ Base url of the CircleCI instance: https://circleci.com
|
||||
✔ CircleCI organization name: valet-labs
|
||||
Environment variables successfully updated.
|
||||
```
|
||||
|
||||
## Verify your environment
|
||||
|
||||
To verify our environment is configured correctly, we are going to run the `update` CLI command. The `update` CLI command will download the latest version of Valet to your codespace.
|
||||
|
||||
1. In the codespace terminal run the following command:
|
||||
|
||||
```bash
|
||||
gh valet update
|
||||
```
|
||||
|
||||
2. You should see a confirmation that you were logged into the GitHub Container Registry and Valet was updated to the latest version.
|
||||
|
||||
```console
|
||||
$ gh valet update
|
||||
Login Succeeded
|
||||
latest: Pulling from valet-customers/valet-cli
|
||||
Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb
|
||||
Status: Image is up to date for ghcr.io/valet-customers/valet-cli:latest
|
||||
ghcr.io/valet-customers/valet-cli:latest
|
||||
```
|
||||
|
||||
### Next lab
|
||||
|
||||
[Perform an audit of CircleCI](./2-audit.md)
|
||||
@@ -0,0 +1,230 @@
|
||||
# Perform an audit of CircleCI
|
||||
|
||||
In this lab, you will use the `audit` command to get a high-level view of all projects in a CircleCI organization.
|
||||
|
||||
The `audit` command will perform the following steps:
|
||||
1. Fetch all of the projects defined in an CircleCI organization.
|
||||
2. Convert each pipeline to their equivalent GitHub Actions workflow.
|
||||
3. Generate a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configure-credentials-for-valet).
|
||||
|
||||
## Perform an audit
|
||||
|
||||
You will be performing an `audit` for the __valet-labs__ CircleCI organization that was created for the purposes of these labs. Your environment was configured to use this organization during the [configure lab](./1-configure.md). The remaining information needed to perform an `audit` is:
|
||||
|
||||
1. Where do we want to store the result?
|
||||
- __tmp/audit__. This can be any path within the working directory that Valet commands are executed from.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Navigate to the codespace terminal.
|
||||
2. Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet audit circle-ci --output-dir tmp/audit
|
||||
```
|
||||
|
||||
3. The command will list all the files written to disk in green when the command succeeds.
|
||||
|
||||
## Inspect the output files
|
||||
|
||||
1. Find the `audit_summary.md` file in the file explorer.
|
||||
2. Right-click the `audit_summary.md` file and select `Open Preview`.
|
||||
3. This file contains details that summarize what percentage of your pipelines were converted automatically.
|
||||
|
||||
### Review audit summary
|
||||
|
||||
#### Pipelines
|
||||
|
||||
The pipeline summary section contains high level statistics regarding the conversion rate done by Valet:
|
||||
|
||||
```md
|
||||
## Pipelines
|
||||
|
||||
Total: **6**
|
||||
|
||||
- Successful: **6 (100%)**
|
||||
- Partially successful: **0 (0%)**
|
||||
- Unsupported: **0 (0%)**
|
||||
- Failed: **0 (0%)**
|
||||
|
||||
### Job types
|
||||
|
||||
Supported: **6 (100%)**
|
||||
|
||||
- 2.1: **5**
|
||||
- 2: **1**
|
||||
```
|
||||
|
||||
Here are some key terms that can appear in the “Pipelines” section:
|
||||
|
||||
- __Successful__ pipelines had 100% of the pipeline constructs and individual items converted automatically to their GitHub Actions equivalent.
|
||||
- __Partially successful__ pipelines had all of the pipeline constructs converted, however, there were some individual items that were not converted automatically to their GitHub Actions equivalent.
|
||||
- __Failed pipelines__ encountered a fatal error when being converted. This can occur for one of three reasons:
|
||||
- The pipeline was misconfigured and not valid in CircleCI.
|
||||
- Valet encountered an internal error when converting it.
|
||||
- There was an unsuccessful network response, often due to invalid credentials, that caused the pipeline to be inaccessible.
|
||||
|
||||
The "Job types" section will summarize which types of pipelines are being used and which are supported or unsupported by Valet.
|
||||
|
||||
#### Build steps
|
||||
|
||||
The build steps summary section presents an overview of the individual build steps that are used across all pipelines and how many were automatically converted by Valet.
|
||||
|
||||
```md
|
||||
### Build steps
|
||||
|
||||
Total: **42**
|
||||
|
||||
Known: **42 (100%)**
|
||||
|
||||
- run: **10**
|
||||
- checkout: **6**
|
||||
- circleci_node_install_packages: **6**
|
||||
- circleci_node_job_test: **5**
|
||||
- store_artifacts: **3**
|
||||
- circleci_python_install_packages: **3**
|
||||
- circleci_ruby_install_deps: **2**
|
||||
- restore_cache: **2**
|
||||
- attach_workspace: **1**
|
||||
- persist_to_workspace: **1**
|
||||
- store_test_results: **1**
|
||||
- circleci_ruby_rspec_test: **1**
|
||||
- circleci_ruby_rubocop_check: **1**
|
||||
|
||||
Actions: **45**
|
||||
|
||||
- run: **23**
|
||||
- actions/checkout@v2: **7**
|
||||
- actions/cache@v2: **6**
|
||||
- actions/upload-artifact@v2: **5**
|
||||
- ruby/setup-ruby@v1: **2**
|
||||
- actions/download-artifact@v2: **1**
|
||||
- ./.github/actions/greeting: **1**
|
||||
```
|
||||
|
||||
Here are some key terms that can appear in "Build steps" section:
|
||||
|
||||
- A __known__ build step is a step that was automatically converted to an equivalent action.
|
||||
- An __unknown__ build step is a step that was not automatically converted to an equivalent action.
|
||||
- An __unsupported__ build step is a step that is either:
|
||||
- A step that is fundamentally not supported by GitHub Actions.
|
||||
- A step that is configured in a way that is incompatible with GitHub Actions.
|
||||
- An __action__ is a list of the actions that were used in the converted workflows. This is important for the following scenarios:
|
||||
- Gathering the list of actions to sync to your appliance if you use GitHub Enterprise Server.
|
||||
- Defining an organization-level allowlist of actions that can be used. This list of actions is a comprehensive list of which actions their security and/or compliance teams will need to review.
|
||||
|
||||
There is an equivalent breakdown of build triggers, environment variables, and other uncategorized items displayed in the audit summary file.
|
||||
|
||||
#### Manual Tasks
|
||||
|
||||
The manual tasks summary section presents an overview of the manual tasks that you will need to perform that Valet is not able to complete automatically.
|
||||
|
||||
```md
|
||||
### Manual tasks
|
||||
|
||||
Total: **1**
|
||||
|
||||
Self hosted runners: **1**
|
||||
|
||||
- `medium+`: **1**
|
||||
```
|
||||
|
||||
Here are some key terms that can appear in “Manual tasks” section:
|
||||
|
||||
- A __secret__ refers to a repository or organization level secret that is used by the converted pipelines. These secrets will need to be created manually in Actions in order for these pipelines to function properly.
|
||||
- A __self-hosted runner__ refers to a label of a runner that is referenced by a converted pipeline that is not a GitHub-hosted runner. You will need to manually define these runners in order for these pipelines to function properly.
|
||||
|
||||
#### Files
|
||||
|
||||
The final section of the audit report provides a manifest of all of the files that are written to disk during the audit. These files include:
|
||||
|
||||
```md
|
||||
### Successful
|
||||
|
||||
#### valet-labs/circleci-hello-world
|
||||
|
||||
- [valet-valet-labs/circleci-hello-world/say-hello-workflow.yml](valet-labs/circleci-hello-world/say-hello-workflow.yml)
|
||||
- [valet-labs/circleci-hello-world.config.json](valet-labs/circleci-hello-world.config.json)
|
||||
- [valet-labs/circleci-hello-world.source.yml](valet-labs/circleci-hello-world.source.yml)
|
||||
|
||||
#### valet-labs/circleci-command-example
|
||||
|
||||
- [valet-labs/circleci-command-example/my-workflow.yml](valet-labs/circleci-command-example/my-workflow.yml)
|
||||
- [.github/actions/greeting/action.yml](.github/actions/greeting/action.yml)
|
||||
- [valet-labs/circleci-command-example.config.json](valet-labs/circleci-command-example.config.json)
|
||||
- [valet-labs/circleci-command-example.source.yml](valet-labs/circleci-command-example.source.yml)
|
||||
|
||||
#### valet-labs/circleci-node-example
|
||||
|
||||
- [valet-labs/circleci-node-example/sample.yml](valet-labs/circleci-node-example/sample.yml)
|
||||
- [valet-labs/circleci-node-example.config.json](valet-labs/circleci-node-example.config.json)
|
||||
- [valet-labs/circleci-node-example.source.yml](valet-labs/circleci-node-example.source.yml)
|
||||
|
||||
#### valet-labs/circleci-python-example
|
||||
|
||||
- [valet-labs/circleci-python-example/sample.yml](valet-labs/circleci-python-example/sample.yml)
|
||||
- [valet-labs/circleci-python-example.config.json](valet-labs/circleci-python-example.config.json)
|
||||
- [valet-labs/circleci-python-example.source.yml](valet-labs/circleci-python-example.source.yml)
|
||||
|
||||
#### valet-labs/circleci-demo-java-spring
|
||||
|
||||
- [valet-labs/circleci-demo-java-spring/workflow.yml](valet-labs/circleci-demo-java-spring/workflow.yml)
|
||||
- [valet-labs/circleci-demo-java-spring.config.json](valet-labs/circleci-demo-java-spring.config.json)
|
||||
- [valet-labs/circleci-demo-java-spring.source.yml](valet-labs/circleci-demo-java-spring.source.yml)
|
||||
|
||||
#### valet-labs/circleci-demo-ruby-rails
|
||||
|
||||
- [valet-labs/circleci-demo-ruby-rails/build_and_test.yml](valet-labs/circleci-demo-ruby-rails/build_and_test.yml)
|
||||
- [valet-labs/circleci-demo-ruby-rails.config.json](valet-labs/circleci-demo-ruby-rails.config.json)
|
||||
- [valet-labs/circleci-demo-ruby-rails.source.yml](valet-labs/circleci-demo-ruby-rails.source.yml)
|
||||
```
|
||||
|
||||
Each pipeline will have a variety of files written that include:
|
||||
|
||||
- The original pipeline as it was defined in GitHub.
|
||||
- Any network responses used to convert a pipeline.
|
||||
- The converted workflow.
|
||||
- Stack traces that can used to troubleshoot a failed pipeline conversion
|
||||
|
||||
## Inspect the workflow usage csv file
|
||||
|
||||
1. Open the `tmp/audit/workflow_usage.csv` file in the file explorer.
|
||||
2. This file contains a comma-separated list of all actions, secrets, and runners that are used by each successfully converted pipeline:
|
||||
|
||||
```csv
|
||||
Pipeline,Action,File path
|
||||
valet-labs/circleci-hello-world,actions/checkout@v2,/data/tmp/audit/valet-labs/circleci-hello-world/say-hello-workflow.yml
|
||||
valet-labs/circleci-command-example,./.github/actions/greeting,/data/tmp/audit/valet-labs/circleci-command-example/my-workflow.yml
|
||||
valet-labs/circleci-node-example,actions/checkout@v2,/data/tmp/audit/valet-labs/circleci-node-example/sample.yml
|
||||
valet-labs/circleci-node-example,actions/cache@v2,/data/tmp/audit/valet-labs/circleci-node-example/sample.yml
|
||||
valet-labs/circleci-python-example,actions/checkout@v2,/data/tmp/audit/valet-labs/circleci-python-example/sample.yml
|
||||
valet-labs/circleci-python-example,actions/cache@v2,/data/tmp/audit/valet-labs/circleci-python-example/sample.yml
|
||||
valet-labs/circleci-demo-java-spring,actions/checkout@v2,/data/tmp/audit/valet-labs/circleci-demo-java-spring/workflow.yml
|
||||
valet-labs/circleci-demo-java-spring,actions/cache@v2,/data/tmp/audit/valet-labs/circleci-demo-java-spring/workflow.yml
|
||||
valet-labs/circleci-demo-java-spring,actions/upload-artifact@v2,/data/tmp/audit/valet-labs/circleci-demo-java-spring/workflow.yml
|
||||
valet-labs/circleci-demo-java-spring,actions/download-artifact@v2,/data/tmp/audit/valet-labs/circleci-demo-java-spring/workflow.yml
|
||||
valet-labs/circleci-demo-ruby-rails,ruby/setup-ruby@v1,/data/tmp/audit/valet-labs/circleci-demo-ruby-rails/build_and_test.yml
|
||||
valet-labs/circleci-demo-ruby-rails,actions/checkout@v2,/data/tmp/audit/valet-labs/circleci-demo-ruby-rails/build_and_test.yml
|
||||
valet-labs/circleci-demo-ruby-rails,actions/cache@v2,/data/tmp/audit/valet-labs/circleci-demo-ruby-rails/build_and_test.yml
|
||||
|
||||
Pipeline,Secret,File path
|
||||
|
||||
|
||||
Pipeline,Runner,File path
|
||||
```
|
||||
|
||||
The contents of this file can be useful in answering questions similar to the following:
|
||||
|
||||
- What workflows will depend on which actions?
|
||||
- What workflows use an action that must go through a security review?
|
||||
- What workflows use specific secrets?
|
||||
- What workflows use specific runners?
|
||||
|
||||
### Next lab
|
||||
|
||||
[Forecast potential build runner usage](3-forecast.md)
|
||||
@@ -0,0 +1,119 @@
|
||||
# Forecast potential build runner usage
|
||||
|
||||
In this lab you will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from completed pipeline runs in CircleCI.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
|
||||
## Perform a forecast
|
||||
|
||||
Answer the following questions before running the `forecast` command:
|
||||
|
||||
1. What is the date you want to start forecasting from?
|
||||
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were triggered in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
|
||||
2. Where do you want to store the results?
|
||||
- **tmp/forecast_reports**
|
||||
|
||||
### Steps
|
||||
|
||||
1. Navigate to your codespace terminal
|
||||
2. Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet forecast circle-ci --output-dir tmp/forecast_reports --start-date 2022-09-02
|
||||
```
|
||||
|
||||
3. The command will list all the files written to disk when the command succeeds.
|
||||
|
||||
```console
|
||||
$ gh valet forecast circle-ci --output-dir tmp/forecast_reports --start-date 2022-09-02
|
||||
[2022-08-20 22:08:20] Logs: 'tmp/forecast/log/valet-20220916-021004.log'
|
||||
[2022-08-20 22:08:20] Forecasting 'http://app.circleci.com/pipelines/github/valet-labs'
|
||||
[2022-08-20 22:08:20] Output file(s):
|
||||
[2022-08-20 22:08:20] tmp/forecast/jobs/09-16-2022-02-10_jobs_0.json
|
||||
[2022-08-20 22:08:20] tmp/forecast/forecast_report.md
|
||||
```
|
||||
|
||||
## Review the forecast report
|
||||
|
||||
The forecast report, logs, and completed job data will be located within the `tmp/forecast_reports` folder.
|
||||
|
||||
1. Find the `forecast_report.md` file in the file explorer.
|
||||
2. Right-click the `forecast_report.md` file and select `Open Preview`.
|
||||
3. This file contains metrics used to forecast potential GitHub Actions usage.
|
||||
|
||||
### Total
|
||||
|
||||
The "Total" section of the forecast report contains high level statistics related to all the jobs completed after the `--start-date` CLI option:
|
||||
|
||||
```md
|
||||
- Job count: **18**
|
||||
- Pipeline count: **13**
|
||||
|
||||
- Execution time
|
||||
|
||||
- Total: **46 minutes**
|
||||
- Median: **1 minutes**
|
||||
- P90: **6 minutes**
|
||||
- Min: **0 minutes**
|
||||
- Max: **10 minutes**
|
||||
|
||||
- Queue time
|
||||
|
||||
- Median: **0 minutes**
|
||||
- P90: **0 minutes**
|
||||
- Min: **0 minutes**
|
||||
- Max: **0 minutes**
|
||||
|
||||
- Concurrent jobs
|
||||
|
||||
- Median: **0**
|
||||
- P90: **0**
|
||||
- Min: **0**
|
||||
- Max: **4**
|
||||
```
|
||||
|
||||
Here are some key terms of items defined in the forecast report:
|
||||
|
||||
- The `job count` is the total number of completed jobs.
|
||||
- The `pipeline count` is the number of unique pipelines used.
|
||||
- `Execution time` describes the amount of time a runner spent on a job. This metric can be used to help plan for the cost of GitHub-hosted runners.
|
||||
- This metric is correlated to how much you should expect to spend in GitHub Actions. This will vary depending on the hardware used for these minutes. You can use the [Actions pricing calculator](https://github.com/pricing/calculator) to estimate a dollar amount.
|
||||
- `Queue time` metrics describe the amount of time a job spent waiting for a runner to be available to execute it.
|
||||
- `Concurrent jobs` metrics describe the amount of jobs running at any given time. This metric can be used to define the number of runners a customer should configure.
|
||||
|
||||
Additionally, these metrics are defined for each queue of runners defined in CircleCI. This is especially useful if there are a mix of hosted/self-hosted runners or high/low spec machines to see metrics specific to different types of runners.
|
||||
|
||||
## Forecasting multiple providers
|
||||
|
||||
You can examine the available options for the `forecast` command by running `gh valet forecast --help`. When you do this you will see the `--source-file-path` option:
|
||||
|
||||
```console
|
||||
$ gh valet forecast -h
|
||||
Options:
|
||||
--source-file-path <source-file-path> (REQUIRED) The file path(s) to existing jobs data.
|
||||
-o, --output-dir <output-dir> (REQUIRED) The location for any output files.
|
||||
--start-date <start-date> The start date of the forecast analysis in YYYY-MM-DD format. [default: 9/12/2022 12:42:39 PM]
|
||||
--time-slice <time-slice> The time slice in seconds to use for computing concurrency metrics. [default: 60]
|
||||
--credentials-file <credentials-file> The file containing the credentials to use.
|
||||
--no-telemetry Boolean value to disallow telemetry.
|
||||
--no-ssl-verify Disable ssl certificate verification.
|
||||
--no-http-cache Disable caching of http responses.
|
||||
-?, -h, --help Show help and usage information
|
||||
```
|
||||
|
||||
You can use the `--source-file-path` CLI option to combine data from multiple reports into a single report. This becomes useful if you use multiple CI/CD providers and want to get a holistic view of the runner usage. This works by using the `.json` files generated by `forecast` commands as space-delimited values for the `--source-file-path` CLI option. Optionally, this value could be a glob pattern to dynamically specify the list of files (e.g. `**/*.json`).
|
||||
|
||||
Run the following command from within the codespace terminal:
|
||||
|
||||
```bash
|
||||
gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/forecast-combined
|
||||
```
|
||||
|
||||
You can now inspect the output of the command to see a forecast report using all of the files matching the `tmp/**/jobs/*.json` pattern.
|
||||
|
||||
## Next steps
|
||||
|
||||
[Perform a dry-run of a CircleCI pipeline](4-dry-run.md)
|
||||
@@ -0,0 +1,197 @@
|
||||
# Perform a dry-run migration of a CircleCI pipeline
|
||||
|
||||
In this lab you will use the `dry-run` command to convert a CircleCI pipeline to its equivalent GitHub Actions workflow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [audit lab](./2-audit.md).
|
||||
|
||||
## Perform a dry run
|
||||
|
||||
You will be performing a dry run migration against a CircleCI project. Answer the following questions before running this command:
|
||||
|
||||
1. What project do you want to convert?
|
||||
- __circleci-demo-ruby-rails__. This is one of the sample projects avaiable in the CircleCI valet-labs organization.
|
||||
|
||||
2. Where do you want to store the result?
|
||||
- __tmp/dry-run__. This can be any path within the working directory that Valet commands are executed from.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Navigate to your codespace terminal
|
||||
2. Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet dry-run circle-ci --output-dir tmp/dry-run --circle-ci-project circleci-demo-ruby-rails
|
||||
```
|
||||
|
||||
3. The command will list all the files written to disk when the command succeeds.
|
||||
|
||||
```console
|
||||
$ gh valet dry-run circle-ci --output-dir tmp/dry-run --circle-ci-project circleci-demo-ruby-rails --circle-ci-organization valet-labs
|
||||
[2022-09-19 19:46:03] Logs: 'tmp/dry-run/log/valet-20220919-194603.log'
|
||||
[2022-09-19 19:46:05] Output file(s):
|
||||
[2022-09-19 19:46:05] tmp/dry-run/valet-labs/circleci-demo-ruby-rails/build_and_test.yml
|
||||
```
|
||||
|
||||
4. View the converted workflow:
|
||||
- Find `tmp/dry-run/valet-labs/circleci-demo-ruby-rails` in the file explorer pane in your codespace.
|
||||
- Click `build_and_test.yml` to open.
|
||||
|
||||
## Inspect the output files
|
||||
|
||||
The files generated from the `dry-run` command represent the equivalent Actions workflow for the CircleCI project. The CircleCI configuration and converted workflow can be seen below:
|
||||
|
||||
<details>
|
||||
<summary><em>CircleCI configuration 👇</em></summary>
|
||||
|
||||
```yaml
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
ruby: circleci/ruby@1.1.0
|
||||
node: circleci/node@2
|
||||
|
||||
jobs:
|
||||
build:
|
||||
docker:
|
||||
- image: cimg/ruby:2.7.5-node
|
||||
steps:
|
||||
- checkout
|
||||
- ruby/install-deps
|
||||
# Store bundle cache
|
||||
- node/install-packages:
|
||||
pkg-manager: yarn
|
||||
cache-key: "yarn.lock"
|
||||
test:
|
||||
parallelism: 3
|
||||
docker:
|
||||
- image: cimg/ruby:2.7.5-node
|
||||
- image: circleci/postgres:9.5-alpine
|
||||
environment:
|
||||
POSTGRES_USER: circleci-demo-ruby
|
||||
POSTGRES_DB: rails_blog_test
|
||||
POSTGRES_PASSWORD: ""
|
||||
environment:
|
||||
BUNDLE_JOBS: "3"
|
||||
BUNDLE_RETRY: "3"
|
||||
PGHOST: 127.0.0.1
|
||||
PGUSER: circleci-demo-ruby
|
||||
PGPASSWORD: ""
|
||||
RAILS_ENV: test
|
||||
steps:
|
||||
- checkout
|
||||
- ruby/install-deps
|
||||
- node/install-packages:
|
||||
pkg-manager: yarn
|
||||
cache-key: "yarn.lock"
|
||||
- run:
|
||||
name: Wait for DB
|
||||
command: dockerize -wait tcp://localhost:5432 -timeout 1m
|
||||
- run:
|
||||
name: Database setup
|
||||
command: bundle exec rails db:schema:load --trace
|
||||
# Run rspec in parallel
|
||||
- ruby/rspec-test
|
||||
- ruby/rubocop-check
|
||||
|
||||
workflows:
|
||||
version: 2
|
||||
build_and_test:
|
||||
jobs:
|
||||
- build
|
||||
- test:
|
||||
requires:
|
||||
- build
|
||||
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><em>Converted workflow 👇</em></summary>
|
||||
|
||||
```yaml
|
||||
name: valet-labs/circleci-demo-ruby-rails/build_and_test
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: cimg/ruby:2.7.5-node
|
||||
steps:
|
||||
- name: Set up bundler cache
|
||||
uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: 3.0.2
|
||||
bundler-cache: true
|
||||
- uses: actions/checkout@v2
|
||||
- run: bundle check || bundle install
|
||||
env:
|
||||
BUNDLE_DEPLOYMENT: true
|
||||
- id: yarn-cache-dir-path
|
||||
run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: "${{ steps.yarn-cache-dir-path.outputs.dir }}"
|
||||
key: "${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}"
|
||||
restore-keys: "${{ runner.os }}-yarn-"
|
||||
- run: yarn install --frozen-lockfile
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: cimg/ruby:2.7.5-node
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:9.5-alpine
|
||||
env:
|
||||
POSTGRES_USER: circleci-demo-ruby
|
||||
POSTGRES_DB: rails_blog_test
|
||||
POSTGRES_PASSWORD: ''
|
||||
needs:
|
||||
- build
|
||||
env:
|
||||
BUNDLE_JOBS: '3'
|
||||
BUNDLE_RETRY: '3'
|
||||
PGHOST: 127.0.0.1
|
||||
PGUSER: circleci-demo-ruby
|
||||
PGPASSWORD: ''
|
||||
RAILS_ENV: test
|
||||
steps:
|
||||
- name: Set up bundler cache
|
||||
uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: 3.0.2
|
||||
bundler-cache: true
|
||||
- uses: actions/checkout@v2
|
||||
- run: bundle check || bundle install
|
||||
env:
|
||||
BUNDLE_DEPLOYMENT: true
|
||||
- id: yarn-cache-dir-path
|
||||
run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: "${{ steps.yarn-cache-dir-path.outputs.dir }}"
|
||||
key: "${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}"
|
||||
restore-keys: "${{ runner.os }}-yarn-"
|
||||
- run: yarn install --frozen-lockfile
|
||||
- name: Wait for DB
|
||||
run: dockerize -wait tcp://localhost:5432 -timeout 1m
|
||||
- name: Database setup
|
||||
run: bundle exec rails db:schema:load --trace
|
||||
- run: bundle exec rspec spec --profile 10 --format RspecJunitFormatter --out /tmp/test-results/rspec/results.xml --format progress
|
||||
- run: bundle exec rubocop --format progress
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
Despite these two pipelines using different syntax they will function equivalently.
|
||||
|
||||
## Next lab
|
||||
|
||||
[Use custom transformers to customize Valet's behavior](./5-custom-transformers.md)
|
||||
@@ -0,0 +1,198 @@
|
||||
# Using custom transformers to customize Valet's behavior
|
||||
|
||||
In this lab you will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:
|
||||
|
||||
1. Convert items that are not automatically converted.
|
||||
2. Convert items that were automatically converted using different actions.
|
||||
3. Convert environment variable values differently.
|
||||
4. Convert references to runners to use a different runner name in Actions.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
|
||||
## Perform a dry-run
|
||||
|
||||
You will be performing a `dry-run` command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:
|
||||
|
||||
```bash
|
||||
gh valet dry-run circle-ci --output-dir tmp/dry-run --circle-ci-project circleci-node-example
|
||||
```
|
||||
|
||||
The converted workflow that is generated by the above command can be seen below:
|
||||
|
||||
<details>
|
||||
<summary><em>Converted workflow 👇</em></summary>
|
||||
|
||||
```yaml
|
||||
name: valet-labs/circleci-node-example/sample
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
jobs:
|
||||
setup:
|
||||
runs-on:
|
||||
- self-hosted
|
||||
- large
|
||||
env:
|
||||
COVERAGE_DIR: "./tmp/cov"
|
||||
steps:
|
||||
- run: mkdir -p $COVERAGE_DIR
|
||||
node_test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- id: npm-cache-dir
|
||||
run: echo "::set-output name=dir::$(npm config get cache)"
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: "${{ steps.npm-cache-dir.outputs.dir }}"
|
||||
key: "${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}"
|
||||
restore-keys: "${{ runner.os }}-node-"
|
||||
- run: npm ci
|
||||
- run: npm run test
|
||||
# # This item has no matching transformer
|
||||
# - codecov_codecov_upload:
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
_Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the fundamentals of the `dry-run` command.
|
||||
|
||||
## Custom transformers for an unknown step
|
||||
|
||||
The converted workflow above contains a `codecov_codecov_upload` step that was not automatically converted. Answer the following questions before writing a custom transformer:
|
||||
|
||||
1. What is the "identifier" of the step to customize?
|
||||
- __codecov_codecov_upload__
|
||||
|
||||
2. What is the desired Actions syntax to use instead?
|
||||
- After some research, you have determined that the [Codecov action](https://github.com/marketplace/actions/codecov) in the marketplace will provide similar functionality:
|
||||
|
||||
```yaml
|
||||
- uses: codecov/codecov-action@v3
|
||||
with:
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
```
|
||||
|
||||
Now you can begin to write the custom transformer. Custom transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
||||
|
||||
```bash
|
||||
touch transformers.rb && code transformers.rb
|
||||
```
|
||||
|
||||
Next, you will define a `transform` method for the `codecov_codecov_upload` identifier by adding the following code to `transformers.rb`:
|
||||
|
||||
```ruby
|
||||
transform "codecov_codecov_upload" do |_item|
|
||||
{
|
||||
name: "Upload coverage to Codecov",
|
||||
uses: "codecov/codecov-action@v3",
|
||||
with: {
|
||||
token: "${{ secrets.CODECOV_TOKEN }}"
|
||||
}
|
||||
}
|
||||
end
|
||||
```
|
||||
|
||||
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in CircleCI.
|
||||
|
||||
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
|
||||
|
||||
```bash
|
||||
gh valet dry-run circle-ci --output-dir tmp/dry-run --circle-ci-project circleci-node-example --custom-transformers transformers.rb
|
||||
```
|
||||
|
||||
The converted workflow that is generated by the above command will now use the custom logic for the `codecov_codecov_upload` step.
|
||||
|
||||
```diff
|
||||
- # # This item has no matching transformer
|
||||
- # - codecov_codecov_upload:
|
||||
+ - name: upload coverage to Codecov
|
||||
+ uses: codecov/codecov-action@v3
|
||||
+ with:
|
||||
+ token: "${{ secrets.CODECOV_TOKEN }}"
|
||||
```
|
||||
|
||||
_Note_: If you were unsure what the data structure of `item` was, you could use the following code in the custom transformer to print `item` to the console:
|
||||
|
||||
```ruby
|
||||
transform "codecov_codecov_upload" do |item|
|
||||
puts item
|
||||
end
|
||||
```
|
||||
|
||||
## Custom transformers for environment variables
|
||||
|
||||
You can use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `COVERAGE_DIR` environment variable to be `$RUNNER_TEMP/cov` instead of `./tmp/cov`.
|
||||
|
||||
To do this, add the following code to the `transformers.rb` file.
|
||||
|
||||
```ruby
|
||||
env "COVERAGE_DIR", "$RUNNER_TEMP/cov"
|
||||
```
|
||||
|
||||
In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.
|
||||
|
||||
Now you can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `COVERAGE_DIR` environment variable will be set to `$RUNNER_TEMP/cov`:
|
||||
|
||||
```diff
|
||||
env:
|
||||
- COVERAGE_DIR: "./tmp/cov"
|
||||
+. COVERAGE_DIR: "$RUNNER_TEMP/cov"
|
||||
```
|
||||
|
||||
## Custom transformers for resource class
|
||||
|
||||
You can also use custom transformers to change the runner for a job that defines a `resource_class` attribute. In the example pipeline, the `setup` job uses a `resource_class` of `large` to dictate the machine used to execute the job. In this scenario, you may want to use this value to perform the `setup` job on a runner with the label of `some-large-runner` in Actions.
|
||||
|
||||
To do this, add the following code to the `transformers.rb` file.
|
||||
|
||||
```ruby
|
||||
runner "large", "some-large-runner"
|
||||
```
|
||||
|
||||
In this example, the first parameter to the `runner` method is the resource class value and the second is the label(s) of the runner to use in Actions.
|
||||
|
||||
Now you can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` will be set to `some-large-runner`:
|
||||
|
||||
```diff
|
||||
setup:
|
||||
runs-on:
|
||||
- - self-hosted
|
||||
- - large
|
||||
+ - some-large-runner
|
||||
```
|
||||
|
||||
At this point, the file contents of `transformers.rb` should match this:
|
||||
|
||||
<details>
|
||||
<summary><em>Custom transformers 👇</em></summary>
|
||||
|
||||
```ruby
|
||||
env "COVERAGE_DIR", "$RUNNER_TEMP/cov"
|
||||
runner "large", "some-large-runner"
|
||||
|
||||
transform "codecov_codecov_upload" do |_item|
|
||||
{
|
||||
name: "Upload coverage to Codecov",
|
||||
uses: "codecov/codecov-action@v3",
|
||||
with: { token: "${{ secrets.CODECOV_TOKEN }}" }
|
||||
}
|
||||
end
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
That's it! Congratulations, you have overridden Valet's default behavior by customizing the conversion of:
|
||||
|
||||
- Unknown steps
|
||||
- Environment variables
|
||||
- Runners
|
||||
|
||||
## Next lab
|
||||
|
||||
[Perform a production migration of a CircleCI pipeline](6-migrate.md)
|
||||
@@ -0,0 +1,54 @@
|
||||
# Perform a production migration of a CircleCI pipeline
|
||||
|
||||
In this lab, you will use the `migrate` command to convert a CircleCI pipeline and open a pull request with the equivalent Actions workflow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
Answer the following questions before running a `migrate` command:
|
||||
|
||||
1. What project do you want to migrate?
|
||||
- __circleci-hello-world__
|
||||
2. Where do you want to store the logs?
|
||||
- __tmp/migrate__
|
||||
3. What is the URL for the GitHub repository to add the workflow to?
|
||||
- __this repository__. The URL should follow the pattern <https://github.com/:owner/:repo> with `:owner` and `:repo` replaced with your values.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Run the following `migrate` command in the codespace terminal. Ensure the values in `--target-url` for `:owner` and `:repo` are replaced with your values:
|
||||
|
||||
```bash
|
||||
gh valet migrate circle-ci --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --circle-ci-project circleci-hello-world
|
||||
```
|
||||
|
||||
2. The command will write the URL to the pull request that was created when the command succeeds.
|
||||
|
||||
```console
|
||||
$ gh valet migrate circle-ci --target-url https://github.com/:owner/:repo --output-dir tmp/migrate --circle-ci-project circleci-hello-world
|
||||
[2022-08-20 22:08:20] Logs: 'tmp/migrate/log/valet-20220916-014033.log'
|
||||
[2022-08-20 22:08:20] Pull request: 'https://github.com/:owner/:repo/pull/1'
|
||||
```
|
||||
|
||||
3. Open the generated pull request in a new browser tab.
|
||||
|
||||
### Inspect the pull request
|
||||
|
||||
The first thing to notice about the pull request is that there is a list of manual steps to complete.
|
||||
|
||||
Next, you can inspect the "Files changed" in this pull request and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this pull request.
|
||||
|
||||
|
||||
|
||||
Finally, you can merge the pull request once your review has completed. You can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub.
|
||||
|
||||
At this point, the migration has completed and you have successfully migrated a CircleCI pipeline to Actions!
|
||||
|
||||
### Next Lab
|
||||
|
||||
This concludes all labs for migrating CircleCI pipelines to Actions with Valet!
|
||||
@@ -0,0 +1,75 @@
|
||||
# CircleCI to Actions migrations powered by Valet
|
||||
|
||||
The instructions below will guide you through configuring a GitHub Codespace environment that will be used in subsequent labs that demonstrate how to use Valet to migrate CircleCI pipelines to GitHub Actions.
|
||||
|
||||
These steps **must** be completed prior to starting other labs.
|
||||
|
||||
## Create your own repository for these labs
|
||||
|
||||
1. Ensure that you have created a repository using the [valet-customers/labs](https://github.com/valet-customers/labs) as a template.
|
||||
|
||||
## Configure your Codespace
|
||||
|
||||
1. Start a new Codespace.
|
||||
|
||||
- Click the `Code` with button down arrow above repository on the repository's landing page.
|
||||
- Click the `Codespaces` tab
|
||||
- Click `Create codespaces on main` to create the codespace.
|
||||
- After the Codespace has initialized there will be a terminal present.
|
||||
|
||||
2. Verify the Valet CLI is installed and working. More information on the Valet extension for the official GitHub CLI can be found [here](https://github.com/github/gh-valet).
|
||||
|
||||
- Run the following command in the codespace's terminal:
|
||||
|
||||
```bash
|
||||
gh valet version
|
||||
```
|
||||
|
||||
- Verify the output is similar to below.
|
||||
|
||||
```console
|
||||
$ gh valet version
|
||||
gh version 2.14.3 (2022-07-26)
|
||||
gh valet github/gh-valet v0.1.12
|
||||
valet-cli unknown
|
||||
```
|
||||
|
||||
- If `gh valet version` did not produce similar output, refer to the troubleshooting [guide](#troubleshoot-the-valet-cli).
|
||||
|
||||
## Labs for CircleCI
|
||||
|
||||
Perform the following labs to test-drive Valet
|
||||
|
||||
1. [Configure credentials for Valet](1-configure.md)
|
||||
2. [Perform an audit of CircleCI](2-audit.md)
|
||||
3. [Forecast potential build runner usage](3-forecast.md)
|
||||
4. [Perform a dry-run of a CircleCI pipeline](4-dry-run.md)
|
||||
5. [Use custom transformers to customize Valet's behavior](5-custom-transformers.md)
|
||||
6. [Perform a production migration of a CircleCI pipeline](6-migrate.md)
|
||||
|
||||
## Troubleshoot the Valet CLI
|
||||
|
||||
The CLI extension for Valet can be manually installed by following these steps:
|
||||
|
||||
- Verify you are in the codespace terminal
|
||||
- Run this command from within the codespace's terminal:
|
||||
|
||||
```bash
|
||||
gh extension install github/gh-valet
|
||||
```
|
||||
|
||||
- Verify the result of the install contains:
|
||||
|
||||
```console
|
||||
$ gh extension install github/gh-valet
|
||||
✓ Installed extension github/gh-valet
|
||||
```
|
||||
|
||||
- If you get an error similar to the image below, then click the link in the terminal output to authorize the token.
|
||||
- Restart the codespace after clicking the link.
|
||||
|
||||
- Verify Valet CLI extension is installed and working by running the following command from the codespace's terminal:
|
||||
|
||||
```bash
|
||||
gh valet version
|
||||
```
|
||||
+39
-27
@@ -13,40 +13,52 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
|
||||
```
|
||||
|
||||
2. Open the GitLab server in a new browser tab:
|
||||
- Click the `PORTS` tab in the codespace terminal window.
|
||||
- In the `PORTS` tab find the row for port 80.
|
||||
- Hover over the address under the `Local Address` column and click the globe to "open in browser".
|
||||
- Click the `PORTS` tab in the codespace terminal window.
|
||||
- In the `PORTS` tab find the row for port 80.
|
||||
- Hover over the address under the `Local Address` column and click the globe to "open in browser".
|
||||
|
||||
3. Create a GitLab personal access token (PAT):
|
||||
- Authenticate with the GitLab server using the following credentials:
|
||||
- Username: `root`
|
||||
- Password: `valet-labs!`
|
||||
- Follow the GitLab [instructions](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token) to generate a PAT.
|
||||
- Ensure the token has the `read_api` scope.
|
||||
- Copy the generated token and save it in a safe location.
|
||||
- Authenticate with the GitLab server using the following credentials:
|
||||
- Username: `root`
|
||||
- Password: `valet-labs!`
|
||||
- Follow the GitLab [instructions](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token) to generate a PAT.
|
||||
- Ensure the token has the `read_api` scope.
|
||||
- Copy the generated token and save it in a safe location.
|
||||
|
||||
4. Create a GitHub personal access token (PAT):
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Name your token in the `Note` field.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
|
||||
5. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal window.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `GitLab CI`, press the spacebar to select, and then press enter to continue.
|
||||
- At the prompt, enter your GitHub username and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 4 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 4 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the GitLab CI token prompt, enter the GitLab CI access token from step 3 and press enter.
|
||||
- At the GitLab CI URL prompt, enter `http://localhost` and press enter.
|
||||
- Select the `TERMINAL` tab from within the codespace terminal window.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `GitLab CI`, press the spacebar to select, and then press enter to continue.
|
||||
- At the GitHub handle prompt, enter the GitHub handle used to generate the GitHub PAT in step 2 and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 4 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 4 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the GitLab CI token prompt, enter the GitLab CI access token from step 3 and press enter.
|
||||
- At the GitLab CI URL prompt, enter `http://localhost` and press enter.
|
||||
|
||||
|
||||
```console
|
||||
$ gh valet configure
|
||||
✔ Which CI providers are you configuring?: GitLab CI
|
||||
Enter the following values (leave empty to omit):
|
||||
✔ GitHub handle used to authenticate with the GitHub Container Registry: mona
|
||||
✔ Personal access token to authenticate with the GitHub Container Registry: ***************
|
||||
✔ Personal access token for GitHub: ***************
|
||||
✔ Base url of the GitHub instance: https://github.com
|
||||
✔ Private token for GitLab: ***************
|
||||
✔ Base url of the GitLab instance: http://localhost
|
||||
Environment variables successfully updated.
|
||||
```
|
||||
|
||||
## Verify your environment
|
||||
|
||||
|
||||
+45
-1
@@ -2,7 +2,10 @@
|
||||
|
||||
In this lab, you will use the `audit` command to get a high-level view of all pipelines in a GitLab server.
|
||||
|
||||
The `audit` command operates by fetching all of the pipelines defined in a GitLab server, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
The `audit` command will perform the following steps:
|
||||
1. Fetch all of the projects defined in a GitLab group.
|
||||
2. Convert each pipeline to their equivalent GitHub Actions workflow.
|
||||
3. Generate a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -178,6 +181,47 @@ Each pipeline will have a variety of files written that include:
|
||||
- The converted workflow.
|
||||
- Stack traces that can be used to troubleshoot a failed pipeline conversion
|
||||
|
||||
## Inspect the workflow usage csv file
|
||||
|
||||
1. Open the `tmp/audit/workflow_usage.csv` file in the file explorer.
|
||||
2. This file contains a comma-separated list of all actions, secrets, and runners that are used by each successfully converted pipeline:
|
||||
|
||||
```csv
|
||||
Pipeline,Action,File path
|
||||
valet/included-files-example,actions/checkout@v2,/data/tmp/audit/valet/included-files-example.yml
|
||||
valet/terraform-example,actions/checkout@v2,/data/tmp/audit/valet/terraform-example.yml
|
||||
valet/child-parent-example,actions/checkout@v2,/data/tmp/audit/valet/child-parent-example.yml
|
||||
valet/child-parent-example,./.github/workflows/a-.gitlab-ci.yml,/data/tmp/audit/valet/child-parent-example.yml
|
||||
valet/child-parent-example,./.github/workflows/b-.gitlab-ci.yml,/data/tmp/audit/valet/child-parent-example.yml
|
||||
valet/include-file-example,actions/checkout@v2,/data/tmp/audit/valet/include-file-example.yml
|
||||
valet/basic-pipeline-example,actions/checkout@v2,/data/tmp/audit/valet/basic-pipeline-example.yml
|
||||
valet/gatsby-example,actions/checkout@v2,/data/tmp/audit/valet/gatsby-example.yml
|
||||
valet/gatsby-example,actions/cache@v2,/data/tmp/audit/valet/gatsby-example.yml
|
||||
valet/gatsby-example,actions/upload-artifact@v2,/data/tmp/audit/valet/gatsby-example.yml
|
||||
valet/gatsby-example,actions/download-artifact@v2,/data/tmp/audit/valet/gatsby-example.yml
|
||||
valet/gatsby-example,JamesIves/github-pages-deploy-action@4.1.5,/data/tmp/audit/valet/gatsby-example.yml
|
||||
valet/android-example,actions/checkout@v2,/data/tmp/audit/valet/android-example.yml
|
||||
valet/android-example,actions/upload-artifact@v2,/data/tmp/audit/valet/android-example.yml
|
||||
valet/android-example,actions/download-artifact@v2,/data/tmp/audit/valet/android-example.yml
|
||||
valet/dotnet-example,actions/checkout@v2,/data/tmp/audit/valet/dotnet-example.yml
|
||||
valet/dotnet-example,actions/upload-artifact@v2,/data/tmp/audit/valet/dotnet-example.yml
|
||||
valet/dotnet-example,actions/download-artifact@v2,/data/tmp/audit/valet/dotnet-example.yml
|
||||
valet/node-example,actions/checkout@v2,/data/tmp/audit/valet/node-example.yml
|
||||
valet/node-example,actions/cache@v2,/data/tmp/audit/valet/node-example.yml
|
||||
valet/rails-example,actions/checkout@v2,/data/tmp/audit/valet/rails-example.yml
|
||||
|
||||
Pipeline,Secret,File path
|
||||
valet/rails-example,${{ secrets.PASSWORD }},/data/tmp/audit/valet/rails-example.yml
|
||||
|
||||
Pipeline,Runner,File path
|
||||
```
|
||||
|
||||
The contents of this file can be useful in answering questions similar to the following:
|
||||
- What workflows will depend on which actions?
|
||||
- What workflows use an action that must go through a security review?
|
||||
- What workflows use specific secrets?
|
||||
- What workflows use specific runners?
|
||||
|
||||
### Next lab
|
||||
|
||||
[Forecast potential build runner usage](3-forecast.md)
|
||||
|
||||
+13
-1
@@ -85,7 +85,19 @@ Additionally, these metrics are defined for each queue of runners defined in Git
|
||||
|
||||
You can examine the available options for the `forecast` command by running `gh valet forecast --help`. When you do this you will see the `--source-file-path` option:
|
||||
|
||||
|
||||
```console
|
||||
$ gh valet forecast -h
|
||||
Options:
|
||||
--source-file-path <source-file-path> (REQUIRED) The file path(s) to existing jobs data.
|
||||
-o, --output-dir <output-dir> (REQUIRED) The location for any output files.
|
||||
--start-date <start-date> The start date of the forecast analysis in YYYY-MM-DD format. [default: 9/12/2022 12:42:39 PM]
|
||||
--time-slice <time-slice> The time slice in seconds to use for computing concurrency metrics. [default: 60]
|
||||
--credentials-file <credentials-file> The file containing the credentials to use.
|
||||
--no-telemetry Boolean value to disallow telemetry.
|
||||
--no-ssl-verify Disable ssl certificate verification.
|
||||
--no-http-cache Disable caching of http responses.
|
||||
-?, -h, --help Show help and usage information
|
||||
```
|
||||
|
||||
You can use the `--source-file-path` CLI option to combine data from multiple reports into a single report. This becomes useful if you use multiple CI/CD providers and want to get a holistic view of the runner usage. This works by using the `.json` files generated by `forecast` commands as space-delimited values for the `--source-file-path` CLI option. Optionally, this value could be a glob pattern to dynamically specify the list of files (e.g. `**/*.json`).
|
||||
|
||||
|
||||
+8
-2
@@ -211,7 +211,13 @@ include:
|
||||
- local: /config/test.gitlab-ci.yml
|
||||
```
|
||||
|
||||
The output of the `dry-run` command can be seen below:
|
||||
Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet dry-run gitlab --output-dir tmp/dry-run --namespace valet --project included-files-example
|
||||
```
|
||||
|
||||
The output of the command above can be seen below:
|
||||
|
||||
```yaml
|
||||
name: valet/included-files-example
|
||||
@@ -246,7 +252,7 @@ jobs:
|
||||
- run: echo "this is from a local file"
|
||||
```
|
||||
|
||||
It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date.
|
||||
It's important to note that Valet converted this into a single workflow without templates. This is because of fundamental differences in how GitLab templates and GitHub Actions templates (i.e. Reusable Workflows and Composite Actions) function in regards to job ordering. Unfortunately, elements of reusability will be sacrificed in order for the converted pipelines to function the same. It is likely that the output of Valet could be refactored to use [reusable workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) at a later date.
|
||||
|
||||
As an added challenge, try constructing and running the `dry-run` command yourself. Hint, you should only have to change the project name.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ In this lab you will build upon the `dry-run` command to override Valet's defaul
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start you GitLab server.
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start your GitLab server.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
|
||||
@@ -63,7 +63,7 @@ The converted workflow above contains an `artifacts.terraform` step that was not
|
||||
- __artifacts.terraform__
|
||||
|
||||
2. What is the desired Actions syntax to use instead?
|
||||
- After some research, you have determined that the following bash script will provide similar functionality:
|
||||
- After some research, you have determined that the following action will provide similar functionality:
|
||||
|
||||
```yaml
|
||||
- uses: actions/upload-artifact@v3
|
||||
@@ -74,15 +74,37 @@ The converted workflow above contains an `artifacts.terraform` step that was not
|
||||
Now you can begin to write the custom transformer. Custom transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
||||
|
||||
```bash
|
||||
code transformers.rb
|
||||
touch transformers.rb && code transformers.rb
|
||||
```
|
||||
|
||||
Next, you will define a `transform` method for the `artifacts.terraform` identifier by adding the following code to `transformers.rb`:
|
||||
To build this custom transformer, you first need to inspect the `item` keyword to programmatically use the file path of the terraform json in the `actions/upload-artifact@v3` step.
|
||||
|
||||
To do this, you will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
|
||||
|
||||
```ruby
|
||||
transform "artifacts.terraform" do |item|
|
||||
puts "This is the item: #{item}"
|
||||
end
|
||||
```
|
||||
|
||||
The `transform` method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab.
|
||||
|
||||
Now, we can perform a `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
|
||||
|
||||
```console
|
||||
$ gh valet dry-run gitlab --output-dir tmp --namespace valet --project terraform-example --custom-transformers transformers.rb
|
||||
[2022-09-20 17:47:55] Logs: 'tmp/log/valet-20220920-174755.log'
|
||||
This is the item: $PLAN_JSON
|
||||
[2022-09-20 17:47:56] Output file(s):
|
||||
[2022-09-20 17:47:56] tmp/valet/terraform-example.yml
|
||||
```
|
||||
|
||||
Now that you know the data structure of `item`, you can access the file path programmatically by editing the custom transformer to the following:
|
||||
|
||||
```ruby
|
||||
transform "artifacts.terraform" do |item|
|
||||
{
|
||||
uses: "actions/upload-artifact@v2",
|
||||
uses: "actions/upload-artifact@v3",
|
||||
with: {
|
||||
path: item
|
||||
}
|
||||
@@ -90,8 +112,6 @@ transform "artifacts.terraform" do |item|
|
||||
end
|
||||
```
|
||||
|
||||
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in GitLab.
|
||||
|
||||
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
|
||||
|
||||
```bash
|
||||
@@ -102,19 +122,11 @@ The converted workflow that is generated by the above command will now use the c
|
||||
|
||||
```diff
|
||||
- # # 'artifacts.terraform' was not transformed because there is no suitable equivalent in GitHub Actions
|
||||
+ uses: actions/upload-artifact@v2
|
||||
+ uses: actions/upload-artifact@v3
|
||||
+ with:
|
||||
+ path: "$PLAN_JSON"
|
||||
```
|
||||
|
||||
_Note_: If you were unsure what the data structure of `item` was, you could use the following code in the custom transformer to print `item` to the console:
|
||||
|
||||
```ruby
|
||||
transform "artifacts.terraform" do |item|
|
||||
puts item
|
||||
end
|
||||
```
|
||||
|
||||
## Custom transformers for environment variables
|
||||
|
||||
You can also use custom transformers to edit the values of environment variables in converted workflows. In this example, you will update the `PLAN_JSON` environment variable to be `custom_plan.json` instead of `plan.json`.
|
||||
@@ -136,12 +148,40 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C
|
||||
+ PLAN_JSON: custom_plan.json
|
||||
```
|
||||
|
||||
## Custom transformers for runners
|
||||
|
||||
Finally, you can use custom transformers to dictate which runners the converted workflows should use. To do this, answer the following questions:
|
||||
|
||||
1. What is label of the runner in GitLab to update?
|
||||
- __:default__. This is a special keyword to define the default runner to use. You can optional target specific `tags` in a job.
|
||||
|
||||
2. What is the label of the runner in Actions to use instead?
|
||||
- __custom-runner__
|
||||
|
||||
With these questions answered, you can add the following code to the `transformers.rb` file:
|
||||
|
||||
```ruby
|
||||
runner :default, "custom-runner"
|
||||
```
|
||||
|
||||
In this example, the first parameter to the `runner` method is the GitLab label and the second is the Actions runner label.
|
||||
|
||||
Now you can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
|
||||
|
||||
```diff
|
||||
runs-on:
|
||||
- - ubuntu-latest
|
||||
+ - custom-runner
|
||||
```
|
||||
|
||||
At this point, the file contents of `transformers.rb` should match this:
|
||||
|
||||
<details>
|
||||
<summary><em>Custom transformers 👇</em></summary>
|
||||
|
||||
```ruby
|
||||
runner :default, "custom-runner"
|
||||
|
||||
env "PLAN_JSON", "custom_plan.json"
|
||||
|
||||
transform "artifacts.terraform" do |item|
|
||||
|
||||
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a GitLab pipeline and
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a GitLab server.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
4. Completed the [custom transformers lab](./5-custom-transformers.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
|
||||
+42
-28
@@ -7,40 +7,54 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
|
||||
## Configuring credentials
|
||||
|
||||
1. Open the Jenkins server in a new browser tab:
|
||||
- Click the `PORTS` tab in the codespace terminal window.
|
||||
- In the `PORTS` tab, find the row for port 8080.
|
||||
- Hover over the address under the `Local Address` column and click the globe to "open in browser".
|
||||
- Click the `PORTS` tab in the codespace terminal window.
|
||||
- In the `PORTS` tab, find the row for port 8080.
|
||||
- Hover over the address under the `Local Address` column and click the globe to "open in browser".
|
||||
|
||||
2. Create a Jenkins API token:
|
||||
- In the top right menu bar, click the `admin` button.
|
||||
- In the left panel, click the `Configure` gear.
|
||||
- In the `API token` section, click the `Add new Token` button and click `Generate`.
|
||||
- Copy the generated API token and save it in a safe location.
|
||||
- In the top right menu bar, click the `admin` button.
|
||||
- In the left panel, click the `Configure` gear.
|
||||
- In the `API token` section, click the `Add new Token` button and click `Generate`.
|
||||
- Copy the generated API token and save it in a safe location.
|
||||
|
||||
|
||||
|
||||
|
||||
3. Create a GitHub personal access token (PAT):
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
4. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal window.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `Jenkins`, press the spacebar to select, and then press enter to continue.
|
||||
- At the prompt, enter your GitHub username and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 3 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 3 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the Jenkins token prompt, enter the Jenkins access token from step 2 and press enter.
|
||||
- At the Jenkins URL prompt, enter `http://localhost:8080/` and press enter.
|
||||
- At the Personal access token to fetch source code in GitHub prompt, press enter to accept the default value.
|
||||
- Open github.com in a new browser tab.
|
||||
- In the top right corner of the UI, click your profile photo and click `Settings`.
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Name your token in the `Note` field.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
|
||||
|
||||
4. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal window.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `Jenkins`, press the spacebar to select, and then press enter to continue.
|
||||
- At the GitHub handle prompt, enter the GitHub username used to generate the GitHub PAT in step 3 and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 3 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 3 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
- At the Jenkins token prompt, enter the Jenkins access token from step 2 and press enter.
|
||||
- At the Jenkins username prompt, enter `admin` and press enter.
|
||||
- At the Jenkins URL prompt, enter `http://localhost:8080/` and press enter.
|
||||
|
||||
```console
|
||||
$ gh valet configure
|
||||
✔ Which CI providers are you configuring?: Jenkins
|
||||
Enter the following values (leave empty to omit):
|
||||
✔ GitHub handle used to authenticate with the GitHub Container Registry: mona
|
||||
✔ Personal access token to authenticate with the GitHub Container Registry: ***************
|
||||
✔ Personal access token for GitHub: ***************
|
||||
✔ Base url of the GitHub instance: https://github.com
|
||||
✔ Personal access token for Jenkins: ***************
|
||||
✔ Username of Jenkins user: admin
|
||||
✔ Base url of the Jenkins instance: https://localhost
|
||||
Environment variables successfully updated.
|
||||
```
|
||||
|
||||
## Verify your environment
|
||||
|
||||
|
||||
+36
-1
@@ -2,7 +2,10 @@
|
||||
|
||||
In this lab, you will use the `audit` command to get a high-level view of all pipelines in a Jenkins server.
|
||||
|
||||
The `audit` command operates by fetching all of the pipelines defined in a Jenkins server, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
The `audit` command will perform the following steps:
|
||||
1. Fetch all of the projects defined in a Jenkins server.
|
||||
2. Convert each pipeline to their equivalent GitHub Actions workflow.
|
||||
3. Generate a report that summarizes how complete and complex of a migration is possible with Valet.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -235,6 +238,38 @@ Each pipeline will have a variety of files written that include:
|
||||
- The converted workflow.
|
||||
- Stack traces that can used to troubleshoot a failed pipeline conversion
|
||||
|
||||
## Inspect the workflow usage csv file
|
||||
|
||||
1. Open the `tmp/audit/workflow_usage.csv` file in the file explorer.
|
||||
2. This file contains a comma-separated list of all actions, secrets, and runners that are used by each successfully converted pipeline:
|
||||
|
||||
```csv
|
||||
Pipeline,Action,File path
|
||||
demo_pipeline,actions/checkout@v2,/data/tmp/audit/demo_pipeline.yml
|
||||
demo_pipeline,actions/upload-artifact@v2,/data/tmp/audit/demo_pipeline.yml
|
||||
demo_pipeline,EnricoMi/publish-unit-test-result-action@v1.7,/data/tmp/audit/demo_pipeline.yml
|
||||
monas_dev_work/monas_freestyle,actions/checkout@v2,/data/tmp/audit/monas_dev_work/monas_freestyle.yml
|
||||
monas_dev_work/monas_pipeline,actions/checkout@v2,/data/tmp/audit/monas_dev_work/monas_pipeline.yml
|
||||
test_freestyle_project,actions/checkout@v2,/data/tmp/audit/test_freestyle_project.yml
|
||||
test_pipeline,actions/checkout@v2,/data/tmp/audit/test_pipeline.yml
|
||||
test_pipeline,EnricoMi/publish-unit-test-result-action@v1.7,/data/tmp/audit/test_pipeline.yml
|
||||
|
||||
Pipeline,Secret,File path
|
||||
monas_dev_work/monas_freestyle,${{ secrets.SECRET_TEST_EXPRESSION_VAR }},/data/tmp/audit/monas_dev_work/monas_freestyle.yml
|
||||
test_freestyle_project,${{ secrets.EXPRESSION_FIRST_VAR }},/data/tmp/audit/test_freestyle_project.yml
|
||||
|
||||
Pipeline,Runner,File path
|
||||
demo_pipeline,TeamARunner,/data/tmp/audit/demo_pipeline.yml
|
||||
test_freestyle_project,DemoRunner,/data/tmp/audit/test_freestyle_project.yml
|
||||
test_pipeline,TeamARunner,/data/tmp/audit/test_pipeline.yml
|
||||
```
|
||||
|
||||
The contents of this file can be useful in answering questions similar to the following:
|
||||
- What workflows will depend on which actions?
|
||||
- What workflows use an action that must go through a security review?
|
||||
- What workflows use specific secrets?
|
||||
- What workflows use specific runners?
|
||||
|
||||
## Next lab
|
||||
|
||||
[Forecast potential build runner usage](3-forecast.md)
|
||||
|
||||
+10
-14
@@ -95,21 +95,17 @@ Additionally, these metrics are defined for each queue of runners defined in Jen
|
||||
You can examine the available options for the `forecast` command by running `gh valet forecast jenkins --help`. When you do this you will see the `--source-file-path` option:
|
||||
|
||||
```console
|
||||
$ gh valet forecast jenkins --help
|
||||
$ gh valet forecast -h
|
||||
Options:
|
||||
-u, --jenkins-instance-url <jenkins-instance-url> The URL of the Jenkins CI instance.
|
||||
-n, --jenkins-username <jenkins-username> Username for the Jenkins instance.
|
||||
-t, --jenkins-access-token <jenkins-access-token> Access token for the Jenkins instance.
|
||||
-f, --folders <folders> Folders to forecast in the instance
|
||||
--source-file-path <source-file-path> The file path(s) to existing jobs data.
|
||||
-o, --output-dir <output-dir> (REQUIRED) The location for any output files.
|
||||
--start-date <start-date> The start date of the forecast analysis in YYYY-MM-DD format. [default: 9/9/2022 2:14:15 AM]
|
||||
--time-slice <time-slice> The time slice in seconds to use for computing concurrency metrics. [default: 60]
|
||||
--credentials-file <credentials-file> The file containing the credentials to use.
|
||||
--no-telemetry Boolean value to disallow telemetry.
|
||||
--no-ssl-verify Disable ssl certificate verification.
|
||||
--no-http-cache Disable caching of http responses.
|
||||
-?, -h, --help Show help and usage information
|
||||
--source-file-path <source-file-path> (REQUIRED) The file path(s) to existing jobs data.
|
||||
-o, --output-dir <output-dir> (REQUIRED) The location for any output files.
|
||||
--start-date <start-date> The start date of the forecast analysis in YYYY-MM-DD format. [default: 9/12/2022 12:42:39 PM]
|
||||
--time-slice <time-slice> The time slice in seconds to use for computing concurrency metrics. [default: 60]
|
||||
--credentials-file <credentials-file> The file containing the credentials to use.
|
||||
--no-telemetry Boolean value to disallow telemetry.
|
||||
--no-ssl-verify Disable ssl certificate verification.
|
||||
--no-http-cache Disable caching of http responses.
|
||||
-?, -h, --help Show help and usage information
|
||||
```
|
||||
|
||||
You can use the `--source-file-path` CLI option to combine data from multiple reports into a single report. This becomes useful if you use multiple CI/CD providers and wanted to get a holistic view of the runner usage. This works by using the `.json` files generated by `forecast` commands as space-delimited values for the `--source-file-path` CLI option. Optionally, this value could be a glob pattern to dynamically specify the list of files (e.g. `**/*.json`).
|
||||
|
||||
@@ -75,7 +75,7 @@ _Note_: You can refer to the previous [lab](./4-dry-run.md) to learn about the f
|
||||
|
||||
## Custom transformers for an unknown step
|
||||
|
||||
The converted workflow above contains a `sleep` step was not automatically converted. Answer the following questions before writing a custom transformer:
|
||||
The converted workflow above contains a `sleep` step that was not automatically converted. Answer the following questions before writing a custom transformer:
|
||||
|
||||
1. What is the "identifier" of the step to customize?
|
||||
- __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile.
|
||||
@@ -92,7 +92,7 @@ The converted workflow above contains a `sleep` step was not automatically conve
|
||||
Now you can begin to write the custom transformer. Custom transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
|
||||
|
||||
```bash
|
||||
code transformers.rb
|
||||
touch transformers.rb && code transformers.rb
|
||||
```
|
||||
|
||||
Next, you will define a `transform` method for the `sleep` identifier by adding the following code to `transformers.rb`:
|
||||
@@ -198,7 +198,7 @@ Now you can perform another `dry-run` command with the `--custom-transformers` C
|
||||
+ - uses: actions/upload-artifact@v3
|
||||
+ with:
|
||||
+ name: junit-artifact
|
||||
+ path: path/to/artifact/world.txt
|
||||
+ path: "**/target/*.xml"
|
||||
```
|
||||
|
||||
## Custom transformers for environment variables
|
||||
|
||||
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a Jenkins pipeline an
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment and start a Jenkins server.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./4-dry-run.md).
|
||||
4. Completed the [custom transformers lab](./5-custom-transformers.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
@@ -44,8 +43,6 @@ The first thing we should notice about the pull request is that there is a list
|
||||
|
||||
|
||||
|
||||
Next, review the workflow you are adding by clicking on the `Files changed` tab. This is where you double check that everything looks good. If it didn't, you could push commits with the required changes, prior to merging.
|
||||
|
||||
Next, you can inspect the "Files changed" in this pull request and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this pull request.
|
||||
|
||||
Finally, you can merge the pull request once your review has completed. You can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub.
|
||||
|
||||
@@ -9,6 +9,8 @@ To get started:
|
||||
1. Use the `valet-customers/labs` repository as a template to [generate](https://github.com/valet-customers/labs/generate) a new GitHub repository.
|
||||
2. Select which learning path to begin with. There are currently learning paths for:
|
||||
- [Migrations from Azure DevOps to GitHub Actions](/azure_devops/readme.md)
|
||||
- [Migrations from CircleCI to GitHub Actions](/circle_ci/readme.md)
|
||||
- [Migrations from GitLab CI to GitHub Actions](/gitlab/readme.md)
|
||||
- [Migrations from Jenkins to GitHub Actions](/jenkins/readme.md)
|
||||
- [Migrations from Travis CI to GitHub Actions](/travis/readme.md)
|
||||
3. Each learning path describes how to configure your codespace, bootstrap a CI/CD environment, and troubleshoot the Valet CLI.
|
||||
|
||||
@@ -12,6 +12,7 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
|
||||
- In the left panel, click `Developer Settings`.
|
||||
- Click `Personal access tokens` and then `Legacy tokens` (if present).
|
||||
- Click `Generate new token` and then `Generate new legacy token`. You may be required to authenticate with GitHub during this step.
|
||||
- Name your token in the `Note` field.
|
||||
- Select the following scopes: `workflow` and `read:packages`.
|
||||
- Click `Generate token`.
|
||||
- Copy the generated PAT and save it in a safe location.
|
||||
@@ -19,13 +20,15 @@ You will need to complete all of the setup instructions [here](./readme.md#confi
|
||||
3. Create a Travis CI personal access token (PAT):
|
||||
- Open app.travis-ci.com in a new browser tab.
|
||||
- Click on your profile icon in the top right hand corner to reveal a dropdown menu.
|
||||
- Click `Settings`.
|
||||
- Click on the `Settings` tab.
|
||||
- Click on the `COPY TOKEN` button under "API authentication header" and save it in a safe location.
|
||||
- Click on the `COPY TOKEN` button under the "API authentication" header and save it in a safe location.
|
||||
|
||||
2. Run the `configure` CLI command:
|
||||
- Select the `TERMINAL` tab from within the codespace terminal.
|
||||
- Run the following command: `gh valet configure`.
|
||||
- Use the down arrow key to highlight `Travis CI`, press the spacebar to select, and then press enter to continue.
|
||||
- At the GitHub handle prompt, enter the GitHub username used to generate the GitHub PAT in step 3 and press enter.
|
||||
- At the GitHub Container Registry prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub PAT prompt, enter the GitHub PAT generated in step 2 and press enter.
|
||||
- At the GitHub URL prompt, enter the GitHub instance URL or press enter to accept the default value (`https://github.com`).
|
||||
|
||||
+30
-1
@@ -15,7 +15,7 @@ The `audit` command operates by performing the following:
|
||||
|
||||
## Perform an audit
|
||||
|
||||
You will be performing an audit against the **labs-data** Travis CI organization that was created for the purposes of this lab. Your environment was configured to use this organization during the [configure lab](./1-configure.md). The remaining information needed to perform an `audit` is:
|
||||
You will be performing an audit against the **valet-labs** Travis CI organization that was created for the purposes of this lab. Your environment was configured to use this organization during the [configure lab](./1-configure.md). The remaining information needed to perform an `audit` is:
|
||||
|
||||
1. Where do we want to store the result?
|
||||
- **tmp/audit**. This can be any path within the working directory that Valet commands are executed from.
|
||||
@@ -316,6 +316,35 @@ Each pipeline will have a variety of files written that include:
|
||||
- The converted workflow.
|
||||
- Stack traces that can used to troubleshoot a failed pipeline conversion
|
||||
|
||||
## Inspect the workflow usage csv file
|
||||
|
||||
1. Open the `tmp/audit/workflow_usage.csv` file in the file explorer.
|
||||
2. This file contains a comma-separated list of all actions, secrets, and runners that are used by each successfully converted pipeline:
|
||||
|
||||
```csv
|
||||
Pipeline,Action,File path
|
||||
valet-labs/travisci-php-example,actions/checkout@v2,/data/tmp/audit/valet-labs/travisci-php-example.yml
|
||||
valet-labs/travisci-php-example,shivammathur/setup-php@v2,/data/tmp/audit/valet-labs/travisci-php-example.yml
|
||||
valet-labs/travisci-php-example,rectalogic/notify-irc@v1,/data/tmp/audit/valet-labs/travisci-php-example.yml
|
||||
valet-labs/travisci-nodejs-example,actions/checkout@v2,/data/tmp/audit/valet-labs/travisci-nodejs-example.yml
|
||||
valet-labs/travisci-ruby-example,actions/checkout@v2,/data/tmp/audit/valet-labs/travisci-ruby-example.yml
|
||||
valet-labs/travisci-ruby-example,ruby/setup-ruby@v1,/data/tmp/audit/valet-labs/travisci-ruby-example.yml
|
||||
valet-labs/travisci-python-example,actions/checkout@v2,/data/tmp/audit/valet-labs/travisci-python-example.yml
|
||||
valet-labs/travisci-deploy-example,actions/checkout@v2,/data/tmp/audit/valet-labs/travisci-deploy-example.yml
|
||||
valet-labs/travisci-deploy-example,desiderati/github-action-pushover@v1,/data/tmp/audit/valet-labs/travisci-deploy-example.yml
|
||||
|
||||
Pipeline,Secret,File path
|
||||
valet-labs/travisci-deploy-example,${{ secrets.PUSHOVER_USER_KEY }},/data/tmp/audit/valet-labs/travisci-deploy-example.yml
|
||||
|
||||
Pipeline,Runner,File path
|
||||
```
|
||||
|
||||
The contents of this file can be useful in answering questions similar to the following:
|
||||
- What workflows will depend on which actions?
|
||||
- What workflows use an action that must go through a security review?
|
||||
- What workflows use specific secrets?
|
||||
- What workflows use specific runners?
|
||||
|
||||
### Next lab
|
||||
|
||||
[Forecast potential build runner usage](3-forecast.md)
|
||||
|
||||
@@ -12,7 +12,7 @@ In this lab you will use the `forecast` command to forecast potential GitHub Act
|
||||
Answer the following questions before running the `forecast` command:
|
||||
|
||||
1. What is the date you want to start forecasting from?
|
||||
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were trigger in the demo Travis CI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
|
||||
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were triggered in the demo Travis CI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
|
||||
2. Where do you want to store the results?
|
||||
- **tmp/forecast**
|
||||
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ You will be performing a dry-run against a TravisCI project. Answer the followin
|
||||
```
|
||||
|
||||
4. View the converted workflow:
|
||||
- Find `tmp/dry-run` in the file explorer pane in your codespace.
|
||||
- Find `tmp/dry-run/valet-labs` in the file explorer pane in your codespace.
|
||||
- Click `travisci-ruby-example.yml` to open.
|
||||
|
||||
## Inspect the output files
|
||||
|
||||
@@ -126,7 +126,7 @@ This method can use any valid ruby syntax and should return a `Hash`, or an arra
|
||||
Now you can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
|
||||
|
||||
```bash
|
||||
gh valet dry-run travis-ci --travis-ci-repository "deploy-example" --output-dir tmp/travis/dry-run --custom-transformers transformers.rb
|
||||
gh valet dry-run travis-ci --travis-ci-repository "travisci-deploy-example" --output-dir tmp/travis/dry-run --custom-transformers transformers.rb
|
||||
```
|
||||
|
||||
The converted workflow that is generated by the above command will now use the custom logic for the `codedeploy` step.
|
||||
@@ -188,8 +188,8 @@ Finally, you can use custom transformers to dictate which runners converted work
|
||||
1. What is the label of the runner in Travis to update?
|
||||
- __linux__
|
||||
|
||||
2. What is the label of the runner in Actions to use instead?
|
||||
- __new-runner__
|
||||
2. What is/are the label(s) of the runner(s) in Actions to use instead?
|
||||
- __new-runner__, __self-hosted__
|
||||
|
||||
With these questions answered, you can add the following code to the `transformers.rb` file:
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ In this lab, you will use the `migrate` command to convert a Travis CI pipeline
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./3-dry-run.md).
|
||||
4. Completed the [custom transformers lab](./4-custom-transformers.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user