actions/importer-labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #39 from valet-customers/ethanis/jenkins-labs-edits

Browse Source 
Copywriting for Jenkins labs
This commit is contained in:
Ethan Dennis
2022-09-06 13:38:56 -07:00
committed by GitHub
parent 91b8ab1a08 0b3f64ae5d
commit 35fb47d5f0
13 changed files with 918 additions and 1083 deletions
Download Patch File Download Diff File Expand all files Collapse all files
jenkins/1-configure.md
+70
View File
@@ -0,0 +1,70 @@
# Configure credentials for Valet
In this lab, you will use the `configure` CLI command to set the required credentials and information for Valet to use when working with Jenkins and GitHub.
You will need to complete all of the setup instructions [here](./readme.md#configure-your-codespace) prior to performing this lab.
- [Configuring credentials](#configuring-credentials)
- [Verify your environment](#verify-your-environment)
- [Next Lab](#next-lab)
## Configuring credentials
1. Open the Jenkins server in a new browser tab:
1. Click the `PORTS` tab in the codespace terminal window.
2. In the `PORTS` tab find the row for port 8080.
3. Hover over the address under the `Local Address` column and click the globe to "open in browser".
2. Create a Jenkins API token:
1. Click the `admin` button in the top right menu bar.
2. Click on the `Configure` gear located on the left hand panel.
3. Click the `Add new Token` button in the `API token` section and click `Generate`.
4. Copy the generated API token and save in a safe location.
![img](https://user-images.githubusercontent.com/19557880/184041667-d06cb7f2-a885-474e-b728-7567314aeaf3.png)
3. Create a GitHub Personal Access Token (PAT):
1. Open github.com in a new browser tab.
2. Click your profile photo in the top right of the UI and click `Settings`.
3. Click on `Developer Settings` in the left hand panel.
4. Click `Personal Access Tokens` and then `Legacy tokens` (if present).
5. Click `Generate new token` and then `Legacy tokens`. You may be required to authenticate with GitHub during this step.
6. Select the following scopes: `read:packages` and `workflow`.
7. Click `Generate token`.
8. Copy the generated PAT and save in a safe location.
4. Run the `configure` CLI command:
- Select the `TERMINAL` tab from within the codespace terminal window.
- Run the following command: `gh valet configure`.
- Using the down arrow key to highlight `Jenkins`, press the spacebar to select, and then hit enter to continue.
- At the prompt enter your GitHub Username and press enter.
- At the GitHub Container Registry prompt enter the GitHub PAT generated in step 3 and press enter.
- At the GitHub PAT prompt enter the GitHub PAT generated in step 3 and press enter.
- At the GitHub url prompt enter the GitHub instance url or hit enter to accept the default value (`https://github.com`).
- At the Jenkins token prompt enter the Jenkins access token from step 2 and press enter.
- At the Jenkins url prompt enter `http://localhost:8080/` and press enter.
- At the Personal access token to fetch source code in GitHub prompt hit enter to accept the default value.
![img](https://user-images.githubusercontent.com/19557880/184041328-ce54ea22-b0cd-4c84-b02c-10ad7b09ad89.png)
## Verify your environment
To verify our environment is configured correctly, we are going to run the `update` CLI command. The `update` CLI command will download the latest version of Valet to your codespace.
1. In the codespace terminal run the following command:
```bash
gh valet update
```
2. You should see a confirmation that you were logged into the GitHub Container Registry and Valet was updated to the latest version.
```bash
Login Succeeded
latest: Pulling from valet-customers/valet-cli
Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb
Status: Image is up to date for ghcr.io/valet-customers/valet-cli:latest
ghcr.io/valet-customers/valet-cli:latest
```
### Next Lab
[Perform an audit of a Jenkins server](2-audit.md#perform-an-audit-of-a-jenkins-server)
jenkins/2-audit.md
+111
View File
@@ -0,0 +1,111 @@
# Perform an audit of a Jenkins server
In this lab, you will use the `audit` command to get a high-level view of all pipelines in a Jenkins server.
This `audit` command operates by fetching all of the pipelines defined in a Jenkins server, converting each to their equivalent GitHub Actions workflow, and writing a report that summarizes how complete and complex of a migration is possible with Valet.
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
## Perform an audit
We will be performing an audit against a preconfigured Jenkins server. We will need to answer the following questions before running this command:
1. Do we want to audit the entire Jenkins instance or just a single folder?
- In this example we will be auditing the entire Jenkins instance, but in the future if you wanted to configure a specific folder to be audited add the `-f <folder_path>` flag to the audit command.
2. Where do we want to store the result?
- __./tmp/audit__. This can be any path within the working directory that Valet commands are executed from.
### Steps
1. Navigate to the codespace terminal.
2. Run the following command from the root directory:
```bash
gh valet audit jenkins --output-dir tmp/audit
```
3. The command will list all the files written to disk in green when the command succeeds.
![img](https://user-images.githubusercontent.com/19557880/184682347-b19760fa-36a6-423e-a445-bb30eda5ac59.png)
## Inspect the output files
The audit summary, logs, config files, jenkinsfiles, and transformed workflows will be located within the `tmp/audit` folder.
1. Find the `audit_summary.md` file in the file explorer.
2. Right-click the `audit_summary.md` file and select `Open Preview`.
3. This file contains details that summarizes what percentage of your pipelines were converted automatically.
### Review the audit summary
#### Pipelines
The pipeline summary section contains high level statistics regarding the conversion rate done by Valet:
![img](https://user-images.githubusercontent.com/19557880/184683664-81985baf-5c03-4765-a067-f4023416e3ea.png)
Here are some key terms in the “Pipelines” section in the above example:
- __Successful__ pipelines had 100% of the pipeline constructs and individual items converted automatically to their GitHub Actions equivalent.
- __Partially successful__ pipelines had all of the pipeline constructs converted, however, there were some individual items (e.g. build tasks or build triggers) that were not converted automatically to their GitHub Actions equivalent.
- __Unsupported__ pipelines are definition types that are not supported by Valet. The following Jenkins pipeline types are supported:
- Flow Definition
- Project (declarative Jenkinsfile pipelines)
- Multibranch Project
- __Failed pipelines__ encountered a fatal error when being converted. This can occur for one of three reasons:
- The pipeline was misconfigured and not valid in Jenkins.
- Valet encountered an internal error when converting it.
- There was an unsuccessful network response, often due to invalid credentials, that caused the pipeline to be inaccessible.
The “Job types” section will summarize which types of pipelines are being used and which are supported or unsupported by Valet.
#### Build steps
The build steps summary section presents an overview of the individual build steps that are used across all pipelines and how many were automatically converted by Valet.
![img](https://user-images.githubusercontent.com/19557880/184684062-69ab0bde-5e32-45f8-a7dd-ed4655872975.png)
Here are some key terms in the “Build steps” section in the above example:
- A __known__ build step is a step that was automatically converted to an equivalent action.
- An __unknown__ build step is a step that was not automatically converted to an equivalent action.
- An __unsupported__ build step is a step that is either:
- A step that is fundamentally not supported by GitHub Actions.
- A step that is configured in a way that is incompatible with GitHub Actions.
- An __action__ is a list of the actions that were used in the converted workflows. This is important for the following scenarios:
- Gathering the list of actions to sync to your appliance if you use GitHub Enterprise Server.
- Defining an organization-level allowlist of actions that can be used. This list of actions is a comprehensive list of which actions their security and/or compliance teams will need to review.
There is an equivalent breakdown of build triggers, environment variables, and other uncategorized items displayed in the audit summary file.
#### Manual Tasks
The manual tasks summary section presents an overview of the manual tasks that you will need to perform that Valet is not able to complete automatically.
![img](https://user-images.githubusercontent.com/19557880/184684249-9accfd94-c2df-4891-af56-dcff66beb557.png)
Here are some key terms in the “Manual tasks” section in the above example:
- A __secret__ refers to a repository or organization level secret that is used by the converted pipelines. These secrets will need to be created manually in Actions in order for these pipelines to function properly.
- A __self-hosted runner__ refers to a label of a runner that is referenced by a converted pipeline that is not a GitHub-hosted runner. You will need to manually define these runners in order for these pipelines to function properly.
#### Files
The final section of the audit report provides a manifest of all of the files that are written to disk during the audit. These files include:
![img](https://user-images.githubusercontent.com/19557880/184684416-b3db774e-4ab8-46e0-91ad-e503632df5cb.png)
Each pipeline will have a variety of files written that include:
- The original pipeline as it was defined in Jenkins.
- Any network responses used to convert a pipeline.
- The converted workflow.
- Stack traces that can used to troubleshoot a failed pipeline conversion
## Next lab
[Perform a dry-run of a Jenkins pipeline](3-dry-run.md)
jenkins/3-dry-run.md
+144
View File
@@ -0,0 +1,144 @@
# Perform a dry-run of a Jenkins pipeline
In this lab you will use the `dry-run` command to convert a Jenkins pipeline to its equivalent GitHub Actions workflow.
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [audit lab](./2-audit.md).
## Perform a dry-run
We will be performing a dry-run against a pipeline in the preconfigured Jenkins server. We will need to answer the following questions before running this command:
1. What is the name of the pipeline we want to convert?
- __test_pipeline__
2. What is the URL of the pipeline we want to convert?
- __<http://localhost:8080/job/test_pipeline>__
3. Where do we want to store the result?
- __./tmp/dry-run-lab__. This can be any path within the working directory that Valet commands are executed from.
### Steps
1. Navigate to the codespace terminal
2. Run the following command from the root directory:
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run
```
3. The command will list all the files written to disk when the command succeeds.
![img](https://user-images.githubusercontent.com/19557880/184935603-5c2d4dfe-66ef-4cb1-9398-e96954ca72e3.png)
4. View the converted workflow:
- Find `./tmp/dry-run` in the file explorer pane in codespaces.
- Click `test_pipeline.yml` to open
## Inspect the output files
The files generated from the `dry-run` command represent the equivalent Actions workflow for the given Jenkins pipeline. The Jenkins pipeline and converted workflow can be seen below:
<details>
<summary><em>Jenkins Pipeline 👇</em></summary>
```groovy
pipeline {
agent {
label 'TeamARunner'
}
environment {
DISABLE_AUTH = 'true'
DB_ENGINE = 'sqlite'
}
stages {
stage('build') {
steps {
echo "Database engine is ${DB_ENGINE}"
sleep 80
echo "DISABLE_AUTH is ${DISABLE_AUTH}"
}
}
stage('test') {
steps{
junit '**/target/*.xml'
}
}
}
}
```
</details>
<details>
<summary><em>Converted workflow 👇</em></summary>
```yaml
name: test_pipeline
on:
push:
paths: "*"
schedule:
- cron: 0-29/10 * * * *
env:
DISABLE_AUTH: 'true'
DB_ENGINE: sqlite
jobs:
build:
runs-on:
- self-hosted
- TeamARunner
steps:
- name: checkout
uses: actions/checkout@v2
- name: echo message
run: echo "Database engine is ${{ env.DB_ENGINE }}"
# # This item has no matching transformer
# - sleep:
# - key: time
# value:
# isLiteral: true
# value: 80
- name: echo message
run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
test:
runs-on:
- self-hosted
- TeamARunner
needs: build
steps:
- name: checkout
uses: actions/checkout@v2
- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1.7
if: always()
with:
files: "**/target/*.xml"
```
</details>
These 2 pipelines function equivantly despite using different syntax. In this case, the pipeline conversion was “partially successful” (i.e. there were item(s) not automatically converted) and the unconverted item was placed as comment in the location the Jenkins pipeline used it. For example:
```diff
- sleep 80
+ # # This item has no matching transformer
+ # - sleep:
+ # - key: time
+ # value:
+ # isLiteral: true
+ # value: 80
```
In the next lab, we'll learn how to override Valet's default behavior and customize the converted workflow that is generate.
Try running the `dry-run` command for different pipelines in the Jenkins server. As a hint, you just have to change the `--source-url` CLI option.
## Next lab
[Use custom transformers to customize Valet's behavior](4-custom-transformers.md)
jenkins/4-custom-transformers.md
+293
View File
@@ -0,0 +1,293 @@
# Use custom transformers to customize Valet's behavior
In this lab we will build upon the `dry-run` command to override Valet's default behavior and customize the converted workflow using "custom transformers". Custom transformers can be used to:
1. Convert items that are not automatically converted.
2. Convert items that were automatically converted using different actions.
3. Convert environment variable values differently.
4. Convert references to runners to use a different runner name in Actions.
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [dry-run lab](./3-dry-run.md).
## Perform a dry-run
We will be performing a `dry-run` command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run
```
The converted workflow that is generated by the above command can be seen below
<details>
<summary><em>Converted workflow 👇</em></summary>
```yaml
name: test_pipeline
on:
push:
paths: "*"
schedule:
- cron: 0-29/10 * * * *
env:
DISABLE_AUTH: 'true'
DB_ENGINE: sqlite
jobs:
build:
runs-on:
- self-hosted
- TeamARunner
steps:
- name: checkout
uses: actions/checkout@v2
- name: echo message
run: echo "Database engine is ${{ env.DB_ENGINE }}"
# # This item has no matching transformer
# - sleep:
# - key: time
# value:
# isLiteral: true
# value: 80
- name: echo message
run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
test:
runs-on:
- self-hosted
- TeamARunner
needs: build
steps:
- name: checkout
uses: actions/checkout@v2
- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1.7
if: always()
with:
files: "**/target/*.xml"
```
</details>
_Note_: You can refer to the previous [lab](./3-dry-run.md) to learn about the fundamentals of the `dry-run` command.
## Custom transformers for an unknown step
The converted workflow above contains a `sleep` step was not automatically converted. We will need to answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __sleep__. The identifier will be the key of a key value pair within the step of a Jenkinsfile.
2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the following bash script will provide similar functionality:
```yaml
- name: Sleep for 80 seconds
run: sleep 80s
shell: bash
```
Now we can begin to write the custom transformer. Customer transformers use a DSL built on top of Ruby and should be defined in a file with the `.rb` file extension. You can create this file by running the following command in your codespace terminal:
```bash
code transformers.rb
```
Next, we will define a `transform` method for the `sleep` identifier by adding the following code to `transformers.rb`:
```ruby
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
```
This method can use any valid ruby syntax and should return a `Hash` that represents the YAML that should be generated for a given step. Valet will use this method to convert a step with the provided identifier and will use the `item` parameter for the original values configured in Jenkins.
Now, we can perform another `dry-run` command and use the `--custom-transformers` CLI option to provide this custom transformer. Run the following command within your codespace terminal:
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o tmp/jenkins/dry-run --custom-transformers transformers.rb
```
Open the workflow that is generated and inspect the contents. Now, the `sleep` step is converted and uses the customized behavior!
```diff
- # # This item has no matching transformer
- # - sleep:
- # - key: time
- # value:
- # isLiteral: true
- # value: 80
+ - name: Sleep for 80 seconds
+ run: sleep 80s
+ shell: bash
```
## Custom transformers for a known step
We can also override Valet's default behavior. In this scenario, we may not desire to use the third-party action for publishing junit test results that is used by default. Again, we will need to answer the following questions before writing a custom transformer:
1. What is the "identifier" of the step to customize?
- __junit__
2. What is the desired Actions syntax to use instead?
- After some research, we have determined that the uploading test results as an artifact will be suitable:
```yaml
- uses: actions/upload-artifact@v3
with:
name: junit-artifact
path: path/to/artifact/world.txt
```
We will build this custom transformer similar to the previous custom transformer, however, we first need to inspect the `item` keyword to programmatically use the file path to junit's test results in the `actions/upload-artifact@v3` step.
To do this, we will print `item` to the console. You can achieve this by adding the following custom transformer to `transformers.rb`:
```ruby
transform "junit" do |item|
puts "This is the item: #{item}"
end
```
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. The output of the `dry-run` command should look similar to this:
![img](https://user-images.githubusercontent.com/19557880/186782050-65ece0c4-52a3-4f88-818f-0f860c50c2b7.png)
Now that we know the data structure of `item` we can access the file path programmatically by editing the custom transformer to following:
```ruby
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
{
"uses" => "actions/upload-artifact@v3",
"with" => {
"name" => "junit-artifact",
"path" => file_path
}
}
end
```
_Note_: `transformers.rb` should contain a `transform` method for both `sleep` and `junit`.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `EnricoMi/publish-unit-test-result-action@v1.7` action will have been replaced with the customized steps.
```diff
- - name: Publish test results
- uses: EnricoMi/publish-unit-test-result-action@v1.7
- if: always()
- with:
- files: "**/target/*.xml"
+ - uses: actions/upload-artifact@v3
+ with:
+ name: junit-artifact
+ path: path/to/artifact/world.txt
```
## Custom transformers for environment variables
We can also use custom transformers to edit the values of environment variables in converted workflows. In our example, we will be updating the `DB_ENGINE` environment variable to be `mongodb` instead of `sqlite`.
To do this, add the following code to the `transformers.rb` file.
```ruby
env "DB_ENGINE", "mongodb"
```
In this example, the first parameter to the `env` method is the environment variable name and the second is the updated value.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `DB_ENGINE` environment variable will be set to `mongodb`:
```diff
env:
DISABLE_AUTH: 'true'
- DB_ENGINE: sqlite
+ DB_ENGINE: mongodb
```
## Custom transformers for runners
Finally, we can use custom transformers to dictate which runners converted workflows should use. To do this we will need to answer the following questions:
1. What is label of the runner in Jenkins to update?
- __TeamARunner__
2. What is the label of the runner in Actions to use instead?
- __ubuntu-latest__
With these questions answered, we can add the following code to the `transformers.rb` file:
```ruby
runner "TeamARunner", "ubuntu-latest"
```
In this example, the first parameter to the `runner` method is the Jenkins label and the second is the Actions runner label.
Now, we can perform another `dry-run` command with the `--custom-transformers` CLI option. When you open the converted workflow the `runs-on` statement will use the customized runner label:
```diff
runs-on:
- - self-hosted
- - TeamARunner
+ - ubuntu-latest
```
At this point of the lab the file contents of `transformers.rb` should match this:
<details>
<summary><em>Custom transformers 👇</em></summary>
```ruby
runner "TeamARunner", "ubuntu-latest"
env "DB_ENGINE", "mongodb"
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
[
{
"uses": "actions/upload-artifact@v3",
"with": {
"name": "my-artifact",
"path": file_path
}
}
]
end
```
</details>
Thats it! Congratulations you have overridden Valet's default behavior by customizing the conversion of:
- Unknown steps
- Known steps
- Environment variables
- Runners
## Next lab
[Perform a production migration of a Jenkins pipeline](5-migrate.md)
jenkins/5-migrate.md
+55
View File
@@ -0,0 +1,55 @@
# Perform a production migration of a Jenkins pipeline
In this lab, you will use the `migrate` command to convert a Jenkins pipeline and open a pull request with the equivalent Actions workflow.
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
3. Completed the [dry-run lab](./3-dry-run.md).
4. Completed the [custom transformers lab](./4-custom-transformers.md).
## Performing a migration
We need to answer the following questions before running a `migrate` command:
1. What is the source URL of the pipeline we want to convert?
- __<http://localhost:8080/monas_dev_work/job/monas_freestyle>__
2. Where do we want to store the result?
- __./tmp/migrate__
3. What is the URL for the GitHub repository to add the workflow to?
- __this repository__. The URL should should follow the pattern <https://github.com/:owner/:repo> with `:owner` and `:repo` replaced with your values.
### Steps
1. Run the following `migrate` command in the codespace terminal:
```bash
gh valet migrate jenkins --target-urlhttps://github.com/:owner/:repo --output-dir ./tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle
```
2. The command will write the URL to the pull request that was created when the command succeeds.
![img](https://user-images.githubusercontent.com/19557880/185509412-ab64d92d-2a56-4d5a-bbb4-35a41a2ca48c.png)
3. Open the generated pull request in a new browser tab.
### Inspect the pull request
The first thing we should notice about the PR is that there is a list of manual steps for us to complete:
![img](https://user-images.githubusercontent.com/19557880/186784161-b7882ac4-ac99-4462-b69f-f49b9202527b.png)
Next, let's review the workflow we are adding by clicking on `Files changed` tab. This is where you would double check everything looks good. If it didn't you could push commits with the required changes, prior to merging.
Next, you can inspect the "Files changed" in this PR and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this PR.
Finally, you can merge the PR once your review has completed. We can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub.
![img](https://user-images.githubusercontent.com/19557880/185509704-90243ec5-e77f-4baf-a9b2-d9a4d9fda199.png)
At this point, the migration has completed and you have successfully migrated a Jenkins pipeline to Actions!
### Next lab
[Forecast potential build runner usage](6-forecast.md)
jenkins/6-forecast.md
+175
View File
@@ -0,0 +1,175 @@
# Forecast the runner usage of a Jenkins instance
In this lab you will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from completed pipeline runs in your Jenkins server.
## Prerequisites
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](./1-configure-lab.md#configuring-credentials).
## Perform a forecast
We will need to answer the following questions before running the `forecast` command:
1. Do we want to forecast the entire Jenkins server or a single folder?
- We will be forecasting the entire Jenkins server but we could optionally limit this by using the `-f <folder_path>` CLI option.
2. What is the date we want to start forecasting from?
- __2022-08-02__. This date is needed as it is prior to when the data was seeded in Jenkins for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
3. Where do we want to store the results?
- `./tmp/forecast_reports`
### Steps
1. Navigate to the codespace terminal
2. Run the following command from the root directory:
```bash
gh valet forecast jenkins --output-dir ./tmp/forecast_reports --start-date 2022-08-02
```
3. The command will list all the files written to disk when the command succeeds.
![img](https://user-images.githubusercontent.com/19557880/186223037-18556c82-5a29-4434-bc17-4b906d704967.png)
## Review the forecast report
The forecast report, logs, and completed job data will be located within the `tmp/forecast_reports` folder.
1. Find the `forecast_report.md` file in the file explorer.
2. Right-click the `forecast_report.md` file and select `Open Preview`.
3. This file contains metrics used to forecast potential GitHub Actions usage.
<!-- <details>
<summary>example forecast_report.md</summary>
# Forecast report for [Jenkins](http://localhost:8080)
- Valet version: __0.1.0.13448(2222622ecf83e372990e1657b1293e1ac552be21)__
- Performed at: __8/23/22 at 17:11__
- Date range: __8/2/22 - 8/23/22__
## Total
- Job count: __73__
- Pipeline count: __6__
- Execution time
- Total: __27,057 minutes__
- Median: __2 minutes__
- P90: __19 minutes__
- Min: __0 minutes__
- Max: __15,625 minutes__
- Queue time
- Median: __0 minutes__
- P90: __0 minutes__
- Min: __0 minutes__
- Max: __0 minutes__
- Concurrent jobs
- Median: __1__
- P90: __3__
- Min: __0__
- Max: __29__
---
## N/A
- Job count: __73__
- Pipeline count: __6__
- Execution time
- Total: __27,057 minutes__
- Median: __2 minutes__
- P90: __19 minutes__
- Min: __0 minutes__
- Max: __15,625 minutes__
- Queue time
- Median: __0 minutes__
- P90: __0 minutes__
- Min: __0 minutes__
- Max: __0 minutes__
- Concurrent jobs
- Median: __1__
- P90: __3__
- Min: __0__
- Max: __29__
> Note: Concurrent jobs are calculated by using a sliding window of 1m 0s.
</details> -->
### Total
The "Total" section of the forecast report contains high level statistics related to all the jobs completed after the `--start-date` CLI option:
```md
- Job count: __73__
- Pipeline count: __6__
- Execution time
- Total: __27,057 minutes__
- Median: __2 minutes__
- P90: __19 minutes__
- Min: __0 minutes__
- Max: __15,625 minutes__
- Queue time
- Median: __0 minutes__
- P90: __0 minutes__
- Min: __0 minutes__
- Max: __0 minutes__
- Concurrent jobs
- Median: __1__
- P90: __3__
- Min: __0__
- Max: __29__
```
Here are some key terms of items defined in the forecast report:
- The `job count` is the total number of completed jobs.
- The `pipeline count` is the number of unique pipelines used.
- `Execution time` describes the amount of time a runner spent on a job. This metric can be used to help plan for the cost of GitHub hosted runners.
- This metric is correlated to how much you should expect to spend in GitHub Actions. This will vary depending on the hardware used for these minutes and the [Actions pricing calculator](https://github.com/pricing/calculator) should be used to estimate a dollar amount.
- `Queue time` metrics describe the amount of time a job spent waiting for a runner to be available to execute it.
- `Concurrent jobs` metrics describe the amount of jobs running at any given time. This metric can be used to define the number of runners a customer should configure.
Additionally, these metrics are defined for each queue of runners defined in Jenkins. This is especially useful if there are a mix of hosted/self-hosted runners or high/low spec machines to see metrics specific to different types of runners.
## Forecasting multiple providers
We can examine the available options for the `forecast` command by running `gh valet forecast --help`. When you do this you will see the `--source-file-path` option:
![img](https://user-images.githubusercontent.com/19557880/186263140-f02c6cab-7979-417c-bdfe-b9590e9c5597.png)
The `--source-file-path` CLI option can be used to combine data from multiple reports into a single report. This becomes useful if you use multiple CI/CD providers and wanted to get a holistic view of the runner usage. This works by using the `.json` files generated by `forecast` commands as space-delimited values for the `--source-file-path` CLI option. Optionally, this value could be a glob pattern to dynamically specify the list of files (e.g. `**/*.json`).
Run the following command from within the codespace terminal:
```bash
gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast
```
You can now inspect the output of the command to see a forecast report using all of the files matching the `tmp/**/jobs/*.json` pattern.
## Next steps
This concludes all labs for migrating Jenkins pipelines to Actions with Valet!
jenkins/readme.md
+70 -58
View File
@@ -1,88 +1,100 @@
# Valet labs for Jenkins
# Jenkins to Actions migrations powered by Valet
This lab bootstraps a Valet environment using GitHub Codespaces and enables you to spin up a Jenkins instance against which to run the Valet CI/CD migration tool.
The instructions below will guide you through configuring a GitHub Codespace environment that will be used in subsequent labs that demonstrate how to use Valet to migrate Jenkins pipelines to GitHub Actions.
- [Use this repo as a template](#repo-template)
- [Use Valet with a codespace](#use-valet-with-a-codespace)
- [Bootstrap Jenkins](#bootstrap-jenkins)
- [Labs for Jenkins](#labs-for-jenkins)
- [Troubleshoot GH Valet extension](#troubleshoot-gh-valet-extension)
- [Troubleshooting Jenkins](#troubleshooting-jenkins)
These steps **must** be completed prior to starting other labs.
## Repo template
## Create your own repository for these labs
1. Verify you are in your own Repository created from the landing page [Valet Labs](https://github.com/valet-customers/labs).
1. Ensure that you have created a repository using the [valet-customers/labs](https://github.com/valet-customers/labs) as a template.
## Use Valet with a codespace
## Configure your Codespace
1. Start the codespace
- Click the `Code` with button down arrow above repository on the repository's landing page.
- Click the `Codespaces` tab
- Click `Create codespaces on main` to create the codespace. If you are in another branch then the `main` branch, the codespace will button will have the current branch specified.
- Wait a couple minutes, then verify that the codespace starts up. Once it is fully booted up, the terminal should be present.
2. Verify Valet CLI is installed and working. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet)
- Verify Valet CLI is installed and working
- Run the following command in the Visual Studio Code terminal:
1. Start a new Codespace.
```
gh valet version
```
- Click the `Code` with button down arrow above repository on the repository's landing page.
- Click the `Codespaces` tab
- Click `Create codespaces on main` to create the codespace.
- After the Codespace has initialized there will be a terminal present.
- Verify the output looks like the below image. Note the valet version will be different than below as the latest version gets pulled down.
- If `gh valet version` did not produce a similar image with a version please follow these instructions [Troubleshoot GH Valet extension](#troubleshoot-gh-valet-extension)
2. Verify the Valet CLI is installed and working. More information on the Valet extension for the official GitHub CLI can be found [here](https://github.com/github/gh-valet).
<img width="836" alt="Screen Shot 2022-08-10 at 1 45 20 PM" src="https://user-images.githubusercontent.com/19557880/186771327-631e8839-3614-4ab7-8108-818b5a0c6e93.png">
- Run the following command in the codespace's terminal:
## Bootstrap Jenkins
```bash
gh valet version
```
1. Run the Jenkins setup script. This script will setup Jenkins and ensure it is ready to use. In general, this script should be run first if you are starting a new codespace or restarting an existing one.
- Verify the output is similar to the image below. The version information may differ than what is shown below.
- If `gh valet version` did not produce similar output then please follow the troubleshooting [guide](#troubleshoot-the-valet-cli).
- Navigate to the terminal within your Codespace.
- Run the following command to kick off the creation of your Jenkins instance:
![img](https://user-images.githubusercontent.com/19557880/186771327-631e8839-3614-4ab7-8108-818b5a0c6e93.png)
```
## Bootstrap a Jenkins server
1. Execute the Jenkins setup script that will start a container with a Jenkins server running inside of it. This script should be executed when starting a new Codespace or restarting an existing one.
- Run the following command from the codespace's terminal to start a Jenkins server:
```bash
./jenkins/bootstrap/setup.sh
```
- After a couple seconds, a pop-up box should appear with a link to the forwarded URL for your Jenkins instance.
- You can also access the forwarded URL by going to the `Ports` tab in your terminal. Right click on the URL listed under the `Local Address` and clicking the `Open in Browser` tab.
- Once you have navigated to the url, the following default credentials have been assigned:
- After some time, a pop-up box should appear with a link to the URL for your Jenkins server.
- You can also access the URL by going to the `Ports` tab in your terminal. Right click on the URL listed under the `Local Address` and click the `Open in Browser` tab.
- username: `admin`
- password: `password`
2. Open the Jenkins server in your browser and use the following credentials to authenticate:
2. Click the `Sign in` button and you should now see your new Jenkins instance with a few pre-populated pipelines.
- Username: `admin`
- Password: `password`
- Once authenticated, you should see a Jenkins server with a few predefined pipelines.
## Labs for Jenkins
Perform the following labs to test-drive Valet
- [Configure Valet to work with Jenkins](valet-configure-lab.md#configure-valet-to-work-with-jenkins)
- [Audit Jenkins using the Valet audit command](valet-audit-lab.md#audit-jenkins-pipelines-using-the-valet-audit-command)
- [Dry-run the migration of an Jenkins pipeline to GitHub Actions](valet-dry-run-lab.md#dry-run-the-migration-of-a-jenkins-pipeline-to-github-actions)
- [Using Custom Transformers in a dry-run](valet-custom-transformers-lab.md#using-custom-transformers-in-a-dry-run)
- [Migrate an Jenkins Project to GitHub Actions](valet-migrate-lab.md#migrate-a-jenkins-project-to-github-actions)
- [Forecast the usage of a Jenkins namespace](valet-forecast-lab.md#forecast-the-runner-usage-of-a-jenkins-instance)
1. [Configure credentials for Valet](1-configure.md)
2. [Perform an audit of a Jenkins server](2-audit.md)
3. [Perform a dry-run of a Jenkins pipeline](3-dry-run.md)
4. [Use custom transformers to customize Valet's behavior](4-custom-transformers.md)
5. [Perform a production migration of a Jenkins pipeline](5-migrate.md)
6. [Forecast potential build runner usage](6-forecast.md)
## Troubleshoot GH Valet extension
## Troubleshoot the Valet CLI
Manually Install the GitHub CLI Valet extension. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet)
The CLI extension for Valet can be manually installed by following these steps:
- Verify you are in the codespace terminal
- Run this command to install the GitHub Valet extension
- `gh extension install github/gh-valet`
- Verify the result of the install is: `✓ Installed extension github/gh-valet`
- If you get a similar error to the following, click the link to authorize the token
- Restart Codespace after clicking the link
- Verify Valet CLI is installed and working by running `gh valet version`
- Run this command from within the codespace's terminal:
![linktolcickauth](https://user-images.githubusercontent.com/26442605/169588015-9414404f-82b6-4d0f-89d4-5f0e6941b029.png)
## Troubleshooting Jenkins
```bash
gh extension install github/gh-valet
```
1. Navigate to the Docker tab on your left hand side.
2. Under the `Containers` tab you should see a Docker container `jenkins:valet` listed with a green play button ▶
- If you see the `jenkins:valet` container, but it has a red stopped symbol next to it ▢, right click on the container and click on `start`, the container should begin running again.
- If the container does not start even after trying to manually start it, right click on the `jenkins:valet` container and click the `remove` button. Next continue by following all the #bootstrap-jenkins instructions again.
- Verify the result of the install contains:
<img width="985" alt="Screen Shot 2022-08-09 at 3 06 46 PM" src="https://user-images.githubusercontent.com/19557880/183770210-c0386616-656e-4fe9-9324-b410ad62c406.png">
```bash
✓ Installed extension github/gh-valet
```
- If you get an error similar to the image below, then click the link in the terminal output to authorize the token.
- Restart the codespace after clicking the link.
![img](https://user-images.githubusercontent.com/26442605/169588015-9414404f-82b6-4d0f-89d4-5f0e6941b029.png)
- Verify Valet CLI extension is installed and working by running the following command from the codespace's terminal:
```bash
gh valet version
```
## Troubleshooting the Jenkins server
Follow these steps if the Jenkins server does not start correctly after running the setup script:
1. Navigate to the `Docker` tab on the left side of the codespace.
2. Under the `Containers` tab you should see a docker container named `jenkins:valet` listed with a green play button ▶
- If you see the `jenkins:valet` container, but it has a red stopped symbol next to it ▢, right click on the container and click on `start`, the container should begin running again.
- If the container does not start even after trying to manually start it, right click on the `jenkins:valet` container and click the `remove` button. Then, attempt to start the Jenkins server again by following the steps [here](#bootstrap-a-jenkins-server).
![img](https://user-images.githubusercontent.com/19557880/183770210-c0386616-656e-4fe9-9324-b410ad62c406.png)
jenkins/valet-audit-lab.md
-124
View File
@@ -1,124 +0,0 @@
# Audit Jenkins pipelines using the Valet audit command
In this lab, you will use Valet to `audit` a Jenkins organization. The `audit` command can be used to scan a CI server and output a summary of the current pipelines.
What happens behind the scenes is that Valet will perform a `dry-run` on each of the Jenkins pipelines. Once that is complete, Valet will perform an aggregation of all of the transformed workflows. This aggregate summary can be used as a planning tool and help understand how complete of a migration is possible with Valet.
By the end of this lab you will have performed an audit on the demo Jenkins instance, and have a good understanding of the components that make up an audit.
- [Prerequisites](#prerequisites)
- [Perform an audit](#perform-an-audit)
- [View audit output](#view-audit-output)
- [Review the pipelines](#review-the-pipelines)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed the steps [here](../jenkins/readme.md#valet-labs-for-jenkins) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](../jenkins/valet-configure-lab.md#configure-valet-to-work-with-jenkins) to configure the Valet CLI.
## Perform an audit
We will be performing an audit against a preconfigured Jenkins instance. Before running the command we need to collect some information:
1. Do we want to audit the entire Jenkins instance, or just a single folder? __In this example we will be auditing the entire Jenkins instance, but in the future if you wanted to configure a specific folder to be audited add the `-f <folder_path>` flag to the audit command__
2. Where do we want to store the result? __./tmp/audit. This can be any valid path on the system. In the case of codespaces it is generally best to use `./tmp/SOME_DIRECTORY_HERE` so the files show in explorer__
### Steps
1. Navigate to the codespace terminal.
2. Now, from root directory, run the following Valet audit command:
```
gh valet audit jenkins --output-dir tmp/audit
```
3. Valet will log the output files in green when the audit is successful
### Example
<img src="https://user-images.githubusercontent.com/19557880/184682347-b19760fa-36a6-423e-a445-bb30eda5ac59.png" alt="valet-audit-1"/>
## View audit output
The audit summary, logs, config files, jenkinsfiles, and transformed Actions Workflows should all be located within the `tmp/audit` folder.
1. Under the `audit` folder find the `audit_summary.md`
2. Right-click the `audit_summary.md` file and select `Open Preview`
3. The file contains details about your current pipelines and what can be migrated 100% automatically vs. what will need some manual intervention or aren't supported by GitHub Actions.
4. Review the file, it should look like the image below:
### Example
<img src="https://user-images.githubusercontent.com/19557880/184682836-3b8155ae-f302-491e-8ce6-27cc57f96468.png" alt="valet-audit-2"/>
## Review the pipelines
### Pipelines
The audit summary starts by giving a summary of the types of pipelines that were extracted from Jenkins.
- It shows that there are a total of 7 pipelines extracted.
- 42% pipelines were successful. This means that Valet knew how to map all the constructs of the Jenkins pipeline to a GitHub Actions equivalent. All of the build plugins and triggers that are referenced were all successfully converted into a GitHub Actions equivalent.
- 42% pipelines were partially successful. This means that Valet knew how to map all the constructs of the Jenkins pipeline but there may be a plugin that was referenced that Valet wasn't able to automatically map to a Github Actions equivalent.
- 1% of these pipelines were unsupported. This means that the pipeline type is fundamentally unsupported by Valet. This is most likely a Jenkins scripted pipeline.
- 0% of these fail altogether. If there were any pipelines that would fall under this category, that would mean that those pipelines were misconfigured or there was an issue with Valet.
Under the `Job types` section, we can see that the `audit` command is able to support the conversion of project, freestyle (flow-definition), and multibranch pipelines from Jenkins and convert them to a GitHub Actions workflow. Valet does not support converting [scripted pipelines](https://www.jenkins.io/doc/book/pipeline/syntax/#scripted-pipeline) (e.g. pure Groovy).
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184683664-81985baf-5c03-4765-a067-f4023416e3ea.png" alt="valet-audit-3" height="400"/>
### Build steps
Under the `Build steps` section we can see a breakdown of the build steps that were used in these pipelines.
- <b>Supported:</b> 12/16 discrete build steps are considered known by Valet. When Valet encounters a build step of this type, it knows exactly how to map that into a GitHub Actions equivalent.
- <b>Unknown:</b> 3/16 discrete build steps are considered unknown by Valet. When Valet encounters a build step of this type, it does not yet know to map this automatically to a GitHub Action equivalent.
- <b>Unsupported:</b> 1/16 discrete build steps are considered unsupported by Valet. This could mean one of three things:
1. The way that plugin was configured for a given job is unsupported.
2. The plugin itself is fundamentally not supported in GitHub Actions.
3. It's supported by default in GitHub Actions.
Under the `Actions` section we have the list of the Actions that were used in order to implement the transformation of all of these build steps. Valet is a planning tool that can help in facilitating the migration into GitHub Actions and this list of Actions is a great place to understand what dependencies you would be taking on third-party Actions after this migration.
For example, if you are doing things like setting up the allow list of third-party Actions in a GitHub Enterprise server instance this list of Actions is a fantastic place to begin security reviews and audits of what third-party actions to depend on.
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184684062-69ab0bde-5e32-45f8-a7dd-ed4655872975.png" alt="valet-audit-4"/>
### Trigger, Environment, Other
Similar to `Build steps`, there are `Trigger`, `Environment`, and a catch all `Other` section that breakdown each of their uses across the audited pipelines.
### Example
<img src="https://user-images.githubusercontent.com/19557880/184684174-43caff58-6083-45e1-a36e-6899d99c136b.png" alt="valet-audit-5" height="600"/>
### Manual Tasks
Under the Manual task section you will find a list of all the manual tasks that the pipelines would surface in a migration. Manual tasks are Valet's way of indicating tasks a user needs to do in order for a pipeline to be functional, such as adding `secrets`, or setting up a `self-hosted` runner. We will see how these manual tasks appear on a pull request when we do a migration in a lab later on.
### Example
<img src="https://user-images.githubusercontent.com/19557880/184684249-9accfd94-c2df-4891-af56-dcff66beb557.png" alt="valet-audit-5" height="200"/>
### Files
At the end of the Audit Summary page you will find a list of all of the files that were written to disk. Generally, for any given pipeline, you’ll find 2 or 3 associated files. In these files are the actual converted GitHub Actions workflows.
In addition, you’ll see a file that shows the raw JSON data that we pull from Jenkins as well as any associated Jenkinsfiles for a given job. These files are really useful for engineering teams to help debug any issues and to understand what may have gone on in a transformation.
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184684416-b3db774e-4ab8-46e0-91ad-e503632df5cb.png" alt="valet-audit-6" height="700"/>
### Next Lab
[Dry-run the migration of a Jenkins pipeline to GitHub Actions](valet-dry-run-lab.md)
jenkins/valet-configure-lab.md
-78
View File
@@ -1,78 +0,0 @@
# Configure Valet to work with Jenkins
In this lab, you will use the Valet `configure` command to set up the required information to communicate with the Jenkins and GitHub instances. The `configure` command can be used for all of the supported providers, in this lab we will be focusing on Jenkins.
- [Prerequisites](#prerequisites)
- [Configuring Valet](#configuring-valet)
- [Verify Valet Works](#verify-valet-works)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed the steps [here](../jenkins/readme.md#valet-labs-for-jenkins) to set up your Codespace environment and start a Jenkins server.
## Configuring Valet
1. Login to the Jenkins instance to generate a personal access token:
1. Click the `PORTS` tab in the codespace terminal window.
2. In the `PORTS` tab find the row for port 8080.
3. Hover over the address under the `Local Address` column, and click the globe to "open in browser".
4. Login to the Jenkins server and generate a Jenkins API token.
- Click the `admin` button located within the top right menu.
- Click on the `configure` gear located on the left hand panel.
- Under the `API token` section, click `Add new Token`.
- Add a default name to your token, then click `Generate`.
- Copy the token that was generated and record token for a later step.
![configure-result](https://user-images.githubusercontent.com/19557880/184041667-d06cb7f2-a885-474e-b728-7567314aeaf3.png)
2. Create a GitHub personal access token (PAT).
- Navigate to your GitHub `Settings` - click your profile photo and then click `Settings`.
- Go to `Developer Settings`
- Go to `Personal Access Tokens` -> `Legacy tokens (if present)`
- Click `Generate new token` -> `Legacy tokens (if present)`. If required, provide your password.
- Select at least these scopes: `read packages` and `workflow`. Optionally, provide a text in the **Note** field and change the expiration.
- Click `Generate token`
- Copy the token somewhere safe and temporary.
3. Run Valet configure commands
- In the codespace terminal window click back to the `TERMINAL` tab.
- Within the terminal, ensure you are in the root directory.
- Run `gh valet configure`.
- Use the down arrow key to highlight `Jenkins`, press the spacebar to select, then hit enter to accept.
- At the prompt enter your GitHub Username and press enter.
- At the GitHub Container Registry prompt enter the GitHub PAT generated in step 2 and press enter
- At the GitHub PAT prompt enter the GitHub PAT generated in step 2 and press enter.
- At the GitHub url prompt enter the GitHub instance url or hit enter to accept the default, if you are using github.com then the default is the right choice.
- At the Jenkins token prompt enter the Jenkins access token from step 1 and press enter.
- At the Jenkins url prompt enter `http://localhost:8080/` and press enter.
- At the Personal access token to fetch source code in GitHub prompt, if any of your Jenkins pipelines have source code in a GitHub repository enter the GitHub PAT that would have access to these files.
4. If all went well you should see a similar output in your terminal:
![configure-result](https://user-images.githubusercontent.com/19557880/184041328-ce54ea22-b0cd-4c84-b02c-10ad7b09ad89.png)
## Verify Valet Works
To verify Valet works we are going to run a `update` and `dry-run` command. We will go further into details about the `dry-run` command in a later lab, but for now we want to get the latest version of Valet and confirm that Valet can perform a dry-run with no errors.
1. In the codespace terminal update Valet by running `gh valet update`
2. In the terminal you should see a confirmation that it logged into the GitHub Container Registry and pulled the latest version.
```
Login Succeeded
latest: Pulling from valet-customers/valet-cli
Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb
Status: Image is up to date for ghcr.io/valet-customers/valet-cli:latest
ghcr.io/valet-customers/valet-cli:latest
```
3. Next, lets run the dry-run command in the codespaces terminal, to verify we can talk to Jenkins
```
gh valet dry-run jenkins --source-url https://localhost:8080/job/test_pipeline/ --output-dir ./tmp/dry-run-lab
```
4. In the terminal you should see the command was successful, if not it is a good time to practice the configure command again and make sure the access tokens values are correct and were generated with the correct permissions.
![configure-dry-run](https://user-images.githubusercontent.com/19557880/184255620-8e9b120e-5de0-41df-9cb6-c52028de3b0f.png)
### Next Lab
[Audit Jenkins using the Valet audit command](valet-audit-lab.md#audit-jenkins-pipelines-using-the-valet-audit-command)
jenkins/valet-custom-transformers-lab.md
-401
View File
@@ -1,401 +0,0 @@
# Using Custom Transformers in a `dry-run`
In this lab we want to do a `dry-run` of the `test_pipeline` pipeline, however, after a closer inspection, we discovered that we will need to customize how Valet transforms:
1. Steps that are unsupported by Valet, but are essential to our devops process.
2. Steps in which the GitHub action chosen by Valet does not meet our current needs.
3. All pipelines that have the environment variable `DB_ENGINE` must be changed to a different value: `mongodb`.
4. The self-hosted runners before used the `TeamARunner` label in Jenkins but will run on the `ubuntu-latest` runner in Actions.
These customizations will be present in many of our company's pipelines and an automated way to apply these changes would be ideal. In this lab, we will use the `--custom-transformers` flag to change the behavior of Valet using its DSL built on top of the Ruby language.
- [Prerequisites](#prerequisites)
- [Perform a dry-run](#perform-a-dry-run)
- [Custom transformers for an unknown step](#custom-transformers-for-an-unknown-step)
- [Custom transformers for an known step](#custom-transformers-for-a-known-step)
- [Custom transformers for environment variables](#custom-transformers-for-environment-variables)
- [Custom transformers for runners](#custom-transformers-for-runners)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed the steps [here](../jenkins/readme.md#valet-labs-for-jenkins) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](../jenkins/valet-configure-lab.md#configure-valet-to-work-with-jenkins) to configure the Valet CLI.
3. Completed the [dry-run lab](../jenkins/valet-dry-run-lab.md#dry-run-the-migration-of-a-jenkins-pipeline-to-github-actions).
## Perform a dry-run
- Let’s run the `dry-run` command to see what information we can get from the generated action yaml.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run
```
- Open the resulting GitHub Actions workflow by navigating to `tmp/valet/test_pipeline.yml` from the explorer
__Click to Expand__
<details>
<summary><em>Actions Workflow</em></summary>
```yaml
name: test_pipeline
on:
push:
paths: "*"
schedule:
- cron: 0-29/10 * * * *
env:
DISABLE_AUTH: 'true'
DB_ENGINE: sqlite
jobs:
check_env_variables:
runs-on:
- self-hosted
- TeamARunner
steps:
- name: checkout
uses: actions/checkout@v2
- name: echo message
run: echo "Database engine is ${{ env.DB_ENGINE }}"
# # This item has no matching transformer
# - sleep:
# - key: time
# value:
# isLiteral: true
# value: 80
- name: echo message
run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
build:
runs-on:
- self-hosted
- TeamARunner
needs: check_env_variables
steps:
- name: checkout
uses: actions/checkout@v2
- name: Set up JDK 1.11
uses: actions/setup-java@v1
with:
java-version: '1.11'
settings-path: "${{ github.workspace }}"
- name: sh
shell: bash
run: mvn clean package
test:
runs-on:
- self-hosted
- TeamARunner
needs: build
steps:
- name: checkout
uses: actions/checkout@v2
- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1.7
if: always()
with:
files: "**/target/*.xml"
```
</details>
Go back to the previous lab [dry-run Jenkins pipeline](../jenkins/valet-dry-run-lab.md#dry-run-the-migration-of-a-jenkins-pipeline-to-github-actions) if you need a review of the details of the dry-run.
## Custom transformers for an unknown step
We can see in the center of the transformed workflow that the `sleep` step was not transformed. In order for us to write a custom transformer for this we need to know the identifier. In general, the identifier will be the key of a key/value pair within the step of a Jenkinsfile, which in this case is `sleep`. This is how our custom transformer will target the correct step.
After some research we have decided that a simple bash script will meet our needs:
```yaml
- name: Sleep for 80 seconds
run: sleep 80s
shell: bash
```
Now that we know the final yaml needed for the transformer, we can start to write the ruby file.The custom transformers file can have any name, but it is recommended that you use an `.rb` extension so the codespaces editor knows it is a ruby file and can provide syntax highlighting.
- Create a new file where you will put the custom transformers logic. It can have any file name, but it is recommended that you use an `.rb` extension so the codespaces editor knows it is a ruby file and can provide syntax highlighting. For this example we will call it `transformers.rb`.
In the custom transformers file we will add a `transform` method. This is a special method that Valet exposes, that takes the identifier we determined earlier and returns a ruby hash of the final YAML for the pipeline.
The ruby hash can be thought of as the JSON representation of the YAML we want. Valet will call that method when it encounters the identifier and pass in an `item`. The `item` is the key of that key/value pair defined for that step in Jenkins. In this case the item is the settings of the sleep command.
```ruby
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
```
Let’s run the `dry-run` command again with `--custom-transformer transformers.rb` to see if our custom transformer worked.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run --custom-transformers transformers.rb
```
When you open the file you should see the following changes in the GitHub Actions Workflow, note how the sleep timer step is no longer commented out!
```diff
- # # This item has no matching transformer
- # - sleep:
- # - key: time
- # value:
- # isLiteral: true
- # value: 80
+ - name: Sleep for 80 seconds
+ run: sleep 80s
+ shell: bash
```
## Custom transformers for a known step
Next lets address the third-party GitHub action `EnricoMi/publish-unit-test-result-action@v1.7` that Valet chose for the `junit` step. This action does not meet our needs because our CTO has dictated that our pipelines take no dependencies on third-party actions. We can customize the action that we would like Valet to use instead.
After some research we find that the following action will upload an artifact with the junit test results without depending on a third party action.
```yaml
- uses: actions/upload-artifact@v3
with:
name: junit-artifact
path: path/to/artifact/world.txt
```
We will build this custom transformer similar to how we built the previous custom transformer. This time, we need to add some additional logic. We would like to programmatically assign the file path to junit's test results to our upload action.
The challenge is, that we are unsure what `item` represents (what the incoming JSON object looks like from Jenkins). In order to troubleshoot this, you could use some basic ruby to print `item` to the terminal. You can achieve this by adding the following line in the transform method: `puts "This is the item: #{item}"`.
You are able to add multiple custom transformers to the same file, so you can add this one right below the previous custom transformer.
```ruby
transform "junit" do |item|
puts "This is the item: #{item}"
end
```
Let’s run the `dry-run` command to see what information is outputted in the terminal.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run --custom-transformers transformers.rb
```
You should see something like the following outputted into the terminal:
<img width="1113" alt="Screen Shot 2022-08-17 at 10 25 53 AM" src="https://user-images.githubusercontent.com/19557880/186782050-65ece0c4-52a3-4f88-818f-0f860c50c2b7.png">
Now that we know the structure of the incoming JSON blob for `item` we can access the file path programmatically.
```ruby
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
[
{
"uses": "actions/upload-artifact@v3",
"with": {
"name": "junit-artifact",
"path": file_path
}
}
]
end
```
Your file should include both custom transformers. Ensure it matches the file below:
__Click to Expand__
<details>
<summary><em>Custom Transformer file</em></summary>
```ruby
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
[
{
"uses": "actions/upload-artifact@v3",
"with": {
"name": "junit-artifact",
"path": file_path
}
}
]
end
```
</details>
Let’s run the `dry-run` command again to see if our custom transformer worked.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run --custom-transformers transformers.rb
```
When you open the file you should see the`EnricoMi/publish-unit-test-result-action@v1.7` action has been replaced with the customized steps.
```diff
- - name: Publish test results
- uses: EnricoMi/publish-unit-test-result-action@v1.7
- if: always()
- with:
- files: "**/target/*.xml"
+ - uses: actions/upload-artifact@v3
+ with:
+ name: junit-artifact
+ path: path/to/artifact/world.txt
```
## Custom transformers for environment variables
But wait! There's more we need to customize to make these pipelines fit our needs. Several of our pipelines will need to have the value of the `DB_ENGINE` environment variable changed from `sqlite` to `mongodb`.
Valet allows you to replace values of `variables` by using the `env` method. Let’s replace the value for `DB_ENGINE` by using the below line. The first value of the `env` method is the target variable name and the second is the new value to be used.
```ruby
env "DB_ENGINE", "mongodb"
```
You can keep adding these transformer values to your `transformer.rb` file. Your transformer file should look like the one below:
__Click to Expand__
<details>
<summary><em>Custom Transformer file</em></summary>
```ruby
env "DB_ENGINE", "mongodb"
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
[
{
"uses": "actions/upload-artifact@v3",
"with": {
"name": "my-artifact",
"path": file_path
}
}
]
end
```
</details>
Let’s run the `dry-run` command again to see if our custom transformer worked.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run --custom-transformers transformers.rb
```
When you open the outputed file you should see that the value for `DB_ENGINE` has been changed from `sqlite` to `mongodb`.
```diff
env:
DISABLE_AUTH: 'true'
- DB_ENGINE: sqlite
+ DB_ENGINE: mongodb
```
## Custom transformers for runners
Lastly, several of our pipelines will need to have the runner label updated from `TeamARunner` to `ubuntu-latest`.
Valet offers the ability to provide customized mapping between build agents in the source CI instance and their equivalent GitHub Actions runner labels by using the `runner` method. The syntax for this is similar to the `env` method.
The `runner` methods first parameter is the Jenkins agent label and the second parameter is the GitHub Actions runner label(s) to map too. As an example, the following could be used:
```ruby
runner "TeamARunner", "ubuntu-latest"
```
You can keep adding these transformer methods to your `transformer.rb` file. Your transformer file should look like the one below:
__Click to Expand__
<details>
<summary><em>Custom Transformer file</em></summary>
```ruby
runner "TeamARunner", "ubuntu-latest"
env "DB_ENGINE", "mongodb"
transform "sleep" do |item|
wait_time = item["arguments"][0]["value"]["value"]
{
"name": "Sleep for #{wait_time} seconds",
"run": "sleep #{wait_time}s",
"shell": "bash"
}
end
transform "junit" do |item|
test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
file_path = test_results.dig("value", "value")
[
{
"uses": "actions/upload-artifact@v3",
"with": {
"name": "my-artifact",
"path": file_path
}
}
]
end
```
</details>
Let’s run the `dry-run` command again to see if our custom transformer worked.
```bash
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run --custom-transformers transformers.rb
```
When you open the file you should see that the value for `TeamARunner` has been changed to `ubuntu-latest`.
```diff
runs-on:
- - self-hosted
- - TeamARunner
+ - ubuntu-latest
```
Thats it! Congratulations you have customized transforming:
- unsupported steps
- steps that are supported but don't meet your requirements
- environment variables
- runners
## Next Lab
[Migrating a Jenkins Pipeline](../jenkins/valet-migrate-lab.md#migrate-a-jenkins-project-to-github-actions)
jenkins/valet-dry-run-lab.md
-183
View File
@@ -1,183 +0,0 @@
# Dry-run the migration of a Jenkins pipeline to GitHub Actions
In this lab, you will use the Valet `dry-run` command to convert a Jenkins pipeline to its equivalent GitHub Actions workflow.
The end result of this command will be the actions workflow written to your local filesystem.
- [Prerequisites](#prerequisites)
- [Perform a dry-run](#perform-a-dry-run)
- [Review dry-run output](#review-dry-run-output)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed the steps [here](../jenkins/readme.md#valet-labs-for-jenkins) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](../jenkins/valet-configure-lab.md#configure-valet-to-work-with-jenkins) to configure the Valet CLI.
3. Completed the [audit lab](../Jenkins/valet-audit-lab.md#audit-jenkins-pipelines-using-the-valet-audit-command).
## Perform a dry-run
We will be performing a dry-run against a preconfigured pipeline in the Jenkins instance. Before running the command we need to collect some information:
1. What is the name of the pipeline we want to convert? __test_pipeline__
2. What is the source URL of the pipeline we want to convert? __<http://localhost:8080/job/test_pipeline>__
3. Where do we want to store the result? __./tmp/dry-run-lab. This can be any valid path on the system. In the case of codespaces it is generally best to use `./tmp/SOME_DIRECTORY_HERE` so the files show in explorer__
### Steps
1. Navigate to the codespace terminal
2. Run the dry-run command using the values determined above
```
gh valet dry-run jenkins --source-url http://localhost:8080/job/test_pipeline -o .tmp/jenkins/dry-run
```
3. When the command finishes the output files should be printed to the terminal.
<img width="915" alt="Screen Shot 2022-08-16 at 9 54 26 AM" src="https://user-images.githubusercontent.com/19557880/184935603-5c2d4dfe-66ef-4cb1-9398-e96954ca72e3.png">
4. Open generated actions workflow
- Find `./tmp/dry-run-lab/valet` in the file explorer pane in codespaces.
- Click `test_pipeline.yml` to open
<img width="234" alt="Screen Shot 2022-08-16 at 9 55 44 AM" src="https://user-images.githubusercontent.com/19557880/184935840-d4bdcbc9-75e5-4918-a055-28b765eac50c.png">
## Review dry-run output
The dry-run output will show you the GitHub Actions yaml that would be migrated to GitHub with the `migrate` command. We will now take a quick look at what was generated.
__Click to Expand__
<details>
<summary><em>Jenkins Pipeline</em> </summary>
```yaml
pipeline {
agent {
label 'TeamARunner'
}
environment {
DISABLE_AUTH = 'true'
DB_ENGINE = 'sqlite'
}
stages {
stage('build') {
steps {
echo "Database engine is ${DB_ENGINE}"
sleep 80
echo "DISABLE_AUTH is ${DISABLE_AUTH}"
}
}
stage('test') {
steps{
junit '**/target/*.xml'
}
}
}
}
```
</details>
<details>
<summary><em>Actions Workflow</em></summary>
```yaml
name: test_pipeline
on:
push:
paths: "*"
schedule:
- cron: 0-29/10 * * * *
env:
DISABLE_AUTH: 'true'
DB_ENGINE: sqlite
jobs:
build:
runs-on:
- self-hosted
- TeamARunner
steps:
- name: checkout
uses: actions/checkout@v2
- name: echo message
run: echo "Database engine is ${{ env.DB_ENGINE }}"
# # This item has no matching transformer
# - sleep:
# - key: time
# value:
# isLiteral: true
# value: 80
- name: echo message
run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
test:
runs-on:
- self-hosted
- TeamARunner
needs: build
steps:
- name: checkout
uses: actions/checkout@v2
- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1.7
if: always()
with:
files: "**/target/*.xml"
```
</details>
In the Jenkins pipeline we have 2 stages and 4 steps that run on the self-hosted runner labeled `TeamARunner`.
In the Actions workflow we have the same steps and the stages are now being enforced using the `needs` keyword. We can see this if we examine the `test` job, it has `needs: build`, which makes it depend on the `build` job.
```diff
- stages: test
+ needs: build
```
The `agent` in the Jenkins pipeline has been transformed to `runs-on` on each of the jobs.
```diff
- agent {
- label 'TeamARunner'
- }
+ runs-on:
+ - self-hosted
+ - TeamARunner
```
And the `echo` commands remain mostly the same
```diff
- echo "Database engine is ${DB_ENGINE}"
+ - name: echo message
+ run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
```
Note how Valet was not able to find a suitable conversion for the `sleep` command, and it added a comment to the yaml so this information was not lost, and it could be addressed manually later if needed.
```diff
- sleep 80
+ # # This item has no matching transformer
+ # - sleep:
+ # - key: time
+ # value:
+ # isLiteral: true
+ # value: 80
```
Lastly, the `junit` command was transformed using a third party action `EnricoMi/publish-unit-test-result-action@v1.7`
```diff
- junit '**/target/*.xml'
+ - name: Publish test results
+ uses: EnricoMi/publish-unit-test-result-action@v1.7
+ if: always()
+ with:
+ files: "**/target/*.xml"
```
Try constructing and running the `dry-run` command yourself. Hint, you should just have to change the project name.
## Next Lab
[Using Custom Transformers in a dry-run](valet-custom-transformers-lab.md#using-custom-transformers-in-a-dry-run)
jenkins/valet-forecast-lab.md
-181
View File
@@ -1,181 +0,0 @@
# Forecast the runner usage of a Jenkins instance
In this lab we will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from the historical pipeline data in our Jenkins instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab.
- [Prerequisites](#prerequisites)
- [Prepare for forecast](#prepare-for-forecast)
- [Perform a forecast](#perform-a-forecast)
- [Review forecast report](#review-forecast-report)
- [Forecasting multiple providers](#forecasting-multiple-providers)
- [Next steps](#next-steps)
## Prerequisites
1. Followed [steps](../jenkins/readme.md#valet-labs-for-jenkins) to set up your codespace environment and start your Jenkins instance.
2. Completed the [configure lab](../jenkins/valet-configure-lab.md#configure-valet-to-work-with-jenkins) to configure the Valet CLI.
## Prepare for forecast
Before we can run the forecast we need to answer a few questions so we can construct the correct command.
1. Do we want to forecast the entire Jenkins instance, or just a single folder?
- In this example we will be auditing the entire Jenkins instance, but in the future if you wanted to configure a specific folder to be audited add the `-f <folder_path>` flag to the forecast command.
2. What is the date we want to start forecasting from?
- __2022-08-02__. This date is before the time the data was populated in the Jenkins server running in these labs. This value defaults to the date one week ago, however, you should ensure a date is used that will capture enough data to get a representative view of typical usage.
3. Where do we want to store the results?
- `./tmp/forecast_reports`. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the workspace.
## Perform a forecast
- Using the answers above we get the following `forecast` command:
```
gh valet forecast jenkins --output-dir ./tmp/forecast_reports --start-date 2022-08-02
```
- Run the command in the codespace terminal.
- Verify that the command output is similar to this.
![forecast_output](https://user-images.githubusercontent.com/19557880/186223037-18556c82-5a29-4434-bc17-4b906d704967.png)
## Review forecast report
Open the forecast report and review the calculated metrics.
- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and right-click, and select __Open Preview__.
![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png)
- The file should be similar to this.
<details>
<summary>example forecast_report.md</summary>
# Forecast report for [Jenkins](http://localhost:8080)
- Valet version: __0.1.0.13448(2222622ecf83e372990e1657b1293e1ac552be21)__
- Performed at: __8/23/22 at 17:11__
- Date range: __8/2/22 - 8/23/22__
## Total
- Job count: __73__
- Pipeline count: __6__
- Execution time
- Total: __27,057 minutes__
- Median: __2 minutes__
- P90: __19 minutes__
- Min: __0 minutes__
- Max: __15,625 minutes__
- Queue time
- Median: __0 minutes__
- P90: __0 minutes__
- Min: __0 minutes__
- Max: __0 minutes__
- Concurrent jobs
- Median: __1__
- P90: __3__
- Min: __0__
- Max: __29__
---
## N/A
- Job count: __73__
- Pipeline count: __6__
- Execution time
- Total: __27,057 minutes__
- Median: __2 minutes__
- P90: __19 minutes__
- Min: __0 minutes__
- Max: __15,625 minutes__
- Queue time
- Median: __0 minutes__
- P90: __0 minutes__
- Min: __0 minutes__
- Max: __0 minutes__
- Concurrent jobs
- Median: __1__
- P90: __3__
- Min: __0__
- Max: __29__
> Note: Concurrent jobs are calculated by using a sliding window of 1m 0s.
</details>
### Metric Definitions
| Name | Description |
| ----- | ----------- |
| Median | The __middle__ value |
| P90 | 90% of the values are less than or equal to |
| Min | The lowest value |
| Max | The highest value |
### Total Section
- This section shows the metrics for all of the jobs run within the Jenkins instance from 08/02/2022 to the time the command was executed.
## Total
- Job count: __73__
- Pipeline count: __6__
---
We can see there were 73 completed jobs across 6 unique pipelines. A pipeline can have one or more jobs and a pipeline may be executed multiple times in the date range included in the forecast.
For example `monas_freestyle` contains 1 job.
![demo_pipeline](https://user-images.githubusercontent.com/19557880/186261368-d4dbbe8d-71e0-4084-bbbb-7557e9dbbb86.png)
Here are some key terms of items defined in the forecast report:
- The `job count` is the total number of completed jobs.
- The `pipeline count` is the number of unique pipelines used.
- `Execution time` describes the amount of time a runner spent on a job. This metric can be used for a customer to help set expectations for the cost of GitHub hosted runners.
- This metric is correlated to the amount of spend a customer should expect in GitHub Actions. This will vary greatly depending on the hardware the customer uses for these minutes and the Actions pricing calculator should be used to get an estimate of the approximate spend the customer should expect.
- Looking closer we can see during our forecast timeframe the total job run time was 27,057 minutes with 90% of the jobs finishing under 20 minutes, and the longest job taking 15,625 minutes. The `min` is 0 because the quickest job took less than a minute and was rounded down to 0.
- `Queue time` metrics describe the amount of time a job spent waiting for a runner to be available to execute it.
- `Concurrent jobs` metrics describe the amount of jobs running at any given time. This metric can be used to define the number of runners a customer should configure.
Additionally, these metrics are defined for each queue of runners that a customer has defined in the CI/CD platform. This is especially useful for customers that use a mix of hosted and self-hosted runners to see runner utilization metrics that are specific to different types of runners.
### Runner Group Sections
- The preceding section shows the same metrics as the `Total` section, but are grouped by runner group. A runner group is a machine (or group of machines) that each job runs on
- In this case we do not have any runner groups, so the metrics match under `N/A` match the `Total` section. If there were different groups we could possibly identify runner types that needed to be increased or decreased when moving to GitHub Actions.
## Forecasting multiple providers
If we examine the help for the `forecast` command by running `gh valet forecast --help` we can see the option: `--source-file-path`
![forecast-help](https://user-images.githubusercontent.com/19557880/186263140-f02c6cab-7979-417c-bdfe-b9590e9c5597.png)
Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as Azure DevOps and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a space-delimited list of data file paths or a glob pattern that will match all of the data files we want to include and combine into a single report. We will use a glob pattern, which in general should match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the previous value used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`!
- run `gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast`
- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider, it only operates on the data files it finds.
![combined-report](https://user-images.githubusercontent.com/19557880/186264213-b3201710-8093-4ae5-9aef-5c7f95cc3951.png)
## Next steps
This concludes the Valet labs for Jenkins! If you are interested exploring the power of Valet more, you can leverage the demo Jenkins Instance and modify and add new projects that more closely match your needs and try out the commands again!
jenkins/valet-migrate-lab.md
-58
View File
@@ -1,58 +0,0 @@
# Migrate a Jenkins Project to GitHub Actions
In this lab, we will use the Valet `migrate` command to migrate a Jenkins pipeline to GitHub Actions.
The previous commands used in the labs, such as `audit` and `dry-run` have prepared us to run a migration.
The `migrate` command will transform the Jenkins pipeline into a GitHub Actions workflow like the `dry-run` command did, but instead of writing these files locally, it will open a pull request with the files.
The pull request will contain a checklist of `Manual Tasks` if required. These tasks are changes that Valet could not do on our behalf, like creating a runner or adding a secret to a repository.
- [Prerequisites](#prerequisites)
- [Preparing for migration](#preparing-for-migration)
- [Performing the migration](#performing-a-migration)
- [Reviewing the pull request](#reviewing-the-pull-request)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed the steps [here](../jenkins/readme.md#valet-labs-for-jenkins) to set up your Codespace environment and start a Jenkins server.
2. Completed the [configure lab](../jenkins/valet-configure-lab.md#configure-valet-to-work-with-jenkins) to configure the Valet CLI.
3. Completed the [dry-run lab](../jenkins/valet-dry-run-lab.md#dry-run-the-migration-of-a-jenkins-pipeline-to-github-actions).
4. Completed the [dry-run lab with custom transformer](../jenkins/valet-custom-transformers-lab.md#using-custom-transformers-in-a-dry-run).
## Preparing for migration
Before running the command we need to collect some information:
1. What is the name of the pipeline we want to convert? __monas_dev_work/job/monas_freestyle__
2. What is the source URL of the pipeline we want to convert? __<http://localhost:8080/monas_dev_work/job/monas_freestyle>__
3. Where do we want to store the result? __./tmp/migrate__.
4. What is the URL for the GitHub repository we want to add the workflow too? __this repo!__. *When constructing the value for the migrate command it should match this url <https://github.com/GITHUB-ORG-USERNAME-HERE/GITHUB-REPO-NAME-HERE> with `GITHUB-ORG-USERNAME` and `GITHUB-REPO-NAME` substitued with your values*
## Performing a migration
1. Run `migrate` command using the information collected above, make sure to update the `--target-url` value with the information from step 4.
```bash
gh valet migrate jenkins --target-url https://github.com/GITHUB-ORG-USERNAME-HERE/GITHUB-REPO-NAME-HERE --output-dir ./tmp/migrate --source-url http://localhost:8080/job/monas_dev_work/job/monas_freestyle
```
2. Valet will create a pull request directly in the target GitHub repository.
3. Open the pull request by clicking the green pull request link in the output of the migrate command, if you have trouble clicking it you can always copy and paste the url in your browser.
![pr-screen-shot](https://user-images.githubusercontent.com/19557880/185509412-ab64d92d-2a56-4d5a-bbb4-35a41a2ca48c.png)
## Reviewing the pull request
- Let's first look at the `Conversation` tab of the PR. It tells us we have a manual task to perform before the GitHub Actions workflow is functional. We need to add a secret. We can use the GitHub [documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) for secrets and add a `actions` secret for `EXPRESSION_FIRST_VAR` with any value.
![Manual steps](https://user-images.githubusercontent.com/19557880/186784161-b7882ac4-ac99-4462-b69f-f49b9202527b.png)
- Next, let's review the workflow we are adding by clicking on `Files changed` tab. This is where you would double check everything looks good. If it didn't you could push commits with the required changes, prior to merging.
- Now our review is completed we want to go back to the `Conversation` tab and click `Merge pull request`
- Once the PR is merged the new workflow should start and we can view it by clicking `Actions` in the top navigation
<img width="1119" alt="actions-screen-shot" src="https://user-images.githubusercontent.com/19557880/185509704-90243ec5-e77f-4baf-a9b2-d9a4d9fda199.png">
- The migration is complete and we have successfully transformed a pipeline from Jenkins into a GitHub Actions workflow, added it to a GitHub Repository, and run the workflow in Actions.
### Next Lab
[Forecast Jenkins Usage](../jenkins/valet-forecast-lab.md#forecast-the-runner-usage-of-a-jenkins-instance)