actions/importer-labs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merged with feature branch

Browse Source
This commit is contained in:
Begona Guereca
2022-08-17 15:42:22 -07:00
parent 7e86c1935b 9e9f2db93b
commit fd3cbea48c
9 changed files with 322 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
jenkins/bootstrap/setup.sh
Executable
+33
View File
@@ -0,0 +1,33 @@
#!/bin/bash
container_name="valet"
username="admin"
password="password"
echo "Building Jenkins instance!"
if [ "$(docker ps -a | grep jenkins:$container_name)" ]; then
echo -e "Jenkins is running"
docker container start jenkins
else
echo -e "\nStarting a new Jenkins container"
# Build jenkins image from docker compose file
docker build -t jenkins:$container_name .
# Build container
docker run -d --name jenkins -p 8080:8080 --env JENKINS_ADMIN_ID=$username --env JENKINS_ADMIN_PASSWORD=$password jenkins:$container_name
fi
# allow valet to talk to Jenkins by removing network isolation between containers
export DOCKER_ARGS="--network=host"
grep -q "export DOCKER_ARGS=" ~/.bashrc || echo 'export DOCKER_ARGS="--network=host"' >> ~/.bashrc
echo -e "\nWaiting for Jenkins to start..."
while ! curl -s http://localhost:8080/ > /dev/null; do
echo -e "."
sleep 5
done
echo -e '\nJenkins is up and running!'
echo -e "\nUsername: admin"
echo -e "\bPassword: password"
jenkins/readme.md
+71 -1
View File
@@ -1 +1,71 @@
# Coming Soon!
# Valet labs for Jenkins
This lab bootstraps a Valet environment using GitHub Codespaces and enables you to spin up a Jenkins instance against which to run the Valet CI/CD migration tool.
- [Use this Repo as a template](#repo-template)
- [Prerequisites](#prerequisites)
- [Use Valet with a codespace](#use-valet-with-a-codespace)
- [Bootstrap Jenkins](#bootstrap-jenkins)
## Repo template
1. Verify you are in your own Repository created from the landing page [Valet Labs](https://github.com/valet-customers/labs).
## Use Valet with a codespace
1. Start the codespace
- Click the `Code` with button down arrow above repository on the repository's landing page.
- Click the `Codespaces` tab
- Click `Create codespaces on main` to create the codespace. If you are in another branch then the `main` branch, the codespace will button will have the current branch specified.
- Wait a couple minutes, then verify that the codespace starts up. Once it is fully booted up, the termininal should be present.
2. Verify Valet CLI is installed and working. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet)
- Verify Valet CLI is installed and working
- Run `gh valet version` in the Visual Studio Code terminal and verify the output looks like the below image. Note the valet version will be different than below as the latest version gets pulled down.
- If `gh valet version` did not produce a similar image with a version please follow these instructions [Troubleshoot GH Valet extension](#troubleshoot-gh-valet-extension)
<img width="836" alt="Screen Shot 2022-08-10 at 1 45 20 PM" src="https://user-images.githubusercontent.com/19557880/184017284-04233b97-d25e-48cf-be51-f7088c779fd5.png">
## Bootstrap Jenkins
1. Run the Jenkins setup script. This script will setup Jenkins and ensure it is ready to use. In general, this script should be run first if you are starting a new codespace or restarting an existing one.
- Navigate to the terminal within your Codespace.
- Run `source jenkins/bootstrap/setup.sh` to kick off the creation of your Jenkins instance.
- After a couple seconds, a pop-up box should appear with a link to the forwarded URL for your Jenkins instance.
- You can also access the forwarded URL by going to the `Ports` tab in your terminal. Right click on the URL listed under the `Local Address` and clicking the `Open in Browser` tab.
- Once you have navigated to the url, the following default credentials have been assigned:
- username: `admin`
- password: `password`
2. Click the `Sign in` button and you should now see your new Jenkins instance with a few pre-populataed pipelines.
## Labs for Jenkins
Perform the following labs to test-drive Valet
- TBD
## Troubleshoot GH Valet extension
Manually Install the GitHub CLI Valet extension. More information on the [GitHub Valet CLI extension](https://github.com/github/gh-valet)
- Verify you are in the codespace terminal
- Run this command to install the GitHub Valet extension
- `gh extension install github/gh-valet`
- Verify the result of the install is: `✓ Installed extension github/gh-valet`
- If you get a similiar error to the following, click the link to authorize the token
- Restart Codespace after clicking the link
- Verify Valet CLI is installed and working by running `gh valet version`
![linktolcickauth](https://user-images.githubusercontent.com/26442605/169588015-9414404f-82b6-4d0f-89d4-5f0e6941b029.png)
## Troubleshooting Jenkins
1. Navigate to the Docker tab on your left hand side.
2. Under the `Containers` tab you should see a Docker container `jenkins:valet` listed with a green play button ▶
- If you see the `jenkins:valet` container, but it has a red stopped symbol next to it ▢, right click on the container and click on `start`, the container should begin running again.
- If the container does not start even after trying to manually start it, right click on the `jenkins:valet` container and click the `remove` button. Next continue by following all the #bootstrap-jenkins instructions again.
<img width="985" alt="Screen Shot 2022-08-09 at 3 06 46 PM" src="https://user-images.githubusercontent.com/19557880/183770210-c0386616-656e-4fe9-9324-b410ad62c406.png">
jenkins/valet-audit-lab.md
+124
View File
@@ -0,0 +1,124 @@
# Audit Jenkins pipelines using the Valet audit command
In this lab, you will use Valet to `audit` a Jenkins organization. The `audit` command can be used to scan a CI server and output a summary of the current pipelines.
What happens behind the scenes is that Valet will perform a `dry-run` on each of the Jenkins pipelines. Once that is complete, Valet will perform an aggregation of all of the transformed workflows. This aggregate summary can be used as a planning tool and help understand how complete of a migration is possible with Valet.
By the end of this lab you will have performed an audit on the demo Jenkins instance, and have a good understanding of the components that make up an audit.
- [Prerequisites](#prerequisites)
- [Perform an audit](#perform-an-audit)
- [View audit output](#view-audit-output)
- [Review the pipelines](#review-the-pipelines)
- [Next Lab](#next-lab)
## Prerequisites
1. Follow all steps [here](../jenkins#readme) to set up your environment.
2. Follow all steps [here](../jenkins#valet-configure-lab) to configure Valet.
## Perform an audit
We will be performing an audit against a preconfigured Jenkins instance. Before running the command we need to collect some information:
1. Do we want to audit the entire Jenkins instance, or just a single folder? __In this example we will be auditing the entire Jenkins instance, but in the future if you wanted to configure a specific folder to be audited add the `-f <folder_path>` flag to the audit command__
2. Where do we want to store the result? __./tmp/audit. This can be any valid path on the system. In the case of codespaces it is generally best to use `./tmp/SOME_DIRECTORY_HERE` so the files show in explorer__
### Steps
1. Navigate to the codespace terminal.
2. Now, from root directory, run the following Valet audit command:
```
gh valet audit jenkins --output-dir tmp/audit
```
3. Valet will log the output files in green when the audit is successful
### Example
<img src="https://user-images.githubusercontent.com/19557880/184682347-b19760fa-36a6-423e-a445-bb30eda5ac59.png" alt="valet-audit-1"/>
## View audit output
The audit summary, logs, config files, jenkinsfiles, and transformed Actions Workflows should all be located within the `tmp/audit` folder.
1. Under the `audit` folder find the `audit_summary.md`
2. Right-click the `audit_summary.md` file and select `Open Preview`
3. The file contains details about your current pipelines and what can be migrated 100% automatically vs. what will need some manual intervention or aren't supported by GitHub Actions.
4. Review the file, it should look like the image below:
### Example
<img src="https://user-images.githubusercontent.com/19557880/184682836-3b8155ae-f302-491e-8ce6-27cc57f96468.png" alt="valet-audit-2"/>
## Review the pipelines
### Pipelines
The audit summary starts by giving a summary of the types of pipelines that were extracted from Jenkins.
- It shows that there are a total of 7 pipelines extracted.
- 42% pipelines were successful. This means that Valet knew how to map all the constructs of the Jenkins pipeline to a GitHub Actions equivalent. All of the build pluggins and triggers that are referenced were all successfully converted into a GitHub Actions equivalent.
- 42% pipelines were partially successful. This means that Valet knew how to map all the constructs of the Jenkins pipeline but there may be a plugin that was referenced that Valet wasn't able to automatically map to a Github Actions equivalent.
- 1% of these pipelines were unsupported. This means that the pipeline type is fundamentally unsupported by Valet. This is most likely a Jenkins scripted pipeline.
- 0% of these fail altogether. If there were any pipelines that would fall under this category, that would mean that those pipelines were misconfigured or there was an issue with Valet.
Under the `Job types` section, we can see that the `audit` command is able to support the conversion of project, freestyle (flow-defintion), and multibranch pipelines from Jenkins and convert them to a GitHub Actions workflow. Valet does not support converting [scripted pipelines](https://www.jenkins.io/doc/book/pipeline/syntax/#scripted-pipeline) (e.g. pure Groovy).
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184683664-81985baf-5c03-4765-a067-f4023416e3ea.png" alt="valet-audit-3" height="400"/>
### Build steps
Under the `Build steps` section we can see a breakdown of the build steps that were used in these pipelines.
- <b>Supported:</b> 12/16 discrete build steps are considered known by Valet. When Valet encounters a build step of this type, it knows exactly how to map that into a GitHub Actions equivalent.
- <b>Unknown:</b> 3/16 discrete build steps are considered unknown by Valet. When Valet enounters a build step of this type, it does not yet know to map this automatically to a GitHub Action equivalent.
- <b>Unsupported:</b> 1/16 discrete build steps are considered unsupported by Valet. This could mean one of three things:
1. The way that plugin was configured for a given job is unsupported.
2. The plugin itself is fundamentally not supported in GitHub Actions.
3. It's supported by default in GitHub Actions.
Under the `Actions` section we have the list of the Actions that were used in order to implement the transformation of all of these build steps. Valet is a planning tool that can help in facilitating the migration into GitHub Actions and this list of Actions is a great place to understand what dependencies you would be taking on third-party Actions after this migration.
For example, if you are doing things like setting up the allow list of third-party Actions in a GitHub Enterprise server instance this list of Actions is a fantastic place to begin security reviews and audits of what third-party actions to depend on.
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184684062-69ab0bde-5e32-45f8-a7dd-ed4655872975.png" alt="valet-audit-4"/>
### Trigger, Environment, Other
Similar to `Build steps`, there are `Trigger`, `Environment`, and a catch all `Other` section that breakdown each of their uses accross the audited pipelines.
### Example
<img src="https://user-images.githubusercontent.com/19557880/184684174-43caff58-6083-45e1-a36e-6899d99c136b.png" alt="valet-audit-5" height="600"/>
### Manual Tasks
Under the Manual task section you will find a list of all the manual tasks that the pipelines would surface in a migration. Manual tasks are Valet's way of indicating tasks a user needs to do in order for a pipeline to be functional, such as adding `secrets`, or setting up a `self-hosted` runner. We will see how these manual tasks appear on a pull request when we do a migration in a lab later on.
### Example
<img src="https://user-images.githubusercontent.com/19557880/184684249-9accfd94-c2df-4891-af56-dcff66beb557.png" alt="valet-audit-5" height="200"/>
### Files
At the end of the Audit Summary page you will find a list of all of the files that were written to disk. Generally, for any given pipeline, you’ll find 2 or 3 associated files. In these files are the actual converted GitHub Actions workflows.
In addition, you’ll see a file that shows the raw JSON data that we pull from Jenkins as well as any associated Jenkinsfiles for a given job. These files are really useful for engineering teams to help debug any issues and to understand what may have gone on in a transformation.
#### Example
<img src="https://user-images.githubusercontent.com/19557880/184684416-b3db774e-4ab8-46e0-91ad-e503632df5cb.png" alt="valet-audit-6" height="700"/>
### Next Lab
[Dry run the migration of a Jenkins pipeline to GitHub Actions](valet-dry-run-lab.md)
jenkins/valet-configure-lab.md
+78
View File
@@ -0,0 +1,78 @@
# Configure Valet to work with Jenkins
In this lab, you will use the Valet `configure` command to set up the required information to communicate with the Jenkins and GitHub instances. The `configure` command can be used for all of the supported providers, in this lab we will be focusing on Jenkins.
- [Prerequisites](#prerequisites)
- [Configuring Valet](#configuring-valet)
- [Verify Valet Works](#verify-valet-works)
- [Next Lab](#next-lab)
## Prerequisites
1. Followed [steps](../Jenkins#readme) to set up your codespace environment and start your Jenkins instanace.
## Configuring Valet
1. Login to the Jenkins instance to generate a personal access token:
1. Click the `PORTS` tab in the codespace terminal window.
2. In the `PORTS` tab find the row for port 8080.
3. Hover over the address under the `Local Address` column, and click the globe to "open in browser".
4. Login to the Jenkins server and generate a Jenkins API token.
- Click the `admin` button located within the top right menu.
- Click on the `configure` gear located on the left hand panel.
- Under the `API token` section, click `Add new Token`.
- Add a defaualt name to your token, then click `Generate`.
- Copy the token that was generated and record token for a later step.
![configure-result](https://user-images.githubusercontent.com/19557880/184041667-d06cb7f2-a885-474e-b728-7567314aeaf3.png)
2. Create a GitHub personal access token (PAT).
- Navigate to your GitHub `Settings` - click your profile photo and then click `Settings`.
- Go to `Developer Settings`
- Go to `Personal Access Tokens` -> `Legacy tokens (if present)`
- Click `Generate new token` -> `Legacy tokens (if present)`. If required, provide your password.
- Select at least these scopes: `read packages` and `workflow`. Optionally, provide a text in the **Note** field and change the expiration.
- Click `Generate token`
- Copy the token somewhere safe and temporary.
3. Run Valet configure commands
- In the codespace terminal window click back to the `TERMINAL` tab.
- Within the terminal, ensure you are in the root dirrectory.
- Run `gh valet configure`.
- Use the down arrow key to highlight `Jenkins`, press the spacebar to select, then hit enter to accept.
- At the prompt enter your GitHub Username and press enter.
- At the GitHub Container Registry prompt enter the GitHub PAT generated in step 3 and press enter
- At the GitHub PAT prompt enter the GitHub PAT generated in step 3 and press enter.
- At the GitHub url prompt enter the GitHub instance url or hit enter to accept the default, if you are using github.com then the default is the right choice.
- At the Jenkins token prompt enter the Jenkins access token from step 2 and press enter.
- At the Jenkins url prompt enter `http://localhost:8080/` and press enter.
- At the Personal access token to fetch source code in GitHub prompt, if any of your Jenkins pipelines have source code in a GitHub repository enter the GitHub PAT that would have acess to these files.
4. If all went well you should see a similar output in your terminal:
![configure-result](https://user-images.githubusercontent.com/19557880/184041328-ce54ea22-b0cd-4c84-b02c-10ad7b09ad89.png)
## Verify Valet Works
To verify Valet works we are going to run a `update` and `dry-run` command. We will go further into details about the `dry-run` command in a later lab, but for now we want to get the latest version of Valet and confirm that Valet can perform a dry-run with no errors.
1. In the codespace terminal update Valet by running `gh valet update`
2. In the terminal you should see a confirmation that it logged into the GitHub Container Registry and pulled the latest version.
```
Login Succeeded
latest: Pulling from valet-customers/valet-cli
Digest: sha256:a7d00dee8a37e25da59daeed44b1543f476b00fa2c41c47f48deeaf34a215bbb
Status: Image is up to date for ghcr.io/valet-customers/valet-cli:latest
ghcr.io/valet-customers/valet-cli:latest
```
3. Next, lets run the dry-run command in the codespaces terminal, to verify we can talk to Jenkins
```
gh valet dry-run jenkins --source-url https://localhost:8080/job/test_pipeline/ --output-dir ./tmp/dry-run-lab
```
4. In the terminal you should see the command was successful, if not it is a good time to practice the configure command again and make sure the access tokens values are correct and were generated with the correct permissions.
![configure-dry-run](https://user-images.githubusercontent.com/19557880/184255620-8e9b120e-5de0-41df-9cb6-c52028de3b0f.png)
### Next Lab
TODO