Merge branch 'feature/circleci-labs' into circleci/custom-transformer
This commit is contained in:
j-dunham
@@ -0,0 +1,193 @@
|
||||
# Perform a dry-run of a CircleCI pipeline
|
||||
|
||||
In this lab you will use the `dry-run` command to convert a CircleCI pipeline to its equivalent GitHub Actions workflow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your Codespace environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [audit lab](./2-audit.md).
|
||||
|
||||
## Perform a dry run
|
||||
|
||||
You will be performing a dry-run against a CircleCI project. Answer the following questions before running this command:
|
||||
|
||||
1. What project do you want to convert?
|
||||
- __circleci-demo-ruby-rails__. This is one of the sample projects avaiable in the CircleCI labs-data organization.
|
||||
|
||||
2. Where do you want to store the result?
|
||||
- __./tmp/dry-run-lab__. This can be any path within the working directory that Valet commands are executed from.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Navigate to your codespace terminal
|
||||
2. Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet dry-run circle-ci --output-dir ./tmp/dry-run-lab --circle-ci-project circleci-demo-ruby-rails
|
||||
```
|
||||
|
||||
3. The command will list all the files written to disk when the command succeeds.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
4. View the converted workflow:
|
||||
- Find `./tmp/dry-run-lab/labs-data/circleci-demo-ruby-rails` in the file explorer pane in your codespace.
|
||||
- Click `build_and_test.yml` to open.
|
||||
|
||||
## Inspect the output files
|
||||
|
||||
The files generated from the `dry-run` command represent the equivalent Actions workflow for the CircleCI project. The CircleCI configuration and converted workflow can be seen below:
|
||||
|
||||
<details>
|
||||
<summary><em>CircleCI configuration 👇</em></summary>
|
||||
|
||||
```yaml
|
||||
version: 2.1
|
||||
|
||||
orbs:
|
||||
ruby: circleci/ruby@1.1.0
|
||||
node: circleci/node@2
|
||||
|
||||
jobs:
|
||||
build:
|
||||
docker:
|
||||
- image: cimg/ruby:2.7.5-node
|
||||
steps:
|
||||
- checkout
|
||||
- ruby/install-deps
|
||||
# Store bundle cache
|
||||
- node/install-packages:
|
||||
pkg-manager: yarn
|
||||
cache-key: "yarn.lock"
|
||||
test:
|
||||
parallelism: 3
|
||||
docker:
|
||||
- image: cimg/ruby:2.7.5-node
|
||||
- image: circleci/postgres:9.5-alpine
|
||||
environment:
|
||||
POSTGRES_USER: circleci-demo-ruby
|
||||
POSTGRES_DB: rails_blog_test
|
||||
POSTGRES_PASSWORD: ""
|
||||
environment:
|
||||
BUNDLE_JOBS: "3"
|
||||
BUNDLE_RETRY: "3"
|
||||
PGHOST: 127.0.0.1
|
||||
PGUSER: circleci-demo-ruby
|
||||
PGPASSWORD: ""
|
||||
RAILS_ENV: test
|
||||
steps:
|
||||
- checkout
|
||||
- ruby/install-deps
|
||||
- node/install-packages:
|
||||
pkg-manager: yarn
|
||||
cache-key: "yarn.lock"
|
||||
- run:
|
||||
name: Wait for DB
|
||||
command: dockerize -wait tcp://localhost:5432 -timeout 1m
|
||||
- run:
|
||||
name: Database setup
|
||||
command: bundle exec rails db:schema:load --trace
|
||||
# Run rspec in parallel
|
||||
- ruby/rspec-test
|
||||
- ruby/rubocop-check
|
||||
|
||||
workflows:
|
||||
version: 2
|
||||
build_and_test:
|
||||
jobs:
|
||||
- build
|
||||
- test:
|
||||
requires:
|
||||
- build
|
||||
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><em>Converted workflow 👇</em></summary>
|
||||
|
||||
```yaml
|
||||
name: labs-data/circleci-demo-ruby-rails/build_and_test
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: cimg/ruby:2.7.5-node
|
||||
steps:
|
||||
- name: Set up bundler cache
|
||||
uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: 3.0.2
|
||||
bundler-cache: true
|
||||
- uses: actions/checkout@v2
|
||||
- run: bundle check || bundle install
|
||||
env:
|
||||
BUNDLE_DEPLOYMENT: true
|
||||
- id: yarn-cache-dir-path
|
||||
run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: "${{ steps.yarn-cache-dir-path.outputs.dir }}"
|
||||
key: "${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}"
|
||||
restore-keys: "${{ runner.os }}-yarn-"
|
||||
- run: yarn install --frozen-lockfile
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: cimg/ruby:2.7.5-node
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:9.5-alpine
|
||||
env:
|
||||
POSTGRES_USER: circleci-demo-ruby
|
||||
POSTGRES_DB: rails_blog_test
|
||||
POSTGRES_PASSWORD: ''
|
||||
needs:
|
||||
- build
|
||||
env:
|
||||
BUNDLE_JOBS: '3'
|
||||
BUNDLE_RETRY: '3'
|
||||
PGHOST: 127.0.0.1
|
||||
PGUSER: circleci-demo-ruby
|
||||
PGPASSWORD: ''
|
||||
RAILS_ENV: test
|
||||
steps:
|
||||
- name: Set up bundler cache
|
||||
uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: 3.0.2
|
||||
bundler-cache: true
|
||||
- uses: actions/checkout@v2
|
||||
- run: bundle check || bundle install
|
||||
env:
|
||||
BUNDLE_DEPLOYMENT: true
|
||||
- id: yarn-cache-dir-path
|
||||
run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
|
||||
- uses: actions/cache@v2
|
||||
with:
|
||||
path: "${{ steps.yarn-cache-dir-path.outputs.dir }}"
|
||||
key: "${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}"
|
||||
restore-keys: "${{ runner.os }}-yarn-"
|
||||
- run: yarn install --frozen-lockfile
|
||||
- name: Wait for DB
|
||||
run: dockerize -wait tcp://localhost:5432 -timeout 1m
|
||||
- name: Database setup
|
||||
run: bundle exec rails db:schema:load --trace
|
||||
- run: bundle exec rspec spec --profile 10 --format RspecJunitFormatter --out /tmp/test-results/rspec/results.xml --format progress
|
||||
- run: bundle exec rubocop --format progress
|
||||
```
|
||||
</details>
|
||||
|
||||
Despite these two pipelines using different syntax they will function equivalently.
|
||||
|
||||
## Next lab
|
||||
|
||||
[Use custom transformers to customize Valet's behavior](./4-custom-transformers.md)
|
||||
@@ -0,0 +1,51 @@
|
||||
# Perform a production migration of a CircleCI pipeline
|
||||
|
||||
In this lab, you will use the `migrate` command to convert a CircleCI pipeline and open a pull request with the equivalent Actions workflow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
3. Completed the [dry-run lab](./3-dry-run.md).
|
||||
4. Completed the [custom transformers lab](./4-custom-transformers.md).
|
||||
|
||||
## Performing a migration
|
||||
|
||||
Answer the following questions before running a `migrate` command:
|
||||
|
||||
1. What project do you want to migrate?
|
||||
- __circleci-hello-world__
|
||||
2. Where do you want to store the logs?
|
||||
- __./tmp/migrate__
|
||||
3. What is the URL for the GitHub repository to add the workflow to?
|
||||
- __this repository__. The URL should follow the pattern <https://github.com/:owner/:repo> with `:owner` and `:repo` replaced with your values.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Run the following `migrate` command in the codespace terminal. Ensure the values in `--target-url` for `:owner` and `:repo` are replaced with your values:
|
||||
|
||||
```bash
|
||||
gh valet migrate circle-ci --target-url https://github.com/:owner/:repo --output-dir ./tmp/migrate --circle-ci-project circleci-hello-world
|
||||
```
|
||||
|
||||
2. The command will write the URL to the pull request that was created when the command succeeds.
|
||||
|
||||
|
||||
|
||||
3. Open the generated pull request in a new browser tab.
|
||||
|
||||
### Inspect the pull request
|
||||
|
||||
The first thing to notice about the pull request is that there is a list of manual steps to complete.
|
||||
|
||||
Next, you can inspect the "Files changed" in this pull request and see the converted workflow that is being added. Any additional changes or code reviews that were needed should be done in this pull request.
|
||||
|
||||
|
||||
|
||||
Finally, you can merge the pull request once your review has completed. You can then view the workflow running by selecting the "Actions" menu in the top navigation bar in GitHub.
|
||||
|
||||
At this point, the migration has completed and you have successfully migrated a CircleCI pipeline to Actions!
|
||||
|
||||
### Next Lab
|
||||
|
||||
[Forecast potential build runner usage](6-forecast.md)
|
||||
@@ -0,0 +1,99 @@
|
||||
# Forecast potential build runner usage
|
||||
|
||||
In this lab you will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from completed pipeline runs in CircleCI.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Followed the steps [here](./readme.md#configure-your-codespace) to set up your GitHub Codespaces environment.
|
||||
2. Completed the [configure lab](./1-configure.md#configuring-credentials).
|
||||
|
||||
## Perform a forecast
|
||||
|
||||
Answer the following questions before running the `forecast` command:
|
||||
1. What is the date you want to start forecasting from?
|
||||
- **2022-09-02**. This date is needed as it is prior to when the bulk of builds were trigger in the demo CircleCI organization for these labs. This value defaults to the date one week ago, however, you should use a start date that will show a representative view of typical usage.
|
||||
2. Where do you want to store the results?
|
||||
- **./tmp/forecast_reports**
|
||||
|
||||
### Steps
|
||||
|
||||
1. Navigate to your codespace terminal
|
||||
2. Run the following command from the root directory:
|
||||
|
||||
```bash
|
||||
gh valet forecast circle-ci --output-dir ./tmp/forecast_reports --start-date 2022-09-02
|
||||
```
|
||||
|
||||
3. The command will list all the files written to disk when the command succeeds.
|
||||
|
||||
|
||||
|
||||
## Review the forecast report
|
||||
|
||||
The forecast report, logs, and completed job data will be located within the `tmp/forecast_reports` folder.
|
||||
|
||||
1. Find the `forecast_report.md` file in the file explorer.
|
||||
2. Right-click the `forecast_report.md` file and select `Open Preview`.
|
||||
3. This file contains metrics used to forecast potential GitHub Actions usage.
|
||||
|
||||
### Total
|
||||
|
||||
The "Total" section of the forecast report contains high level statistics related to all the jobs completed after the `--start-date` CLI option:
|
||||
|
||||
```md
|
||||
- Job count: **18**
|
||||
- Pipeline count: **13**
|
||||
|
||||
- Execution time
|
||||
|
||||
- Total: **46 minutes**
|
||||
- Median: **1 minutes**
|
||||
- P90: **6 minutes**
|
||||
- Min: **0 minutes**
|
||||
- Max: **10 minutes**
|
||||
|
||||
- Queue time
|
||||
|
||||
- Median: **0 minutes**
|
||||
- P90: **0 minutes**
|
||||
- Min: **0 minutes**
|
||||
- Max: **0 minutes**
|
||||
|
||||
- Concurrent jobs
|
||||
|
||||
- Median: **0**
|
||||
- P90: **0**
|
||||
- Min: **0**
|
||||
- Max: **4**
|
||||
```
|
||||
|
||||
Here are some key terms of items defined in the forecast report:
|
||||
|
||||
- The `job count` is the total number of completed jobs.
|
||||
- The `pipeline count` is the number of unique pipelines used.
|
||||
- `Execution time` describes the amount of time a runner spent on a job. This metric can be used to help plan for the cost of GitHub-hosted runners.
|
||||
- This metric is correlated to how much you should expect to spend in GitHub Actions. This will vary depending on the hardware used for these minutes. You can use the [Actions pricing calculator](https://github.com/pricing/calculator) to estimate a dollar amount.
|
||||
- `Queue time` metrics describe the amount of time a job spent waiting for a runner to be available to execute it.
|
||||
- `Concurrent jobs` metrics describe the amount of jobs running at any given time. This metric can be used to define the number of runners a customer should configure.
|
||||
|
||||
Additionally, these metrics are defined for each queue of runners defined in CircleCI. This is especially useful if there are a mix of hosted/self-hosted runners or high/low spec machines to see metrics specific to different types of runners.
|
||||
|
||||
## Forecasting multiple providers
|
||||
|
||||
You can examine the available options for the `forecast` command by running `gh valet forecast --help`. When you do this you will see the `--source-file-path` option:
|
||||
|
||||
|
||||
|
||||
You can use the `--source-file-path` CLI option to combine data from multiple reports into a single report. This becomes useful if you use multiple CI/CD providers and want to get a holistic view of the runner usage. This works by using the `.json` files generated by `forecast` commands as space-delimited values for the `--source-file-path` CLI option. Optionally, this value could be a glob pattern to dynamically specify the list of files (e.g. `**/*.json`).
|
||||
|
||||
Run the following command from within the codespace terminal:
|
||||
|
||||
```bash
|
||||
gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast
|
||||
```
|
||||
|
||||
You can now inspect the output of the command to see a forecast report using all of the files matching the `tmp/**/jobs/*.json` pattern.
|
||||
|
||||
## Next steps
|
||||
|
||||
This concludes all labs for migrating CircleCI pipelines to Actions with Valet!
|
||||
Reference in New Issue
Block a user