From 6f662ac9b641a608f338aa73f329b81be72d05f3 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Wed, 17 Aug 2022 11:14:05 -0400 Subject: [PATCH 01/19] Create valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 gitlab/valet-forecast-lab.md diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md new file mode 100644 index 0000000..9f4d59c --- /dev/null +++ b/gitlab/valet-forecast-lab.md @@ -0,0 +1,20 @@ +# Forecast the usage of a GitLab namespace +## TBD + +- [Prerequisites](#prerequisites) +- [Perform a forecast](#perform-a-forecast) +- [Review forecast output](#review-forecast-output) + +## Prerequisites + +1. Followed [steps](../gitlab#readme) to set up your codespace environment. +2. Completed the [configure lab](../gitlab/valet-configure-lab.md) + + +## Perform a forecast +### TBD + + + +## Review forecast output +### TBD From 143fac003d488976c6cb5d2dab6bdb43f8f9caa6 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Wed, 17 Aug 2022 13:29:05 -0400 Subject: [PATCH 02/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 9f4d59c..f605cb2 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -1,7 +1,8 @@ # Forecast the usage of a GitLab namespace -## TBD +In this lab we will us the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data in the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in this lab. - [Prerequisites](#prerequisites) +- [Prepare for forecast](#prepare-for-forecast) - [Perform a forecast](#perform-a-forecast) - [Review forecast output](#review-forecast-output) @@ -10,6 +11,10 @@ 1. Followed [steps](../gitlab#readme) to set up your codespace environment. 2. Completed the [configure lab](../gitlab/valet-configure-lab.md) +## Prepare for forecast +Before we can run the forecast we need to answer a few questions so we can construct the correct command +1) What namespace do we want to run the forecast for? __Valet__. +2) What timeframe do we want to forecast? ## Perform a forecast ### TBD From 483df416256737a0d6705344a389630b0f454fbd Mon Sep 17 00:00:00 2001 From: j-dunham Date: Wed, 17 Aug 2022 14:17:01 -0400 Subject: [PATCH 03/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index f605cb2..53e2a37 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -1,5 +1,5 @@ # Forecast the usage of a GitLab namespace -In this lab we will us the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data in the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in this lab. +In this lab we will us the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data from the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. - [Prerequisites](#prerequisites) - [Prepare for forecast](#prepare-for-forecast) @@ -13,8 +13,9 @@ In this lab we will us the `forecast` command to forecast potential GitHub Actio ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command -1) What namespace do we want to run the forecast for? __Valet__. -2) What timeframe do we want to forecast? +1) What namespace do we want to run the forecast for? __Valet__. This is the only group in the demo GitLab instance. +2) What is the date we want to start forecasting from? __08-02-2022__. This is around but before the time the data was populated on our demo GitLab instance. +3) Where do we want to store the results? __./tmp/forecast__. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace. ## Perform a forecast ### TBD From 33679df2cefc9b56c3e1165ef0c247e21cde7f69 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Wed, 17 Aug 2022 16:20:51 -0400 Subject: [PATCH 04/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 95 +++++++++++++++++++++++++++++++++--- 1 file changed, 89 insertions(+), 6 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 53e2a37..1d44fae 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -10,17 +10,100 @@ In this lab we will us the `forecast` command to forecast potential GitHub Actio 1. Followed [steps](../gitlab#readme) to set up your codespace environment. 2. Completed the [configure lab](../gitlab/valet-configure-lab.md) +3. Ran the setup script in the terminal to make sure the GitLab instance is ready + ``` + source gitlab/bootstrap/setup.sh + ``` ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command -1) What namespace do we want to run the forecast for? __Valet__. This is the only group in the demo GitLab instance. -2) What is the date we want to start forecasting from? __08-02-2022__. This is around but before the time the data was populated on our demo GitLab instance. -3) Where do we want to store the results? __./tmp/forecast__. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace. +1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ +2) What is the date we want to start forecasting from? __08-02-2022. This is around but before the time the data was populated on our demo GitLab instance.__ +3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ ## Perform a forecast -### TBD - - +- Construct the command using the values from the questions above, it should look like: +``` +gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 08-02-2022 +``` +- Run the command in the codespace terminal +- Verify that you command outputed a similar result + ![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) ## Review forecast output +We will now review the `forecast` outputed files +### forecast_report.md +- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and click to open + ![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png) +- The file should be similar to this +
+ example forecast_report.md + + ``` + # Forecast report for [GitLab](http://localhost/valet) + + - Valet version: **0.1.0.13432(03b5bc9370a8f0073c0cc1a4b25f6b81d0005c0f)** + - Performed at: **8/17/22 at 20:00** + - Date range: **2/8/22 - 8/17/22** + + ## Total + + - Job count: **57** + - Pipeline count: **15** + + - Execution time + + - Total: **135 minutes** + - Median: **0 minutes** + - P90: **7 minutes** + - Min: **0 minutes** + - Max: **10 minutes** + + - Queue time + + - Median: **0 minutes** + - P90: **5 minutes** + - Min: **0 minutes** + - Max: **42 minutes** + + - Concurrent jobs + + - Median: **0** + - P90: **0** + - Min: **0** + - Max: **9** + + --- + + ## gitlab-runner + + - Job count: **57** + - Pipeline count: **15** + + - Execution time + + - Total: **135 minutes** + - Median: **0 minutes** + - P90: **7 minutes** + - Min: **0 minutes** + - Max: **10 minutes** + + - Queue time + + - Median: **0 minutes** + - P90: **5 minutes** + - Min: **0 minutes** + - Max: **42 minutes** + + - Concurrent jobs + + - Median: **0** + - P90: **0** + - Min: **0** + - Max: **9** + + > Note: Concurrent jobs are calculated by using a sliding window of 1m 0s. + ``` + +
### TBD From 159aa07cf9b57cebd7e1fe0adc29b55dcad56730 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Wed, 17 Aug 2022 16:46:19 -0400 Subject: [PATCH 05/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 1d44fae..9acc5d3 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -9,14 +9,14 @@ In this lab we will us the `forecast` command to forecast potential GitHub Actio ## Prerequisites 1. Followed [steps](../gitlab#readme) to set up your codespace environment. -2. Completed the [configure lab](../gitlab/valet-configure-lab.md) -3. Ran the setup script in the terminal to make sure the GitLab instance is ready +2. Completed the [configure lab](../gitlab/valet-configure-lab.md). +3. Ran the setup script in the terminal to make sure the GitLab instance is ready . ``` source gitlab/bootstrap/setup.sh ``` ## Prepare for forecast -Before we can run the forecast we need to answer a few questions so we can construct the correct command +Before we can run the forecast we need to answer a few questions so we can construct the correct command. 1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ 2) What is the date we want to start forecasting from? __08-02-2022. This is around but before the time the data was populated on our demo GitLab instance.__ 3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ @@ -26,20 +26,19 @@ Before we can run the forecast we need to answer a few questions so we can const ``` gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 08-02-2022 ``` -- Run the command in the codespace terminal -- Verify that you command outputed a similar result +- Run the command in the codespace terminal. +- Verify that you command outputed a similar result. ![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) -## Review forecast output -We will now review the `forecast` outputed files -### forecast_report.md -- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and click to open +## Review forecast report +Lets open the forecast report and look at the calculated metrics. +- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and click to open. ![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png) -- The file should be similar to this +- The file should be similar to this.
example forecast_report.md - ``` + ```yaml # Forecast report for [GitLab](http://localhost/valet) - Valet version: **0.1.0.13432(03b5bc9370a8f0073c0cc1a4b25f6b81d0005c0f)** @@ -104,6 +103,11 @@ We will now review the `forecast` outputed files > Note: Concurrent jobs are calculated by using a sliding window of 1m 0s. ``` + ### Metrics + + ### Sections + - `Total` shows the metrics using all of the known runners. + - `gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading.
### TBD From a10aa250b73537987c1b4fd21ae7be37584722bf Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 09:33:15 -0400 Subject: [PATCH 06/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 9acc5d3..095fe22 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -1,10 +1,10 @@ # Forecast the usage of a GitLab namespace -In this lab we will us the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data from the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. +In this lab we will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data from the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. - [Prerequisites](#prerequisites) - [Prepare for forecast](#prepare-for-forecast) - [Perform a forecast](#perform-a-forecast) -- [Review forecast output](#review-forecast-output) +- [Review forecast report](#review-forecast-report) ## Prerequisites @@ -27,18 +27,17 @@ Before we can run the forecast we need to answer a few questions so we can const gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 08-02-2022 ``` - Run the command in the codespace terminal. -- Verify that you command outputed a similar result. +- Verify that the command output is similar to this. ![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) ## Review forecast report -Lets open the forecast report and look at the calculated metrics. -- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and click to open. +Now we will open the forecast report and review the calculated metrics. +- From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and right-click, and select __Open Preview__. ![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png) - The file should be similar to this.
example forecast_report.md - ```yaml # Forecast report for [GitLab](http://localhost/valet) - Valet version: **0.1.0.13432(03b5bc9370a8f0073c0cc1a4b25f6b81d0005c0f)** @@ -102,12 +101,14 @@ Lets open the forecast report and look at the calculated metrics. - Max: **9** > Note: Concurrent jobs are calculated by using a sliding window of 1m 0s. - ``` - ### Metrics - - ### Sections - - `Total` shows the metrics using all of the known runners. - - `gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. - +
+ +### Metrics + +### Sections +- `Total` shows the metrics using all of the known runners. +- `gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. + + ### TBD From c1f50521a713fa99abe4c12dde26a78335641563 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 09:51:28 -0400 Subject: [PATCH 07/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 095fe22..c0bcc30 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -105,7 +105,13 @@ Now we will open the forecast report and review the calculated metrics. ### Metrics - +| Name | Description | +| ----- | ----------- | +| Median | The __middle__ value | +| P90 | 90% of the values are less than or equal too | +| Min | The lowest value | +| Max | The highest value | + ### Sections - `Total` shows the metrics using all of the known runners. - `gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. From 8727b195a7460c5bc60cab72cc5eb74058ad7ffe Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 11:12:31 -0400 Subject: [PATCH 08/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 41 ++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index c0bcc30..a1695fc 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -18,7 +18,7 @@ In this lab we will use the `forecast` command to forecast potential GitHub Acti ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command. 1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ -2) What is the date we want to start forecasting from? __08-02-2022. This is around but before the time the data was populated on our demo GitLab instance.__ +2) What is the date we want to start forecasting from? __08-02-2022. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understand of the typical usage. Too little data and the metrics might not give a accurrate picture__ 3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ ## Perform a forecast @@ -31,7 +31,7 @@ gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet - ![forecast_output](https://user-images.githubusercontent.com/18723510/185232893-1ed46bca-f310-47dc-804c-40c13737f231.png) ## Review forecast report -Now we will open the forecast report and review the calculated metrics. +Open the forecast report and review the calculated metrics. - From the codespace explorer pane find `./tmp/forecast_reports/forecast_report.md` and right-click, and select __Open Preview__. ![forecast_explorer](https://user-images.githubusercontent.com/18723510/185234641-948a551b-316f-4cce-9e7d-4c078ae11a04.png) - The file should be similar to this. @@ -104,7 +104,7 @@ Now we will open the forecast report and review the calculated metrics. -### Metrics +### Metric Definitions | Name | Description | | ----- | ----------- | | Median | The __middle__ value | @@ -112,9 +112,38 @@ Now we will open the forecast report and review the calculated metrics. | Min | The lowest value | | Max | The highest value | -### Sections -- `Total` shows the metrics using all of the known runners. -- `gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. +### Total Section +- This section shows the metrics for all of the jobs run in projects in the `valet` group during the time between today and 08/02/2022. + ## Total + + - Job count: **57** + - Pipeline count: **15** + --- + We can see we ran 15 pipelines that contained 57 jobs. The number of jobs is expected to be larger than pipelines because a pipeline is typically a collection of jobs. For example `basic-pipeline-example` contains 6 jobs + ![basic-pipeline-jobs](https://user-images.githubusercontent.com/18723510/185423928-ec1b13b5-01fc-4e48-bbe5-0a77be7cecea.png) + +- `Execution time` shows the metrics for the time a job took to ran. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quick job took less than a minute and was rounded down to 0. + - Execution time + - Total: **135 minutes** + - Median: **0 minutes** + - P90: **7 minutes** + - Min: **0 minutes** + - Max: **10 minutes** + +- `Queue time` shows the metrics for how long jobs waited for a runner to be available. + - Queue time + - Median: **0 minutes** + - P90: **5 minutes** + - Min: **0 minutes** + - Max: **42 minutes** +- `Concurent jobs` show the metrics for how many jobs were run at the same time. + - Concurrent jobs + - Median: **0** + - P90: **0** + - Min: **0** + - Max: **9** +### Runner Section +- The data`gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. ### TBD From 622af908f9dd706d8e24287fba72aac703deb337 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 13:29:18 -0400 Subject: [PATCH 09/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index a1695fc..715dfd7 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -113,7 +113,7 @@ Open the forecast report and review the calculated metrics. | Max | The highest value | ### Total Section -- This section shows the metrics for all of the jobs run in projects in the `valet` group during the time between today and 08/02/2022. +- This section shows the metrics for all of the jobs run in projects contained in the `valet` group, from 08/02/2022 to the time the command was executed. ## Total - Job count: **57** @@ -122,7 +122,7 @@ Open the forecast report and review the calculated metrics. We can see we ran 15 pipelines that contained 57 jobs. The number of jobs is expected to be larger than pipelines because a pipeline is typically a collection of jobs. For example `basic-pipeline-example` contains 6 jobs ![basic-pipeline-jobs](https://user-images.githubusercontent.com/18723510/185423928-ec1b13b5-01fc-4e48-bbe5-0a77be7cecea.png) -- `Execution time` shows the metrics for the time a job took to ran. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quick job took less than a minute and was rounded down to 0. +- `Execution time` shows the metrics for the time a job __took to run__. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quick job took less than a minute and was rounded down to 0. - Execution time - Total: **135 minutes** - Median: **0 minutes** @@ -130,20 +130,20 @@ Open the forecast report and review the calculated metrics. - Min: **0 minutes** - Max: **10 minutes** -- `Queue time` shows the metrics for how long jobs waited for a runner to be available. +- `Queue time` shows the metrics for how long jobs __waited__ for a runner to be available. - Queue time - Median: **0 minutes** - P90: **5 minutes** - Min: **0 minutes** - Max: **42 minutes** -- `Concurent jobs` show the metrics for how many jobs were run at the same time. +- `Concurent jobs` show the metrics for how many jobs were run at the __same time__. - Concurrent jobs - Median: **0** - P90: **0** - Min: **0** - Max: **9** -### Runner Section -- The data`gitlab-runner` shows the metrics for the runners in the gitlab-runner group, if there were other groups they would show here under their own group heading. +### Runner Group Sections +- The preceeding section show the same metrics as the `Total` section ### TBD From 91781b2553406f37cac5d9f3710da3c9d18fe005 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 13:59:59 -0400 Subject: [PATCH 10/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 715dfd7..af762d1 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -143,7 +143,29 @@ Open the forecast report and review the calculated metrics. - Min: **0** - Max: **9** ### Runner Group Sections -- The preceeding section show the same metrics as the `Total` section - - -### TBD +- The preceeding section show the same metrics as the `Total` section, but grouped by runner groups +- In this case we only have one runner group `gitlab-runner` so the metrics will match the `Total` section + ## gitlab-runner + + - Job count: **57** + - Pipeline count: **15** + + - Execution time + - Total: **135 minutes** + - Median: **0 minutes** + - P90: **7 minutes** + - Min: **0 minutes** + - Max: **10 minutes** + + - Queue time + - Median: **0 minutes** + - P90: **5 minutes** + - Min: **0 minutes** + - Max: **42 minutes** + + - Concurrent jobs + - Median: **0** + - P90: **0** + - Min: **0** + - Max: **9** + From 3a40e2cb3e8ecd1279d3aa81554ce458e01772a6 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 16:57:20 -0400 Subject: [PATCH 11/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index af762d1..bd10fb8 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -1,16 +1,17 @@ # Forecast the usage of a GitLab namespace -In this lab we will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from historical pipeline data from the GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. +In this lab we will use the `forecast` command to forecast potential GitHub Actions usage by computing metrics from the historical pipeline data in our GitLab instance. The metrics will be stored on disk in a markdown file and include job metrics for execution time, queue time, and concurrency. We will look at each of these metrics in more depth later in this lab. - [Prerequisites](#prerequisites) - [Prepare for forecast](#prepare-for-forecast) - [Perform a forecast](#perform-a-forecast) - [Review forecast report](#review-forecast-report) +- [Review additional files](#review-additional-files) ## Prerequisites 1. Followed [steps](../gitlab#readme) to set up your codespace environment. 2. Completed the [configure lab](../gitlab/valet-configure-lab.md). -3. Ran the setup script in the terminal to make sure the GitLab instance is ready . +3. Ran the setup script in the terminal to make sure the GitLab instance is ready. ``` source gitlab/bootstrap/setup.sh ``` @@ -18,11 +19,11 @@ In this lab we will use the `forecast` command to forecast potential GitHub Acti ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command. 1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ -2) What is the date we want to start forecasting from? __08-02-2022. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understand of the typical usage. Too little data and the metrics might not give a accurrate picture__ +2) What is the date we want to start forecasting from? __08-02-2022. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understand of the typical usage. Too little data and the metrics might not give an accurrate picture__ 3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ ## Perform a forecast -- Construct the command using the values from the questions above, it should look like: +- Using the answers above we get the following `forecast` command: ``` gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 08-02-2022 ``` @@ -122,7 +123,7 @@ Open the forecast report and review the calculated metrics. We can see we ran 15 pipelines that contained 57 jobs. The number of jobs is expected to be larger than pipelines because a pipeline is typically a collection of jobs. For example `basic-pipeline-example` contains 6 jobs ![basic-pipeline-jobs](https://user-images.githubusercontent.com/18723510/185423928-ec1b13b5-01fc-4e48-bbe5-0a77be7cecea.png) -- `Execution time` shows the metrics for the time a job __took to run__. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quick job took less than a minute and was rounded down to 0. +- `Execution time` shows the metrics for the time a job __took to run__. Looking closer we can see during our forecast timeframe the total job run time was 135 minutes with 90% of the jobs finishing under 7 minutes, and the longest job taking 10 minutes. The `min` is 0 because the quickest job took less than a minute and was rounded down to 0. - Execution time - Total: **135 minutes** - Median: **0 minutes** @@ -136,15 +137,15 @@ Open the forecast report and review the calculated metrics. - P90: **5 minutes** - Min: **0 minutes** - Max: **42 minutes** -- `Concurent jobs` show the metrics for how many jobs were run at the __same time__. +- `Concurrent jobs` show the metrics for how many jobs were run at the __same time__. - Concurrent jobs - Median: **0** - P90: **0** - Min: **0** - Max: **9** ### Runner Group Sections -- The preceeding section show the same metrics as the `Total` section, but grouped by runner groups -- In this case we only have one runner group `gitlab-runner` so the metrics will match the `Total` section +- The preceding section show the same metrics as the `Total` section, but are grouped by runner group +- In this case we only have one runner group `gitlab-runner` so the metrics match the `Total` section. If there were different groups we could possibly identify runner types that needed to be increased or decreased when moving to GitHub Actions ## gitlab-runner - Job count: **57** @@ -169,3 +170,7 @@ Open the forecast report and review the calculated metrics. - Min: **0** - Max: **9** +## Review additional files +When Valet performs a `forecast` the main file of interest is the `forecast_report.md` but there are some additional files created in the results directory, that might help if troubleshooting is needed. In the `jobs` directory a `.json` file was created. This file is a simple JSON file containing the job objects retrieved from the GitLab instance. This file is generally not something you would look at but if you thought you were missing a particular job or jobs in the metrics, you could look here and confirm. Valet also generates logs and writes them to the `log` directory. Again, this is not something you would typically look at but might be useful to know. + + From e44dbce4e750da31b94a21c471c36307d9331079 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 17:44:50 -0400 Subject: [PATCH 12/19] Update gitlab/valet-forecast-lab.md Co-authored-by: Luke Engle <99493186+luke-engle@users.noreply.github.com> --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index bd10fb8..0a0a961 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -19,7 +19,7 @@ In this lab we will use the `forecast` command to forecast potential GitHub Acti ## Prepare for forecast Before we can run the forecast we need to answer a few questions so we can construct the correct command. 1) What namespace do we want to run the forecast for? __valet. This is the only group in the demo GitLab instance.__ -2) What is the date we want to start forecasting from? __08-02-2022. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understand of the typical usage. Too little data and the metrics might not give an accurrate picture__ +2) What is the date we want to start forecasting from? __2022-08-02. This date is before the time the data was populated on our demo GitLab instance. In practice, this should be a date that will give you enough data to get a good understanding of the typical usage. Too little data and the metrics might not give an accurate picture__ 3) Where do we want to store the results? __./tmp/forecast_reports. This can be any valid path on the system, but for simplicity it is recommend to use a directory in the root of the codespace workspace.__ ## Perform a forecast From 4e53e084f5ccd8fcf2b629137c6effecf8768a62 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 17:44:56 -0400 Subject: [PATCH 13/19] Update gitlab/valet-forecast-lab.md Co-authored-by: Luke Engle <99493186+luke-engle@users.noreply.github.com> --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 0a0a961..a5c009a 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -25,7 +25,7 @@ Before we can run the forecast we need to answer a few questions so we can const ## Perform a forecast - Using the answers above we get the following `forecast` command: ``` -gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 08-02-2022 +gh valet forecast gitlab --output-dir ./tmp/forecast_reports --namespace valet --start-date 2022-08-02 ``` - Run the command in the codespace terminal. - Verify that the command output is similar to this. From a7930a9ec724bdabd90c26f3505f4f0987b3aeb8 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 17:45:03 -0400 Subject: [PATCH 14/19] Update gitlab/valet-forecast-lab.md Co-authored-by: Luke Engle <99493186+luke-engle@users.noreply.github.com> --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index a5c009a..c56d28e 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -109,7 +109,7 @@ Open the forecast report and review the calculated metrics. | Name | Description | | ----- | ----------- | | Median | The __middle__ value | -| P90 | 90% of the values are less than or equal too | +| P90 | 90% of the values are less than or equal to | | Min | The lowest value | | Max | The highest value | From e4521e38d70e7c204dd63f44cbcda7bb7126a546 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 17:45:30 -0400 Subject: [PATCH 15/19] Update gitlab/valet-forecast-lab.md Co-authored-by: Luke Engle <99493186+luke-engle@users.noreply.github.com> --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index c56d28e..c81f679 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -144,7 +144,7 @@ Open the forecast report and review the calculated metrics. - Min: **0** - Max: **9** ### Runner Group Sections -- The preceding section show the same metrics as the `Total` section, but are grouped by runner group +- The preceding section shows the same metrics as the `Total` section, but are grouped by runner group. A runner group is a machine (or group of machines) that each job runs on - In this case we only have one runner group `gitlab-runner` so the metrics match the `Total` section. If there were different groups we could possibly identify runner types that needed to be increased or decreased when moving to GitHub Actions ## gitlab-runner From d5484f7e94c74d1cec3e24a30583e90bec93aad1 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Thu, 18 Aug 2022 17:47:08 -0400 Subject: [PATCH 16/19] Update gitlab/valet-forecast-lab.md Co-authored-by: Luke Engle <99493186+luke-engle@users.noreply.github.com> --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index c81f679..213eabf 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -114,7 +114,7 @@ Open the forecast report and review the calculated metrics. | Max | The highest value | ### Total Section -- This section shows the metrics for all of the jobs run in projects contained in the `valet` group, from 08/02/2022 to the time the command was executed. +- This section shows the metrics for all of the jobs run in projects contained in the `valet` namespace, from 08/02/2022 to the time the command was executed. ## Total - Job count: **57** From 0fa8379a34c8836f9b8147011719dde86dafe4fb Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 19 Aug 2022 11:11:12 -0400 Subject: [PATCH 17/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 213eabf..fd55b1c 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -6,6 +6,8 @@ In this lab we will use the `forecast` command to forecast potential GitHub Acti - [Perform a forecast](#perform-a-forecast) - [Review forecast report](#review-forecast-report) - [Review additional files](#review-additional-files) +- [Forecasting multiple providers](#forecasting-multiple-providers) +- [Next steps](#next-steps) ## Prerequisites @@ -170,7 +172,15 @@ Open the forecast report and review the calculated metrics. - Min: **0** - Max: **9** -## Review additional files -When Valet performs a `forecast` the main file of interest is the `forecast_report.md` but there are some additional files created in the results directory, that might help if troubleshooting is needed. In the `jobs` directory a `.json` file was created. This file is a simple JSON file containing the job objects retrieved from the GitLab instance. This file is generally not something you would look at but if you thought you were missing a particular job or jobs in the metrics, you could look here and confirm. Valet also generates logs and writes them to the `log` directory. Again, this is not something you would typically look at but might be useful to know. +## Forecasting multiple providers +If we examine the help for the `forecast` command by running `gh valet forecast --help` we can see a new option `--source-file-path` +![forecast-help](https://user-images.githubusercontent.com/18723510/185643813-8a56710a-b5e6-4f62-a073-67d548cade1c.png) +Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as GitLab and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a glob pattern that will match all of the data files we want to include and combines them in a report. Most likely, the glob pattern will match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the value we used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`! +- run `gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast` +- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider it only operates the data files. +![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png) + +## Next steps +This concludes the Valet labs for GitLab, if you are interested exploring the power of Valet more. You can leverage the demo GitLab Instance and modify and add new projects that more closely match your needs and try out the commands again! From 0548019005ec9128662ba6d92ea12fe60dfe1945 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 19 Aug 2022 11:14:18 -0400 Subject: [PATCH 18/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index fd55b1c..86839e6 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -179,7 +179,7 @@ If we examine the help for the `forecast` command by running `gh valet forecast Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as GitLab and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a glob pattern that will match all of the data files we want to include and combines them in a report. Most likely, the glob pattern will match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the value we used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`! - run `gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast` -- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider it only operates the data files. +- Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider, it only operates on the data files it finds. ![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png) ## Next steps From 81863ea058b40cbc779fa923042b014eb6f0a525 Mon Sep 17 00:00:00 2001 From: j-dunham Date: Fri, 19 Aug 2022 15:01:23 -0400 Subject: [PATCH 19/19] Update valet-forecast-lab.md --- gitlab/valet-forecast-lab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitlab/valet-forecast-lab.md b/gitlab/valet-forecast-lab.md index 86839e6..09aacf5 100644 --- a/gitlab/valet-forecast-lab.md +++ b/gitlab/valet-forecast-lab.md @@ -177,7 +177,7 @@ If we examine the help for the `forecast` command by running `gh valet forecast ![forecast-help](https://user-images.githubusercontent.com/18723510/185643813-8a56710a-b5e6-4f62-a073-67d548cade1c.png) -Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as GitLab and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a glob pattern that will match all of the data files we want to include and combines them in a report. Most likely, the glob pattern will match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the value we used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`! +Using `--source-file-path` we can combine data from multiple forecast runs into a single report. This becomes useful if we are using multiple CI/CD providers, such as GitLab and Jenkins, and wanted to get a holistic view of the runner usage across the providers. The way this works is the forecast command creates a `.json` file in a `jobs` directory for each command execution. The `--source-file-path` takes a space-delimited list of data file paths or a glob pattern that will match all of the data files we want to include and combine into a single report. We will use a glob pattern, which in general should match `OUTPUT_DIR/**/jobs/*.json` where the `OUTPUT_DIR` is the previous value used for `--output-dir`, which in this lab was `./tmp/forecast_reports`. We do not have multiple providers but we can still try it out because we have a data file at `tmp/forecast_reports/jobs/`! - run `gh valet forecast --source-file-path tmp/**/jobs/*.json -o tmp/combined-forecast` - Now we have a new report that was generated from all the data files that matched the glob pattern. Note this command does not introspect the CI/CD provider, it only operates on the data files it finds. ![combined-report](https://user-images.githubusercontent.com/18723510/185647504-ada354ed-4ac7-4d43-b2d5-e5f9cd1656dd.png)