Compare commits

..

1 Commits

Author SHA1 Message Date
Image generation service account. d55730c808 Updating readme file for win19 version 20230830.3.0 2023-08-31 08:37:48 +00:00
681 changed files with 21884 additions and 24010 deletions
+4 -9
View File
@@ -36,22 +36,17 @@ body:
attributes:
label: Runner images affected
options:
- label: Ubuntu 20.04
- label: Ubuntu 22.04
- label: Ubuntu 24.04
- label: Ubuntu Slim
- label: macOS 11
- label: macOS 12
- label: macOS 13
- label: macOS 13 Arm64
- label: macOS 14
- label: macOS 14 Arm64
- label: macOS 15
- label: macOS 15 Arm64
- label: macOS 26 Arm64
- label: Windows Server 2019
- label: Windows Server 2022
- label: Windows Server 2025
- type: textarea
attributes:
label: Mitigation ways
description: Steps or options for impact mitigation
validations:
required: true
+3 -9
View File
@@ -19,19 +19,13 @@ body:
attributes:
label: Runner images affected
options:
- label: Ubuntu 20.04
- label: Ubuntu 22.04
- label: Ubuntu 24.04
- label: Ubuntu Slim
- label: macOS 11
- label: macOS 12
- label: macOS 13
- label: macOS 13 Arm64
- label: macOS 14
- label: macOS 14 Arm64
- label: macOS 15
- label: macOS 15 Arm64
- label: macOS 26 Arm64
- label: Windows Server 2019
- label: Windows Server 2022
- label: Windows Server 2025
- type: textarea
attributes:
label: Image version and build link
+4 -10
View File
@@ -57,19 +57,13 @@ body:
attributes:
label: Runner images where you need the tool
options:
- label: Ubuntu 20.04
- label: Ubuntu 22.04
- label: Ubuntu 24.04
- label: Ubuntu Slim
- label: macOS 11
- label: macOS 12
- label: macOS 13
- label: macOS 13 Arm64
- label: macOS 14
- label: macOS 14 Arm64
- label: macOS 15
- label: macOS 15 Arm64
- label: macOS 26 Arm64
- label: Windows Server 2019
- label: Windows Server 2022
- label: Windows Server 2025
- type: textarea
attributes:
label: Can this tool be installed during the build?
@@ -82,4 +76,4 @@ body:
- type: input
attributes:
label: Are you willing to submit a PR?
description: We accept contributions!
description: We accept contributions!
-39
View File
@@ -1,39 +0,0 @@
# GitHub Copilot Instructions for Actions Runner Images Repository
## Scope and goals
- This repository serves as the source for building GitHub Actions runner and Azure DevOps agent images for Windows, Ubuntu, and macOS. You can find exact versions in the [Available Images](../README.md#available-images) section of README.md. Windows and Ubuntu images build on Azure infrastructure using Packer; macOS images use Anka virtualization.
- Emphasize best practices for contributing to open-source projects, including code style, commit messages, and pull request etiquette.
- Prefer clarity and correctness over creativity. If information is missing, ask clarifying questions or insert TODOs instead of guessing.
## Code and command instructions
- Follow the code style guide in [CONTRIBUTING.md](../CONTRIBUTING.md#code-style-guide) for Bash and PowerShell scripts, including naming conventions, file structure, and indentation rules.
- Focus on re-using helpers when writing scripts. Windows, Linux and Ubuntu scripts have helper functions available to simplify installation and validation.
- Always confirm versions and installation paths against existing toolset files and installation scripts.
## Output format
- Use GitHub Flavored Markdown only. Avoid raw HTML unless necessary.
- One H1 (`#`) per page, followed by logical, sequential headings (`##`, `###`, …).
- Use fenced code blocks with language identifiers (` ```bash `, ` ```json `, ` ```yaml `, etc.).
- Use blockquote callouts for notes:
> [!NOTE] Context or nuance
> [!TIP] Helpful hint
> [!WARNING] Risks or breaking changes
> [!IMPORTANT] Critical requirement for functionality
## Style and tone
- Audience: Open-source contributors, GitHub Actions maintainers, and developers building custom runner images. Assume familiarity with CI/CD concepts, Packer, and basic infrastructure provisioning, but explain platform-specific details (Azure for Windows/Ubuntu, Anka for macOS) when relevant.
- Voice: Second person ("you"), active voice, imperative for operational steps.
- Be concise: short paragraphs and sentences. Prefer lists and step-by-steps, especially for operational procedures and troubleshooting.
- Use inclusive, accessible language. Avoid idioms, sarcasm, and culturally specific references.
- English: en-US (spelling, punctuation, and units).
## Safety and integrity
- Do not expose sensitive credentials (API tokens, Azure subscription IDs, etc.) in code examples.
- Do not fabricate tool versions, installation paths, or software availability without verifying against toolset files or actual installation scripts.
- Always call out assumptions and limitations explicitly, especially for changes affecting runner image behavior or software availability.
- If ambiguous requests are made about image modifications, ask clarifying questions about target OS, tool versions, and compatibility requirements before proceeding.
@@ -1,22 +0,0 @@
name: Check Outdated Version Pinning
on:
schedule:
- cron: '0 12 * * 1' # Run at 12:00 UTC every Monday
permissions:
issues: write
contents: read
jobs:
check-pinning-dates:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Validate JSON Schema
shell: pwsh
run: ./helpers/CheckOutdatedVersionPinning.ps1
env:
GH_TOKEN: ${{ github.token }}
+5 -5
View File
@@ -34,17 +34,17 @@ jobs:
strategy:
fail-fast: false
matrix:
language: [ 'python' ]
language: [ 'python', 'ruby' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support
steps:
- name: Checkout repository
uses: actions/checkout@v5
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -55,7 +55,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
uses: github/codeql-action/autobuild@v2
# ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -69,4 +69,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v2
+8 -5
View File
@@ -10,12 +10,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Create release for ${{ github.event.client_payload.ReleaseBranchName }}
uses: ncipollo/release-action@b7eabc95ff50cbeeedec83973935c8f306dfcd0b #v1.20.0
uses: actions/create-release@v1.1.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag: ${{ github.event.client_payload.ReleaseBranchName }}
name: ${{ github.event.client_payload.ReleaseTitle }}
tag_name: ${{ github.event.client_payload.ReleaseBranchName }}
release_name: ${{ github.event.client_payload.ReleaseTitle }}
body: ${{ github.event.client_payload.ReleaseBody }}
prerelease: ${{ github.event.client_payload.Prerelease }}
commit: ${{ github.event.client_payload.Commitish }}
allowUpdates: true
commitish: ${{ github.event.client_payload.Commitish }}
+10 -24
View File
@@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v3
with:
fetch-depth: 0
@@ -18,44 +18,30 @@ jobs:
run: |
git checkout ${{ github.event.client_payload.ReleaseBranchName }}
git branch ${{ github.event.client_payload.ReleaseBranchName }}-docs
git push origin ${{ github.event.client_payload.ReleaseBranchName }}-docs --force
git push origin ${{ github.event.client_payload.ReleaseBranchName }}-docs
- name: Create pull request for ${{ github.event.client_payload.ReleaseBranchName }}
id: create-pr
uses: actions/github-script@v8
uses: actions/github-script@v2
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const pulls = await github.rest.pulls.list({
let response = await github.pulls.create({
owner: context.repo.owner,
repo: context.repo.repo,
head: `${context.repo.owner}:${{ github.event.client_payload.ReleaseBranchName }}-docs`,
title: "${{ github.event.client_payload.PullRequestTitle }}",
head: "${{ github.event.client_payload.ReleaseBranchName }}-docs",
base: "${{ github.event.client_payload.PullRequestBase }}",
state: 'open'
body: `${{ github.event.client_payload.PullRequestBody }}`
});
if (pulls.data.length > 0) {
console.log(`Pull request already exists: ${pulls.data[0].html_url}`);
return pulls.data[0].number;
} else {
console.log('No existing pull request found, creating new one');
let response = await github.rest.pulls.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: "${{ github.event.client_payload.PullRequestTitle }}",
head: "${{ github.event.client_payload.ReleaseBranchName }}-docs",
base: "${{ github.event.client_payload.PullRequestBase }}",
body: `${{ github.event.client_payload.PullRequestBody }}`
});
return response.data.number;
}
return response.data.number
- name: Request reviewers
uses: actions/github-script@v8
uses: actions/github-script@v2
with:
github-token: ${{secrets.PRAPPROVAL_SECRET}}
script: |
github.rest.pulls.requestReviewers({
github.pulls.requestReviewers({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: ${{ steps.create-pr.outputs.result }},
+37 -78
View File
@@ -1,112 +1,71 @@
name: Create SBOM for the release
run-name: Collecting SBOM for ${{ github.event.client_payload.agentSpec || 'unknown image' }} - ${{ github.event.client_payload.imageVersion || 'unknown version' }}
name: Create and upload a SBOM to release assets
# Inherited variables:
# github.event.client_payload.agentSpec - Current YAML Label
# github.event.client_payload.ReleaseID - Current release ID
# github.event.client_payload.imageVersion - AzDO image version "major.minor"
# github.event.client_payload.ReleaseBranchName - Necessary to identify workflow run
#
# Current SYFT tool issues:
# macOS (major): prompt privilegies that blocking process indefinetely (https://github.com/anchore/syft/issues/1367)
on:
repository_dispatch:
types: [generate-sbom]
defaults:
run:
shell: pwsh
jobs:
sbom-check:
outputs:
check_status: ${{ steps.check.outputs.status }}
runs-on: ubuntu-latest
env:
RELEASE_ID: ${{ github.event.client_payload.ReleaseID }}
steps:
- name: Check SBOM asset for release ${{ env.RELEASE_ID }}
id: check
shell: pwsh
run: |
$apiUrl = "https://api.github.com/repos/actions/runner-images/releases/$env:RELEASE_ID"
$response = Invoke-RestMethod -Uri $apiUrl -Method Get -SkipHttpErrorCheck
if ($response.message -ilike "Not Found") {
echo "status=release_not_found" >> $env:GITHUB_OUTPUT
Write-Error "Release $env:RELEASE_ID wasn't found"
exit 1
}
foreach ($asset in $response.assets) {
if ($asset.name -like '*sbom*') {
echo "status=sbom_exists" >> $env:GITHUB_OUTPUT
return "Release $env:RELEASE_ID already contains a SBOM"
}
}
Write-Host "Release has been found, SBOM is not attached, starting generation."
echo "status=okay" >> $env:GITHUB_OUTPUT
building-sbom:
needs: sbom-check
if: ${{ needs.sbom-check.outputs.check_status == 'okay' }}
#Checking image version on available runner
version-check:
runs-on: ${{ github.event.client_payload.agentSpec }}
env:
AGENT_SPEC: ${{ github.event.client_payload.agentSpec }}
RELEASE_ID: ${{ github.event.client_payload.ReleaseID }}
IMAGE_VERSION: ${{ github.event.client_payload.imageVersion }}
steps:
- name: Available image version check
- name: Available image version check for ${{ github.event.client_payload.ReleaseBranchName }}
run: |
$expectedVersion = $env:IMAGE_VERSION
$runnerVersion = $env:ImageVersion
# Split versions by dot
$expectedParts = $expectedVersion.Split('.')
$runnerParts = $runnerVersion.Split('.')
# Determine what parts to compare
$minLength = [Math]::Min($expectedParts.Length, $runnerParts.Length)
$expectedComparable = $expectedParts[0..($minLength-1)] -join '.'
$runnerComparable = $runnerParts[0..($minLength-1)] -join '.'
# Perform the comparison
if ($expectedComparable -ne $runnerComparable) {
throw "Version mismatch: Expected version '$expectedVersion' doesn't match runner version '$runnerVersion'"
$imageVersionComponents = $env:ImageVersion.Split('.')
$imageMajorVersion = $imageVersionComponents[0]
$imageMinorVersion = $imageVersionComponents[1]
if ("$imageMajorVersion.$imageMinorVersion" -ne '${{ github.event.client_payload.imageVersion }}') {
throw "Current runner $imageMajorVersion.$imageMinorVersion image version doesn't match ${{ github.event.client_payload.imageVersion }}."
}
#Install and run SYFT, compress SBOM, upload it to release assets
create-sbom:
needs: version-check
runs-on: ${{ github.event.client_payload.agentSpec }}
steps:
#Installation section
- name: Install SYFT tool on Windows
if: ${{ runner.os == 'Windows' }}
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b C:/syft
- name: Install SYFT tool on Ubuntu
if: ${{ runner.os == 'Linux' }}
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b D:/syft v0.84.1
- name: Install SYFT tool on Ubuntu or macOS
if: ${{ runner.os != 'Windows' }}
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
- name: Install SYFT v1.24.0 on macOS
if: ${{ runner.os == 'macOS' }}
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin v1.24.0
#Running section.
- name: Run SYFT on Windows
if: ${{ runner.os == 'Windows' }}
run: C:/syft/syft dir:C:/ -vv -o spdx-json=sbom.json
run: D:/syft/syft dir:C:/ -vv -o spdx-json=sbom.json
- name: Run SYFT on Ubuntu
if: ${{ runner.os == 'Linux' }}
run: syft dir:/ -vv -o spdx-json=sbom.json
- name: Run SYFT on macOS
if: ${{ runner.os == 'macOS' }}
# Skip protected folders to avoid prompt privileges that block process indefinitely (https://github.com/anchore/syft/issues/1367)
run: sudo syft dir:/ -vv -o spdx-json=sbom.json --exclude ./Users --exclude ./System/Volumes --exclude ./private
shell: bash
#Preparing artifact (raw SBOM.json is too big)
- name: Compress SBOM file
run: Compress-Archive sbom.json sbom.json.zip
- uses: actions/upload-artifact@v4
#Upload artifact action
- uses: actions/upload-artifact@v3
with:
name: sbom-${{ env.AGENT_SPEC }}-${{ env.IMAGE_VERSION }}
name: sbom-${{ github.event.client_payload.agentSpec }}-${{ github.event.client_payload.imageVersion }}
path: sbom.json.zip
if-no-files-found: warn
#Upload release asset action
#Might be changed to softprops/action-gh-release after additional check
- name: Upload release asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: "https://uploads.github.com/repos/actions/runner-images/releases/${{ env.RELEASE_ID }}/assets{?name,label}"
upload_url: "https://uploads.github.com/repos/actions/runner-images/releases/${{ github.event.client_payload.ReleaseID }}/assets{?name,label}"
asset_path: ./sbom.json.zip
asset_name: sbom.${{ env.AGENT_SPEC }}.json.zip
asset_content_type: application/zip
asset_name: sbom.${{ github.event.client_payload.agentSpec }}.json.zip
asset_content_type: application/zip
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
steps:
- name: Checkout Code
uses: actions/checkout@v5
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Lint Code Base
uses: github/super-linter/slim@v7
uses: github/super-linter/slim@v4
env:
VALIDATE_ALL_CODEBASE: false
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+5 -5
View File
@@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v3
with:
fetch-depth: 0
@@ -24,11 +24,11 @@ jobs:
sleep 30
- name: Approve pull request by GitHub-Actions bot
uses: actions/github-script@v8
uses: actions/github-script@v2
with:
github-token: ${{secrets.PRAPPROVAL_SECRET}}
script: |
github.rest.pulls.createReview({
github.pulls.createReview({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: ${{ github.event.client_payload.PullRequestNumber }},
@@ -36,11 +36,11 @@ jobs:
});
- name: Merge pull request for ${{ github.event.client_payload.ReleaseBranchName }}
uses: actions/github-script@v8
uses: actions/github-script@v2
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.rest.pulls.merge({
github.pulls.merge({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: ${{ github.event.client_payload.PullRequestNumber }},
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
steps:
- name: Checkout Repository
uses: actions/checkout@v5
uses: actions/checkout@v3
- name: Run Software Report module tests
shell: pwsh
@@ -1,119 +0,0 @@
name: Trigger Build workflow
on:
workflow_call:
inputs:
image_type:
required: true
type: string
defaults:
run:
shell: pwsh
jobs:
trigger-workflow:
runs-on: ubuntu-latest
outputs:
ci_workflow_run_id: ${{ steps.resolve.outputs.ci_workflow_run_id }}
ci_workflow_run_url: ${{ steps.resolve.outputs.ci_workflow_run_url }}
env:
CI_PR_TOKEN: ${{ secrets.CI_PR_TOKEN }}
PR_TITLE: ${{ github.event.pull_request.title }}
CI_REPO: ${{ vars.CI_REPO }}
steps:
- name: Checkout Code
uses: actions/checkout@v5
- name: Trigger Build workflow
run: |
Import-Module ./helpers/GitHubApi.psm1
$gitHubApi = Get-GithubApi -Repository "${env:CI_REPO}" -AccessToken "${env:CI_PR_TOKEN}"
$eventType = "trigger-${{ inputs.image_type }}-build"
[string] $prGuid = New-Guid
$clientPayload = @{
pr_title = "${env:PR_TITLE} - " + $prGuid
custom_repo = "${{ github.event.pull_request.head.repo.full_name }}"
custom_repo_commit_hash = "${{ github.event.pull_request.head.sha }}"
}
$gitHubApi.DispatchWorkflow($eventType, $clientPayload)
"PR_GUID=$prGuid" | Out-File -Append -FilePath $env:GITHUB_ENV
- name: Resolve Workflow Run ID
id: resolve
run: |
Import-Module ./helpers/GitHubApi.psm1
$gitHubApi = Get-GithubApi -Repository "${env:CI_REPO}" -AccessToken "${env:CI_PR_TOKEN}"
$workflowFileName = $("{0}.yml" -f "${{ inputs.image_type }}").ToLower()
$WorkflowSearchPattern = "${env:PR_GUID}"
# It might take a few minutes for the action to start
$attempt = 1
do {
$workflowRuns = $gitHubApi.GetWorkflowRuns($WorkflowFileName).workflow_runs
$workflowRunId = ($workflowRuns | Where-Object {$_.display_title -match $WorkflowSearchPattern}).id | Select-Object -First 1
if (-not ([string]::IsNullOrEmpty($workflowRunId))) {
$workflowRun = $gitHubApi.GetWorkflowRun($workflowRunId)
Write-Host "Found the workflow run with ID $workflowRunId on attempt $attempt. Workflow run link: $($workflowRun.html_url)"
"ci_workflow_run_id=$workflowRunId" | Out-File -Append -FilePath $env:GITHUB_OUTPUT
"ci_workflow_run_url=$($workflowRun.html_url)" | Out-File -Append -FilePath $env:GITHUB_OUTPUT
break
}
Write-Host "Workflow run for $WorkflowSearchPattern pattern not found on attempt $attempt."
$attempt += 1
Start-Sleep 30
} until ($attempt -eq 10)
if ([string]::IsNullOrEmpty($workflowRunId)) {
throw "Failed to find a workflow run for '$WorkflowSearchPattern'."
}
wait-completion:
runs-on: ubuntu-latest
needs: trigger-workflow
steps:
- name: Checkout Code
uses: actions/checkout@v5
- name: Wait for workflow completion
env:
CI_PR_TOKEN: ${{ secrets.CI_PR_TOKEN }}
CI_REPO: ${{ vars.CI_REPO }}
run: |
./helpers/WaitWorkflowCompletion.ps1 `
-WorkflowRunId "${{ needs.trigger-workflow.outputs.ci_workflow_run_id }}" `
-Repository "${env:CI_REPO}" `
-AccessToken "${env:CI_PR_TOKEN}"
- name: Add Summary
if: always()
run: |
"# Test Partner Image" >> $env:GITHUB_STEP_SUMMARY
"| Key | Value |" >> $env:GITHUB_STEP_SUMMARY
"| :-----------: | :--------: |" >> $env:GITHUB_STEP_SUMMARY
"| Workflow Run | [Link](${{ needs.trigger-workflow.outputs.ci_workflow_run_url }}) |" >> $env:GITHUB_STEP_SUMMARY
"| Workflow Result | $env:CI_WORKFLOW_RUN_RESULT |" >> $env:GITHUB_STEP_SUMMARY
" " >> $env:GITHUB_STEP_SUMMARY
cancel-workflow:
runs-on: ubuntu-latest
needs: [trigger-workflow, wait-completion]
if: cancelled()
steps:
- name: Checkout Code
uses: actions/checkout@v5
- name: Cancel workflow
env:
CI_PR_TOKEN: ${{ secrets.CI_PR_TOKEN }}
CI_REPO: ${{ vars.CI_REPO }}
run: |
Import-Module ./helpers/GitHubApi.psm1
$gitHubApi = Get-GithubApi -Repository "${env:CI_REPO}" -AccessToken "${env:CI_PR_TOKEN}"
$gitHubApi.CancelWorkflowRun("${{ needs.trigger-workflow.outputs.ci_workflow_run_id }}")
-20
View File
@@ -1,20 +0,0 @@
name: Trigger Ubuntu22.04 CI
run-name: Ubuntu22.04 - ${{ github.event.pull_request.title }}
on:
pull_request_target:
types: labeled
paths:
- 'images/ubuntu/**'
defaults:
run:
shell: pwsh
jobs:
Ubuntu_2204:
if: github.event.label.name == 'CI ubuntu-all' || github.event.label.name == 'CI ubuntu-2204'
uses: ./.github/workflows/trigger-ubuntu-win-build.yml
with:
image_type: 'ubuntu2204'
secrets: inherit
-20
View File
@@ -1,20 +0,0 @@
name: Trigger Ubuntu24.04 CI
run-name: Ubuntu24.04 - ${{ github.event.pull_request.title }}
on:
pull_request_target:
types: labeled
paths:
- 'images/ubuntu/**'
defaults:
run:
shell: pwsh
jobs:
Ubuntu_2404:
if: github.event.label.name == 'CI ubuntu-all' || github.event.label.name == 'CI ubuntu-2404'
uses: ./.github/workflows/trigger-ubuntu-win-build.yml
with:
image_type: 'ubuntu2404'
secrets: inherit
+5 -3
View File
@@ -10,17 +10,19 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Update release for ${{ github.event.client_payload.ReleaseBranchName }}
uses: actions/github-script@v8
uses: actions/github-script@v2
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const response = await github.rest.repos.getReleaseByTag({
const response = await github.repos.getReleaseByTag({
owner: context.repo.owner,
repo: context.repo.repo,
tag: "${{ github.event.client_payload.ReleaseBranchName }}"
});
github.rest.repos.updateRelease({
github.repos.updateRelease({
owner: context.repo.owner,
repo: context.repo.repo,
release_id: response.data.id,
@@ -1,20 +0,0 @@
name: Validate JSON Schema
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
validate-json-schema:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Validate JSON Schema
shell: pwsh
run: ./helpers/CheckJsonSchema.ps1
-20
View File
@@ -1,20 +0,0 @@
name: Trigger Windows19 CI
run-name: Windows2019 - ${{ github.event.pull_request.title }}
on:
pull_request_target:
types: labeled
paths:
- 'images/windows/**'
defaults:
run:
shell: pwsh
jobs:
Windows_2019:
if: github.event.label.name == 'CI windows-all' || github.event.label.name == 'CI windows-2019'
uses: ./.github/workflows/trigger-ubuntu-win-build.yml
with:
image_type: 'windows2019'
secrets: inherit
-20
View File
@@ -1,20 +0,0 @@
name: Trigger Windows22 CI
run-name: Windows2022 - ${{ github.event.pull_request.title }}
on:
pull_request_target:
types: labeled
paths:
- 'images/windows/**'
defaults:
run:
shell: pwsh
jobs:
Windows_2022:
if: github.event.label.name == 'CI windows-all' || github.event.label.name == 'CI windows-2022'
uses: ./.github/workflows/trigger-ubuntu-win-build.yml
with:
image_type: 'windows2022'
secrets: inherit
-20
View File
@@ -1,20 +0,0 @@
name: Trigger Windows25 CI
run-name: Windows2025 - ${{ github.event.pull_request.title }}
on:
pull_request_target:
types: labeled
paths:
- 'images/windows/**'
defaults:
run:
shell: pwsh
jobs:
Windows_2025:
if: github.event.label.name == 'CI windows-all' || github.event.label.name == 'CI windows-2025'
uses: ./.github/workflows/trigger-ubuntu-win-build.yml
with:
image_type: 'windows2025'
secrets: inherit
+3 -17
View File
@@ -1,6 +1,4 @@
{
"files.trimFinalNewlines": true,
"files.insertFinalNewline": true,
"powershell.codeFormatting.addWhitespaceAroundPipe": true,
"powershell.codeFormatting.alignPropertyValuePairs": true,
"powershell.codeFormatting.autoCorrectAliases": true,
@@ -17,18 +15,6 @@
"powershell.codeFormatting.whitespaceBetweenParameters": true,
"powershell.codeFormatting.whitespaceInsideBrace": true,
"shellcheck.exclude": [
"SC1090","SC2096"
],
"shellcheck.customArgs": [
"-x"
],
"json.schemas": [
{
"fileMatch": [
"**/toolset-*.json"
],
"url": "./schemas/toolset-schema.json"
}
]
}
"SC1091"
]
}
+31 -222
View File
@@ -1,4 +1,4 @@
# Contributing
## Contributing
[fork]: https://github.com/actions/runner-images/fork
[pr]: https://github.com//actions/runner-images/compare
@@ -6,245 +6,54 @@
Hi there! We're thrilled that you'd like to contribute to this project. Your help is essential for keeping it great.
Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [MIT](LICENSE.md) license.
Please note that this project is released with a [Contributor Code of Conduct][code-of-conduct]. By participating in this project, you agree to abide by its terms.
## Contents
- [Submitting a pull request](#submitting-a-pull-request)
- [Adding a new tool to an image](#adding-a-new-tool-to-an-image)
- [Code style guide](#code-style-guide)
Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [MIT](LICENSE.md).
Please note that this project is released with a [Contributor Code of Conduct][code-of-conduct]. By participating in this project you agree to abide by its terms.
## Submitting a pull request
1. [Fork][fork] and clone the repository.
1. Create a new branch: `git checkout -b my-branch-name`.
1. Make your changes, ensuring that they include steps to install, validate post-install, and update the software report (please see [Adding a new tool to an image](#adding-a-new-tool-to-an-image) for details).
1. Test your changes by [creating an image and deploying a VM](docs/create-image-and-azure-resources.md).
1. Push to your fork and [submit a pull request][pr].
1. [Fork][fork] and clone the repository
1. Create a new branch: `git checkout -b my-branch-name`
1. Make your changes, ensure that they include steps to install, validate post-install and update software report (please see [How to add new tool](CONTRIBUTING.md#how-to-add-new-tool) for details).
1. Test your changes by [creating VHD and deploying a VM](docs/create-image-and-azure-resources.md).
1. Push to your fork and [submit a pull request][pr]
Here are a few things you can do that will increase the likelihood of your pull request being accepted:
- Follow the style guide for [Powershell](https://github.com/PoshCode/PowerShellPracticeAndStyle) when writing Windows scripts. There is currently no set style for the Shell scripts that run Linux installs :soon:.
- Include complete details of why this is needed in the PR description.
- Include complete details of why this is needed in the PR description.
- Keep your change as focused as possible. If there are multiple changes you would like to make that are not dependent upon each other, consider submitting them as separate pull requests.
- Write [good commit messages](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
- Write [good commit messages](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
- For new tools:
- Make sure that the tool satisfies the [Software Guidelines](README.md#software-guidelines).
- Create an issue and get approval from us to add this tool to the image before creating the pull request.
## Adding a new tool to an image
- Make sure that the tool satisfies [Software Guidelines](README.md#software-guidelines).
- Create an issue and get an approval from us to add this tool to the image before creating the pull request.
## How to add new tool
### General rules
- For every new tool, add validation scripts and update the software report script to ensure that it is included in the documentation.
- If the tool is available on multiple platforms (macOS, Windows, Linux), make sure you include it on as many as possible.
- If installing multiple versions of the tool, consider putting the list of versions in the corresponding `toolset.json` file. This will help other customers configure their builds flexibly. See [toolset-windows-2019.json](images/windows/toolsets/toolset-2019.json) as an example.
- Use consistent naming across all files.
- Validation scripts should be simple and shouldn't change the image content.
- For every new tool add validation scripts and update software report script to make sure that it is included to documentation
- If the tool is available in other platforms (macOS, Windows, Linux), make sure you include it in as many as possible.
- If installing a few versions of the tool, consider putting the list of versions in the corresponding `toolset.json` file. It will help other customers to configure their builds flexibly. See [toolset-windows-2019.json](images/win/toolsets/toolset-2019.json) as example.
- Use consistent naming across all files
- Validation scripts should be simple and shouldn't change image content
### Windows
- Add a script that will install the tool and put the script in the `scripts/build` folder.
There are a bunch of helper functions that could simplify your code: `Install-ChocoPackage`, `Install-Binary`, `Install-VSIXFromFile`, `Install-VSIXFromUrl`, `Invoke-DownloadWithRetry`, `Test-IsWin19`, `Test-IsWin22` (find the full list of helpers in [ImageHelpers.psm1](images/windows/scripts/helpers/ImageHelpers.psm1)).
- Add a script that will validate the tool installation and put the script in the `scripts/tests` folder.
We use [Pester v5](https://github.com/pester/pester) for validation scripts. If the tests for the tool are complex enough, create a separate `*.Tests.ps1`. Otherwise, use `Tools.Tests.ps1` for simple tests.
Add `Invoke-PesterTests -TestFile <testFileName> [-TestName <describeName>]` at the end of the installation script to ensure that your tests will be run.
- Add changes to the software report generator `images/windows/scripts/docs-gen/Generate-SoftwareReport.ps1`. The software report generator is used to generate an image's README file, e.g. [Windows2019-Readme.md](images/windows/Windows2019-Readme.md) and uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
- Add a script that will install the tool and put the script in the `scripts/Installers` folder.
There are a bunch of helper functions that could simplify your code: `Choco-Install`, `Install-Binary`, `Install-VsixExtension`, `Start-DownloadWithRetry`, `Test-IsWin19`, `Test-IsWin22` (find the full list of helpers in [ImageHelpers.psm1](images/win/scripts/ImageHelpers/ImageHelpers.psm1)).
- Add a script that will validate the tool installation and put the script in the `scripts/Tests` folder.
We use [Pester v5](https://github.com/pester/pester) for validation scripts. If the tests for the tool are complex enough, create a separate `*.Tests.ps1`. Otherwise, use `Tools.Tests.ps1` for simple tests.
Add `Invoke-PesterTests -TestFile <testFileName> [-TestName <describeName>]` at the end of the installation script to make sure that your tests will be run.
- Add changes to the software report generator `images/win/scripts/SoftwareReport/SoftwareReport.Generator.ps1`. The software report generator is used to generate an image's README file, e.g. [Windows2019-Readme.md](images/win/Windows2019-Readme.md) and uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
### Ubuntu
- Add a script that will install and validate the tool and put the script in the `scripts/build` folder.
Use existing scripts such as [github-cli.sh](images/ubuntu/scripts/build/github-cli.sh) as a starting point.
- Use [helpers](images/ubuntu/scripts/helpers/install.sh) to simplify the installation process.
- The validation part should `exit 1` if there is any issue with the installation.
- Add changes to the software report generator `images/ubuntu/scripts/docs-gen/Generate-SoftwareReport.ps1`. The software report generator is used to generate an image's README file, e.g. [Ubuntu2204-Readme.md](images/ubuntu/Ubuntu2204-Readme.md) and it uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
- Add script that will install and validate the tool and put the script in the `scripts/installers` folder.
Use existing scripts such as [github-cli.sh](images/linux/scripts/installers/github-cli.sh) as a starting point.
- Use [helpers](images/linux/scripts/helpers/install.sh) to simplify installation process.
- Validation part should `exit 1` if any issue with installation.
- Add changes to the software report generator `images/linux/scripts/SoftwareReport/SoftwareReport.Generator.ps1`. The software report generator is used to generate an image's README file, e.g. [Ubuntu2004-Readme.md](images/linux/Ubuntu2004-README.md) and it uses [MarkdownPS](https://github.com/Sarafian/MarkdownPS).
### macOS
The macOS source lives in this repository and is available for everyone. However, the macOS image-generation CI doesn't support external contributions yet, so we are not able to accept pull requests for now.
We are in the process of preparing the macOS CI to accept contributions. Until then, we appreciate your patience and ask that you continue to make tool requests by filing issues.
## Code style guide
The principles of clean code apply to all languages. The main points are:
- Use meaningful names for variables, functions, files, etc.
- Keep functions short and simple.
- Use comments to explain what the code does.
- Use a consistent code style, naming convention, and file structure.
### File structure
- Each file should have a header with a title and a short description of the file.
- Each file should have a newline at the end.
- Use blank lines to separate logical blocks of code, but don't abuse blank lines:
- Don't add a blank line in the beginning and end of a block or function.
- Don't add blank lines between logically connected statements.
- Avoid trailing whitespace.
### Bash scripts
#### Naming convention for bash scripts
- Use lowercase letters for variable names.
- Use uppercase letters for constants.
- Use underscores to separate words in variable names.
#### Bash script structure
Each script should start with the following shebang:
```bash
#!/bin/bash -e
```
> TODO: do we need to set pipefail?
This will make the script exit if any command fails.
After the shebang, add a header with the following format:
```bash
################################################################################
## File: <filename>
## Desc: <short description of what the script does>
################################################################################
```
Then import helpers that are used in the script.
For Linux:
```bash
source $HELPER_SCRIPTS/os.sh
source $HELPER_SCRIPTS/install.sh
source $HELPER_SCRIPTS/etc-environment.sh
```
For macOS:
```bash
source ~/utils/utils.sh
```
> [!NOTE]
> You don't need to import all helpers, only the ones that are used in the script.
After that, add the script code.
### Indentations and line breaks in bash scripts
- Use 4 spaces for indentation.
- Use 1 space between `if`/`for`/`while` and `[[` and between `[[` and the condition.
- Place `then`/`do` on the new line.
- For short `if`/`for`/`while` statements, use the one-line format.
- Break long pipelines using `\`.
### Other recommendations for bash scripts
- For command substitution, use `$()` instead of backticks.
- Use `[[` instead of `[` for conditional expressions.
- Prefer using long options instead of short keys, but there are exceptions, e.g.:
- `tar -xzf`
- `apt-get -yqq`
- `curl -sSLf`
- `wget -qO-`
### PowerShell scripts
#### Naming convention for PowerShell scripts
- Use camelCase for variable names.
- Use uppercase letters for constants.
- Use `Verb-Noun` and PascalCase for function names.
### PowerShell script structure
Each script should start with the following header:
```powershell
################################################################################
## File: <filename>
## Desc: <short description of what the script does>
################################################################################
```
Then declare functions that are used in the script.
> TODO: do we need to set the error action preference and progress preference?
>
> ```powershell
> $ErrorActionPreference = "Stop"
> $ProgressPreference = "SilentlyContinue"
> ```
For Linux and macOS, import helpers that are used in the script:
For Linux:
```powershell
Import-Module "$env:HELPER_SCRIPTS/Tests.Helpers.psm1" -DisableNameChecking
```
For macOS:
```powershell
Import-Module "$env:HOME/image-generation/helpers/Common.Helpers.psm1"
Import-Module "$env:HOME/image-generation/helpers/Xcode.Helpers.psm1" -DisableNameChecking
```
> [!NOTE]
> You don't need to import all helpers, only the ones that are used in the script.
After that, add the script code.
### Indentations and line breaks in PowerShell scripts
- Use 4 spaces for indentation.
- Use 1 space between `if`/`elseif`/`foreach` and `(` but not between `(` and the condition.
- Add a space before and after pipe `|` and redirection `>` operators.
- Align properties in hash tables.
- Use [1TBS](https://en.wikipedia.org/wiki/Indentation_style#Variant:_1TBS_(OTBS)) style for curly braces:
- If block of statement is long, then place it on the new line, indent it, and add a closing curly brace on the new line.
- If block of statement is short, then place it on the same line as the statement.
```powershell
function Show-Example1 {
$exampleVariable = Get-ChildItem $env:TEMP
$exampleVariable | ForEach-Object {
$itemName = $_.Name
$itemPath = $_.FullName
}
}
$Example2 | Some-Function -Arguments @{Parameter1 = "Disabled"}
```
- Avoid using aliases.
- Break long pipelines using backticks or use [splatting](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting?view=powershell-7.3):
```powershell
# Instead of this
Copy-Item -Path "test.txt" -Destination "test2.txt" -WhatIf
# you can use this
$HashArguments = @{
Path = "test.txt"
Destination = "test2.txt"
WhatIf = $true
}
Copy-Item @HashArguments
```
When using backticks be extra careful with trailing whitespace as they can cause errors.
### Other recommendations for PowerShell scripts
- Verify exit codes of commands.
- When writing a function, provide a docstring that describes what the function does.
macOS source lives in this repository and available for everyone. However, macOS image-generation CI doesn't support external contributions yet so we are not able to accept pull-requests for now.
We are in the process of preparing macOS CI to accept contributions. Until then, we appreciate your patience and ask you continue to make tool requests by filing issues.
## Resources
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2025 GitHub
Copyright (c) 2023 GitHub
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+202 -217
View File
@@ -1,217 +1,202 @@
# GitHub Actions Runner Images
**Table of Contents**
- [About](#about)
- [Available Images](#available-images)
- [Announcements](#announcements)
- [Image Definitions](#image-definitions)
- [Image Releases](#image-releases)
- [Software and Image Support](#software-and-image-support)
- [How to Interact with the Repo](#how-to-interact-with-the-repo)
- [FAQs](#faqs)
## About
This repository contains the source code used to create the VM images for [GitHub-hosted runners](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners) used for Actions, as well as for [Microsoft-hosted agents](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent) used for Azure Pipelines.
To build a VM machine from this repo's source, see the [instructions](docs/create-image-and-azure-resources.md).
## Available Images
| Image | YAML Label | Included Software |
| --------------------|---------------------|--------------------|
| Ubuntu 24.04 | `ubuntu-latest` or `ubuntu-24.04` | [ubuntu-24.04] |
| Ubuntu 22.04 | `ubuntu-22.04` | [ubuntu-22.04] |
| macOS 26 Arm64 `beta` | `macos-26` or `macos-26-xlarge` | [macOS-26-arm64] |
| macOS 15 | `macos-latest-large`, `macos-15-large`, or `macos-15-intel` | [macOS-15] |
| macOS 15 Arm64 | `macos-latest`, `macos-15`, or `macos-15-xlarge` | [macOS-15-arm64] |
| macOS 14 | `macos-14-large`| [macOS-14] |
| macOS 14 Arm64 | `macos-14` or `macos-14-xlarge`| [macOS-14-arm64] |
| macOS 13 ![Deprecated](https://img.shields.io/badge/-Deprecated-red) | `macos-13` or `macos-13-large` | [macOS-13] |
| macOS 13 Arm64 ![Deprecated](https://img.shields.io/badge/-Deprecated-red) | `macos-13-xlarge` | [macOS-13-arm64] |
| Windows Server 2025 | `windows-latest` or `windows-2025` | [windows-2025] |
| Windows Server 2022 | `windows-2022` | [windows-2022] |
| Windows Server 2019 ![Deprecated](https://img.shields.io/badge/-Deprecated-red) | `windows-2019` | [windows-2019] |
### Label scheme
- In general the `-latest` label is used for the latest OS image version that is GA
- Before moving the`-latest` label to a new OS version we will announce the change and give sufficient lead time for users to update their workflows
[ubuntu-24.04]: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md
[ubuntu-22.04]: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md
[windows-2019]: https://github.com/actions/runner-images/blob/main/images/windows/Windows2019-Readme.md
[windows-2025]: https://github.com/actions/runner-images/blob/main/images/windows/Windows2025-Readme.md
[windows-2022]: https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md
[macOS-13]: https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md
[macOS-13-arm64]: https://github.com/actions/runner-images/blob/main/images/macos/macos-13-arm64-Readme.md
[macOS-14]: https://github.com/actions/runner-images/blob/main/images/macos/macos-14-Readme.md
[macOS-14-arm64]: https://github.com/actions/runner-images/blob/main/images/macos/macos-14-arm64-Readme.md
[macOS-15]: https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md
[macOS-15-arm64]: https://github.com/actions/runner-images/blob/main/images/macos/macos-15-arm64-Readme.md
[macOS-26-arm64]: https://github.com/actions/runner-images/blob/main/images/macos/macos-26-arm64-Readme.md
[self-hosted runners]: https://help.github.com/en/actions/hosting-your-own-runners
## Announcements
See notable upcoming changes by viewing issues with the [Announcement](https://github.com/actions/runner-images/labels/Announcement) label.
## Image Definitions
### Beta
The purpose of a Beta is to collect feedback on an image before it is released to GA. The goal of a Beta is to identify and fix any potential issues that exist on that
image. Images are updated on a weekly cadence. Any workflows that run on a beta image do not fall under the customer [SLA](https://github.com/customer-terms/github-online-services-sla) in place for Actions.
Customers choosing to use Beta images are encouraged to provide feedback in the runner-images repo by creating an issue. A Beta may take on different availability, i.e. public vs private.
### GA
A GA (General Availability) image has been through a Beta period and is deemed ready for general use. Images are updated on a weekly cadence. In order to be moved to
GA the image must meet the following criteria:
1. Has been through a Beta period (public or private)
2. Most major software we install on the image has a compatible
version for the underlying OS and
3. All major bugs reported during the Beta period have been addressed.
This image type falls under the customer [SLA](https://github.com/customer-terms/github-online-services-sla) for actions. GA images are eventually deprecated according to our guidelines as we only support the
latest 2 versions of an OS.
#### Latest Migration Process
GitHub Actions and Azure DevOps use the `-latest` YAML label (ex: `ubuntu-latest`, `windows-latest`, and `macos-latest`). These labels point towards the newest stable OS version available.
The `-latest` migration process is gradual and happens over 1-2 months in order to allow customers to adapt their workflows to the newest OS version. During this process, any workflow using the `-latest` label, may see changes in the OS version in their workflows or pipelines. To avoid unwanted migration, users can specify a specific OS version in the yaml file (ex: macos-14, windows-2022, ubuntu-22.04).
## Image Releases
*How to best follow along with changes*
1. Find the latest releases for this repository [here.](https://github.com/actions/runner-images/releases)
2. Subscribe to the releases coming out of this repository, instructions [here.](https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#configuring-your-watch-settings-for-an-individual-repository)
3. Upcoming changes: A pre-release is created when the deployment of an image has started. As soon as the deployment is finished, the pre-release is converted to a release. If you have subscribed to releases, you will get notified of pre-releases as well.
- You can also track upcoming changes using the [awaiting-deployment](https://github.com/actions/runner-images/labels/awaiting-deployment) label.
4. For high impact changes, we will post these in advance to the GitHub Changelog on our [blog](https://github.blog/changelog/) and on [twitter](https://twitter.com/GHchangelog).
- Ex: breaking changes, GA or deprecation of images
*Cadence*
- We typically deploy weekly updates to the software on the runner images.
## Software and Image Support
### Support Policy
- Tools and versions will typically be removed 6 months after they are deprecated or have reached end-of-life
- We support (at maximum) 2 GA images and 1 beta image at a time. We begin the deprecation process of the oldest image label once the newest OS image label has been released to GA.
- The images generally contain the latest versions of packages installed except for Ubuntu LTS where we mostly rely on the Canonical-provided repositories.
- Popular tools can have several versions installed side-by-side with the following strategy:
| Tool name | Installation strategy |
|-----------|-----------------------|
| Docker images | not more than 3 latest LTS OS\tool versions. New images or new versions of current images are added using the standard tool request process |
| Java | all LTS versions |
| Node.js | 3 latest LTS versions |
| Go | 3 latest minor versions |
| Python <br/> Ruby | 5 most popular `major.minor` versions |
| PyPy | 3 most popular `major.minor` versions |
| .NET Core | 2 latest LTS versions and 1 latest version. For each feature version only latest patch is installed. Note for [Ubuntu images see details.](./docs/dotnet-ubuntu.md) |
| GCC <br/> GNU Fortran <br/> Clang <br/> GNU C++ | 3 latest major versions |
| Android NDK | 1 latest non-LTS, 2 latest LTS versions |
| Xcode | - only one major version of Xcode will be supported per macOS version <br/> - all minor versions of the supported major version will be available <br/> - beta and RC versions will be provided "as-is" in the latest available macOS image only no matter of beta/GA status of the image <br/> - when a new patch version is released, the previous patch version will be replaced |
| Xcode Platforms | - only three major.minor versions of platform tools and simulator runtimes will be available for installed Xcode, including beta/RC versions |
### Package managers usage
We use third-party package managers to install software during the image generation process. The table below lists the package managers and the software installed.
> [!NOTE]
> Third-party repositories are re-evaluated every year to identify if they are still useful and secure.
| Operating system | Package manager | Third-party repos and packages |
| :--- | :---: | ---: |
| Ubuntu | [APT](https://wiki.debian.org/Apt) | [docker](https://download.docker.com/linux/ubuntu) <br/> [Eclipse-Temurin (Adoptium)](https://packages.adoptium.net/artifactory/deb/) <br/> [Erlang](https://packages.erlang-solutions.com/ubuntu) <br/> [Firefox](https://ppa.launchpad.net/mozillateam/ppa/ubuntu) <br/> [git-lfs](https://packagecloud.io/install/repositories/github/git-lfs) <br/> [git](https://launchpad.net/~git-core/+archive/ubuntu/ppa) <br/> [Google Cloud CLI](https://packages.cloud.google.com/apt) <br/> [Heroku](https://cli-assets.heroku.com/channels/stable/apt) <br/> [HHvm](https://dl.hhvm.com/ubuntu) <br/> [MongoDB](https://repo.mongodb.org/apt/ubuntu) <br/> [Mono](https://download.mono-project.com/repo/ubuntu) <br/> [MS Edge](https://packages.microsoft.com/repos/edge) <br/> [PostgreSQL](https://apt.postgresql.org/pub/repos/apt/) <br/> [R](https://cloud.r-project.org/bin/linux/ubuntu) |
| | [pipx](https://pypa.github.io/pipx) | ansible-core <br/>yamllint |
| Windows | [Chocolatey](https://chocolatey.org) | No third-party repos installed |
| macOS | [Homebrew](https://brew.sh) | [aws-cli v2](https://github.com/aws/homebrew-tap) </br> [azure/bicep](https://github.com/Azure/homebrew-bicep) </br> [mongodb/brew](https://github.com/mongodb/homebrew-brew) |
| | [pipx](https://pypa.github.io/pipx/) | yamllint |
### Image Deprecation Policy
- Images begin the deprecation process of the oldest image label once a new GA OS version has been released.
- Deprecation process begins with an announcement that sets a date for deprecation
- As it gets closer to the date, GitHub begins doing scheduled brownouts of the image
- During this time there will be an Announcement pinned in the repo to remind users of the deprecation.
- Finally GitHub will deprecate the image and it will no longer be available
### Preinstallation Policy
In general, these are the guidelines we follow when deciding what to pre-install on our images:
- Popularity: widely-used tools and ecosystems will be given priority.
- Latest Technology: recent versions of tools will be given priority.
- Deprecation: end-of-life tools and versions will not be added.
- Licensing: MIT, Apache, or GNU licenses are allowed.
- Time & Space on the Image: we will evaluate how much time is saved and how much space is used by having the tool pre-installed.
- Support: If a tool requires the support of more than one version, we will consider the cost of this maintenance.
### Default Version Update Policy
- In general, once a new version is installed on the image, we announce the default version update 2 weeks prior to deploying it.
- For potentially dangerous updates, we may extend the timeline up to 1 month between the announcement and deployment.
## How to Interact with the Repo
- **Issues**: To file a bug report, or request tools to be added/updated, please [open an issue using the appropriate template](https://github.com/actions/runner-images/issues/new/choose)
- **Discussions**: If you want to share your thoughts about image configuration, installed software, or bring a new idea, please create a new topic in a [discussion](https://github.com/actions/runner-images/discussions) for a corresponding category. Before making a new discussion please make sure no similar topics were created earlier.
- For general questions about using the runner images or writing your Actions workflow, please open requests in the [GitHub Actions Community Forum](https://github.community/c/github-actions/41).
## FAQs
<details>
<summary><b><i>What images are available for GitHub Actions and Azure DevOps?</b></i></summary>
The availability of images for GitHub Actions and Azure DevOps is the same. However, deprecation policies may differ. See documentation for more details:
- [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources)
- [Azure DevOps](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#software)
</details>
<details>
<summary><b><i>What image version is used in my build?</b></i></summary>
Usually, image deployment takes 2-3 days, and documentation in the `main` branch is only updated when deployment is finished. To find out which image version and what software versions are used in a specific build, see `Set up job` (GitHub Actions) or `Initialize job` (Azure DevOps) step log.
<img width="1440" alt="actions-runner-image" src="https://github.com/actions/runner-images/assets/88318005/922a8bf5-3e4d-4265-9527-b3b51e6bf9c8">
</details>
<details>
<summary><b><i>Looking for other Linux distributions?</b></i></summary>
We do not plan to offer other Linux distributions. We recommend using Docker if you'd like to build using other distributions with the hosted runner images. Alternatively, you can leverage [self-hosted runners] and fully customize your VM image to your needs.
</details>
<details>
<summary><b><i>How do I contribute to the macOS source?</b></i></summary>
macOS source lives in this repository and is available for everyone. However, macOS image-generation CI doesn't support external contributions yet so we are not able to accept pull-requests for now.
We are in the process of preparing macOS CI to accept contributions. Until then, we appreciate your patience and ask you to continue to make tool requests by filing issues.
</details>
<details>
<summary><b><i>How does GitHub determine what tools are installed on the images?</b></i></summary>
For some tools, we always install the latest at the time of the deployment; for others, we pin the tool to specific version(s). For more details please see the [Preinstallation Policy](#preinstallation-policy)
</details>
<details>
<summary><b><i>How do I request that a new tool be pre-installed on the image?</b></i></summary>
Please create an issue and get an approval from us to add this tool to the image before creating the pull request.
</details>
<details>
<summary><b><i>What branch should I use to build custom image?</b></i></summary>
We strongly encourage customers to build their own images using the main branch.
This repository contains multiple branches and releases that serve as document milestones to reflect what software is installed in the images at certain point of time. Current builds are not idempotent and if one tries to build a runner image using the specific tag it is not guaranteed that the build will succeed.
</details>
# GitHub Actions Runner Images
**Table of Contents**
- [About](#about)
- [Available Images](#available-images)
- [Announcements](#announcements)
- [Image Definitions](#image-definitions)
- [Image Releases](#image-releases)
- [Software and Image Support](#software-and-image-support)
- [How to Interact with the Repo](#how-to-interact-with-the-repo)
- [FAQs](#faqs)
## About
This repository contains the source code used to create the VM images for [GitHub-hosted runners](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners) used for Actions, as well as for [Microsoft-hosted agents](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent) used for Azure Pipelines.
To build a VM machine from this repo's source, see the [instructions](docs/create-image-and-azure-resources.md).
## Available Images
| Image | YAML Label | Included Software | Rollout Progress of Latest Image Release |
| --------------------|---------------------|--------------------|---------------------|
| Ubuntu 22.04 | `ubuntu-latest` or `ubuntu-22.04` | [ubuntu-22.04] | [![status22](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=ubuntu22&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=ubuntu22&redirect=1)
| Ubuntu 20.04 | `ubuntu-20.04` | [ubuntu-20.04] | [![status20](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=ubuntu20&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=ubuntu20&redirect=1)
| macOS 13 [beta] | `macos-13` or `macos-13-xl`| [macOS-13] | [![statusumac13](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-13&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-13&redirect=1)
| macOS 12 | `macos-latest`, `macos-latest-xl`, `macos-12`, or `macos-12-xl`| [macOS-12] | [![statusumac12](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-12&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-12&redirect=1)
| macOS 11 | `macos-11`| [macOS-11] | [![statusmac11](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-11&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=macos-11&redirect=1)
| Windows Server 2022 | `windows-latest` or `windows-2022` | [windows-2022] | [![statuswin22](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=windows-2022&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=windows-2022&redirect=1) |
| Windows Server 2019 | `windows-2019` | [windows-2019] | [![statuswin19](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=windows-2019&badge=1)](https://runnerimagesdeploymentstatus.azurewebsites.net/api/status?imageName=windows-2019&redirect=1)
### Label scheme
- In general the `-latest` label is used for the latest OS image version that is GA
- Before moving the`-latest` label to a new OS version we will announce the change and give sufficient lead time for users to update their workflows
[ubuntu-22.04]: https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
[ubuntu-20.04]: https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
[windows-2022]: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md
[windows-2019]: https://github.com/actions/runner-images/blob/main/images/win/Windows2019-Readme.md
[macOS-11]: https://github.com/actions/runner-images/blob/main/images/macos/macos-11-Readme.md
[macOS-12]: https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md
[macOS-13]: https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md
[macOS-10.15]: https://github.com/actions/runner-images/blob/main/images/macos/macos-10.15-Readme.md
[self-hosted runners]: https://help.github.com/en/actions/hosting-your-own-runners
## Announcements
See notable upcoming changes by viewing issues with the [Announcement](https://github.com/actions/runner-images/labels/Announcement) label.
## Image Definitions
### Beta
The purpose of a Beta is to collect feedback on an image before it is released to GA. The goal of a Beta is to identify and fix any potential issues that exist on that
image. Images are updated on a weekly cadence. Any workflows that run on a beta image do not fall under the customer [SLA](https://github.com/customer-terms/github-online-services-sla) in place for Actions.
Customers choosing to use Beta images are encouraged to provide feedback in the runner-images repo by creating an issue. A Beta may take on different availability, i.e. public vs private.
### GA
A GA (General Availability) image has been through a Beta period and is deemed ready for general use. Images are updated on a weekly cadence. In order to be moved to
GA the image must meet the following criteria:
1. Has been through a Beta period (public or private)
2. Most major software we install on the image has a compatible
version for the underlying OS and
3. All major bugs reported during the Beta period have been addressed.
This image type falls under the customer [SLA](https://github.com/customer-terms/github-online-services-sla) for actions. GA images are eventually deprecated according to our guidelines as we only support the
latest 2 versions of an OS.
#### Latest Migration Process
GitHub Actions and Azure DevOps use the `-latest` YAML label (ex: `ubuntu-latest`, `windows-latest`, and `macos-latest`). These labels point towards the newest stable OS version available.
The `-latest` migration process is gradual and happens over 1-2 months in order to allow customers to adapt their workflows to the newest OS version. During this process, any workflow using the `-latest` label, may see changes in the OS version in their workflows or pipelines. To avoid unwanted migration, users can specify a specific OS version in the yaml file (ex: macos-12, windows-2022, ubuntu-22.04).
## Image Releases
*How to best follow along with changes*
1. Find the latest releases for this repository [here.](https://github.com/actions/runner-images/releases)
2. Subscribe to the releases coming out of this repository, instructions [here.](https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#configuring-your-watch-settings-for-an-individual-repository)
3. Upcoming changes: A pre-release is created when the deployment of an image has started. As soon as the deployment is finished, the pre-release is converted to a release. If you have subscribed to releases, you will get notified of pre-releases as well.
- You can also track upcoming changes using the [awaiting-deployment](https://github.com/actions/runner-images/labels/awaiting-deployment) label.
4. For high impact changes, we will post these in advance to the GitHub Changelog on our [blog](https://github.blog/changelog/) and on [twitter](https://twitter.com/GHchangelog).
- Ex: breaking changes, GA or deprecation of images
*Cadence*
- We typically deploy weekly updates to the software on the runner images.
## Software and Image Support
### Support Policy
- Tools and versions will typically be removed 6 months after they are deprecated or have reached end-of-life
- We support (at maximum) 2 GA images and 1 beta image at a time. We begin the deprecation process of the oldest image label once the newest OS image label has been released to GA.
- The images generally contain the latest versions of packages installed except for Ubuntu LTS where we mostly rely on the Canonical-provided repositories.
- Popular tools can have several versions installed side-by-side with the following strategy:
| Tool name | Installation strategy |
|-----------|-----------------------|
| Docker images | not more than 3 latest LTS OS\tool versions. New images or new versions of current images are added using the standard tool request process |
| Java | all LTS versions |
| Node.js | 3 latest LTS versions |
| Go | 3 latest minor versions |
| Python <br/> Ruby | 5 most popular `major.minor` versions |
| PyPy | 3 most popular `major.minor` versions |
| .NET Core | 2 latest LTS versions and 1 latest version. For each feature version only latest patch is installed |
| GCC <br/> GNU Fortran <br/> Clang <br/> GNU C++ | 3 latest major versions |
| Android NDK | 1 latest non-LTS, 2 latest LTS versions |
| Xcode | - all OS compatible versions side-by-side <br/> - for beta, GM versions - latest beta only <br/> - old patch versions are deprecated in 3 months |
### Package managers usage
We use third-party package managers to install software during the image generation process. The table below lists the package managers and the software installed.
> **Note**: third-party repositories are re-evaluated every year to identify if they are still useful and secure.
| Operating system | Package manager | Third-party repos and packages |
| :--- | :---: | ---: |
| Ubuntu | [APT](https://wiki.debian.org/Apt) | [Eclipse-Temurin (Adoptium)](https://packages.adoptium.net/artifactory/deb) </br> [Erlang](https://packages.erlang-solutions.com/ubuntu) </br>[Firefox](https://launchpad.net/~mozillateam/+archive/ubuntu/ppa) </br> [gcc, gfortran](https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test) </br> [git](https://launchpad.net/~git-core/+archive/ubuntu/ppa) </br> [HHvm](https://dl.hhvm.com/ubuntu) </br> [PHP](https://launchpad.net/~ondrej/+archive/ubuntu/php) (Ubuntu 20 only) </br> [Mono](https://download.mono-project.com/repo/ubuntu) </br> [PostgreSQL](https://apt.postgresql.org/pub/repos/apt) </br> [R](https://cloud.r-project.org/bin/linux/ubuntu) |
| | [pipx](https://pypa.github.io/pipx) | ansible-core </br>yamllint |
| Windows | [Chocolatey](https://chocolatey.org) | No third-party repos installed |
| macOS | [Homebrew](https://brew.sh) | [aws-cli v2](https://github.com/aws/homebrew-tap) </br> [azure/bicep](https://github.com/Azure/homebrew-bicep) </br> [mongodb/brew](https://github.com/mongodb/homebrew-brew) |
| | [pipx](https://pypa.github.io/pipx/) | yamllint |
### Image Deprecation Policy
- Images begin the deprecation process of the oldest image label once a new GA OS version has been released.
- Deprecation process begins with an announcement that sets a date for deprecation
- As it gets closer to the date, GitHub begins doing scheduled brownouts of the image
- During this time there will be an Announcement pinned in the repo to remind users of the deprecation.
- Finally GitHub will deprecate the image and it will no longer be available
### Preinstallation Policy
In general, these are the guidelines we follow when deciding what to pre-install on our images:
- Popularity: widely-used tools and ecosystems will be given priority.
- Latest Technology: recent versions of tools will be given priority.
- Deprecation: end-of-life tools and versions will not be added.
- Licensing: MIT, Apache, or GNU licenses are allowed.
- Time & Space on the Image: we will evaluate how much time is saved and how much space is used by having the tool pre-installed.
- Support: If a tool requires the support of more than one version, we will consider the cost of this maintenance.
### Default Version Update Policy
- In general, once a new version is installed on the image, we announce the default version update 2 weeks prior to deploying it.
- For potentially dangerous updates, we may extend the timeline up to 1 month between the announcement and deployment.
## How to Interact with the Repo
- **Issues**: To file a bug report, or request tools to be added/updated, please [open an issue using the appropriate template](https://github.com/actions/runner-images/issues/new/choose)
- **Discussions**: If you want to share your thoughts about image configuration, installed software, or bring a new idea, please create a new topic in a [discussion](https://github.com/actions/runner-images/discussions) for a corresponding category. Before making a new discussion please make sure no similar topics were created earlier.
- For general questions about using the runner images or writing your Actions workflow, please open requests in the [GitHub Actions Community Forum](https://github.community/c/github-actions/41).
## FAQs
<details>
<summary><b><i>What images are available for GitHub Actions and Azure DevOps?</b></i></summary>
The availability of images for GitHub Actions and Azure DevOps is the same. However, deprecation policies may differ. See documentation for more details:
- [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources)
- [Azure DevOps](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#software)
</details>
<details>
<summary><b><i>What image version is used in my build?</b></i></summary>
Usually, image deployment takes 2-3 days, and documentation in the `main` branch is only updated when deployment is finished. To find out which image version and what software versions are used in a specific build, see `Set up job` (GitHub Actions) or `Initialize job` (Azure DevOps) step log.
<img width="1440" alt="actions-runner-image" src="https://user-images.githubusercontent.com/56982181/169595536-91a8a79b-d5e0-47d1-a736-510cff6cfb83.png">
</details>
<details>
<summary><b><i>Looking for other Linux distributions?</b></i></summary>
We do not plan to offer other Linux distributions. We recommend using Docker if you'd like to build using other distributions with the hosted runner images. Alternatively, you can leverage [self-hosted runners] and fully customize your VM image to your needs.
</details>
<details>
<summary><b><i>How do I contribute to the macOS source?</b></i></summary>
macOS source lives in this repository and is available for everyone. However, macOS image-generation CI doesn't support external contributions yet so we are not able to accept pull-requests for now.
We are in the process of preparing macOS CI to accept contributions. Until then, we appreciate your patience and ask you to continue to make tool requests by filing issues.
</details>
<details>
<summary><b><i>How does GitHub determine what tools are installed on the images?</b></i></summary>
For some tools, we always install the latest at the time of the deployment; for others, we pin the tool to specific version(s). For more details please see the [Preinstallation Policy](#preinstallation-policy)
</details>
<details>
<summary><b><i>How do I request that a new tool be pre-installed on the image?</b></i></summary>
Please create an issue and get an approval from us to add this tool to the image before creating the pull request.
</details>
-294
View File
@@ -1,294 +0,0 @@
#!/usr/bin/env bash
################################################################################
## File: diff-image-versions.sh
## Desc: Compare software versions between two runner image releases
## Usage: ./diff-image-versions.sh <os-name> <version1> <version2>
##
## Example:
## ./diff-image-versions.sh ubuntu22 20251102.127 20251125.163
## ./diff-image-versions.sh win25 20251102.77 20251125.122
## ./diff-image-versions.sh macos-14 20251102.0024 20251125.0031
################################################################################
set -euo pipefail
usage() {
cat <<EOF
Usage: $(basename "${0}") <os-name> <version1> <version2>
Compare runner image versions and display software changes.
Arguments:
os-name OS identifier (ubuntu22, ubuntu24, win19, win22, win25,
macos-13, macos-14, macos-15, or arm64 variants)
version1 Earlier version (YYYYMMDD.NNN)
version2 Later version (YYYYMMDD.NNN)
Examples:
$(basename "${0}") ubuntu22 20251102.127 20251125.163
$(basename "${0}") win25 20251102.77 20251125.122
EOF
}
get_readme_path() {
local os_name="${1}"
local os_folder=""
local pattern=""
# Determine OS folder and readme filename pattern
case "${os_name}" in
ubuntu*)
os_folder="ubuntu"
local version="${os_name#ubuntu}"
pattern="Ubuntu${version}04-Readme.md"
;;
win*)
os_folder="windows"
local version="${os_name#win}"
pattern="Windows20${version}-Readme.md"
;;
macos*)
os_folder="macos"
pattern="${os_name}-Readme.md"
;;
*)
echo "Error: Unknown OS '${os_name}'" >&2
echo "Valid: ubuntu*, win*, macos-*" >&2
return 1
;;
esac
local readme_path="images/${os_folder}/${pattern}"
# Verify file exists in git repository
if ! git cat-file -e "HEAD:${readme_path}" 2>/dev/null; then
echo "Error: Readme not found: ${readme_path}" >&2
return 1
fi
echo "${readme_path}"
}
validate_version() {
local version="${1}"
if [[ ! "${version}" =~ ^[0-9]{8}\.[0-9]+$ ]]; then
echo "Error: Invalid version '${version}'" >&2
echo "Format: YYYYMMDD.NNN (e.g., 20251102.127)" >&2
return 1
fi
return 0
}
tag_exists() {
local tag="${1}"
if git rev-parse "${tag}" >/dev/null 2>&1; then
return 0
else
echo "Error: Tag '${tag}' not found" >&2
return 1
fi
}
main() {
# Check arguments
if [[ $# -ne 3 ]]; then
usage
return 1
fi
local os_name="${1}"
local version1="${2}"
local version2="${3}"
# Validate inputs
validate_version "${version1}" || return 1
validate_version "${version2}" || return 1
# Get readme path
local readme_path
readme_path="$(get_readme_path "${os_name}")" || return 1
# Construct git tags
local tag1="${os_name}/${version1}"
local tag2="${os_name}/${version2}"
# Verify tags exist
tag_exists "${tag1}" || return 1
tag_exists "${tag2}" || return 1
# Get release dates
local date1
local date2
date1=$(git log -1 --format="%ci" "${tag1}" | cut -d' ' -f1)
date2=$(git log -1 --format="%ci" "${tag2}" | cut -d' ' -f1)
# Calculate days between releases
local days_diff
days_diff=$(( ($(date -d "${date2}" +%s) - $(date -d "${date1}" +%s)) / 86400 ))
# Display header
echo "================================================================================"
echo "Comparing: ${os_name}"
echo " From: ${version1} (${date1})"
echo " To: ${version2} (${date2})"
echo " Span: ${days_diff} days"
echo "================================================================================"
echo ""
# Perform diff with minimal context (only changed lines with colors)
# ANSI codes: ^[[31m (red for -), ^[[32m (green for +), ^[[36m (cyan for @@)
# Filter to show only lines starting with red/green (additions/deletions)
local diff_output
diff_output=$(git diff --color=always --unified=0 "${tag1}:${readme_path}" "${tag2}:${readme_path}" | \
grep -E $'^\x1b\\[(31|32)m' | \
grep -v -E $'^\x1b\\[1m(---|\\+\\+\\+)')
if [[ -n "${diff_output}" ]]; then
# Extract announcements from both versions
local announcements1
local announcements2
announcements1=$(git show "${tag1}:${readme_path}" | sed -n '/| Announcements |/,/^\*\*\*$/p' | grep -E '^\| \[' | sed 's/^| \[/• [/' | sed 's/ |$//' || true)
announcements2=$(git show "${tag2}:${readme_path}" | sed -n '/| Announcements |/,/^\*\*\*$/p' | grep -E '^\| \[' | sed 's/^| \[/• [/' | sed 's/ |$//' || true)
# Show announcement changes
if [[ "${announcements1}" != "${announcements2}" ]]; then
echo "📢 Announcement Changes:"
echo "────────────────────────────────────────────────────────────────────────────────"
if [[ -n "${announcements2}" ]]; then
echo "${announcements2}"
else
echo "(no announcements)"
fi
echo "────────────────────────────────────────────────────────────────────────────────"
echo ""
fi
# Extract cached tools sections
local cached_tools1
local cached_tools2
cached_tools1=$(git show "${tag1}:${readme_path}" | sed -n '/^### Cached Tools$/,/^###[^#]/p' | head -n -1 || true)
cached_tools2=$(git show "${tag2}:${readme_path}" | sed -n '/^### Cached Tools$/,/^###[^#]/p' | head -n -1 || true)
# Show cached tools changes
if [[ "${cached_tools1}" != "${cached_tools2}" ]]; then
local cached_diff
cached_diff=$(git diff --color=always --unified=2 --no-index \
<(echo "${cached_tools1}") <(echo "${cached_tools2}") 2>/dev/null | \
grep -E $'(^\x1b\\[(31|32)m[-+]| #### )' | \
sed -r 's/\x1b\[m$//' || true)
if [[ -n "${cached_diff}" ]]; then
echo "🔧 Cached Tools Changes (setup-* actions):"
echo "────────────────────────────────────────────────────────────────────────────────"
echo "${cached_diff}"
echo "────────────────────────────────────────────────────────────────────────────────"
echo ""
fi
fi
echo "Full Diff:"
echo "────────────────────────────────────────────────────────────────────────────────"
echo "${diff_output}"
echo "────────────────────────────────────────────────────────────────────────────────"
echo ""
# Count changes
local changes
changes=$(echo "${diff_output}" | wc -l)
echo "Changes: ${changes} lines"
# Parse version changes for breaking change analysis
local breaking_changes=()
local removals=()
local additions=()
# Extract clean lines (strip ANSI codes)
while IFS= read -r line; do
if [[ "${line}" =~ ^\-(.+)$ ]]; then
removals+=("${BASH_REMATCH[1]}")
elif [[ "${line}" =~ ^\+(.+)$ ]]; then
additions+=("${BASH_REMATCH[1]}")
fi
done < <(echo "${diff_output}" | sed -r 's/\x1b\[[0-9;]*m//g')
# Detect breaking changes
for removed in "${removals[@]}"; do
local tool_name=""
local old_version=""
local found_match=false
# Try to extract tool name and version (handle various formats)
if [[ "${removed}" =~ ^([^0-9]+[[:space:]]+)([0-9]+\.[0-9]+[^[:space:]]*) ]]; then
tool_name="${BASH_REMATCH[1]}"
old_version="${BASH_REMATCH[2]}"
elif [[ "${removed}" =~ ^([^0-9]+[[:space:]]+v)([0-9]+\.[0-9]+[^[:space:]]*) ]]; then
tool_name="${BASH_REMATCH[1]}"
old_version="${BASH_REMATCH[2]}"
fi
# If we found a semver-style version, look for matching addition
if [[ -n "${tool_name}" && -n "${old_version}" ]]; then
for added in "${additions[@]}"; do
if [[ "${added}" =~ ^${tool_name}([0-9]+\.[0-9]+[^[:space:]]*) ]]; then
local new_version="${BASH_REMATCH[1]}"
found_match=true
# Extract major version for semver comparison
if [[ "${old_version}" =~ ^([0-9]+)\. && "${new_version}" =~ ^([0-9]+)\. ]]; then
local old_major="${BASH_REMATCH[1]}"
local new_major="${BASH_REMATCH[1]}"
[[ "${old_version}" =~ ^([0-9]+)\. ]] && old_major="${BASH_REMATCH[1]}"
[[ "${new_version}" =~ ^([0-9]+)\. ]] && new_major="${BASH_REMATCH[1]}"
if [[ ${new_major} -gt ${old_major} ]]; then
breaking_changes+=("🔴 ${tool_name}${old_version}${new_version} (major version bump)")
fi
fi
break
fi
done
fi
# If no match found and looks like a versioned tool, it's a removal
if [[ ${found_match} == false && -n "${old_version}" ]]; then
breaking_changes+=("${removed} (removed)")
elif [[ ${found_match} == false && "${removed}" =~ [0-9]+\.[0-9]+ ]]; then
breaking_changes+=("${removed} (removed)")
fi
done
# Display breaking changes
if [[ ${#breaking_changes[@]} -gt 0 ]]; then
echo ""
echo "⚠️ Breaking changes detected (${#breaking_changes[@]}):"
echo "--------------------------------------------------------------------------------"
printf '%s\n' "${breaking_changes[@]}"
echo "--------------------------------------------------------------------------------"
fi
else
echo "No changes found."
fi
# Display PR link and commit count
local pr_number
pr_number=$(git log --all --format="%s" --grep="${version2}" | \
grep -oP '\(#\K[0-9]+(?=\))' | head -1)
local commit_count
commit_count=$(git rev-list --count "${tag1}..${tag2}")
echo "Commits: ${commit_count}"
if [[ -n "${pr_number}" ]]; then
echo "PR: https://github.com/actions/runner-images/pull/${pr_number}"
fi
return 0
}
# Execute main function
main "$@"
+109 -162
View File
@@ -1,41 +1,26 @@
# GitHub Actions Runner Images
The runner-images project uses [Packer](https://www.packer.io/) to generate disk images for Windows 2019/2022 and Ubuntu 22.04/24.04.
The runner-images project uses [Packer](https://www.packer.io/) to generate disk images for Windows 2019/2022 and Ubuntu 20.04/22.04.
Each image is configured by a HCL2 Packer template that specifies where to build the image (Azure, in this case),
Each image is configured by a JSON or HCL2 Packer template that specifies where to build the image (Azure in this case)
and what steps to run to install software and prepare the disk.
The Packer process initializes a connection to the Azure subscription using Azure CLI and creates temporary resources
required for the build process: a resource group, network interfaces and a virtual machine from the "clean" image specified in the template.
The Packer process initializes a connection to Azure subscription using Azure CLI and creates temporary resources
required for the build process: resource group, network interfaces and virtual machine from the "clean" image specified in the template.
If the VM deployment succeeds, Packer connects to it using SSH or WinRM and begins executing installation steps from the template one-by-one.
If any step fails, image generation is aborted, and the temporary VM is terminated.
Packer also attempts to clean up all the temporary resources it created (unless otherwise configured).
If the VM deployment succeeds, Packer connects it using ssh or WinRM and begins executing installation steps from the template one-by-one.
If any step fails, image generation is aborted and the temporary VM is terminated.
Packer also attempts to cleanup all the temporary resources it created (unless otherwise configured).
After successful completion of all installation steps, Packer creates a managed image from the temporary VM's disk and deletes the VM.
After successful completion of all installation steps Packer converts snapshot of the temporary VM to VHD image
and uploads it to the specified Azure Storage Account.
- [Build Agent Preparation](#build-agent-preparation)
- [Manual image generation](#manual-image-generation)
- [Manual Image Generation Customization](#manual-image-generation-customization)
- [Network Security](#network-security)
- [Azure Subscription Authentication](#azure-subscription-authentication)
- [Generated Machine Deployment](#generated-machine-deployment)
- [Automated image generation](#automated-image-generation)
- [Required variables](#required-variables)
- [Optional variables](#optional-variables)
- [Builder variables](#builder-variables)
- [Toolset](#toolset)
- [Post-generation scripts](#post-generation-scripts)
- [Running scripts](#running-scripts)
- [Script Details: Ubuntu](#script-details-ubuntu)
- [Script Details: Windows](#script-details-windows)
## Build agent preparation
## Build Agent Preparation
The build agent is a machine where the Packer process will be started.
You can use any physical or virtual machine running Windows or Linux OS.
Of course, you may also use an [Azure VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/quick-create-cli).
In any case, you will need these software installed:
Build agent is a machine where Packer process will be started.
You can use any physical or virtual machine running OS Windows or Linux.
Of course you may also use [Azure VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/quick-create-cli).
In any case you will need these software installed:
- Packer 1.8.2 or higher.
@@ -49,7 +34,7 @@ In any case, you will need these software installed:
For Linux - install the latest version from your distro's package repo.
For Windows - download and install it from [here](https://gitforwindows.org/) or use [Chocolatey](https://chocolatey.org/):
For Windows - download and install it from [here](https://gitforwindows.org/) of use [Chocolatey](https://chocolatey.org/):
```powershell
choco install git -params '"/GitAndUnixToolsOnPath"'
@@ -60,86 +45,85 @@ In any case, you will need these software installed:
In Windows you already have it.
For Linux follow instructions [here](https://learn.microsoft.com/en-us/windows-server/administration/linux-package-repository-for-microsoft-software)
to add Microsoft's Linux Software Repository and then install the `powershell` package.
to add Microsoft's Linux Software Repository and then install package `powershell`.
- Azure CLI.
Follow the instructions [here](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).
Or if you use Windows, you may run this command in Powershell instead:
Follow instructions [here](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).
Or if you use Windows you may run this command in Powershell instead:
```powershell
Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi
Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'; rm .\AzureCLI.msi
```
## Manual image generation
## Automated image generation
This repository includes a script that assists in generating images in Azure.
All you need is an Azure subscription, a resource group in that subscription and a build agent configured as described above.
This repo bundles script that automates image generation process.
You only need a build agent configured as described above and active Azure subscription.
We suggest to start with UbuntuMinimal image because it includes only a minimal set of required software and builds in less then half an hour.
All the commands below should be executed in PowerShell.
All steps here are supposed to run in Powershell.
First, clone the runner-images repository and set the current directory to it:
First, clone runner-images repository and change directory:
```powershell
git clone https://github.com/actions/runner-images.git
Set-Location runner-images
```
Then, import the [GenerateResourcesAndImage](../helpers/GenerateResourcesAndImage.ps1) script from the `helpers` subdirectory:
Then import [GenerateResourcesAndImage](../helpers/GenerateResourcesAndImage.ps1) script from `helpers` subdirectory:
```powershell
Import-Module .\helpers\GenerateResourcesAndImage.ps1
```
Finally, run the `GenerateResourcesAndImage` function, setting the mandatory arguments: image type and where to build and store the resulting managed image:
Finally, run `GenerateResourcesAndImage` function setting mandatory arguments: image type and where to create resources:
- `SubscriptionId` - your Azure Subscription ID;
- `ResourceGroupName` - the name of the resource group that will store the resulting artifact (e.g., "imagegen-test").
The resource group must already exist in your Azure subscription;
- `AzureLocation` - the location where resources will be created (e.g., "East US");
- `ImageType` - the type of image to build (valid options are "Windows2019", "Windows2022", "Windows2025", "Ubuntu2204", "Ubuntu2404").
- `SubscriptionId` - your Azure Subscription ID
- `ResourceGroupName` - name of the resource group that will be created within your subscription (e.g. "imagegen-test")
- `AzureLocation` - location where resources will be created (e.g. "East US")
- `ImageType` - what image to build (we suggest choosing "UbuntuMinimal" here, other valid options are "Windows2019", "Windows2022", "Ubuntu2004", "Ubuntu2204")
This function automatically creates all required Azure resources and initiates the Packer image generation for the selected image type.
This function automatically creates all required Azure resources and kicks off packer image generation for the selected image type.
When the image is ready, you may proceed to [deployment](#generated-machine-deployment).
When image is ready you may proceed to [deployment](#generated-machine-deployment)
## Manual Image Generation Customization
## Image generation customization
The `GenerateResourcesAndImage` function accepts a number of arguments that may assist you in generating an image in your specific environment.
Function `GenerateResourcesAndImage` accepts a bunch of arguments that may help you generating image in your specific environment.
For example, you may want all the resources involved in the image generation process to be tagged.
In this case, pass a HashTable of tags as a value for the `Tags` parameter.
For example, you may want that all the resources involved in image generation process are tagged.
In this case pass a HashTable of tags as a value for `Tags` parameter.
If you don't want the function to authenticate interactively, you should create a Service Principal and invoke the function with the parameters `AzureClientId`, `AzureClientSecret` and `AzureTenantId`.
You can find more details in the [corresponding section below](#azure-subscription-authentication).
If you don't want function to authenticate interactively, you should create Service Principal and invoke the function with parameters `AzureClientId`, `AzureClientSecret` and `AzureTenantId`.
You can find more details [in corresponding section below](#azure-subscription-authentication).
Use `get-help GenerateResourcesAndImage -Detailed` for the complete list of available parameters.
Use `get-help GenerateResourcesAndImage -Detailed` for the complete list of parameters available.
### Network Security
### Network security
To connect to a temporary virtual machine, Packer uses WinRM or SSH.
To connect to a temporary virtual machine Packer uses WinRM or SSH.
If your build agent is located outside of the Azure subscription where the temporary VM is created, a public network interface and public IP address are used.
Make sure that firewalls are configured properly and that WinRM (TCP port 5986) and SSH (TCP port 22) connections are allowed both outgoing for the build agent and incoming for the temporary VM.
Also, if you don't want the temporary VM to be accessible from everywhere, set the `RestrictToAgentIpAddress` parameter value to `$true`
to set up firewall rules allowing access only from your build agent's public IP address.
If your build agent is located outside of the Azure subscription where temporary VM is created, the public network interface and public IP address is used.
Make sure that firewalls are configured properly and WinRM (tcp port 5986) and ssh (tcp port 22) connections are allowed both outgoing for build agent and incoming for temporary VM.
Also if you don't want temporary VM to be accessible from everywhere, set `RestrictToAgentIpAddress` parameter value to `$true`
to setup firewall rules allowing access only from your build agent public IP address.
If your build agent and temporary VM are in the same subscription, you can configure Packer to connect using a private virtual network.
To achieve this, set proper values for the environment variables `VNET_RESOURCE_GROUP`, `VNET_NAME` and `VNET_SUBNET`.
If your build agent and temporary VM are in the same subscription you can configure Packer to connect using private virtual network.
To achieve that set proper values for environment variables `VNET_RESOURCE_GROUP`, `VNET_NAME` and `VNET_SUBNET`.
### Azure Subscription Authentication
### Azure subscription authentication
Packer uses a Service Principal to authenticate in Azure infrastructure.
For more information about Service Principals, refer to the
Packer uses Service Principal to authenticate in Azure infrastructure.
For more information about Service Principals refer to
[Azure documentation](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal).
The `GenerateResourcesAndImage` function is able to create a Service Principal to be used by Packer.
It uses the Connect-AzAccount cmdlet that invokes an interactive authentication process by default.
If you don't want to use interactive authentication, you should create a Service Principal with full read-write permissions for the selected Azure subscription on your own
and provide proper values for the parameters `AzureClientId`, `AzureClientSecret` and `AzureTenantId`.
Function `GenerateResourcesAndImage` is able to create Service Principle to be used by Packer.
It uses Connect-AzAccount cmdlet that invokes interactive authentication process by default.
If you don't want to use interactive authentication you should create Service Principal with full read-write permissions for selected Azure subscription on your own
and provide proper values for parameters `AzureClientId`, `AzureClientSecret` and `AzureTenantId`.
Here is an example of how to create a Service Principal using the Az PowerShell module:
Here is an example of how to create Service Principle using Az Powershell module:
```powershell
$credentials = [Microsoft.Azure.PowerShell.Cmdlets.Resources.MSGraph.Models.ApiV10.MicrosoftGraphPasswordCredential]@{
@@ -161,110 +145,71 @@ Start-Sleep -Seconds 30
}
```
## Generated Machine Deployment
## Generated machine deployment
After successful image generation, a Virtual Machine can be created from the generated image using the [CreateAzureVMFromPackerTemplate](../helpers/CreateAzureVMFromPackerTemplate.ps1) script.
After the successful image generation, Virtual Machine can be created from the generated VHD using [CreateAzureVMFromPackerTemplate](../helpers/CreateAzureVMFromPackerTemplate.ps1) script.
```powershell
Import-Module .\helpers\CreateAzureVMFromPackerTemplate.ps1
CreateAzureVMFromPackerTemplate -SubscriptionId {YourSubscriptionId} -ResourceGroupName {ResourceGroupName} -ManagedImageName "Runner-Image-Ubuntu2204" -VirtualMachineName "testvm1" -AdminUsername "shady1" -AdminPassword "SomeSecurePassword1" -AzureLocation "eastus"
CreateAzureVMFromPackerTemplate -SubscriptionId {YourSubscriptionId} -ResourceGroupName {ResourceGroupName} -TemplateFile "C:\BuildVmImages\temporaryTemplate.json" -VirtualMachineName "testvm1" -AdminUsername "shady1" -AdminPassword "SomeSecurePassword1" -AzureLocation "eastus"
```
Where:
- `SubscriptionId` - the Azure subscription ID where resources will be created;
- `ResourceGroupName` - the Azure resource group name where the Azure virtual machine will be created;
- `ManagedImageName` - the name of the managed image to be used for the virtual machine creation;
- `VirtualMachineName` - the name of the virtual machine to be generated;
- `AdminUserName` - the administrator username for the virtual machine to be created;
- `AdminPassword` - the administrator password for the virtual machine to be created;
- `AzureLocation` - the location where the Azure virtual machine will be provisioned (e.g., "eastus").
- `SubscriptionId` - The Azure subscription Id where resources will be created.
- `ResourceGroupName` - The Azure resource group name where the Azure virtual machine will be created.
- `TemplateFilePath` - The path to the json ARM-template generated by packer during image generation locally.*
- `VirtualMachineName` - The name of the virtual machine to be generated.
- `AdminUserName` - The administrator username for the virtual machine to be created.
- `AdminPassword` - The administrator password for the virtual machine to be created.
- `AzureLocation` - The location where the Azure virtual machine will be provisioned. Example: "eastus"
This function creates an Azure VM and generates network resources in Azure to make the VM accessible.
\* *ARM-template can be obtained from the Packer output. For now, it seems like there is an [Az CLI bug](https://github.com/Azure/azure-cli/issues/5899) with specifying the template through a URI, so download the template from URI, that will be printed at the bottom of image-generation log, and use the local path of the template file.*
## Automated image generation
The function creates an Azure VM from a template and generates network resources in Azure to make the VM accessible.
If you want to generate images automatically (e.g., as a part of a CI/CD pipeline),
you can use Packer directly. To do this, you will need:
## Manual image generation
- a build agent configured as described in the
[Build agent preparation](#build-agent-preparation) section;
- an Azure subscription and Service Principal configured as described in the
[Azure subscription authentication](#azure-subscription-authentication) section;
- a resource group created in your Azure subscription where the managed image will be stored;
- a string to be used as a password for the user used to install software (Windows only).
Then, you can invoke Packer in your CI/CD pipeline using the following commands:
```powershell
packer plugins install github.com/hashicorp/azure 2.2.1
packer build -only "$BuildName*" `
-var "subscription_id=$SubscriptionId" `
-var "client_id=$ClientId" `
-var "client_secret=$ClientSecret" `
-var "install_password=$InstallPassword" `
-var "location=$Location" `
-var "image_os=$ImageOS" `
-var "managed_image_name=$ImageName" `
-var "managed_image_resource_group_name=$ImageResourceGroupName" `
-var "tenant_id=$TenantId" `
$TemplatePath
```
Where:
- `BuildName` - name of the build defined in Packer template's `build{}` block (e.g. "ubuntu-24_04", "windows-2025");
- `SubscriptionId` - your Azure Subscription ID;
- `ClientId` and `ClientSecret` - Service Principal credentials;
- `TenantId` - Azure Tenant ID;
- `InstallPassword` - password for the user used to install software (Windows only);
- `Location` - location where resources will be created (e.g., "East US");
- `ImageOS` - the type of OS that will be deployed as a temporary VM (e.g. "ubuntu24", "win25");
- `ImageName` and `ImageResourceGroupName` - name of the resource group where the managed image will be stored;
- `TemplatePath` - path to the folder with Packer template files (e.g., "images/windows/templates").
If you want more control over image generation process you may run Packer directly. This section describes variables defined in Packer template. Some of them may be set using environment variabes.
### Required variables
The following variables are required to be passed to the Packer process:
| Template var | Env var | Description
| ------------ | ------- | -----------
| `subscription_id` | `ARM_SUBSCRIPTION_ID` | The subscription under which the build will be performed.
| `subscription_id` | `ARM_SUBSCRIPTION_ID` | Subscription under which the build will be performed.
| `client_id` | `ARM_CLIENT_ID` | The Active Directory service principal associated with your builder.
| `client_secret` | `ARM_CLIENT_SECRET` | The password or secret for your service principal; may be omitted if `client_cert_path` is set.
| `client_cert_path` | `ARM_CLIENT_CERT_PATH` | The location of a PEM file containing a certificate and private key for the service principal; may be omitted if `client_secret` is set.
| `location` | `ARM_RESOURCE_LOCATION` | The Azure datacenter in which your VM will be built.
| `managed_image_resource_group_name` | `ARM_RESOURCE_GROUP` | The resource group under which the final artifact will be stored.
| `client_cert_path` | `ARM_CLIENT_CERT_PATH` | The location of a PEM file containing a certificate and private key for service principal; may be omitted if `client_secret` is set.
| `location` | `ARM_RESOURCE_LOCATION` | Azure datacenter in which your VM will build.
| `resource_group` | `ARM_RESOURCE_GROUP` | Resource group under which the final artifact will be stored.
| `storage_account` | `ARM_STORAGE_ACCOUNT` | Storage account under which the final artifact will be stored.
### Optional variables
The following variables are optional:
- `build_resource_group_name` - Specify an existing resource group to run the build in it. By default, a temporary resource group will be created and destroyed as part of the build. If you do not have permission to do so, use build_resource_group_name to specify an existing resource group to run the build in it.
- `object_id` - The object ID for the AAD SP. Will be derived from the oAuth token if empty.
- `tenant_id` - The Active Directory tenant identifier with which your `client_id` and `subscription_id` are associated. If not specified, `tenant_id` will be looked up using `subscription_id`.
- `temp_resource_group_name` - Name assigned to the temporary resource group created during the build. If this value is not set, a random value will be assigned. This resource group is deleted at the end of the build.
- `private_virtual_network_with_public_ip` - This value allows you to set a `virtual_network_name` and obtain a public IP. If this value is not set and `virtual_network_name` is defined Packer is only allowed to be executed from a host on the same subnet / virtual network.
- `virtual_network_name` - Use a pre-existing virtual network for the VM. This option enables private communication with the VM, no public IP address is used or provisioned (unless you set `private_virtual_network_with_public_ip`).
- `virtual_network_resource_group_name` - If `virtual_network_name` is set, this value may also be set. If `virtual_network_name` is set, and this value is not set the builder attempts to determine the resource group containing the virtual network. If the resource group cannot be found, or it cannot be disambiguated, this value should be set.
- `virtual_network_subnet_name` - If `virtual_network_name` is set, this value may also be set. If `virtual_network_name` is set, and this value is not set the builder attempts to determine the subnet to use with the virtual network. If the subnet cannot be found, or it cannot be disambiguated, this value should be set.
- `capture_name_prefix` - VHD prefix. The final artifacts will be named PREFIX-osDisk.UUID and PREFIX-vmTemplate.UUID.
- `managed_image_name` - the name of the managed image to create. If not specified, "Runner-Image-{{ImageType}}" will be used;
- `build_resource_group_name` - specify an existing resource group to run the build in; by default, a temporary resource group will be created and destroyed as part of the build; if you do not have permission to do so, use `build_resource_group_name` to specify an existing resource group to run the build in;
- `object_id` - the object ID for the AAD SP; will be derived from the oAuth token if empty;
- `tenant_id` - the Active Directory tenant identifier with which your `client_id` and `subscription_id` are associated; if not specified, `tenant_id` will be looked up using `subscription_id`;
- `temp_resource_group_name` - the name assigned to the temporary resource group created during the build; if this value is not set, a random value will be assigned; this resource group is deleted at the end of the build;
- `private_virtual_network_with_public_ip` - this value allows you to set a `virtual_network_name` and obtain a public IP; if this value is not set and `virtual_network_name` is defined, Packer is only allowed to be executed from a host on the same subnet / virtual network;
- `virtual_network_name` - use a pre-existing virtual network for the VM; this option enables private communication with the VM, no public IP address is used or provisioned (unless you set `private_virtual_network_with_public_ip`);
- `virtual_network_resource_group_name` - if `virtual_network_name` is set, this value may also be set; if `virtual_network_name` is set, and this value is not set, the builder attempts to determine the resource group containing the virtual network; if the resource group cannot be found, or it cannot be disambiguated, this value should be set;
- `virtual_network_subnet_name` - if `virtual_network_name` is set, this value may also be set; if `virtual_network_name` is set, and this value is not set, the builder attempts to determine the subnet to use with the virtual network; if the subnet cannot be found, or it cannot be disambiguated, this value should be set.
## Builder variables
### Builder variables
The `builders` section contains variables for the `azure-arm` builder used in the project. Most of the builder variables are inherited from the `user variables` section, however, the variables can be overwritten to adjust image-generation performance.
- `vm_size` - the size of the VM used for building; this can be changed when you deploy a VM from your image;
- `image_os` - the type of OS that will be deployed as a temporary VM;
- `image_version` - specify the version of an OS to boot from.
- `vm_size` - Size of the VM used for building. This can be changed when you deploy a VM from your VHD.
- `image_os` - Type of OS that will be deployed as a temporary VM.
- `image_version` - Specify version of an OS to boot from.
**Detailed Azure builders documentation can be found in the [packer documentation](https://www.packer.io/docs/builders/azure).**
**Detailed Azure builders documentation can be found in [packer documentation](https://www.packer.io/docs/builders/azure).**
## Toolset
### Toolset
The configuration for some installed software is located in `toolset.json` files. These files define the list of Ruby, Python, Go versions, the list of PowerShell modules and VS components that will be installed on the image. They can be changed if these tools are not required, to reduce image generation time or image size.
Configuration for some installed software is located in `toolset.json` files. These files define the list of Ruby, Python, Go versions, the list of PowerShell modules and VS components that will be installed to image. They can be changed if these tools are not required to reduce image generation time or image size.
Generated tool versions and details can be found in related projects:
@@ -272,23 +217,23 @@ Generated tool versions and details can be found in related projects:
- [Go](https://github.com/actions/go-versions)
- [Node](https://github.com/actions/node-versions)
## Post-generation scripts
### Post-generation scripts
> :warning: These scripts are intended to be run on a VM deployed in Azure
> :warning: These scripts are intended to run on a VM deployed in Azure
The user, created during the image generation, does not exist in the resulting image. Hence, some configuration files related to the user's home directory need to be changed, as well as the file permissions for some directories. Scripts for that are located in the `post-gen` folder in the repository:
The user, created during the image generation, does not exist in the result VHD hence some configuration files related to the user's home directory need to be changed as well as the file permissions for some directories. Scripts for that are located in the `post-generation` folder in the repository:
- Windows: <https://github.com/actions/runner-images/tree/main/images/windows/assets/post-gen>
- Linux: <https://github.com/actions/runner-images/tree/main/images/ubuntu/assets/post-gen>
- Windows: <https://github.com/actions/runner-images/tree/main/images/win/post-generation>
- Linux: <https://github.com/actions/runner-images/tree/main/images/linux/post-generation>
**Note:** The default user for Linux should have `sudo privileges`.
The scripts are copied to the image during the generation process to the following paths:
The scripts are copied to the VHD during the image generation process to the following paths:
- Windows: `C:\post-generation`
- Linux: `/opt/post-generation`
### Running scripts
#### Running scripts
- Ubuntu
@@ -302,16 +247,18 @@ The scripts are copied to the image during the generation process to the followi
Get-ChildItem C:\post-generation -Filter *.ps1 | ForEach-Object { & $_.FullName }
```
### Script Details: Ubuntu
#### Script details
- **cleanup-logs.sh** - removes all build process logs from the machine;
- **environment-variables.sh** - replaces `$HOME` with the default user's home directory for environment variables related to the default user home directory;
- **homebrew-permissions.sh** - resets the Homebrew repository directory by running `git reset --hard` to make the working tree clean after changing permissions in /home and changes the repository directory owner to the current user;
- **rust-permissions.sh** - fixes permissions for the Rust folder; a detailed issue explanation is provided in [runner-images/issues/572](https://github.com/actions/runner-images/issues/572).
##### Ubuntu
### Script Details: Windows
- **cleanup-logs.sh** - removes all build process logs from the machine
- **environment-variables.sh** - replaces `$HOME` with the default user's home directory for environmental variables related to the default user home directory
- **homebrew-permissions.sh** - Resets homebrew repository directory by running `git reset --hard` to make the working tree clean after chmoding /home and changes the repository directory owner to the current user
- **rust-permissions.sh** - fixes permissions for the Rust folder. Detailed issue explanation is provided in [runner-images/issues/572](https://github.com/actions/runner-images/issues/572).
- **GenerateIISExpressCertificate.ps1** - generates and imports a certificate to run applications with IIS Express through HTTPS;
- **InternetExplorerConfiguration.ps1** - turns off the Internet Explorer Enhanced Security feature;
- **Msys2FirstLaunch.ps1** - initializes the bash user profile in MSYS2;
- **VSConfiguration.ps1** - performs initial Visual Studio configuration.
##### Windows
- **GenerateIISExpressCertificate.ps1** - generates and imports a certificate to run applications with IIS Express through HTTPS
- **InternetExplorerConfiguration** - turns off the Internet Explorer Enhanced Security feature
- **Msys2FirstLaunch.ps1** - initializes bash user profile in MSYS2
- **VSConfiguration.ps1** - performs initial Visual Studio configuration
+27
View File
@@ -0,0 +1,27 @@
# Debugging Failed Packer Builds
## Step 1: Run packer build `-on-error=ask`
When you run the `packer build` command, give it the `-on-error=ask` flag.
By default, `packer build` will delete the resource group as soon as the build fails.
`-on-error=ask` will pause it and wait for your input so you have time to remote in to the VM and diagnose the failure.
When the build fails, you will see this:
![Ask on error screenshot](/docs/resources/askOnError.png "Ask on error screenshot")
## Step 2: Find the resource group name in the build log
At the beginning of the build log (written to console), find the resource group name for the VM:
![Resource group from log screenshot](/docs/resources/resourceGroupName.png "Resource group from log screenshot")
Log into the Azure Portal. Find that resource group under `Resource groups`. You should see the resources for the Packer build:
![Packer resource group in Azure screenshot](/docs/resources/packerResourceGroup.png "Packer resource group in Azure screenshot")
## Step 3: Connect to the VM
Select the VM in the resource group. Click `Connect:`
This will download an RDP file. Open that and enter the credentials found in the JSON file you pass to `packer build`:
![VM credentials screenshot](/docs/resources/vmCredentials.png "VM credentials screenshot")
-34
View File
@@ -1,34 +0,0 @@
# Ubuntu .NET Core Versions
.NET has changed the recommended install methods for Ubuntu from 2404.
This document gives an overview of these change and the impact this has on the `runner-images`.
## .NET Core for Ubuntu 2004 and 2204
2004 and 2204 use the [Microsoft Package repository](https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu-install?tabs=dotnet8&pivots=os-linux-ubuntu-2004) to install .NET deb files built and published by the .NET team.
## .NET Core Versions from Ubuntu 2404
The .NET Core team have worked with Canonical and Ubuntu now provides its own .NET packages.
These are the recommended install path and, as-such what is installed on the image.
> The release of Ubuntu 24.04 is just around the corner. Canonical-produced .NET 6, 7, and 8 packages will be available on day one, for "Noble Numbat". Microsoft will not be publishing .NET packages to the 24.04 feed at packages.microsoft.com.
You can read the [full announcement from .NET team here](https://github.com/dotnet/core/discussions/9258). We'll briefly summarize how this change may impact users of the image.
### [`Feature Bands`](https://learn.microsoft.com/dotnet/core/porting/versioning-sdk-msbuild-vs)
Going forward only the `1xx` feature band will be present in the image as Ubuntu only build and publish this band.
> Most distros, including Ubuntu, stick to the .1xx feature band for the lifetime of a major .NET version. They make this choice because .1xx is (effectively) the "compatibility band". Higher bands can have breaking changes.
> This means there will no longer be packages available for .2xx and later feature bands. Such packages have been exclusively available from Microsoft. If users see an incompatibility between .1xx and higher feature bands, we ask that you please report it in the dotnet/sdk repo. [link: dotnet/core discussion](https://github.com/dotnet/core/discussions/9258)
If you need a higher feature band for your Actions the recommendation is to use the [`setup-dotnet`](https://github.com/actions/setup-dotnet) action to install the desired version.
### .NET MAUI
.NET MAUI is [not included](https://github.com/dotnet/core/discussions/9258#discussioncomment-9548857) in the Ubuntu .NET package. There is work [ongoing to fix.](https://github.com/dotnet/core/discussions/9258#discussioncomment-9548857)
You should be able to resolve this by using the [`setup-dotnet`](https://github.com/actions/setup-dotnet) action to install the desired version.
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 204 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

-42
View File
@@ -1,42 +0,0 @@
$ErrorActionPreference = 'Stop'
# A JSON schema validator which supports outputting line numbers for errors
# this allows us to put annotations on builds for errors in the JSON files
# `Test-Json` built in cmdline doesn't. No existing cli tool supports this
# that I could find either. See: https://github.com/lawrencegripper/gripdev-json-schema-validator
Install-Module -Name GripDevJsonSchemaValidator -Force -Scope CurrentUser
# Find all toolset JSON files
$toolsetFiles = Get-ChildItem -Recurse -Filter "toolset-*.json" | Where-Object { $_.Name -notlike "*schema.json" }
$schemaFilePath = "./schemas/toolset-schema.json"
$toolsetHasErrors = $false
foreach ($file in $toolsetFiles) {
Write-Host ""
Write-Host "🔍 Validating $($file.FullName)" -ForegroundColor Cyan
$validationResult = Test-JsonSchema -SchemaPath $schemaFilePath -JsonPath $file.FullName -PrettyPrint $false
if ($validationResult.Valid) {
Write-Host "✅ JSON is valid." -ForegroundColor Green
} else {
# File has been modified since the commit, enforce validation
$toolsetHasErrors = $true
Write-Host "`n❌ JSON validation failed!" -ForegroundColor Red
Write-Host " Found the following errors:`n" -ForegroundColor Yellow
$validationResult.Errors | ForEach-Object {
Write-Host $_.UserMessage
if ($env:GITHUB_ACTIONS -eq 'true') {
Write-Host "Adding annotation"
Write-Host "::error file=$($file.Name),line=$($_.LineNumber)::$($_.UserMessage.Replace("`n", '%0A'))"
}
}
}
}
if ($toolsetHasErrors) {
Write-Error "One or more toolset JSON files failed schema validation. See the error output above for more details."
} else {
Write-Host "Schema validation completed successfully"
}
-85
View File
@@ -1,85 +0,0 @@
$ErrorActionPreference = 'Stop'
# Find all toolset JSON files
$toolsetFiles = Get-ChildItem -Recurse -Filter "toolset-*.json" | Where-Object { $_.Name -notlike "*schema.json" }
$expiringPins = @()
$now = Get-Date
$warningDays = 30 # Warn if expiring within 30 days
foreach ($file in $toolsetFiles) {
Write-Host "Processing $($file.Name)"
$content = Get-Content $file.FullName | ConvertFrom-Json
# Recursively search for pinnedDetails in the JSON
function Search-PinnedDetails {
param($obj, $path)
$foundPins = @()
if ($obj -is [System.Management.Automation.PSCustomObject]) {
foreach ($prop in $obj.PSObject.Properties) {
if ($prop.Name -eq "pinnedDetails") {
Write-Host "Found pinned version at $path"
$reviewAt = [DateTime]::Parse($prop.Value.'review-at')
$daysUntilExpiry = ($reviewAt - $now).Days
if ($daysUntilExpiry -lt $warningDays) {
Write-Host "Adding to expiringPins array"
$foundPins += @{
Path = $path
File = $file.Name
ReviewAt = $reviewAt
DaysUntilExpiry = $daysUntilExpiry
Reason = $prop.Value.reason
Link = $prop.Value.link
}
}
} else {
$foundPins += Search-PinnedDetails -obj $prop.Value -path "$path.$($prop.Name)"
}
}
} elseif ($obj -is [Array]) {
for ($i = 0; $i -lt $obj.Count; $i++) {
$foundPins += Search-PinnedDetails -obj $obj[$i] -path "$path[$i]"
}
}
return $foundPins
}
$expiringPins += Search-PinnedDetails -obj $content -path $file.Name
}
if ($expiringPins) {
$issueBody = "# Version Pinning Review Required`n`n"
$issueBody += "The following pinned versions need review:`n`n"
foreach ($pin in $expiringPins) {
$status = if ($pin.DaysUntilExpiry -lt 0) { "EXPIRED" } else { "Expiring Soon" }
$issueBody += "## $($status) - $($pin.Path)`n"
$issueBody += "- **File**: $($pin.File)`n"
$issueBody += "- **Review Date**: $($pin.ReviewAt.ToString('yyyy-MM-dd'))`n"
$issueBody += "- **Days until expiry**: $($pin.DaysUntilExpiry)`n"
$issueBody += "- **Reason**: $($pin.Reason)`n"
$issueBody += "- **Original PR**: $($pin.Link)`n`n"
}
if ($env:GITHUB_ACTIONS -eq 'true') {
# In GitHub Actions, create an issue
Write-Host "Creating issue"
$tempFile = [System.IO.Path]::GetTempFileName()
Set-Content -Path $tempFile -Value $issueBody
gh issue create --title "Version Pinning Review Found Expired Pinned Versions" --body-file $tempFile
Remove-Item -Path $tempFile
}
Write-Host "`nIssue Content:`n"
Write-Host $issueBody
}
else {
Write-Host "No expiring pins found."
if ($env:GITHUB_ACTIONS -eq 'true') {
"expired_pins=0" >> $env:GITHUB_OUTPUT
}
}
+6 -15
View File
@@ -12,8 +12,8 @@ Function CreateAzureVMFromPackerTemplate {
.PARAMETER ResourceGroupName
The Azure resource group name where the Azure virtual machine will be created.
.PARAMETER ManagedImageName
The name of the managed image to be used to create the virtual machine.
.PARAMETER TemplatFilePath
The path for the json template generated by packer during image generation locally.
.PARAMETER VirtualMachineName
The name of the virtual machine to be generated.
@@ -28,7 +28,7 @@ Function CreateAzureVMFromPackerTemplate {
The location where the Azure virtual machine will be provisioned. Example: "eastus"
.EXAMPLE
CreateAzureVMFromPackerTemplate -SubscriptionId {SubscriptionId} -ResourceGroupName {ResourceGroupName} -VirtualMachineName "testvm1" -ManagedImageName {ManagedImageName} -AdminUsername "shady1" -AdminPassword "SomeSecurePassword1" -AzureLocation "eastus"
CreateAzureVMFromPackerTemplate -SubscriptionId {YourSubscriptionId} -ResourceGroupName {ResourceGroupName} -TemplateFile "C:\BuildVmImages\temporaryTemplate.json" -VirtualMachineName "testvm1" -AdminUsername "shady1" -AdminPassword "SomeSecurePassword1" -AzureLocation "eastus"
#>
param (
[Parameter(Mandatory = $True)]
@@ -36,7 +36,7 @@ Function CreateAzureVMFromPackerTemplate {
[Parameter(Mandatory = $True)]
[string] $ResourceGroupName,
[Parameter(Mandatory = $True)]
[string] $ManagedImageName,
[string] $TemplateFilePath,
[Parameter(Mandatory = $True)]
[string] $VirtualMachineName,
[Parameter(Mandatory = $True)]
@@ -52,7 +52,7 @@ Function CreateAzureVMFromPackerTemplate {
$vnetName = $env:UserName + "vnet-" + $guid
$subnetName = $env:UserName + "subnet-" + $guid
$nicName = $env:UserName + "nic-" + $guid
$publicIpName = $env:UserName + "pip-" + $guid
$publicIpName = $env:UserName + "pip-" + $guid
Write-Host "Creating a virtual network and subnet"
($vnet = az network vnet create -g $ResourceGroupName -l $AzureLocation -n $vnetName --address-prefixes 10.0.0.0/16 --subnet-name $subnetName --subnet-prefixes 10.0.1.0/24 --subscription $subscriptionId -o json)
@@ -70,16 +70,7 @@ Function CreateAzureVMFromPackerTemplate {
az network nic ip-config update -g $ResourceGroupName -n ipconfig1 --nic-name $nicName --public-ip-address $publicIpId --subscription $subscriptionId
Write-Host "`nCreating the VM"
az vm create `
--resource-group $ResourceGroupName `
--name $VirtualMachineName `
--image $ManagedImageName `
--size $vmSize `
--admin-username $AdminUsername `
--admin-password $AdminPassword `
--nics $networkId `
--subscription $subscriptionId `
--location $AzureLocation
az deployment group create -g $ResourceGroupName -n $VirtualMachineName --subscription $subscriptionId --template-file $templateFilePath --parameters vmSize=$vmSize vmName=$VirtualMachineName adminUserName=$AdminUsername adminPassword=$AdminPassword networkInterfaceId=$networkId
Write-Host "`nCreated in ${ResourceGroupName}:`n vnet ${vnetName}`n subnet ${subnetName}`n nic ${nicName}`n publicip ${publicIpName}`n vm ${VirtualMachineName}"
}
+164 -80
View File
@@ -3,12 +3,12 @@ $ErrorActionPreference = 'Stop'
enum ImageType {
Windows2019 = 1
Windows2022 = 2
Windows2025 = 3
Ubuntu2004 = 3
Ubuntu2204 = 4
Ubuntu2404 = 5
UbuntuMinimal = 5
}
Function Get-PackerTemplate {
Function Get-PackerTemplatePath {
param (
[Parameter(Mandatory = $True)]
[string] $RepositoryRoot,
@@ -17,43 +17,31 @@ Function Get-PackerTemplate {
)
switch ($ImageType) {
# Note: Double Join-Path is required to support PowerShell 5.1
([ImageType]::Windows2019) {
$relativeTemplatePath = Join-Path (Join-Path "windows" "templates") "build.windows-2019.pkr.hcl"
$imageOS = "win19"
$relativeTemplatePath = Join-Path "win" "windows2019.json"
}
([ImageType]::Windows2022) {
$relativeTemplatePath = Join-Path (Join-Path "windows" "templates") "build.windows-2022.pkr.hcl"
$imageOS = "win22"
$relativeTemplatePath = Join-Path "win" "windows2022.json"
}
([ImageType]::Windows2025) {
$relativeTemplatePath = Join-Path (Join-Path "windows" "templates") "build.windows-2025.pkr.hcl"
$imageOS = "win25"
([ImageType]::Ubuntu2004) {
$relativeTemplatePath = Join-Path "linux" "ubuntu2004.json"
}
([ImageType]::Ubuntu2204) {
$relativeTemplatePath = Join-Path (Join-Path "ubuntu" "templates") "build.ubuntu-22_04.pkr.hcl"
$imageOS = "ubuntu22"
$relativeTemplatePath = Join-Path "linux" "ubuntu2204.pkr.hcl"
}
([ImageType]::Ubuntu2404) {
$relativeTemplatePath = Join-Path (Join-Path "ubuntu" "templates") "build.ubuntu-24_04.pkr.hcl"
$imageOS = "ubuntu24"
([ImageType]::UbuntuMinimal) {
$relativeTemplatePath = Join-Path "linux" "ubuntuminimal.pkr.hcl"
}
default { throw "Unknown type of image" }
}
$imageTemplatePath = [IO.Path]::Combine($RepositoryRoot, "images", $relativeTemplatePath)
# Specific template selection using Packer's "-only" functionality
$buildName = [IO.Path]::GetFileName($imageTemplatePath).Split(".")[1]
if (-not (Test-Path $imageTemplatePath)) {
throw "Template for image '$ImageType' doesn't exist on path '$imageTemplatePath'."
}
return [PSCustomObject] @{
"BuildName" = $buildName
"ImageOS" = $imageOS
"Path" = [IO.Path]::GetDirectoryName($imageTemplatePath)
}
return $imageTemplatePath;
}
Function Show-LatestCommit {
@@ -86,11 +74,9 @@ Function GenerateResourcesAndImage {
.PARAMETER SubscriptionId
The Azure subscription id where the Azure resources will be created.
.PARAMETER ResourceGroupName
The name of the resource group to store the resulting artifact. Resource group must already exist.
The name of the resource group to create the Azure resources in.
.PARAMETER ImageType
The type of image to generate. Valid values are: Windows2019, Windows2022, Windows2025, Ubuntu2204, Ubuntu2404.
.PARAMETER ManagedImageName
The name of the managed image to create. The default is "Runner-Image-{{ImageType}}".
The type of image to generate. Valid values are: Windows2019, Windows2022, Ubuntu2004, Ubuntu2204, UbuntuMinimal.
.PARAMETER AzureLocation
The Azure location where the Azure resources will be created. For example: "East US"
.PARAMETER ImageGenerationRepositoryRoot
@@ -106,6 +92,14 @@ Function GenerateResourcesAndImage {
.PARAMETER RestrictToAgentIpAddress
If set, access to the VM used by packer to generate the image is restricted to the public IP address this script is run from.
This parameter cannot be used in combination with the virtual_network_name packer parameter.
.PARAMETER Force
Delete the resource group if it exists without user confirmation.
.PARAMETER ReuseResourceGroup
Reuse the resource group if it exists without user confirmation.
.PARAMETER AllowBlobPublicAccess
Allow public access to the generated image blob.
.PARAMETER EnableHttpsTrafficOnly
Enable https traffic only for the generated image blob.
.PARAMETER OnError
Specify how packer handles an error during image creation.
Options:
@@ -116,10 +110,8 @@ Function GenerateResourcesAndImage {
The default is 'ask'.
.PARAMETER Tags
Tags to be applied to the Azure resources created.
.PARAMETER PluginVersion
Specify the version of the packer Azure plugin to use. The default is "2.2.1".
.EXAMPLE
GenerateResourcesAndImage -SubscriptionId {YourSubscriptionId} -ResourceGroupName "shsamytest1" -ImageGenerationRepositoryRoot "C:\runner-images" -ImageType Ubuntu2204 -AzureLocation "East US"
GenerateResourcesAndImage -SubscriptionId {YourSubscriptionId} -ResourceGroupName "shsamytest1" -ImageGenerationRepositoryRoot "C:\runner-images" -ImageType Ubuntu2004 -AzureLocation "East US"
#>
param (
[Parameter(Mandatory = $True)]
@@ -128,8 +120,6 @@ Function GenerateResourcesAndImage {
[string] $ResourceGroupName,
[Parameter(Mandatory = $True)]
[ImageType] $ImageType,
[Parameter(Mandatory = $False)]
[string] $ManagedImageName = "Runner-Image-$($ImageType)",
[Parameter(Mandatory = $True)]
[string] $AzureLocation,
[Parameter(Mandatory = $False)]
@@ -143,16 +133,26 @@ Function GenerateResourcesAndImage {
[Parameter(Mandatory = $False)]
[string] $AzureTenantId,
[Parameter(Mandatory = $False)]
[string] $PluginVersion = "2.2.1",
[Parameter(Mandatory = $False)]
[switch] $RestrictToAgentIpAddress,
[Parameter(Mandatory = $False)]
[switch] $Force,
[Parameter(Mandatory = $False)]
[switch] $ReuseResourceGroup,
[Parameter(Mandatory = $False)]
[bool] $AllowBlobPublicAccess = $False,
[Parameter(Mandatory = $False)]
[bool] $EnableHttpsTrafficOnly = $False,
[Parameter(Mandatory = $False)]
[ValidateSet("abort", "ask", "cleanup", "run-cleanup-provisioner")]
[string] $OnError = "ask",
[Parameter(Mandatory = $False)]
[hashtable] $Tags = @{}
)
if ($Force -and $ReuseResourceGroup) {
throw "Force and ReuseResourceGroup cannot be used together."
}
Show-LatestCommit -ErrorAction SilentlyContinue
# Validate packer is installed
@@ -162,31 +162,33 @@ Function GenerateResourcesAndImage {
}
# Get template path
$PackerTemplate = Get-PackerTemplate -RepositoryRoot $ImageGenerationRepositoryRoot -ImageType $ImageType
Write-Debug "Template path: $($PackerTemplate.Path)."
$TemplatePath = Get-PackerTemplatePath -RepositoryRoot $ImageGenerationRepositoryRoot -ImageType $ImageType
Write-Debug "Template path: $TemplatePath."
# Prepare list of allowed inbound IP addresses
if ($RestrictToAgentIpAddress) {
$AgentIp = (Invoke-RestMethod https://ipinfo.io/json).ip
$AgentIp = (Invoke-RestMethod http://ipinfo.io/json).ip
if (-not $AgentIp) {
throw "Unable to determine agent IP address."
}
Write-Host "Access to packer generated VM will be restricted to agent IP Address: $AgentIp."
if ($PSVersionTable.PSVersion.Major -eq 5) {
Write-Verbose "PowerShell 5 detected. Replacing double quotes with escaped double quotes in allowed inbound IP addresses."
$AllowedInboundIpAddresses = '[\"{0}\"]' -f $AgentIp
if ($TemplatePath.Contains("pkr.hcl")) {
if ($PSVersionTable.PSVersion.Major -eq 5) {
Write-Verbose "PowerShell 5 detected. Replacing double quotes with escaped double quotes in allowed inbound IP addresses."
$AllowedInboundIpAddresses = '[\"{0}\"]' -f $AgentIp
} else {
$AllowedInboundIpAddresses = '["{0}"]' -f $AgentIp
}
} else {
$AllowedInboundIpAddresses = $AgentIp
}
elseif ($PSVersionTable.PSVersion.Major -eq 7 -and $PSVersionTable.PSVersion.Minor -le 2) {
Write-Verbose "PowerShell 7.0-7.2 detected. Replacing double quotes with escaped double quotes in allowed inbound IP addresses."
$AllowedInboundIpAddresses = '[\"{0}\"]' -f $AgentIp
} else {
if ($TemplatePath.Contains("pkr.hcl")) {
$AllowedInboundIpAddresses = "[]"
} else {
$AllowedInboundIpAddresses = ""
}
else {
$AllowedInboundIpAddresses = '["{0}"]' -f $AgentIp
}
}
else {
$AllowedInboundIpAddresses = "[]"
}
Write-Debug "Allowed inbound IP addresses: $AllowedInboundIpAddresses."
@@ -198,37 +200,34 @@ Function GenerateResourcesAndImage {
Write-Verbose "PowerShell 5 detected. Replacing double quotes with escaped double quotes in tags JSON."
$TagsJson = $TagsJson -replace '"', '\"'
}
elseif ($PSVersionTable.PSVersion.Major -eq 7 -and $PSVersionTable.PSVersion.Minor -le 2) {
Write-Verbose "PowerShell 7.0-7.2 detected. Replacing double quotes with escaped double quotes in tags JSON."
$TagsJson = $TagsJson -replace '"', '\"'
}
Write-Debug "Tags JSON: $TagsJson."
if ($TemplatePath.Contains(".json")) {
Write-Verbose "Injecting tags into packer template."
if ($Tags) {
$BuilderScriptPathInjected = $TemplatePath.Replace(".json", "-temp.json")
$PackerTemplateContent = Get-Content -Path $TemplatePath | ConvertFrom-Json
$PackerTemplateContent.builders | Add-Member -Name "azure_tags" -Value $Tags -MemberType NoteProperty
$PackerTemplateContent | ConvertTo-Json -Depth 3 | Out-File -Encoding Ascii $BuilderScriptPathInjected
$TemplatePath = $BuilderScriptPathInjected
}
}
$InstallPassword = $env:UserName + [System.GUID]::NewGuid().ToString().ToUpper()
Write-Host "Downloading packer plugins..."
& $PackerBinary plugins install github.com/hashicorp/azure $PluginVersion
if ($LastExitCode -ne 0) {
throw "Packer plugins download failed."
}
Write-Host "Validating packer template..."
& $PackerBinary validate `
"-only=$($PackerTemplate.BuildName)*" `
"-var=client_id=fake" `
"-var=client_secret=fake" `
"-var=subscription_id=$($SubscriptionId)" `
"-var=tenant_id=fake" `
"-var=location=$($AzureLocation)" `
"-var=image_os=$($PackerTemplate.ImageOS)" `
"-var=managed_image_name=$($ManagedImageName)" `
"-var=managed_image_resource_group_name=$($ResourceGroupName)" `
"-var=resource_group=$($ResourceGroupName)" `
"-var=storage_account=fake" `
"-var=install_password=$($InstallPassword)" `
"-var=allowed_inbound_ip_addresses=$($AllowedInboundIpAddresses)" `
"-var=azure_tags=$($TagsJson)" `
$PackerTemplate.Path
$TemplatePath
if ($LastExitCode -ne 0) {
throw "Packer template validation failed."
}
@@ -238,10 +237,9 @@ Function GenerateResourcesAndImage {
if ([string]::IsNullOrEmpty($AzureClientId)) {
Write-Verbose "No AzureClientId was provided, will use interactive login."
az login --output none
}
else {
} else {
Write-Verbose "AzureClientId was provided, will use service principal login."
az login --service-principal --username $AzureClientId --password=$AzureClientSecret --tenant $AzureTenantId --output none
az login --service-principal --username $AzureClientId --password $AzureClientSecret --tenant $AzureTenantId --output none
}
az account set --subscription $SubscriptionId
if ($LastExitCode -ne 0) {
@@ -253,8 +251,97 @@ Function GenerateResourcesAndImage {
if ($ResourceGroupExists) {
Write-Verbose "Resource group '$ResourceGroupName' already exists."
}
else {
throw "Resource group '$ResourceGroupName' does not exist."
# Remove resource group if it exists and we are not reusing it
if ($ResourceGroupExists -and -not $ReuseResourceGroup) {
if ($Force) {
# Delete and recreate the resource group
Write-Host "Deleting resource group '$ResourceGroupName'..."
az group delete --name $ResourceGroupName --yes --output none
if ($LastExitCode -ne 0) {
throw "Failed to delete resource group '$ResourceGroupName'."
}
Write-Host "Resource group '$ResourceGroupName' was deleted."
$ResourceGroupExists = $false
} else {
# Resource group already exists, ask the user what to do
$title = "Resource group '$ResourceGroupName' already exists"
$message = "Do you want to delete the resource group and all resources in it?"
$options = @(
[System.Management.Automation.Host.ChoiceDescription]::new("&Yes", "Delete the resource group and all resources in it."),
[System.Management.Automation.Host.ChoiceDescription]::new("&No", "Keep the resource group and continue."),
[System.Management.Automation.Host.ChoiceDescription]::new("&Abort", "Abort execution.")
)
$result = $Host.UI.PromptForChoice($title, $message, $options, 0)
switch ($result) {
0 {
# Delete and recreate the resource group
Write-Host "Deleting resource group '$ResourceGroupName'..."
az group delete --name $ResourceGroupName --yes
if ($LastExitCode -ne 0) {
throw "Failed to delete resource group '$ResourceGroupName'."
}
Write-Host "Resource group '$ResourceGroupName' was deleted."
$ResourceGroupExists = $false
}
1 {
# Keep the resource group and continue
}
2 {
# Stop the current action
Write-Error "User stopped the action."
exit 1
}
}
}
}
# Create resource group
if (-not $ResourceGroupExists) {
Write-Host "Creating resource group '$ResourceGroupName' in location '$AzureLocation'..."
if ($TagsList) {
az group create --name $ResourceGroupName --location $AzureLocation --tags $TagsList --query id
} else {
az group create --name $ResourceGroupName --location $AzureLocation --query id
}
if ($LastExitCode -ne 0) {
throw "Failed to create resource group '$ResourceGroupName'."
}
}
# Generate proper name for the storage account that follows the recommended naming conventions for azure resources
$StorageAccountName = $ResourceGroupName
if ($ResourceGroupName.EndsWith("-rg")) {
$StorageAccountName = $ResourceGroupName.Substring(0, $ResourceGroupName.Length - 3)
}
$StorageAccountName = $StorageAccountName.Replace("-", "").Replace("_", "").Replace("(", "").Replace(")", "").ToLower()
$StorageAccountName += "001"
if ($StorageAccountName.Length -gt 24) {
$StorageAccountName = $StorageAccountName.Substring(0, 24)
}
try {
$StorageAccountId = (az storage account show --name $StorageAccountName --resource-group $ResourceGroupName --query id 2>$null)
$StorageAccountExists = "$StorageAccountId" -ne ""
} catch {
$StorageAccountExists = $false
}
# Create storage account
if ($StorageAccountExists) {
Write-Verbose "Storage account '$StorageAccountName' already exists."
} else {
Write-Host "Creating storage account..."
if ($TagsList) {
az storage account create --name $StorageAccountName --resource-group $ResourceGroupName --location $AzureLocation --sku Standard_LRS --allow-blob-public-access $AllowBlobPublicAccess --https-only $EnableHttpsTrafficOnly --min-tls-version TLS1_2 --tags $TagsList --query id
} else {
az storage account create --name $StorageAccountName --resource-group $ResourceGroupName --location $AzureLocation --sku Standard_LRS --allow-blob-public-access $AllowBlobPublicAccess --https-only $EnableHttpsTrafficOnly --min-tls-version TLS1_2 --query id
}
if ($LastExitCode -ne 0) {
throw "Failed to create storage account '$StorageAccountName'."
}
}
# Create service principal
@@ -267,7 +354,7 @@ Function GenerateResourcesAndImage {
if ($LastExitCode -ne 0) {
throw "Failed to create service principal '$ServicePrincipalName'."
}
$ServicePrincipalAppId = $ServicePrincipal.appId
$ServicePrincipalPassword = $ServicePrincipal.password
$TenantId = $ServicePrincipal.tenant
@@ -275,8 +362,7 @@ Function GenerateResourcesAndImage {
Write-Verbose "Waiting for service principal to propagate..."
Start-Sleep $SecondsToWaitForServicePrincipalSetup
Write-Host "Service principal created with id '$ServicePrincipalAppId'. It will be deleted after the build."
}
else {
} else {
$ServicePrincipalAppId = $AzureClientId
$ServicePrincipalPassword = $AzureClientSecret
$TenantId = $AzureTenantId
@@ -285,19 +371,17 @@ Function GenerateResourcesAndImage {
Write-Debug "Tenant id: $TenantId."
& $PackerBinary build -on-error="$($OnError)" `
-only "$($PackerTemplate.BuildName)*" `
-var "client_id=$($ServicePrincipalAppId)" `
-var "client_secret=$($ServicePrincipalPassword)" `
-var "subscription_id=$($SubscriptionId)" `
-var "tenant_id=$($TenantId)" `
-var "location=$($AzureLocation)" `
-var "image_os=$($PackerTemplate.ImageOS)" `
-var "managed_image_name=$($ManagedImageName)" `
-var "managed_image_resource_group_name=$($ResourceGroupName)" `
-var "resource_group=$($ResourceGroupName)" `
-var "storage_account=$($StorageAccountName)" `
-var "install_password=$($InstallPassword)" `
-var "allowed_inbound_ip_addresses=$($AllowedInboundIpAddresses)" `
-var "azure_tags=$($TagsJson)" `
$PackerTemplate.Path
$TemplatePath
if ($LastExitCode -ne 0) {
throw "Failed to build image."
@@ -306,7 +390,7 @@ Function GenerateResourcesAndImage {
Write-Error $_
} finally {
Write-Verbose "`nCleaning up..."
# Remove ADServicePrincipal and ADApplication
if ($ADCleanupRequired) {
Write-Host "Removing ADServicePrincipal..."
-97
View File
@@ -1,97 +0,0 @@
class GithubApi
{
[string] $Repository
[object] hidden $AuthHeader
GithubApi(
[string] $Repository,
[string] $AccessToken
) {
$this.Repository = $Repository
$this.AuthHeader = $this.BuildAuth($AccessToken)
}
[object] hidden BuildAuth([string]$AccessToken) {
if ([string]::IsNullOrEmpty($AccessToken)) {
return $null
}
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("'':${AccessToken}"))
return @{
Authorization = "Basic ${base64AuthInfo}"
}
}
[string] hidden BuildBaseUrl([string]$Repository, [string]$ApiPrefix) {
return "https://$ApiPrefix.github.com/repos/$Repository"
}
[object] GetWorkflowRuns([string]$WorkflowId) {
$url = "actions/workflows/$WorkflowId/runs"
$response = $this.InvokeRestMethod($url, 'GET', $null, $null)
return $response
}
[object] GetWorkflowRun([string]$WorkflowRunId) {
$url = "actions/runs/$WorkflowRunId"
$response = $this.InvokeRestMethod($url, 'GET', $null, $null)
return $response
}
[object] DispatchWorkflow([string]$EventType, [object]$EventPayload) {
$url = "dispatches"
$body = @{
"event_type" = $EventType
"client_payload" = $EventPayload
} | ConvertTo-Json
$response = $this.InvokeRestMethod($url, 'POST', $null, $body)
return $response
}
[object] CancelWorkflowRun([string]$workflowRunId) {
$url = "actions/runs/$workflowRunId/cancel"
$response = $this.InvokeRestMethod($url, 'POST', $null, $null)
return $response
}
[string] hidden BuildUrl([string]$url, [string]$RequestParams, [string]$ApiPrefix) {
$baseUrl = $this.BuildBaseUrl($this.Repository, $ApiPrefix)
if ([string]::IsNullOrEmpty($RequestParams)) {
return "$($baseUrl)/$($url)"
} else {
return "$($baseUrl)/$($url)?$($requestParams)"
}
}
[object] hidden InvokeRestMethod(
[string] $url,
[string] $Method,
[string] $RequestParams,
[string] $body
) {
$requestUrl = $this.BuildUrl($url, $RequestParams, "api")
$params = @{
Method = $Method
ContentType = "application/json"
Uri = $requestUrl
Headers = @{}
}
if ($this.AuthHeader) {
$params.Headers += $this.AuthHeader
}
if (![string]::IsNullOrEmpty($body)) {
$params.Body = $body
}
$response = Invoke-RestMethod @params
return $response
}
}
function Get-GithubApi {
param (
[string] $Repository,
[string] $AccessToken
)
return [GithubApi]::New($Repository, $AccessToken)
}
-47
View File
@@ -1,47 +0,0 @@
Param (
[Parameter(Mandatory)]
[string] $WorkflowRunId,
[Parameter(Mandatory)]
[string] $Repository,
[Parameter(Mandatory)]
[string] $AccessToken,
[int] $RetryIntervalSeconds = 300,
[int] $MaxRetryCount = 0
)
Import-Module (Join-Path $PSScriptRoot "GitHubApi.psm1")
function Wait-ForWorkflowCompletion($WorkflowRunId, $RetryIntervalSeconds) {
do {
Start-Sleep -Seconds $RetryIntervalSeconds
$workflowRun = $gitHubApi.GetWorkflowRun($WorkflowRunId)
} until ($workflowRun.status -eq "completed")
return $workflowRun
}
$gitHubApi = Get-GithubApi -Repository $Repository -AccessToken $AccessToken
$attempt = 1
do {
$finishedWorkflowRun = Wait-ForWorkflowCompletion -WorkflowRunId $WorkflowRunId -RetryIntervalSeconds $RetryIntervalSeconds
Write-Host "Workflow run finished with result: $($finishedWorkflowRun.conclusion)"
if ($finishedWorkflowRun.conclusion -in ("success", "cancelled", "timed_out")) {
break
} elseif ($finishedWorkflowRun.conclusion -eq "failure") {
if ($attempt -le $MaxRetryCount) {
Write-Host "Workflow run will be restarted. Attempt $attempt of $MaxRetryCount"
$gitHubApi.ReRunFailedJobs($WorkflowRunId)
$attempt += 1
} else {
break
}
}
} while ($true)
Write-Host "Last result: $($finishedWorkflowRun.conclusion)."
"CI_WORKFLOW_RUN_RESULT=$($finishedWorkflowRun.conclusion)" | Out-File -Append -FilePath $env:GITHUB_ENV
if ($finishedWorkflowRun.conclusion -in ("failure", "cancelled", "timed_out")) {
exit 1
}
@@ -13,15 +13,15 @@ class BaseNode {
}
[String] ToMarkdown([Int32] $Level) {
throw "Abstract method 'ToMarkdown(level)' is not implemented for '$($this.GetType().Name)'"
throw "Abtract method 'ToMarkdown(level)' is not implemented for '$($this.GetType().Name)'"
}
[Boolean] IsSimilarTo([BaseNode] $OtherNode) {
throw "Abstract method 'IsSimilarTo' is not implemented for '$($this.GetType().Name)'"
throw "Abtract method 'IsSimilarTo' is not implemented for '$($this.GetType().Name)'"
}
[Boolean] IsIdenticalTo([BaseNode] $OtherNode) {
throw "Abstract method 'IsIdenticalTo' is not implemented for '$($this.GetType().Name)'"
throw "Abtract method 'IsIdenticalTo' is not implemented for '$($this.GetType().Name)'"
}
}
@@ -39,7 +39,7 @@ class BaseToolNode: BaseNode {
}
[String] GetValue() {
throw "Abstract method 'GetValue' is not implemented for '$($this.GetType().Name)'"
throw "Abtract method 'GetValue' is not implemented for '$($this.GetType().Name)'"
}
[Boolean] IsSimilarTo([BaseNode] $OtherNode) {
@@ -53,4 +53,4 @@ class BaseToolNode: BaseNode {
[Boolean] IsIdenticalTo([BaseNode] $OtherNode) {
return $this.IsSimilarTo($OtherNode) -and ($this.GetValue() -eq $OtherNode.GetValue())
}
}
}
@@ -89,7 +89,7 @@ class HeaderNode: BaseNode {
}
[void] AddTable([PSCustomObject[]] $Table) {
$this.AddNode([TableNode]::FromObjectsArray($Table))
$this.AddNode([TableNode]::FromObjectsArray($Table))
}
[void] AddNote([String] $Content) {
@@ -436,4 +436,4 @@ class NoteNode: BaseNode {
[Boolean] IsIdenticalTo([BaseNode] $OtherNode) {
return $this.IsSimilarTo($OtherNode)
}
}
}
+14
View File
@@ -0,0 +1,14 @@
param(
[String] [Parameter (Mandatory=$true)] $RepoUrl,
[String] [Parameter (Mandatory=$true)] $RepoBranch
)
Write-Host "Clean up default repository"
Remove-Item -path './*' -Recurse -Force
Write-Host "Download ${RepoBranch} branch from ${RepoUrl}"
$env:GIT_REDIRECT_STDERR = '2>&1'
git clone $RepoUrl . -b $RepoBranch --single-branch --depth 1
Write-Host "Latest commit:"
git --no-pager log --pretty=format:"Date: %cd; Commit: %H - %s; Author: %an <%ae>" -1
@@ -0,0 +1,165 @@
# Ideally we would use GitHub Actions for this, but since we use self-hosted machines to run image builds
# we need the following features to use GitHub Actions for Images CI:
# - https://github.community/t5/GitHub-Actions/Make-secrets-available-to-builds-of-forks/m-p/30678#M508
# - https://github.community/t5/GitHub-Actions/GitHub-Actions-Manual-Trigger-Approvals/td-p/31504
# - https://github.community/t5/GitHub-Actions/Protecting-github-workflows/td-p/30290
parameters:
- name: job_id
type: string
default: 'generate_image'
- name: image_type
type: string
- name: image_readme_name
type: string
- name: agent_pool
type: object
default:
name: 'ci-agent-pool'
- name: variable_group_name
type: string
default: 'Image Generation Variables'
- name: create_release
type: boolean
default: true
- name: repository_ref
type: string
default: 'self'
jobs:
- job: ${{ parameters.job_id }}
displayName: Image Generation (${{ parameters.image_type }})
timeoutInMinutes: 600
cancelTimeoutInMinutes: 30
pool: ${{ parameters.agent_pool }}
variables:
- group: ${{ parameters.variable_group_name }}
steps:
- checkout: ${{ parameters.repository_ref }}
clean: true
fetchDepth: 0
fetchTags: false
- task: PowerShell@2
displayName: 'Download custom repository'
condition: and(ne(variables['CUSTOM_REPOSITORY_URL'], ''), ne(variables['CUSTOM_REPOSITORY_BRANCH'], ''))
inputs:
targetType: 'filePath'
filePath: ./images.CI/download-repo.ps1
arguments: -RepoUrl $(CUSTOM_REPOSITORY_URL) `
-RepoBranch $(CUSTOM_REPOSITORY_BRANCH)
- task: PowerShell@2
displayName: 'Set image template variables'
inputs:
targetType: 'inline'
script: |
$ImageType = "${{ parameters.image_type }}"
$TemplateDirectoryName = if ($ImageType.StartsWith("ubuntu")) { "linux" } else { "win" }
$TemplateDirectoryPath = Join-Path "images" $TemplateDirectoryName | Resolve-Path
$TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.pkr.hcl"
if ( -not (Test-Path $TemplatePath) ) {
$TemplatePath = Join-Path $TemplateDirectoryPath "$ImageType.json"
}
Write-Host "##vso[task.setvariable variable=TemplateDirectoryPath;]$TemplateDirectoryPath"
Write-Host "##vso[task.setvariable variable=TemplatePath;]$TemplatePath"
- task: PowerShell@2
displayName: 'Build VM'
inputs:
targetType: filePath
filePath: ./images.CI/linux-and-win/build-image.ps1
arguments: -ResourcesNamePrefix $(Build.BuildId) `
-ClientId $(CLIENT_ID) `
-ClientSecret $(CLIENT_SECRET) `
-TemplatePath $(TemplatePath) `
-ResourceGroup $(AZURE_RESOURCE_GROUP) `
-StorageAccount $(AZURE_STORAGE_ACCOUNT) `
-SubscriptionId $(AZURE_SUBSCRIPTION) `
-TenantId $(AZURE_TENANT) `
-Location $(AZURE_LOCATION) `
-VirtualNetworkName $(BUILD_AGENT_VNET_NAME) `
-VirtualNetworkRG $(BUILD_AGENT_VNET_RESOURCE_GROUP) `
-VirtualNetworkSubnet $(BUILD_AGENT_SUBNET_NAME)
env:
PACKER_LOG: 1
PACKER_LOG_PATH: "$(Agent.TempDirectory)/packer-log.txt"
- task: PowerShell@2
displayName: 'Copy image artifacts to the separate directory'
inputs:
targetType: 'inline'
script: |
$readmePath = Join-Path "$(TemplateDirectoryPath)" "${{ parameters.image_readme_name }}"
$softwareReportPath = Join-Path "$(TemplateDirectoryPath)" "software-report.json"
Copy-Item -Path $readmePath -Destination "$(Build.ArtifactStagingDirectory)/"
if (Test-Path $softwareReportPath) {
Copy-Item -Path $softwareReportPath -Destination "$(Build.ArtifactStagingDirectory)/"
}
- task: PowerShell@2
displayName: 'Print markdown software report'
inputs:
targetType: 'inline'
script: |
Get-Content -Path "$(Build.ArtifactStagingDirectory)/${{ parameters.image_readme_name }}"
- task: PowerShell@2
displayName: 'Print json software report'
inputs:
targetType: 'inline'
script: |
$softwareReportPath = "$(Build.ArtifactStagingDirectory)/software-report.json"
if (Test-Path $softwareReportPath) {
Get-Content -Path $softwareReportPath
}
- task: PublishBuildArtifacts@1
inputs:
ArtifactName: 'Built_VM_Artifacts'
displayName: Publish Artifacts
- task: PowerShell@2
displayName: 'Print provisioners duration'
inputs:
targetType: 'filePath'
filePath: ./images.CI/measure-provisioners-duration.ps1
arguments: -PackerLogPath "$(Agent.TempDirectory)/packer-log.txt" `
-PrefixToPathTrim "$(TemplateDirectoryPath)" `
-PrintTopNLongest 25
- ${{ if eq(parameters.create_release, true) }}:
- task: PowerShell@2
displayName: 'Create release for VM deployment'
inputs:
targetType: filePath
filePath: ./images.CI/linux-and-win/create-release.ps1
arguments: -BuildId $(Build.BuildId) `
-Organization $(RELEASE_TARGET_ORGANIZATION) `
-DefinitionId $(RELEASE_TARGET_DEFINITION_ID) `
-Project $(RELEASE_TARGET_PROJECT) `
-ImageName ${{ parameters.image_type }} `
-AccessToken $(RELEASE_TARGET_TOKEN)
- task: PowerShell@2
displayName: 'Clean up resources'
condition: always()
inputs:
targetType: filePath
filePath: ./images.CI/linux-and-win/cleanup.ps1
arguments: -ResourcesNamePrefix $(Build.BuildId) `
-Image ${{ parameters.image_type }} `
-StorageAccount $(AZURE_STORAGE_ACCOUNT) `
-SubscriptionId $(AZURE_SUBSCRIPTION) `
-ClientId $(CLIENT_ID) `
-ClientSecret $(CLIENT_SECRET) `
-TenantId $(AZURE_TENANT)
@@ -0,0 +1,20 @@
schedules:
- cron: "0 0 * * *"
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_type: ubuntu2004
image_readme_name: Ubuntu2004-Readme.md
@@ -0,0 +1,20 @@
schedules:
- cron: "0 0 * * *"
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_type: ubuntu2204
image_readme_name: Ubuntu2204-Readme.md
@@ -0,0 +1,20 @@
schedules:
- cron: "0 0 * * *"
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_type: windows2019
image_readme_name: Windows2019-Readme.md
@@ -0,0 +1,20 @@
schedules:
- cron: "0 0 * * *"
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_type: windows2022
image_readme_name: Windows2022-Readme.md
+14 -28
View File
@@ -1,22 +1,16 @@
param(
[String] [Parameter (Mandatory=$true)] $TemplatePath,
[String] [Parameter (Mandatory=$true)] $BuildTemplateName,
[String] [Parameter (Mandatory=$true)] $ClientId,
[String] [Parameter (Mandatory=$false)] $ClientSecret,
[String] [Parameter (Mandatory=$true)] $ClientSecret,
[String] [Parameter (Mandatory=$true)] $ResourcesNamePrefix,
[String] [Parameter (Mandatory=$true)] $Location,
[String] [Parameter (Mandatory=$true)] $ImageName,
[String] [Parameter (Mandatory=$true)] $ImageResourceGroupName,
[String] [Parameter (Mandatory=$true)] $TempResourceGroupName,
[String] [Parameter (Mandatory=$true)] $ResourceGroup,
[String] [Parameter (Mandatory=$true)] $StorageAccount,
[String] [Parameter (Mandatory=$true)] $SubscriptionId,
[String] [Parameter (Mandatory=$true)] $TenantId,
[String] [Parameter (Mandatory=$true)] $ImageOS, # e.g. "ubuntu22", "ubuntu22" or "win19", "win22", "win25"
[String] [Parameter (Mandatory=$false)] $UseAzureCliAuth = "false",
[String] [Parameter (Mandatory=$false)] $PluginVersion = "2.3.3",
[String] [Parameter (Mandatory=$false)] $VirtualNetworkName,
[String] [Parameter (Mandatory=$false)] $VirtualNetworkRG,
[String] [Parameter (Mandatory=$false)] $VirtualNetworkSubnet,
[String] [Parameter (Mandatory=$false)] $AllowedInboundIpAddresses = "[]",
[hashtable] [Parameter (Mandatory=$false)] $Tags = @{}
[String] [Parameter (Mandatory=$false)] $VirtualNetworkSubnet
)
if (-not (Test-Path $TemplatePath))
@@ -25,9 +19,12 @@ if (-not (Test-Path $TemplatePath))
exit 1
}
$buildName = $($BuildTemplateName).Split(".")[1]
$Image = [io.path]::GetFileName($TemplatePath).Split(".")[0]
$TempResourceGroupName = "${ResourcesNamePrefix}_${Image}"
$InstallPassword = [System.GUID]::NewGuid().ToString().ToUpper()
packer validate -syntax-only $TemplatePath
$SensitiveData = @(
'OSType',
'StorageAccountLocation',
@@ -38,35 +35,24 @@ $SensitiveData = @(
': ->'
)
$azure_tags = $Tags | ConvertTo-Json -Compress
Write-Host "Show Packer Version"
packer --version
Write-Host "Download packer plugins"
packer plugins install github.com/hashicorp/azure $pluginVersion
Write-Host "Validate packer template"
packer validate -syntax-only -only "$buildName*" $TemplatePath
Write-Host "Build $buildName VM"
packer build -only "$buildName*" `
Write-Host "Build $Image VM"
packer build -var "capture_name_prefix=$ResourcesNamePrefix" `
-var "client_id=$ClientId" `
-var "client_secret=$ClientSecret" `
-var "install_password=$InstallPassword" `
-var "location=$Location" `
-var "image_os=$ImageOS" `
-var "managed_image_name=$ImageName" `
-var "managed_image_resource_group_name=$ImageResourceGroupName" `
-var "resource_group=$ResourceGroup" `
-var "storage_account=$StorageAccount" `
-var "subscription_id=$SubscriptionId" `
-var "temp_resource_group_name=$TempResourceGroupName" `
-var "tenant_id=$TenantId" `
-var "virtual_network_name=$VirtualNetworkName" `
-var "virtual_network_resource_group_name=$VirtualNetworkRG" `
-var "virtual_network_subnet_name=$VirtualNetworkSubnet" `
-var "allowed_inbound_ip_addresses=$($AllowedInboundIpAddresses)" `
-var "use_azure_cli_auth=$UseAzureCliAuth" `
-var "azure_tags=$azure_tags" `
-var "run_validation_diskspace=$env:RUN_VALIDATION_FLAG" `
-color=false `
$TemplatePath `
| Where-Object {
+18 -4
View File
@@ -1,12 +1,26 @@
param(
[Parameter (Mandatory=$true)] [string] $TempResourceGroupName
[String] [Parameter (Mandatory=$true)] $Image,
[String] [Parameter (Mandatory=$true)] $ResourcesNamePrefix,
[String] [Parameter (Mandatory=$true)] $StorageAccount,
[String] [Parameter (Mandatory=$true)] $ClientId,
[String] [Parameter (Mandatory=$true)] $ClientSecret,
[String] [Parameter (Mandatory=$true)] $SubscriptionId,
[String] [Parameter (Mandatory=$true)] $TenantId
)
$groupExist = az group exists --name $TempResourceGroupName
az login --service-principal --username $ClientId --password $ClientSecret --tenant $TenantId | Out-Null
$TempResourceGroupName = "${ResourcesNamePrefix}_${Image}"
$groupExist = az group exists --name $TempResourceGroupName --subscription $SubscriptionId
if ($groupExist -eq "true") {
$osDiskName = az deployment group list --resource-group $TempResourceGroupName --query "[].properties.parameters.osDiskName.value" -o tsv
Write-Host "Found a match, deleting temporary files"
az group delete --name $TempResourceGroupName --yes | Out-Null
az group delete --name $TempResourceGroupName --subscription $SubscriptionId --yes | Out-Null
Write-Host "Temporary group was deleted successfully"
Write-Host "Deleting OS disk"
az storage remove --account-name $StorageAccount -c "images" -n "$osDiskName.vhd" --only-show-errors | Out-Null
Write-Host "OS disk deleted"
} else {
Write-Host "No temporary groups found"
}
}
+9 -13
View File
@@ -1,11 +1,10 @@
param(
[Parameter (Mandatory)] [UInt32] $BuildId,
[Parameter (Mandatory)] [string] $Organization,
[Parameter (Mandatory)] [string] $Project,
[Parameter (Mandatory)] [string] $ImageType,
[Parameter (Mandatory)] [string] $ManagedImageName,
[Parameter (Mandatory)] [string] $DefinitionId,
[Parameter (Mandatory)] [string] $AccessToken
[UInt32] [Parameter (Mandatory)] $BuildId,
[String] [Parameter (Mandatory)] $Organization,
[String] [Parameter (Mandatory)] $Project,
[String] [Parameter (Mandatory)] $ImageName,
[String] [Parameter (Mandatory)] $DefinitionId,
[String] [Parameter (Mandatory)] $AccessToken
)
$Body = @{
@@ -14,11 +13,8 @@ $Body = @{
ImageBuildId = @{
value = $BuildId
}
ImageType = @{
value = $ImageType
}
ManagedImageName = @{
value = $ManagedImageName
ImageName = @{
value = $ImageName
}
}
isDraft = "false"
@@ -33,4 +29,4 @@ $headers = @{
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Tls13
$NewRelease = Invoke-RestMethod $URL -Body $Body -Method "POST" -Headers $headers -ContentType "application/json"
Write-Host "Created release: $($NewRelease._links.web.href)"
Write-Host "Created release: $($NewRelease._links.web.href)"
+235
View File
@@ -0,0 +1,235 @@
function Push-AnkaTemplateToRegistry {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $RegistryUrl,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TagName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TemplateName
)
# if registry uuid doesn't match than delete an image in registry
$AnkaCaCrtPath="$HOME/.config/anka/certs/anka-ca-crt.pem"
$images = anka --machine-readable registry --cacert $AnkaCaCrtPath --registry-path $RegistryUrl list | ConvertFrom-Json | ForEach-Object body
$images | Where-Object name -eq $TemplateName | ForEach-Object {
$id = $_.uuid
Show-StringWithFormat "Deleting '$TemplateName[$id]' VM and '$TagName' tag"
$curlCommand='curl -s -X DELETE -k "{0}/registry/vm?id={1}"' -f $RegistryUrl, $id
Invoke-AnkaCommand -Command $curlCommand
}
$command = "anka registry --cacert $AnkaCaCrtPath --registry-path $RegistryUrl push --force --tag $TagName $TemplateName"
Invoke-AnkaCommand -Command $command
}
function Get-AnkaVM {
param(
[string] $VMName
)
$command = "anka --machine-readable list"
if (-not [string]::IsNullOrEmpty($VMName)) {
$command = "anka --machine-readable show $VMName"
}
Invoke-AnkaCommand -Command $command | ConvertFrom-Json | Foreach-Object body
}
function Get-AnkaVMStatus {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName
)
$command = "anka --machine-readable list $VMName"
Invoke-AnkaCommand -Command $command | ConvertFrom-Json | Foreach-Object { $_.body.status }
}
function Get-AnkaVMIPAddress {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName
)
Get-AnkaVM -VMName $VMName | Foreach-Object ip
}
function Invoke-AnkaCommand {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Command
)
$result = bash -c "$Command 2>&1"
if ($LASTEXITCODE -ne 0) {
Write-Error "There is an error during command execution:`n$result"
exit 1
}
$result
}
function New-AnkaVMTemplate {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $InstallerPath,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TemplateName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TemplateUsername,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TemplatePassword,
[Parameter(Mandatory)]
[int] $CPUCount,
[Parameter(Mandatory)]
[int] $RamSizeGb,
[Parameter(Mandatory)]
[int] $DiskSizeGb
)
$env:ANKA_DEFAULT_USER = $TemplateUsername
$env:ANKA_DEFAULT_PASSWD = $TemplatePassword
$env:ANKA_CREATE_SUSPEND = 0
$command = "anka create --cpu-count '$CPUCount' --ram-size '${RamSizeGb}G' --disk-size '${DiskSizeGb}G' --app '$InstallerPath' $TemplateName"
Invoke-AnkaCommand -Command $command
}
function Remove-AnkaVM {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName
)
$command = "anka delete $VMName --yes"
$isTemplateExists = Get-AnkaVM | Where-Object name -eq $VMName
if ($isTemplateExists) {
$null = Invoke-AnkaCommand -Command $command
}
}
function Set-AnkaVMVideoController {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $ShortMacOSVersion,
[ValidateSet("fbuf", "pg")]
[string] $Controller = "pg"
)
$command = "anka modify $VMName set display -c $Controller"
# Apple Metal is available starting from Big Sur
if (-not $ShortMacOSVersion.StartsWith("10.")) {
$null = Invoke-AnkaCommand -Command $command
}
}
function Set-AnkaVMDisplayResolution {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $DisplayResolution
)
$command = "anka modify $VMName set display -r $DisplayResolution"
$null = Invoke-AnkaCommand -Command $command
}
function Start-AnkaVM {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName
)
$command = "anka start $VMName"
$vmStatus = Get-AnkaVMStatus -VMName $VMName
if ($vmStatus -eq "stopped") {
$null = Invoke-AnkaCommand -Command $command
}
}
function Stop-AnkaVM {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName
)
$command = "anka stop $VMName"
$vmStatus = Get-AnkaVMStatus -VMName $VMName
if ($vmStatus -eq "running") {
$null = Invoke-AnkaCommand -Command $command
}
}
function Wait-AnkaVMIPAddress {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName,
[int] $RetryCount = 20,
[int] $Seconds = 60
)
$condition = {
$vmStatus = Get-AnkaVMStatus -VMName $VMName
if ($vmStatus -eq "failed") {
Write-Host "`t [-] $VMName is in failed status"
exit 1
}
Get-AnkaVMIPAddress -VMName $VMName
}
$null = Invoke-WithRetry -BreakCondition $condition -RetryCount $RetryCount -Seconds $Seconds
}
function Wait-AnkaVMSSHService {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName,
[int] $RetryCount = 20,
[int] $Seconds = 60
)
Start-Sleep -Seconds $Seconds
Write-Host "`t[*] Waiting for '$VMName' VM to get an IP address"
Wait-AnkaVMIPAddress -VMName $VMName -RetryCount $RetryCount -Seconds $Seconds
$ipAddress = Get-AnkaVMIPAddress -VMName $VMName
Write-Host "`t[*] The '$ipAddress' IP address for '$VMName' VM"
Write-Host "`t[*] Checking if SSH on a port is open"
$isSSHPortOpen = Test-SSHPort -IPAddress $ipAddress
if (-not $isSSHPortOpen) {
Write-Host "`t[x] SSH port is closed"
exit 1
}
}
@@ -0,0 +1,209 @@
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[version] $MacOSVersion,
[ValidateNotNullOrEmpty()]
[string] $TemplateUsername,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $TemplatePassword,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $RegistryUrl,
[ValidateNotNullOrEmpty()]
[string] $TemplateName,
[bool] $DownloadLatestVersion = $true,
[bool] $BetaSearch = $false,
[bool] $InstallSoftwareUpdate = $true,
[bool] $EnableAutoLogon = $true,
[int] $CPUCount = 6,
[int] $RamSizeGb = 7,
[int] $DiskSizeGb = 300,
[string] $DisplayResolution = "1920x1080",
[string] $TagName = [DateTimeOffset]::Now.ToUnixTimeSeconds()
)
$ErrorActionPreference = "Stop"
$WarningPreference = "SilentlyContinue"
# Import helper modules
Import-Module "$PSScriptRoot/Anka.Helpers.psm1"
Import-Module "$PSScriptRoot/Service.Helpers.psm1"
# Helper functions
function Invoke-EnableAutoLogon {
if (-not $EnableAutoLogon) {
Write-Host "`t[*] Skip configuring AutoLogon"
return
}
$ipAddress = Get-AnkaVMIPAddress -VMName $TemplateName
Wait-AnkaVMSSHService -VMName $TemplateName -Seconds 30
Write-Host "`t[*] Enable AutoLogon"
Enable-AutoLogon -HostName $ipAddress -UserName $TemplateUsername -Password $TemplatePassword
Write-Host "`t[*] Reboot '$TemplateName' VM to enable AutoLogon"
Restart-VMSSH -HostName $ipAddress | Show-StringWithFormat
Wait-AnkaVMSSHService -VMName $TemplateName -Seconds 30
Write-Host "`t[*] Checking if AutoLogon is enabled"
Test-AutoLogon -VMName $TemplateName -UserName $TemplateUsername
}
function Invoke-SoftwareUpdate {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Password
)
if (-not $InstallSoftwareUpdate) {
Write-Host "`t[*] Skip installing software updates"
return
}
$ipAddress = Get-AnkaVMIPAddress -VMName $TemplateName
# Unenroll Seed
Write-Host "`t[*] Reseting the seed before requesting stable versions"
Remove-CurrentBetaSeed -HostName $ipAddress | Show-StringWithFormat
# Install Software Updates
# Security updates may not be able to install(hang, freeze) when AutoLogon is turned off
Write-Host "`t[*] Finding available software"
$newUpdates = Get-SoftwareUpdate -HostName $ipAddress
if (-not $newUpdates) {
Write-Host "`t[*] No Updates Available"
return
}
Write-Host "`t[*] Fetching Software Updates ready to install on '$TemplateName' VM:"
Show-StringWithFormat $newUpdates
$listOfNewUpdates = $($($newUpdates.Split("*")).Split("Title") | Where-Object {$_ -match "Label:"}).Replace("Label: ", '')
Write-Host "`t[*] Installing Software Updates on '$TemplateName' VM:"
Install-SoftwareUpdate -HostName $ipAddress -listOfUpdates $listOfNewUpdates -Password $Password | Show-StringWithFormat
# Check if Action: restart
# Make an array of updates
$listOfNewUpdates = $newUpdates.split('*').Trim('')
foreach ($newupdate in $listOfNewUpdates) {
# Will be True if the value is not Venture, not empty, and contains "Action: restart" words
if ($newupdate.Contains("Action: restart") -and !$newupdate.Contains("macOS Ventura") -and (-not [String]::IsNullOrEmpty($newupdate))) {
Write-Host "`t[*] Sleep 60 seconds before the software updates have been installed"
Start-Sleep -Seconds 60
Write-Host "`t[*] Waiting for loginwindow process"
Wait-LoginWindow -HostName $ipAddress | Show-StringWithFormat
# Re-enable AutoLogon after installing a new security software update
Invoke-EnableAutoLogon
# Check software updates have been installed
$updates = Get-SoftwareUpdate -HostName $ipAddress
if ($updates.Contains("Action: restart")) {
Write-Host "`t[x] Software updates failed to install: "
Show-StringWithFormat $updates
exit 1
}
}
}
Write-Host "`t[*] Show the install history:"
$hUpdates = Get-SoftwareUpdateHistory -HostName $ipAddress
Show-StringWithFormat $hUpdates
}
function Invoke-UpdateSettings {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Password
)
$isConfRequired = $InstallSoftwareUpdate -or $EnableAutoLogon
if (-not $isConfRequired) {
Write-Host "`t[*] Skip additional configuration"
return
}
Write-Host "`t[*] Starting '$TemplateName' VM"
Start-AnkaVM -VMName $TemplateName
Write-Host "`t[*] Waiting for SSH service on '$TemplateName' VM"
Wait-AnkaVMSSHService -VMName $TemplateName -Seconds 30
# Configure AutoLogon
Invoke-EnableAutoLogon
# Install software updates
Invoke-SoftwareUpdate -Password $Password
Write-Host "`t[*] Stopping '$TemplateName' VM"
Stop-AnkaVM -VMName $TemplateName
}
function Test-VMStopped {
$vmStatus = Get-AnkaVMStatus -VMName $TemplateName
if ($vmStatus -ne "stopped") {
Write-Host "`t[x] VM '$TemplateName' state is not stopped. The current state is '$vmStatus'"
exit 1
}
}
# Password is passed as env-var "SSHPASS"
$env:SSHUSER = $TemplateUsername
$env:SSHPASS = $TemplatePassword
Write-Host "`n[#1] Download macOS application installer:"
$shortMacOSVersion = Get-ShortMacOSVersion -MacOSVersion $MacOSVersion
if ([string]::IsNullOrEmpty($TemplateName)) {
$osArch = $(arch)
if ($osArch -eq "arm64") {
$macOSInstaller = Get-MacOSIPSWInstaller -MacOSVersion $MacOSVersion -DownloadLatestVersion $DownloadLatestVersion -BetaSearch $BetaSearch
$TemplateName = "clean_macos_${shortMacOSVersion}_${osArch}_${DiskSizeGb}gb"
} else {
$macOSInstaller = Get-MacOSInstaller -MacOSVersion $MacOSVersion -DownloadLatestVersion $DownloadLatestVersion -BetaSearch $BetaSearch
$TemplateName = "clean_macos_${shortMacOSVersion}_${DiskSizeGb}gb"
}
}
Write-Host "`n[#2] Create a VM template:"
Write-Host "`t[*] Deleting existed template with name '$TemplateName' before creating a new one"
Remove-AnkaVM -VMName $TemplateName
Write-Host "`t[*] Creating Anka VM template with name '$TemplateName' and '$TemplateUsername' user"
Write-Host "`t[*] CPU Count: $CPUCount, RamSize: ${RamSizeGb}G, DiskSizeGb: ${DiskSizeGb}G, InstallerPath: $macOSInstaller, TemplateName: $TemplateName"
New-AnkaVMTemplate -InstallerPath "$macOSInstaller" `
-TemplateName $TemplateName `
-TemplateUsername $TemplateUsername `
-TemplatePassword $TemplatePassword `
-CPUCount $CPUCount `
-RamSizeGb $RamSizeGb `
-DiskSizeGb $DiskSizeGb | Show-StringWithFormat
Write-Host "`n[#3] Configure AutoLogon and/or install software updates:"
Invoke-UpdateSettings -Password $TemplatePassword
Write-Host "`n[#4] Finalization '$TemplateName' configuration and push to the registry:"
Write-Host "`t[*] The '$TemplateName' VM status is stopped"
Test-VMStopped
# Configure graphics settings
Write-Host "`t[*] Enabling Graphics Acceleration with Apple Metal for '$TemplateName' VM"
Set-AnkaVMVideoController -VMName $TemplateName -ShortMacOSVersion $ShortMacOSVersion
Write-Host "`t[*] Setting screen resolution to $DisplayResolution for $TemplateName"
Set-AnkaVMDisplayResolution -VMName $TemplateName -DisplayResolution $DisplayResolution
# Push a VM template (and tag) to the Cloud
Write-Host "`t[*] Pushing '$TemplateName' image with '$TagName' tag to the '$RegistryUrl' registry..."
Push-AnkaTemplateToRegistry -RegistryUrl $registryUrl -TagName $TagName -TemplateName $TemplateName
+456
View File
@@ -0,0 +1,456 @@
function Enable-AutoLogon {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $UserName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Password
)
$url = "https://raw.githubusercontent.com/actions/runner-images/main/images/macos/provision/bootstrap-provisioner/setAutoLogin.sh"
$script = Invoke-RestMethod -Uri $url
$base64 = [Convert]::ToBase64String($script.ToCharArray())
$command = "echo $base64 | base64 --decode > ./setAutoLogin.sh;sudo bash ./setAutoLogin.sh '${UserName}' '${Password}';rm ./setAutoLogin.sh"
Invoke-SSHPassCommand -HostName $HostName -Command $command
}
function Invoke-SoftwareUpdateArm64 {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Password
)
$url = "https://raw.githubusercontent.com/actions/runner-images/main/images/macos/provision/configuration/auto-software-update-arm64.exp"
$script = Invoke-RestMethod -Uri $url
$base64 = [Convert]::ToBase64String($script.ToCharArray())
$command = "echo $base64 | base64 --decode > ./auto-software-update-arm64.exp;chmod +x ./auto-software-update-arm64.exp; ./auto-software-update-arm64.exp ${Password};rm ./auto-software-update-arm64.exp"
Invoke-SSHPassCommand -HostName $HostName -Command $command
}
function Get-AvailableVersions {
param (
[bool] $IsBeta = $false
)
if ($IsBeta) {
$searchPostfix = " beta"
}
$command = { /usr/sbin/softwareupdate --list-full-installers | grep "macOS" }
$condition = { $LASTEXITCODE -eq 0 }
$softwareUpdates = Invoke-WithRetry -Command $command -BreakCondition $condition | Where-Object { $_.Contains("Title: macOS") -and $_ -match $searchPostfix }
$allVersions = $softwareUpdates -replace "(\* )?(Title|Version|Size):" | ConvertFrom-Csv -Header OSName, OSVersion | Select-Object OSName, OSVersion -Unique
$allVersions
}
function Get-AvailableIPSWVersions {
param (
[bool] $IsBeta = $false,
[bool] $IsLatest = $true,
[string] $MacOSCodeNameOrVersion
)
if ($IsBeta) {
$command = { mist list installer "$MacOSCodeNameOrVersion" --include-betas --latest --export "/Applications/export.json"}
} elseif ($IsLatest) {
$command = { mist list installer "$MacOSCodeNameOrVersion" --latest --export "/Applications/export.json" }
} else {
$command = { mist list installer "$MacOSCodeNameOrVersion" --export "/Applications/export.json" }
}
$condition = { $LASTEXITCODE -eq 0 }
Invoke-WithRetry -Command $command -BreakCondition $condition | Out-Null
$softwareList = get-content -Path "/Applications/export.json"
$turgetVersion = ($softwareList | ConvertFrom-Json).version
if ($null -eq $turgetVersion) {
Write-Host "Requested macOS '$MacOSCodeNameOrVersion' version not found in the list of available installers."
$command = { mist list installer "$($MacOSCodeNameOrVersion.split('.')[0])" }
Invoke-WithRetry -Command $command -BreakCondition $condition
exit 1
}
return $turgetVersion
}
function Get-MacOSIPSWInstaller {
param (
[Parameter(Mandatory)]
[version] $MacOSVersion,
[bool] $DownloadLatestVersion = $false,
[bool] $BetaSearch = $false
)
if ($MacOSVersion -eq [version] "12.0") {
$MacOSName = "macOS Monterey"
} elseif ($MacOSVersion -eq [version] "13.0") {
$MacOSName = "macOS Ventura"
} else {
$MacOSName = $MacOSVersion.ToString()
}
Write-Host "`t[*] Finding available full installers"
if ($DownloadLatestVersion -eq $true) {
$targetVersion = Get-AvailableIPSWVersions -IsLatest $true -MacOSCodeNameOrVersion $MacOSName
Write-host "`t[*] The 'DownloadLatestVersion' flag is set to true. Latest macOS version is '$MacOSName' - '$targetVersion' now"
} elseif ($BetaSearch -eq $true) {
$targetVersion = Get-AvailableIPSWVersions -IsBeta $true -MacOSCodeNameOrVersion $MacOSName
Write-host "`t[*] The 'BetaSearch' flag is set to true. Latestbeta macOS version is '$MacOSName' - '$targetVersion' now"
} else {
$targetVersion = Get-AvailableIPSWVersions -MacOSCodeNameOrVersion $MacOSName -IsLatest $false
Write-host "`t[*] The exact version was specified - '$MacOSName' "
}
$installerPathPattern = "/Applications/Install ${macOSName}*.ipsw"
if (Test-Path $installerPathPattern) {
$previousInstallerPath = Get-Item -Path $installerPathPattern
Write-Host "`t[*] Removing '$previousInstallerPath' installation app before downloading the new one"
sudo rm -rf "$previousInstallerPath"
}
# Download macOS installer
$installerDir = "/Applications/"
$installerName = "Install ${macOSName}.ipsw"
Write-Host "`t[*] Requested macOS '$targetVersion' version installer found, fetching it from mist database"
Invoke-WithRetry { mist download firmware "$targetVersion" --output-directory $installerDir --firmware-name "$installerName" } {$LASTEXITCODE -eq 0} | Out-Null
if (Test-Path "$installerDir$installerName") {
$result = "$installerDir$installerName"
} else {
Write-host "`t[*] Requested macOS '$targetVersion' version installer failed to download"
exit 1
}
return $result
}
function Get-MacOSInstaller {
param (
[Parameter(Mandatory)]
[version] $MacOSVersion,
[bool] $DownloadLatestVersion = $false,
[bool] $BetaSearch = $false
)
# Enroll machine to DeveloperSeed if we need beta and unenroll otherwise
$seedutil = "/System/Library/PrivateFrameworks/Seeding.framework/Versions/Current/Resources/seedutil"
if ($BetaSearch) {
Write-Host "`t[*] Beta Version requested. Enrolling machine to DeveloperSeed"
sudo $seedutil enroll DeveloperSeed | Out-Null
} else {
Write-Host "`t[*] Reseting the seed before requesting stable versions"
sudo $seedutil unenroll | Out-Null
}
# Validate there is no softwareupdate at the moment
Test-SoftwareUpdate
# Validate availability OSVersion
Write-Host "`t[*] Finding available full installers"
$availableVersions = Get-AvailableVersions -IsBeta $BetaSearch
if ($DownloadLatestVersion) {
$shortMacOSVersion = Get-ShortMacOSVersion -MacOSVersion $MacOSVersion
$filterSearch = "${shortMacOSVersion}."
$filteredVersions = $availableVersions.Where{ $_.OSVersion.StartsWith($filterSearch) }
if (-not $filteredVersions) {
Write-Host "`t[x] Failed to find any macOS versions using '$filterSearch' search condition"
Show-StringWithFormat $availableVersions
exit 1
}
Show-StringWithFormat $filteredVersions
$osVersions = $filteredVersions.OSVersion | Sort-Object {[version]$_}
$MacOSVersion = $osVersions | Select-Object -Last 1
Write-Host "`t[*] The 'DownloadLatestVersion' flag is set. Latest macOS version is '$MacOSVersion' now"
}
$macOSName = $availableVersions.Where{ $MacOSVersion -eq $_.OSVersion }.OSName.Split(" ")[1]
if (-not $macOSName) {
Write-Host "`t[x] Requested macOS '$MacOSVersion' version not found in the list of available installers. Available versions are:`n$($availableVersions.OSVersion)"
Write-Host "`t[x] Make sure to pass '-BetaSearch `$true' if you need a beta version installer"
exit 1
}
# Clear LastRecommendedMajorOSBundleIdentifier to prevent error during fetching updates
# Install failed with error: Update not found
Update-SoftwareBundle
# Download macOS installer
Write-Host "`t[*] Requested macOS '$MacOSVersion' version installer found, fetching it from Apple Software Update"
Invoke-WithRetry -Command { sudo /usr/local/bin/mist download installer $MacOSVersion application --force --export installer.json --output-directory /Applications } -BreakCondition { $LASTEXITCODE -eq 0 } | Out-Null
if (-not(Test-Path installer.json -PathType leaf)) {
Write-Host "`t[x] Failed to fetch $MacOSVersion macOS"
exit 1
}
$installerPath = (Get-Content installer.json | Out-String | ConvertFrom-Json).options.applicationPath
if (-not $installerPath) {
Write-Host "`t[x] Path not found using '$installerPathPattern'"
exit 1
}
Write-Host "`t[*] Installer successfully downloaded to '$installerPath'"
$installerPath
}
function Get-ShortMacOSVersion {
param (
[Parameter(Mandatory)]
[version] $MacOSVersion
)
# Take Major.Minor version for macOS 10 (10.14 or 10.15) and Major for all further versions
$MacOSVersion.Major -eq 10 ? $MacOSVersion.ToString(2) : $MacOSVersion.ToString(1)
}
function Get-SoftwareUpdate {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName
)
$command = "/usr/sbin/softwareupdate --list"
$result = Invoke-SSHPassCommand -HostName $HostName -Command $command
$result | Where-Object { $_ -match "(Label|Title):" } | Out-String
}
function Get-SoftwareUpdateHistory {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName
)
$command = "/usr/sbin/softwareupdate --history"
Invoke-SSHPassCommand -HostName $HostName -Command $command | Out-String
}
function Install-SoftwareUpdate {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName,
[array] $listOfUpdates,
[string] $Password
)
$osVersion = [Environment]::OSVersion
# If an update is happening on macOS 12 we will use the prepared list of updates, otherwise, we will install all updates.
if ($osVersion.Version.Major -eq "12") {
foreach ($update in $listOfUpdates){
# Filtering updates that contain "Ventura" word
if ($update -notmatch "Ventura") {
$command = "sudo /usr/sbin/softwareupdate --restart --verbose --install '$($update.trim())'"
Invoke-SSHPassCommand -HostName $HostName -Command $command
}
}
} else {
$osArch = $(arch)
if ($osArch -eq "arm64") {
Invoke-SoftwareUpdateArm64 -HostName $HostName -Password $Password
} else {
$command = "sudo /usr/sbin/softwareupdate --all --install --restart --verbose"
Invoke-SSHPassCommand -HostName $HostName -Command $command
}
}
}
function Invoke-SSHPassCommand {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $Command,
[int] $ConnectTimeout = 10,
[int] $ConnectionAttempts = 10,
[int] $ServerAliveInterval = 30
)
$sshArg = @(
"sshpass"
"-e"
"ssh"
"-o UserKnownHostsFile=/dev/null"
"-o StrictHostKeyChecking=no"
"-o ConnectTimeout=$ConnectTimeout"
"-o ConnectionAttempts=$ConnectionAttempts"
"-o LogLevel=ERROR"
"-o ServerAliveInterval=$ServerAliveInterval"
"${env:SSHUSER}@${HostName}"
)
$sshPassOptions = $sshArg -join " "
$result = bash -c "$sshPassOptions \""$Command\"" 2>&1"
if ($LASTEXITCODE -ne 0) {
Write-Error "There is an error during command execution:`n$result"
exit 1
}
$result
}
function Invoke-WithRetry {
param(
[scriptblock] $Command,
[scriptblock] $BreakCondition,
[int] $RetryCount = 20,
[int] $Seconds = 60
)
while ($RetryCount -gt 0) {
if ($Command) {
$result = & $Command
}
if (& $BreakCondition) {
return $result
}
$RetryCount--
if ($RetryCount -eq 0) {
Write-Error "No more attempts left: $BreakCondition"
}
Write-Host "`t [/] Waiting $Seconds seconds before retrying. Retries left: $RetryCount"
Start-Sleep -Seconds $Seconds
}
$result
}
function Restart-VMSSH {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName
)
#
# https://unix.stackexchange.com/questions/58271/closing-connection-after-executing-reboot-using-ssh-command
#
$command = '(sleep 1 && sudo reboot &) && exit'
Invoke-SSHPassCommand -HostName $HostName -Command $command
}
function Show-StringWithFormat {
param(
[Parameter(ValuefromPipeline)]
[object] $string
)
process {
($string | Out-String).Trim().split("`n") | ForEach-Object { Write-Host "`t $_" }
}
}
function Remove-CurrentBetaSeed {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName
)
$command = "sudo /System/Library/PrivateFrameworks/Seeding.framework/Versions/Current/Resources/seedutil unenroll"
Invoke-SSHPassCommand -HostName $HostName -Command $command | Out-String
}
function Test-AutoLogon {
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $UserName
)
Invoke-WithRetry -BreakCondition {
# pwsh crashes if it invokes directly
# https://github.com/dotnet/runtime/issues/59059
$ankaUser = "" | bash -c "anka run $VMName /usr/bin/id -nu"
$UserName -eq $ankaUser
}
}
function Test-SoftwareUpdate {
param (
[string] $UpdateProcessName = "softwareupdate"
)
$command = {
$updateProcess = (Get-Process -Name $UpdateProcessName -ErrorAction SilentlyContinue).id
if ($updateProcess) {
# Workaround to get commandline param as it doesn't work for macOS atm https://github.com/PowerShell/PowerShell/issues/13943
$processName = /bin/ps -o command= $updateProcess
Write-Host "`t[*] Another software update process with '$updateProcess' id is in place with the following arguments '$processName'"
}
}
$condition = {
$null -eq (Get-Process -Name $UpdateProcessName -ErrorAction SilentlyContinue)
}
Invoke-WithRetry -Command $command -BreakCondition $condition
}
function Test-SSHPort {
param(
[Parameter(Mandatory)]
[ipaddress] $IPAddress,
[int] $Port = 22,
[int] $Timeout = 2000
)
Invoke-WithRetry -Command {$true} -BreakCondition {
try {
$client = [System.Net.Sockets.TcpClient]::new()
$client.ConnectAsync($IPAddress, $Port).Wait($Timeout)
}
catch {
$false
}
finally {
$client.Close()
}
}
}
function Wait-LoginWindow {
param (
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string] $HostName,
[int] $RetryCount = 60,
[int] $Seconds = 60
)
$condition = {
$psCommand = "/bin/ps auxww"
$lw = "/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow"
$ctk = "/System/Library/Frameworks/CryptoTokenKit.framework/ctkahp.bundle/Contents/MacOS/ctkahp"
$proc = Invoke-SSHPassCommand -HostName $HostName -Command $psCommand | Out-String
$proc.Contains($lw) -and $proc.Contains($ctk)
}
Invoke-WithRetry -RetryCount $RetryCount -Seconds $Seconds -BreakCondition $condition
}
function Update-SoftwareBundle {
$productVersion = sw_vers -productVersion
if ( $productVersion.StartsWith('11.') ) {
sudo rm -rf /Library/Preferences/com.apple.commerce.plist
sudo /usr/bin/defaults delete /Library/Preferences/com.apple.SoftwareUpdate.plist LastRecommendedMajorOSBundleIdentifier | Out-Null
}
}
@@ -0,0 +1,149 @@
jobs:
- job: Image_generation
displayName: Image Generation (${{ parameters.image_label }})
timeoutInMinutes: 720
pool:
name: Mac-Cloud Image Generation
variables:
- group: Mac-Cloud Image Generation
- group: Mac-Cloud Image Generation Key Vault
steps:
- pwsh: |
$cleanBuildNumber = "$(Build.BuildNumber)" -replace "(.+_unstable)(\.\d+)", '$1'
$virtualMachineName = "${cleanBuildNumber}.$(System.JobAttempt)"
echo "##vso[task.setvariable variable=VirtualMachineName;]$virtualMachineName"
echo "##vso[build.updatebuildnumber]$virtualMachineName"
displayName: Update BuildNumber
- checkout: self
clean: true
fetchDepth: 1
- task: PowerShell@2
displayName: 'Validate contributor permissions'
condition: startsWith(variables['Build.SourceBranch'], 'refs/pull/')
inputs:
targetType: 'filePath'
filePath: ./images.CI/macos/validate-contributor.ps1
pwsh: true
arguments: -RepositoryName "$(Build.Repository.Name)" `
-AccessToken "$(github-feed-token)" `
-SourceBranch "$(Build.SourceBranch)" `
-ContributorAllowList "$(CONTRIBUTOR_ALLOWLIST)"
- task: PowerShell@2
displayName: 'Download custom repository'
condition: and(ne(variables['CUSTOM_REPOSITORY_URL'], ''), ne(variables['CUSTOM_REPOSITORY_BRANCH'], ''))
inputs:
targetType: 'filePath'
filePath: ./images.CI/download-repo.ps1
arguments: -RepoUrl $(CUSTOM_REPOSITORY_URL) `
-RepoBranch $(CUSTOM_REPOSITORY_BRANCH)
- task: DeleteFiles@1
displayName: Clean up self-hosted machine
inputs:
SourceFolder: 'images/macos/provision/log/'
RemoveSourceFolder: true
- task: PowerShell@2
displayName: 'Select datastore'
inputs:
targetType: 'filePath'
filePath: ./images.CI/macos/select-datastore.ps1
arguments: -VMName "$(VirtualMachineName)" `
-VIServer "$(vcenter-server-v2)" `
-VIUserName "$(vcenter-username-v2)" `
-VIPassword '$(vcenter-password-v2)' `
-Cluster "$(esxi-cluster-v2)"
- pwsh: |
$SensitiveData = @(
'IP address:',
'Using ssh communicator to connect:'
)
packer build -on-error=abort `
-var="vcenter_server=$(vcenter-server-v2)" `
-var="vcenter_username=$(vcenter-username-v2)" `
-var='vcenter_password=$(vcenter-password-v2)' `
-var="vcenter_datacenter=$(vcenter-datacenter-v2)" `
-var="cluster_or_esxi_host=$(esxi-cluster-v2)" `
-var="esxi_datastore=$(buildDatastore)" `
-var="output_folder=$(output-folder)" `
-var="vm_username=$(vm-username)" `
-var="vm_password=$(vm-password)" `
-var="github_api_pat=$(github_api_pat)" `
-var="build_id=$(VirtualMachineName)" `
-var="baseimage_name=${{ parameters.base_image_name }}" `
-var="xcode_install_storage_url=$(xcode_install_storage_url)" `
-var="xcode_install_sas=$(xcode_install_sas)" `
-color=false `
${{ parameters.template_path }} `
| Where-Object {
#Filter sensitive data from Packer logs
$currentString = $_
$sensitiveString = $SensitiveData | Where-Object { $currentString -match $_ }
$sensitiveString -eq $null
}
displayName: 'Build VM'
workingDirectory: 'images/macos'
env:
PACKER_LOG: 1
PACKER_LOG_PATH: $(Agent.TempDirectory)/packer-log.txt
- bash: |
echo "Copy software report files"
cp -vR "images/image-output/software-report/." "$(Build.ArtifactStagingDirectory)/"
echo "Put VM name to 'VM_Done_Name' file"
echo "$(VirtualMachineName)" > "$(Build.ArtifactStagingDirectory)/VM_Done_Name"
displayName: Prepare artifact
- bash: |
cat "$(Build.ArtifactStagingDirectory)/systeminfo.md"
displayName: Print markdown software report
- bash: |
cat "$(Build.ArtifactStagingDirectory)/systeminfo.json"
displayName: Print json software report
- task: PublishBuildArtifacts@1
inputs:
ArtifactName: 'Built_VM_Artifacts'
displayName: Publish Artifacts
- task: PowerShell@2
displayName: 'Print provisioners duration'
inputs:
targetType: 'filePath'
filePath: ./images.CI/measure-provisioners-duration.ps1
arguments: -PackerLogPath "$(Agent.TempDirectory)/packer-log.txt" `
-PrintTopNLongest 25
- task: PowerShell@2
displayName: 'Move vm to cold storage and clear datastore tag'
condition: always()
inputs:
targetType: 'filePath'
filePath: ./images.CI/macos/move-vm.ps1
arguments: -VMName "$(VirtualMachineName)" `
-TargetDataStore "${{ parameters.target_datastore }}" `
-VIServer "$(vcenter-server-v2)" `
-VIUserName "$(vcenter-username-v2)" `
-VIPassword '$(vcenter-password-v2)' `
-CpuCount "$(cpu-count)" `
-CoresPerSocketCount "$(cores-per-socket-count)" `
-Memory "$(memory)"
- task: PowerShell@2
displayName: 'Destroy VM (if build canceled only)'
condition: eq(variables['Agent.JobStatus'], 'Canceled')
inputs:
targetType: 'filePath'
filePath: ./images.CI/macos/destroy-vm.ps1
arguments: -VMName "$(VirtualMachineName)" `
-VIServer "$(vcenter-server-v2)" `
-VIUserName "$(vcenter-username-v2)" `
-VIPassword '$(vcenter-password-v2)'
@@ -0,0 +1,23 @@
name: macOS-11_$(date:yyyyMMdd)$(rev:.r)_unstable
schedules:
- cron: '45 0 * * *'
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_label: 'macOS Big Sur'
base_image_name: 'clean-macOS-11-380Gb-runner'
template_path: 'templates/macOS-11.json'
target_datastore: 'ds-image'
@@ -0,0 +1,23 @@
name: macOS-12_$(date:yyyyMMdd)$(rev:.r)_unstable
schedules:
- cron: "0 0 * * *"
displayName: Daily
branches:
include:
- main
always: true
trigger: none
pr:
autoCancel: true
branches:
include:
- main
jobs:
- template: image-generation.yml
parameters:
image_label: 'macOS Monterey'
base_image_name: 'clean-macOS-12-380Gb-runner'
template_path: 'templates/macOS-12.json'
target_datastore: 'ds-image'
+95
View File
@@ -0,0 +1,95 @@
<#
.SYNOPSIS
This script deletes vm from vCenter
.PARAMETER VMName
VM name to delete (Example "macOS-10.15_20201012.4")
.PARAMETER VIServer
vCenter address (Example "10.0.1.16")
.PARAMETER VIUserName
vCenter username (Example "Administrator")
.PARAMETER VIPassword
vCenter password (Example "12345678")
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIServer,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIUserName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIPassword
)
# Import helpers module
Import-Module $PSScriptRoot\helpers.psm1 -DisableNameChecking
# Connection to a vCenter Server system
Connect-VCServer -VIServer $VIServer -VIUserName $VIUserName -VIPassword $VIPassword
# Check vm clone status
$chainId = (Get-VIEvent -Entity $VMName).ChainId
if ($chainId)
{
$task = Get-Task -Status Running | Where-Object { ($_.Name -eq 'CloneVM_Task') -and ($_.ExtensionData.Info.EventChainId -in $chainId) }
if ($task)
{
try
{
Stop-Task -Task $task -Confirm:$false -ErrorAction Stop
Write-Host "The vm '$VMName' clone task has been canceled"
}
catch
{
Write-Host "##vso[task.LogIssue type=error;]Failed to cancel the task"
}
}
}
# Remove a vm
$vm = Get-VM -Name $VMName -ErrorAction SilentlyContinue
if ($vm)
{
$vmState = $vm.PowerState
if ($vmState -ne "PoweredOff")
{
try
{
$null = Stop-VM -VM $vm -Confirm:$false -ErrorAction Stop
Write-Host "The vm '$VMName' has been powered off"
}
catch
{
Write-Host "##vso[task.LogIssue type=error;]Failed to shutdown '$VMName'"
}
}
try
{
Remove-VM -VM $vm -DeletePermanently -Confirm:$false -ErrorAction Stop
Write-Host "The vm '$VMName' has been removed"
}
catch
{
Write-Host "##vso[task.LogIssue type=error;]Failed to remove '$VMName'"
}
}
else
{
Write-Host "VM '$VMName' not found"
}
+38
View File
@@ -0,0 +1,38 @@
<#
.SYNOPSIS
Helper functions to use in images.CI scripts
#>
Function Connect-VCServer
{
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[System.String]$VIUserName,
[Parameter(Mandatory)]
[System.String]$VIPassword,
[Parameter(Mandatory)]
[System.String]$VIServer
)
try
{
# Preference
$global:ProgressPreference = 'SilentlyContinue'
$global:WarningPreference = 'SilentlyContinue'
# Ignore SSL
$null = Set-PowerCLIConfiguration -Scope Session -InvalidCertificateAction Ignore -ParticipateInCEIP $false -Confirm:$false -WebOperationTimeoutSeconds 600
$securePassword = ConvertTo-SecureString -String $VIPassword -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential($VIUserName, $securePassword)
$null = Connect-VIServer -Server $VIServer -Credential $cred -ErrorAction Stop
Write-Host "Connection to the vSphere server has been established"
}
catch
{
Write-Host "##vso[task.LogIssue type=error;]Failed to connect to the vSphere server"
exit 1
}
}
+86
View File
@@ -0,0 +1,86 @@
<#
.SYNOPSIS
This script migrates given VM to another datastore
.PARAMETER VMName
VM name to migrate (Example "macOS-10.15_20201012.4")
.PARAMETER TargetDataStore
Target datastore (Example "ds-image")
.PARAMETER VIServer
vCenter address (Example "10.0.1.16")
.PARAMETER VIUserName
vCenter username (Example "Administrator")
.PARAMETER VIPassword
vCenter password (Example "12345678")
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$TargetDataStore,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIServer,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIUserName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIPassword,
[string]$JobStatus,
[int32]$CpuCount,
[int32]$CoresPerSocketCount,
[int64]$Memory
)
# Import helpers module
Import-Module $PSScriptRoot\helpers.psm1 -DisableNameChecking
# Connection to a vCenter Server system
Connect-VCServer -VIServer $VIServer -VIUserName $VIUserName -VIPassword $VIPassword
# Clear previously assigned tag with VM Name
try {
Remove-Tag $VMName -Confirm:$false
} catch {
Write-Host "Tag with $VMName doesn't exist"
}
$vm = Get-VM $VMName
if (($env:AGENT_JOBSTATUS -and $env:AGENT_JOBSTATUS -eq 'Failed') -or ($JobStatus -and $JobStatus -eq 'failure')) {
try {
if ($vm.PowerState -ne "PoweredOff") {
Stop-VM -VM $vm -Confirm:$false -ErrorAction Stop | Out-Null
}
Set-VM -VM $vm -Name "${VMName}_failed" -Confirm:$false -ErrorAction Stop | Out-Null
Write-Host "VM has been successfully powered off and renamed to [${VMName}_failed]"
} catch {
Write-Host "##vso[task.LogIssue type=error;]Failed to power off and rename VM '$VMName'"
exit 1
}
}
try {
Move-VM -Vm $vm -Datastore $TargetDataStore -ErrorAction Stop | Out-Null
Write-Host "VM has been moved successfully to target datastore '$TargetDataStore'"
} catch {
Write-Host "##vso[task.LogIssue type=error;]Failed to move VM '$VMName' to target datastore '$TargetDataStore'"
exit 1
}
+117
View File
@@ -0,0 +1,117 @@
<#
.SYNOPSIS
This script selects local datastore based on the following rules:
- Name starts with ds-local-Datastore
- Datastore FreespaceGB > 400 Gb
- VM count on the datastore < 2
.PARAMETER VIServer
vCenter address (Example "10.0.1.16")
.PARAMETER VIUserName
vCenter username (Example "Administrator")
.PARAMETER VIPassword
vCenter password (Example "12345678")
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIServer,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIUserName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIPassword,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$Cluster,
[string]$TagCategory = "Busy"
)
# Import helpers module
Import-Module $PSScriptRoot\helpers.psm1 -DisableNameChecking
function Select-DataStore {
param (
[string]$VMName,
[string]$TagCategory,
[string]$TemplateDatastore = "ds-local-Datastore-*",
[string]$Cluster,
[int]$ThresholdInGb = 400,
[int]$VMCount = 2,
[int]$Retries = 5
)
# 1. Name starts with ds-local-Datastore
# 2. FreespaceGB > 400 Gb
# 3. Choose a datastore with the minimal VM count < 2
Write-Host "Start Datastore selection process..."
$clusterHosts = Get-Cluster -Name $Cluster | Get-VMHost
$availableClusterDatastores = $clusterHosts | Get-Datastore -Name $TemplateDatastore | Where-Object -Property State -eq "Available"
$availableDatastores = $availableClusterDatastores `
| Where-Object { $_.FreeSpaceGB -ge $thresholdInGb } `
| Where-Object {
$vmOnDatastore = @((Get-ChildItem -Path $_.DatastoreBrowserPath).Name -notmatch '(^\.|vmkdump)').Count
$vmOnDatastore -lt $vmCount } `
| Group-Object -Property { $vmOnDatastore }
$datastore = $availableDatastores | Select-Object @{n="VmCount";e={$_.Name}},@{n="DatastoreName";e={$_.Group | Get-Random}} -First 1
$buildDatastore = $datastore.DatastoreName
$tag = Get-Tag -Category $TagCategory -Name $VMName -ErrorAction Ignore
if (-not $tag)
{
$tag = New-Tag -Name $VMName -Category $TagCategory
}
New-TagAssignment -Tag $tag -Entity $buildDatastore | Out-Null
# Wait for 60 seconds to check if any other tags are assigned to the same datastore
Start-Sleep -Seconds 60
# If there are no datastores with 0 VMs, take a datastore with 1 VM (index 1 if datastore has 0 VMs and 2 if 1 VM)
$index = 1 + [int]$datastore.VmCount
$tagAssignments = (Get-TagAssignment -Entity $buildDatastore).Tag.Name | Select-Object -First $index
$isAllow = $tagAssignments -contains $VMName
if ($isAllow)
{
Write-Host "Datastore selected successfully"
Write-Host "##vso[task.setvariable variable=buildDatastore;issecret=true]$buildDatastore"
return
}
# Remove the tag if datastore wasn't selected
Remove-Tag $tag -Confirm:$false
$retries--
if ($retries -le 0)
{
Write-Host "##vso[task.LogIssue type=error;]No datastores found for the condition"
exit 1
}
Write-Host "Datastore select failed, $retries left"
Select-DataStore -VMName $VMName -Cluster $Cluster -TagCategory $TagCategory -Retries $retries
}
# Connection to a vCenter Server system
Connect-VCServer -VIServer $VIServer -VIUserName $VIUserName -VIPassword $VIPassword
# Get a target datastore for current deployment
Select-DataStore -VMName $VMName -Cluster $Cluster -TagCategory $TagCategory
+73
View File
@@ -0,0 +1,73 @@
<#
.SYNOPSIS
This script sets resources for VM
.PARAMETER VMName
VM name to resize (Example "macOS-10.15_20201012.4")
.PARAMETER VIServer
vCenter address (Example "10.0.1.16")
.PARAMETER VIUserName
vCenter username (Example "Administrator")
.PARAMETER VIPassword
vCenter password (Example "12345678")
.PARAMETER CpuCount
Target number of CPUs (Example "3")
.PARAMETER CoresPerSocketCount
Target number of cores per socket (Example "3")
.PARAMETER Memory
Target amount of memory in MB (Example "14336")
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VMName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIServer,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIUserName,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$VIPassword,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[int32]$CpuCount,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[int32]$CoresPerSocketCount,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[int64]$Memory
)
# Import helpers module
Import-Module $PSScriptRoot\helpers.psm1 -DisableNameChecking
# Connection to a vCenter Server system
Connect-VCServer -VIServer $VIServer -VIUserName $VIUserName -VIPassword $VIPassword
$vm = Get-VM $VMName
try {
Write-Host "Change CPU count to $CpuCount, cores count to $CoresPerSocketCount, amount of RAM to $Memory"
$vm | Set-VM -NumCPU $CpuCount -CoresPerSocket $CoresPerSocketCount -MemoryMB $Memory -Confirm:$false -ErrorAction Stop | Out-Null
} catch {
Write-Host "##vso[task.LogIssue type=error;]Failed to change specs for VM '$VMName'"
exit 1
}
+63
View File
@@ -0,0 +1,63 @@
param(
[Parameter(Mandatory)] [string] $RepositoryName,
[Parameter(Mandatory)] [string] $AccessToken,
[Parameter(Mandatory)] [string] $SourceBranch,
[Parameter(Mandatory)] [string] $ContributorAllowList
)
function Build-AuthHeader {
param(
[Parameter(Mandatory)] [string] $AccessToken
)
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("'':${AccessToken}"))
return "Basic ${base64AuthInfo}"
}
function Get-PullRequest {
param(
[Parameter(Mandatory)] [string] $RepositoryName,
[Parameter(Mandatory)] [string] $AccessToken,
[Parameter(Mandatory)] [UInt32] $PullRequestNumber
)
$requestUrl = "https://api.github.com/repos/$RepositoryName/pulls/$PullRequestNumber"
$authHeader = Build-AuthHeader -AccessToken $AccessToken
$params = @{
Method = "GET"
ContentType = "application/json"
Uri = $requestUrl
Headers = @{ Authorization = $authHeader }
}
return Invoke-RestMethod @params
}
function Validate-ContributorPermissions {
param(
[Parameter(Mandatory)] [string] $ContributorAllowList,
[Parameter(Mandatory)] [string] $ContributorName
)
$allowedContributors = $ContributorAllowList.Split(",").Trim()
$validСontributor = $allowedContributors | Where-Object { $_ -eq $ContributorName } `
| Select-Object -First 1
if (-not $validСontributor) {
Write-Host "Failed to start this build. '$ContributorName' is an unknown contributor"
Write-Host "Please add '$ContributorName' to the allowed list to run builds"
exit 1
}
}
$pullRequestNumber = $SourceBranch.Split("/")[2]
$pullRequestInfo = Get-PullRequest -RepositoryName $RepositoryName `
-AccessToken $AccessToken `
-PullRequestNumber $pullRequestNumber
$contributorName = $pullRequestInfo.user.login
Validate-ContributorPermissions -ContributorAllowList $ContributorAllowList `
-ContributorName $contributorName
+4 -4
View File
@@ -22,8 +22,8 @@ function Validate-Scripts {
return $ScriptWithoutShebangLine
}
$PathUbuntu = "./images/ubuntu/scripts"
$PathMacOS = "./images/macos"
$PathUbuntu = "./images/linux/scripts"
$PathMacOS = "./images/macos/provision"
$PatternUbuntu = "#!/bin/bash -e"
$PatternMacOS = "#!/bin/bash -e -o pipefail"
$ScriptsWithBrokenShebang = @()
@@ -34,10 +34,10 @@ if ($ScriptsWithBrokenShebang.Length -gt 0) {
$ScriptsWithBrokenShebang | ForEach-Object {
Write-Host "##[error] '$_'"
}
Write-Host "`n`n##[error] Expected shebang for scripts in 'images/ubuntu' folder is '$PatternUbuntu'"
Write-Host "`n`n##[error] Expected shebang for scripts in 'images/linux' folder is '$PatternUbuntu'"
Write-Host "##[error] Expected shebang for scripts in 'images/macos' folder is '$PatternMacOS'"
exit 1
else {
Write-Host "All scripts have correct shebang."
}
}
}
+401
View File
@@ -0,0 +1,401 @@
# Ubuntu 20.04
- OS Version: 20.04.6 LTS
- Kernel Version: 5.15.0-1042-azure
- Image Version: 20230821.1.0
- Systemd version: 245.4-4ubuntu3.22
## Installed Software
### Language and Runtime
- Bash 5.0.17(1)-release
- Clang: 10.0.0, 11.0.0, 12.0.0
- Clang-format: 10.0.0, 11.0.0, 12.0.0
- Clang-tidy: 10.0.0, 11.0.0, 12.0.0
- Dash 0.5.10.2-6
- Erlang 25.3 (Eshell 13.2)
- Erlang rebar3 3.22.1
- GNU C++: 9.4.0, 10.5.0
- GNU Fortran: 9.4.0, 10.5.0
- Julia 1.9.2
- Kotlin 1.9.0-release-358
- Mono 6.12.0.200
- MSBuild 16.10.1.31701 (Mono 6.12.0.200)
- Node.js 18.17.1
- Perl 5.30.0
- Python 3.8.10
- Ruby 2.7.0p0
- Swift 5.8.1
### Package Management
- cpan 1.64
- Helm 3.12.3
- Homebrew 4.1.5
- Miniconda 23.5.2
- Npm 9.6.7
- NuGet 6.6.1.2
- Pip 20.0.2
- Pip3 20.0.2
- Pipx 1.2.0
- RubyGems 3.1.2
- Vcpkg (build from commit c95000e1b)
- Yarn 1.22.19
#### Environment variables
| Name | Value |
| ----------------------- | ---------------------- |
| CONDA | /usr/share/miniconda |
| VCPKG_INSTALLATION_ROOT | /usr/local/share/vcpkg |
#### Homebrew note
```
Location: /home/linuxbrew
Note: Homebrew is pre-installed on image but not added to PATH.
run the eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" command
to accomplish this.
```
### Project Management
- Ant 1.10.7
- Gradle 8.3
- Lerna 7.1.5
- Maven 3.8.8
- Sbt 1.9.3
### Tools
- Ansible 2.13.11
- apt-fast 1.9.12
- AzCopy 10.20.1 - available by `azcopy` and `azcopy10` aliases
- Bazel 6.3.2
- Bazelisk 1.13.2
- Bicep 0.20.4
- Buildah 1.22.3
- CMake 3.27.3
- CodeQL Action Bundles 2.14.1 2.14.2
- Docker Amazon ECR Credential Helper 0.7.1
- Docker Compose v1 1.29.2
- Docker Compose v2 2.20.3
- Docker-Buildx 0.11.2
- Docker Client 24.0.5
- Docker Server 24.0.5
- Fastlane 2.214.0
- Git 2.41.0
- Git LFS 3.4.0
- Git-ftp 1.6.0
- Haveged 1.9.1
- Heroku 8.3.1
- HHVM (HipHop VM) 4.172.1
- jq 1.6
- Kind 0.20.0
- Kubectl 1.28.0
- Kustomize 5.1.1
- Leiningen 2.10.0
- MediaInfo 19.09
- Mercurial 5.3.1
- Minikube 1.31.2
- n 9.1.0
- Newman 5.3.2
- nvm 0.39.4
- OpenSSL 1.1.1f-1ubuntu2.19
- Packer 1.9.4
- Parcel 2.9.3
- PhantomJS 2.1.1 2.1.1
- Podman 3.4.2
- Pulumi 3.78.1
- R 4.3.1
- Skopeo 1.5.0
- Sphinx Open Source Search Server 2.2.11
- SVN 1.13.0
- Terraform 1.5.5
- yamllint 1.32.0
- yq 4.35.1
- zstd 1.5.5
### CLI Tools
- Alibaba Cloud CLI 3.0.174
- AWS CLI 2.13.11
- AWS CLI Session Manager Plugin 1.2.497.0
- AWS SAM CLI 1.95.0
- Azure CLI 2.51.0
- Azure CLI (azure-devops) 0.26.0
- GitHub CLI 2.32.1
- Google Cloud SDK 443.0.0
- Hub CLI 2.14.2
- Netlify CLI 16.1.0
- OpenShift CLI 4.13.9
- ORAS CLI 1.0.1
- Vercel CLI 31.4.0
### Java
| Version | Environment Variable |
| ------------------- | -------------------- |
| 8.0.382+5 | JAVA_HOME_8_X64 |
| 11.0.20+8 (default) | JAVA_HOME_11_X64 |
| 17.0.8+7 | JAVA_HOME_17_X64 |
### PHP Tools
- PHP: 7.4.33, 8.0.30, 8.1.22, 8.2.9
- Composer 2.5.8
- PHPUnit 8.5.33
```
Both Xdebug and PCOV extensions are installed, but only Xdebug is enabled.
```
### Haskell Tools
- Cabal 3.10.1.0
- GHC 9.6.2
- GHCup 0.1.19.4
- Stack 2.11.1
### Rust Tools
- Cargo 1.71.1
- Rust 1.71.1
- Rustdoc 1.71.1
- Rustup 1.26.0
#### Packages
- Bindgen 0.66.1
- Cargo audit 0.17.6
- Cargo clippy 0.1.71
- Cargo outdated 0.13.1
- Cbindgen 0.24.5
- Rustfmt 1.5.2
### Browsers and Drivers
- Google Chrome 116.0.5845.96
- ChromeDriver 116.0.5845.96
- Chromium 116.0.5845.0
- Microsoft Edge 115.0.1901.203
- Microsoft Edge WebDriver 115.0.1901.203
- Selenium server 4.11.0
- Mozilla Firefox 116.0.3
- Geckodriver 0.33.0
#### Environment variables
| Name | Value |
| ----------------- | ------------------------------------- |
| CHROMEWEBDRIVER | /usr/local/share/chromedriver-linux64 |
| EDGEWEBDRIVER | /usr/local/share/edge_driver |
| GECKOWEBDRIVER | /usr/local/share/gecko_driver |
| SELENIUM_JAR_PATH | /usr/share/java/selenium-server.jar |
### .NET Tools
- .NET Core SDK: 6.0.413, 7.0.110, 7.0.203, 7.0.307, 7.0.400
- nbgv 3.6.133+2d32d93cb1
### Databases
- MongoDB 5.0.20
- sqlite3 3.31.1
#### PostgreSQL
- PostgreSQL 14.9
```
User: postgres
PostgreSQL service is disabled by default.
Use the following command as a part of your job to start the service: 'sudo systemctl start postgresql.service'
```
#### MySQL
- MySQL 8.0.34-0ubuntu0.20.04.1
```
User: root
Password: root
MySQL service is disabled by default.
Use the following command as a part of your job to start the service: 'sudo systemctl start mysql.service'
```
#### MS SQL
- sqlcmd 17.10.0001.1
- SqlPackage 162.0.52.1
### Cached Tools
#### Go
- 1.18.10
- 1.19.12
- 1.20.7
- 1.21.0
#### Node.js
- 14.21.3
- 16.20.2
- 18.17.1
#### Python
- 3.6.15
- 3.7.17
- 3.8.17
- 3.9.17
- 3.10.12
- 3.11.4
#### PyPy
- 2.7.18 [PyPy 7.3.12]
- 3.6.12 [PyPy 7.3.3]
- 3.7.13 [PyPy 7.3.9]
- 3.8.16 [PyPy 7.3.11]
- 3.9.17 [PyPy 7.3.12]
- 3.10.12 [PyPy 7.3.12]
#### Ruby
- 2.5.9
- 2.6.10
- 2.7.8
- 3.0.6
- 3.1.4
### PowerShell Tools
- PowerShell 7.2.13
#### PowerShell Modules
- Az: 9.3.0
- Az (Cached): 3.1.0.zip, 4.4.0.zip, 5.9.0.zip, 6.6.0.zip, 7.5.0.zip
- MarkdownPS: 1.9
- Microsoft.Graph: 2.3.0
- Pester: 5.5.0
- PSScriptAnalyzer: 1.21.0
### Web Servers
| Name | Version | ConfigFile | ServiceStatus | ListenPort |
| --------- | ------- | ------------------------- | ------------- | ---------- |
| apache2 | 2.4.41 | /etc/apache2/apache2.conf | inactive | 80 |
| mono-xsp4 | 4.7.1 | /etc/default/mono-xsp4 | active | 8084 |
| nginx | 1.18.0 | /etc/nginx/nginx.conf | inactive | 80 |
### Android
| Package Name | Version |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Android Command Line Tools | 9.0 |
| Android Emulator | 32.1.14 |
| Android SDK Build-tools | 34.0.0<br>33.0.0 33.0.1 33.0.2<br>32.0.0<br>31.0.0<br>30.0.0 30.0.1 30.0.2 30.0.3<br>29.0.0 29.0.1 29.0.2 29.0.3<br>28.0.0 28.0.1 28.0.2 28.0.3<br>27.0.0 27.0.1 27.0.2 27.0.3 |
| Android SDK Platform-Tools | 34.0.4 |
| Android SDK Platforms | android-34 (rev 2)<br>android-33-ext5 (rev 1)<br>android-33-ext4 (rev 1)<br>android-33 (rev 3)<br>android-32 (rev 1)<br>android-31 (rev 1)<br>android-30 (rev 3)<br>android-29 (rev 5)<br>android-28 (rev 6)<br>android-27 (rev 3) |
| Android SDK Tools | 26.1.1 |
| Android Support Repository | 47.0.0 |
| CMake | 3.10.2<br>3.18.1<br>3.22.1 |
| Google Play services | 49 |
| Google Repository | 58 |
| NDK | 23.2.8568313<br>24.0.8215888<br>25.2.9519653 (default) |
| SDK Patch Applier v4 | 1 |
#### Environment variables
| Name | Value |
| ----------------------- | ------------------------------------------- |
| ANDROID_HOME | /usr/local/lib/android/sdk |
| ANDROID_NDK | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_HOME | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_LATEST_HOME | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_ROOT | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_SDK_ROOT | /usr/local/lib/android/sdk |
### Cached Docker images
| Repository:Tag | Digest | Created |
| ----------------------- | ------------------------------------------------------------------------ | ---------- |
| alpine:3.16 | sha256:a8cbb8c69ee71561f4b69c066bad07f7e510caaa523da26fbfc606b10bd7934b | 2023-08-07 |
| alpine:3.17 | sha256:f71a5f071694a785e064f05fed657bf8277f1b2113a8ed70c90ad486d6ee54dc | 2023-08-07 |
| alpine:3.18 | sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a | 2023-08-07 |
| buildpack-deps:bullseye | sha256:930a1ea53e70457cde2c9d449b531979475330c50366fef8d22520d676c2bb0d | 2023-08-16 |
| buildpack-deps:buster | sha256:c6cfa5c691f91c0760341abf073efe1c4b0437ad05375290b3298b8e1219de5b | 2023-08-16 |
| buildpack-deps:stretch | sha256:78e995165a5788c2f55aed6e548d8f6c1534830d4310c870408fccb2da8c5b2e | 2022-06-23 |
| debian:10 | sha256:b86bfc1dd2fb1820fea6c1e0b6e23d155198b518b3a506f6edad71bf6e9a8cef | 2023-08-16 |
| debian:11 | sha256:1beb7cf458bdfe71b5220cb2069eb45e3fc7eb77a1ccfb169eaebf5f6c4809ab | 2023-08-16 |
| debian:9 | sha256:c5c5200ff1e9c73ffbf188b4a67eb1c91531b644856b4aefe86a58d2f0cb05be | 2022-06-23 |
| moby/buildkit:latest | sha256:d91fe5e6dad11f2c2cee6d268d5843ad64bf2e9852365d56b12d02f2315edce5 | 2023-08-02 |
| node:14 | sha256:a158d3b9b4e3fa813fa6c8c590b8f0a860e015ad4e59bbce5744d2f6fd8461aa | 2023-04-12 |
| node:14-alpine | sha256:434215b487a329c9e867202ff89e704d3a75e554822e07f3e0c0f9e606121b33 | 2023-03-29 |
| node:16 | sha256:818b5adc1ee4a04e8ad5efeb70871571efe414315ad7f16844f24f9962ffdc7e | 2023-08-17 |
| node:16-alpine | sha256:a1f9d027912b58a7c75be7716c97cfbc6d3099f3a97ed84aa490be9dee20e787 | 2023-08-10 |
| node:18 | sha256:11e9c297fc51f6f65f7d0c7c8a8581e5721f2f16de43ceff1a199fd3ef609f95 | 2023-08-17 |
| node:18-alpine | sha256:3482a20c97e401b56ac50ba8920cc7b5b2022bfc6aa7d4e4c231755770cf892f | 2023-08-10 |
| ubuntu:16.04 | sha256:1f1a2d56de1d604801a9671f301190704c25d604a416f59e03c04f5c6ffee0d6 | 2021-08-31 |
| ubuntu:18.04 | sha256:152dc042452c496007f07ca9127571cb9c29697f42acbfad72324b2bb2e43c98 | 2023-05-30 |
| ubuntu:20.04 | sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba | 2023-08-01 |
### Installed apt packages
| Name | Version |
| ---------------------- | --------------------------------- |
| acl | 2.2.53-6 |
| aria2 | 1.35.0-1build1 |
| autoconf | 2.69-11.1 |
| automake | 1:1.16.1-4ubuntu6 |
| binutils | 2.34-6ubuntu1.6 |
| bison | 2:3.5.1+dfsg-1 |
| brotli | 1.0.7-6ubuntu0.1 |
| bzip2 | 1.0.8-2 |
| coreutils | 8.30-3ubuntu2 |
| curl | 7.68.0-1ubuntu2.19 |
| dbus | 1.12.16-2ubuntu2.3 |
| dnsutils | 1:9.16.1-0ubuntu2.15 |
| dpkg | 1.19.7ubuntu3.2 |
| dpkg-dev | 1.19.7ubuntu3.2 |
| fakeroot | 1.24-1 |
| file | 1:5.38-4 |
| flex | 2.6.4-6.2 |
| fonts-noto-color-emoji | 0\~20200916-1\~ubuntu20.04.1 |
| ftp | 0.17-34.1 |
| g++ | 4:9.3.0-1ubuntu2 |
| gcc | 4:9.3.0-1ubuntu2 |
| gnupg2 | 2.2.19-3ubuntu2.2 |
| haveged | 1.9.1-6ubuntu1 |
| imagemagick | 8:6.9.10.23+dfsg-2.1ubuntu11.9 |
| iproute2 | 5.5.0-1ubuntu1 |
| iputils-ping | 3:20190709-3 |
| jq | 1.6-1ubuntu0.20.04.1 |
| lib32z1 | 1:1.2.11.dfsg-2ubuntu1.5 |
| libc++-dev | 1:10.0-50\~exp1 |
| libc++abi-dev | 1:10.0-50\~exp1 |
| libc6-dev | 2.31-0ubuntu9.9 |
| libcurl4 | 7.68.0-1ubuntu2.19 |
| libgbm-dev | 21.2.6-0ubuntu0.1\~20.04.2 |
| libgconf-2-4 | 3.2.6-6ubuntu1 |
| libgsl-dev | 2.5+dfsg-6build1 |
| libgtk-3-0 | 3.24.20-0ubuntu1.1 |
| libmagic-dev | 1:5.38-4 |
| libmagickcore-dev | 8:6.9.10.23+dfsg-2.1ubuntu11.9 |
| libmagickwand-dev | 8:6.9.10.23+dfsg-2.1ubuntu11.9 |
| libsecret-1-dev | 0.20.4-0ubuntu1 |
| libsqlite3-dev | 3.31.1-4ubuntu0.5 |
| libtool | 2.4.6-14 |
| libunwind8 | 1.2.1-9ubuntu0.1 |
| libxkbfile-dev | 1:1.1.0-1 |
| libxss1 | 1:1.2.3-1 |
| libyaml-dev | 0.2.2-1 |
| locales | 2.31-0ubuntu9.9 |
| m4 | 1.4.18-4 |
| make | 4.2.1-1.2 |
| mediainfo | 19.09-1build1 |
| mercurial | 5.3.1-1ubuntu1 |
| net-tools | 1.60+git20180626.aebd88e-1ubuntu1 |
| netcat | 1.206-1ubuntu1 |
| openssh-client | 1:8.2p1-4ubuntu0.9 |
| p7zip-full | 16.02+dfsg-7build1 |
| p7zip-rar | 16.02-3build1 |
| parallel | 20161222-1.1 |
| pass | 1.7.3-2 |
| patchelf | 0.10-2build1 |
| pkg-config | 0.29.1-0ubuntu4 |
| pollinate | 4.33-3ubuntu1.20.04.1 |
| python-is-python3 | 3.8.2-4 |
| rpm | 4.14.2.1+dfsg1-1build2 |
| rsync | 3.1.3-8ubuntu0.5 |
| shellcheck | 0.7.0-2build2 |
| sphinxsearch | 2.2.11-2ubuntu2 |
| sqlite3 | 3.31.1-4ubuntu0.5 |
| ssh | 1:8.2p1-4ubuntu0.9 |
| sshpass | 1.06-1 |
| subversion | 1.13.0-3ubuntu0.2 |
| sudo | 1.8.31-1ubuntu1.5 |
| swig | 4.0.1-5build1 |
| tar | 1.30+dfsg-7ubuntu0.20.04.3 |
| telnet | 0.17-41.2build1 |
| texinfo | 6.7.0.dfsg.2-5 |
| time | 1.7-25.1build1 |
| tk | 8.6.9+1 |
| tzdata | 2023c-0ubuntu0.20.04.2 |
| unzip | 6.0-25ubuntu1.1 |
| upx | 3.95-2build1 |
| wget | 1.20.3-1ubuntu2 |
| xorriso | 1.5.2-1 |
| xvfb | 2:1.20.13-1ubuntu1\~20.04.8 |
| xz-utils | 5.2.4-1ubuntu1.1 |
| zip | 3.0-11build1 |
| zsync | 0.6.2-3ubuntu1 |
+384
View File
@@ -0,0 +1,384 @@
# Ubuntu 22.04
- OS Version: 22.04.3 LTS
- Kernel Version: 5.15.0-1041-azure
- Image Version: 20230821.1.0
- Systemd version: 249.11-0ubuntu3.9
## Installed Software
### Language and Runtime
- Bash 5.1.16(1)-release
- Clang: 12.0.1, 13.0.1, 14.0.0
- Clang-format: 12.0.1, 13.0.1, 14.0.0
- Clang-tidy: 12.0.1, 13.0.1, 14.0.0
- Dash 0.5.11+git20210903+057cd650a4ed-3build1
- GNU C++: 9.5.0, 10.5.0, 11.4.0, 12.3.0
- GNU Fortran: 9.5.0, 10.5.0, 11.4.0, 12.3.0
- Julia 1.9.2
- Kotlin 1.9.0-release-358
- Mono 6.12.0.200
- MSBuild 16.10.1.31701 (Mono 6.12.0.200)
- Node.js 18.17.1
- Perl 5.34.0
- Python 3.10.12
- Ruby 3.0.2p107
- Swift 5.8.1
### Package Management
- cpan 1.64
- Helm 3.12.3
- Homebrew 4.1.5
- Miniconda 23.5.2
- Npm 9.6.7
- NuGet 6.6.1.2
- Pip 22.0.2
- Pip3 22.0.2
- Pipx 1.2.0
- RubyGems 3.3.5
- Vcpkg (build from commit c95000e1b)
- Yarn 1.22.19
#### Environment variables
| Name | Value |
| ----------------------- | ---------------------- |
| CONDA | /usr/share/miniconda |
| VCPKG_INSTALLATION_ROOT | /usr/local/share/vcpkg |
#### Homebrew note
```
Location: /home/linuxbrew
Note: Homebrew is pre-installed on image but not added to PATH.
run the eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" command
to accomplish this.
```
### Project Management
- Lerna 7.1.5
- Maven 3.8.8
### Tools
- Ansible 2.15.3
- apt-fast 1.9.12
- AzCopy 10.20.1 - available by `azcopy` and `azcopy10` aliases
- Bazel 6.3.2
- Bazelisk 1.13.2
- Bicep 0.20.4
- Buildah 1.23.1
- CMake 3.27.3
- CodeQL Action Bundles 2.14.1 2.14.2
- Docker Amazon ECR Credential Helper 0.7.1
- Docker Compose v1 1.29.2
- Docker Compose v2 2.20.3
- Docker-Buildx 0.11.2
- Docker Client 24.0.5
- Docker Server 24.0.5
- Fastlane 2.214.0
- Git 2.41.0
- Git LFS 3.4.0
- Git-ftp 1.6.0
- Haveged 1.9.14
- Heroku 8.3.1
- jq 1.6
- Kind 0.20.0
- Kubectl 1.28.0
- Kustomize 5.1.1
- Leiningen 2.10.0
- MediaInfo 21.09
- Mercurial 6.1.1
- Minikube 1.31.2
- n 9.1.0
- Newman 5.3.2
- nvm 0.39.4
- OpenSSL 3.0.2-0ubuntu1.10
- Packer 1.9.4
- Parcel 2.9.3
- Podman 3.4.4
- Pulumi 3.78.1
- R 4.3.1
- Skopeo 1.4.1
- Sphinx Open Source Search Server 2.2.11
- SVN 1.14.1
- Terraform 1.5.5
- yamllint 1.32.0
- yq 4.35.1
- zstd 1.5.5
### CLI Tools
- Alibaba Cloud CLI 3.0.177
- AWS CLI 2.13.11
- AWS CLI Session Manager Plugin 1.2.497.0
- AWS SAM CLI 1.95.0
- Azure CLI 2.51.0
- Azure CLI (azure-devops) 0.26.0
- GitHub CLI 2.32.1
- Google Cloud SDK 443.0.0
- Hub CLI 2.14.2
- Netlify CLI 16.1.0
- OpenShift CLI 4.13.9
- ORAS CLI 1.0.1
- Vercel CLI 31.4.0
### Java
| Version | Environment Variable |
| ------------------- | -------------------- |
| 8.0.382+5 | JAVA_HOME_8_X64 |
| 11.0.20+8 (default) | JAVA_HOME_11_X64 |
| 17.0.8+7 | JAVA_HOME_17_X64 |
### PHP Tools
- PHP: 8.1.2
- Composer 2.5.8
- PHPUnit 8.5.33
```
Both Xdebug and PCOV extensions are installed, but only Xdebug is enabled.
```
### Haskell Tools
- Cabal 3.10.1.0
- GHC 9.6.2
- GHCup 0.1.19.4
- Stack 2.11.1
### Rust Tools
- Cargo 1.71.1
- Rust 1.71.1
- Rustdoc 1.71.1
- Rustup 1.26.0
#### Packages
- Bindgen 0.66.1
- Cargo audit 0.17.6
- Cargo clippy 0.1.71
- Cargo outdated 0.13.1
- Cbindgen 0.24.5
- Rustfmt 1.5.2
### Browsers and Drivers
- Google Chrome 116.0.5845.96
- ChromeDriver 116.0.5845.96
- Chromium 116.0.5845.0
- Microsoft Edge 115.0.1901.203
- Microsoft Edge WebDriver 115.0.1901.203
- Selenium server 4.11.0
- Mozilla Firefox 116.0.3
- Geckodriver 0.33.0
#### Environment variables
| Name | Value |
| ----------------- | ------------------------------------- |
| CHROMEWEBDRIVER | /usr/local/share/chromedriver-linux64 |
| EDGEWEBDRIVER | /usr/local/share/edge_driver |
| GECKOWEBDRIVER | /usr/local/share/gecko_driver |
| SELENIUM_JAR_PATH | /usr/share/java/selenium-server.jar |
### .NET Tools
- .NET Core SDK: 6.0.413, 7.0.110, 7.0.203, 7.0.307, 7.0.400
- nbgv 3.6.133+2d32d93cb1
### Databases
- sqlite3 3.37.2
#### PostgreSQL
- PostgreSQL 14.9
```
User: postgres
PostgreSQL service is disabled by default.
Use the following command as a part of your job to start the service: 'sudo systemctl start postgresql.service'
```
#### MySQL
- MySQL 8.0.34-0ubuntu0.22.04.1
```
User: root
Password: root
MySQL service is disabled by default.
Use the following command as a part of your job to start the service: 'sudo systemctl start mysql.service'
```
#### MS SQL
- sqlcmd 17.10.0001.1
- SqlPackage 162.0.52.1
### Cached Tools
#### Go
- 1.18.10
- 1.19.12
- 1.20.7
- 1.21.0
#### Node.js
- 14.21.3
- 16.20.2
- 18.17.1
#### Python
- 3.7.17
- 3.8.17
- 3.9.17
- 3.10.12
- 3.11.4
#### PyPy
- 3.7.13 [PyPy 7.3.9]
- 3.8.16 [PyPy 7.3.11]
- 3.9.17 [PyPy 7.3.12]
- 3.10.12 [PyPy 7.3.12]
#### Ruby
- 3.1.4
### PowerShell Tools
- PowerShell 7.2.13
#### PowerShell Modules
- Az: 9.3.0
- MarkdownPS: 1.9
- Microsoft.Graph: 2.3.0
- Pester: 5.5.0
- PSScriptAnalyzer: 1.21.0
### Web Servers
| Name | Version | ConfigFile | ServiceStatus | ListenPort |
| ------- | ------- | ------------------------- | ------------- | ---------- |
| apache2 | 2.4.52 | /etc/apache2/apache2.conf | inactive | 80 |
| nginx | 1.18.0 | /etc/nginx/nginx.conf | inactive | 80 |
### Android
| Package Name | Version |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Android Command Line Tools | 9.0 |
| Android Emulator | 32.1.14 |
| Android SDK Build-tools | 34.0.0<br>33.0.0 33.0.1 33.0.2<br>32.0.0<br>31.0.0<br>30.0.0 30.0.1 30.0.2 30.0.3<br>29.0.0 29.0.1 29.0.2 29.0.3<br>28.0.0 28.0.1 28.0.2 28.0.3<br>27.0.0 27.0.1 27.0.2 27.0.3 |
| Android SDK Platform-Tools | 34.0.4 |
| Android SDK Platforms | android-34 (rev 2)<br>android-33-ext5 (rev 1)<br>android-33-ext4 (rev 1)<br>android-33 (rev 3)<br>android-32 (rev 1)<br>android-31 (rev 1)<br>android-30 (rev 3)<br>android-29 (rev 5)<br>android-28 (rev 6)<br>android-27 (rev 3) |
| Android SDK Tools | 26.1.1 |
| Android Support Repository | 47.0.0 |
| CMake | 3.10.2<br>3.18.1<br>3.22.1 |
| Google Play services | 49 |
| Google Repository | 58 |
| NDK | 23.2.8568313<br>24.0.8215888<br>25.2.9519653 (default) |
| SDK Patch Applier v4 | 1 |
#### Environment variables
| Name | Value |
| ----------------------- | ------------------------------------------- |
| ANDROID_HOME | /usr/local/lib/android/sdk |
| ANDROID_NDK | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_HOME | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_LATEST_HOME | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_NDK_ROOT | /usr/local/lib/android/sdk/ndk/25.2.9519653 |
| ANDROID_SDK_ROOT | /usr/local/lib/android/sdk |
### Cached Docker images
| Repository:Tag | Digest | Created |
| ----------------------- | ------------------------------------------------------------------------ | ---------- |
| alpine:3.16 | sha256:a8cbb8c69ee71561f4b69c066bad07f7e510caaa523da26fbfc606b10bd7934b | 2023-08-07 |
| alpine:3.17 | sha256:f71a5f071694a785e064f05fed657bf8277f1b2113a8ed70c90ad486d6ee54dc | 2023-08-07 |
| alpine:3.18 | sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a | 2023-08-07 |
| buildpack-deps:bullseye | sha256:930a1ea53e70457cde2c9d449b531979475330c50366fef8d22520d676c2bb0d | 2023-08-16 |
| buildpack-deps:buster | sha256:c6cfa5c691f91c0760341abf073efe1c4b0437ad05375290b3298b8e1219de5b | 2023-08-16 |
| debian:10 | sha256:b86bfc1dd2fb1820fea6c1e0b6e23d155198b518b3a506f6edad71bf6e9a8cef | 2023-08-16 |
| debian:11 | sha256:1beb7cf458bdfe71b5220cb2069eb45e3fc7eb77a1ccfb169eaebf5f6c4809ab | 2023-08-16 |
| moby/buildkit:latest | sha256:d91fe5e6dad11f2c2cee6d268d5843ad64bf2e9852365d56b12d02f2315edce5 | 2023-08-02 |
| node:14 | sha256:a158d3b9b4e3fa813fa6c8c590b8f0a860e015ad4e59bbce5744d2f6fd8461aa | 2023-04-12 |
| node:14-alpine | sha256:434215b487a329c9e867202ff89e704d3a75e554822e07f3e0c0f9e606121b33 | 2023-03-29 |
| node:16 | sha256:818b5adc1ee4a04e8ad5efeb70871571efe414315ad7f16844f24f9962ffdc7e | 2023-08-17 |
| node:16-alpine | sha256:a1f9d027912b58a7c75be7716c97cfbc6d3099f3a97ed84aa490be9dee20e787 | 2023-08-10 |
| node:18 | sha256:11e9c297fc51f6f65f7d0c7c8a8581e5721f2f16de43ceff1a199fd3ef609f95 | 2023-08-17 |
| node:18-alpine | sha256:3482a20c97e401b56ac50ba8920cc7b5b2022bfc6aa7d4e4c231755770cf892f | 2023-08-10 |
| ubuntu:18.04 | sha256:152dc042452c496007f07ca9127571cb9c29697f42acbfad72324b2bb2e43c98 | 2023-05-30 |
| ubuntu:20.04 | sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba | 2023-08-01 |
| ubuntu:22.04 | sha256:ec050c32e4a6085b423d36ecd025c0d3ff00c38ab93a3d71a460ff1c44fa6d77 | 2023-08-04 |
### Installed apt packages
| Name | Version |
| ---------------------- | ----------------------------------- |
| acl | 2.3.1-1 |
| aria2 | 1.36.0-1 |
| autoconf | 2.71-2 |
| automake | 1:1.16.5-1.3 |
| binutils | 2.38-4ubuntu2.3 |
| bison | 2:3.8.2+dfsg-1build1 |
| brotli | 1.0.9-2build6 |
| bzip2 | 1.0.8-5build1 |
| coreutils | 8.32-4.1ubuntu1 |
| curl | 7.81.0-1ubuntu1.13 |
| dbus | 1.12.20-2ubuntu4.1 |
| dnsutils | 1:9.18.12-0ubuntu0.22.04.2 |
| dpkg | 1.21.1ubuntu2.2 |
| dpkg-dev | 1.21.1ubuntu2.2 |
| fakeroot | 1.28-1ubuntu1 |
| file | 1:5.41-3 |
| flex | 2.6.4-8build2 |
| fonts-noto-color-emoji | 2.038-0ubuntu1 |
| ftp | 20210827-4build1 |
| g++ | 4:11.2.0-1ubuntu1 |
| gcc | 4:11.2.0-1ubuntu1 |
| gnupg2 | 2.2.27-3ubuntu2.1 |
| haveged | 1.9.14-1ubuntu1 |
| imagemagick | 8:6.9.11.60+dfsg-1.3ubuntu0.22.04.3 |
| iproute2 | 5.15.0-1ubuntu2 |
| iputils-ping | 3:20211215-1 |
| jq | 1.6-2.1ubuntu3 |
| lib32z1 | 1:1.2.11.dfsg-2ubuntu9.2 |
| libc++-dev | 1:14.0-55\~exp2 |
| libc++abi-dev | 1:14.0-55\~exp2 |
| libc6-dev | 2.35-0ubuntu3.1 |
| libcurl4 | 7.81.0-1ubuntu1.13 |
| libgbm-dev | 23.0.4-0ubuntu1\~22.04.1 |
| libgconf-2-4 | 3.2.6-7ubuntu2 |
| libgsl-dev | 2.7.1+dfsg-3 |
| libgtk-3-0 | 3.24.33-1ubuntu2 |
| libmagic-dev | 1:5.41-3 |
| libmagickcore-dev | 8:6.9.11.60+dfsg-1.3ubuntu0.22.04.3 |
| libmagickwand-dev | 8:6.9.11.60+dfsg-1.3ubuntu0.22.04.3 |
| libsecret-1-dev | 0.20.5-2 |
| libsqlite3-dev | 3.37.2-2ubuntu0.1 |
| libssl-dev | 3.0.2-0ubuntu1.10 |
| libtool | 2.4.6-15build2 |
| libunwind8 | 1.3.2-2build2.1 |
| libxkbfile-dev | 1:1.1.0-1build3 |
| libxss1 | 1:1.2.3-1build2 |
| libyaml-dev | 0.2.2-1build2 |
| locales | 2.35-0ubuntu3.1 |
| lz4 | 1.9.3-2build2 |
| m4 | 1.4.18-5ubuntu2 |
| make | 4.3-4.1build1 |
| mediainfo | 22.03-1 |
| mercurial | 6.1.1-1ubuntu1 |
| net-tools | 1.60+git20181103.0eebece-1ubuntu5 |
| netcat | 1.218-4ubuntu1 |
| openssh-client | 1:8.9p1-3ubuntu0.3 |
| p7zip-full | 16.02+dfsg-8 |
| p7zip-rar | 16.02-3build1 |
| parallel | 20210822+ds-2 |
| pass | 1.7.4-5 |
| patchelf | 0.14.3-1 |
| pkg-config | 0.29.2-1ubuntu3 |
| pollinate | 4.33-3ubuntu2 |
| python-is-python3 | 3.9.2-2 |
| rpm | 4.17.0+dfsg1-4build1 |
| rsync | 3.2.7-0ubuntu0.22.04.2 |
| shellcheck | 0.8.0-2 |
| sphinxsearch | 2.2.11-8 |
| sqlite3 | 3.37.2-2ubuntu0.1 |
| ssh | 1:8.9p1-3ubuntu0.3 |
| sshpass | 1.09-1 |
| subversion | 1.14.1-3ubuntu0.22.04.1 |
| sudo | 1.9.9-1ubuntu2.4 |
| swig | 4.0.2-1ubuntu1 |
| tar | 1.34+dfsg-1ubuntu0.1.22.04.1 |
| telnet | 0.17-44build1 |
| texinfo | 6.8-4build1 |
| time | 1.9-0.1build2 |
| tk | 8.6.11+1build2 |
| tzdata | 2023c-0ubuntu0.22.04.2 |
| unzip | 6.0-26ubuntu3.1 |
| upx | 3.96-3 |
| wget | 1.21.2-2ubuntu1 |
| xorriso | 1.5.4-2 |
| xvfb | 2:21.1.4-2ubuntu1.7\~22.04.1 |
| xz-utils | 5.2.5-2ubuntu1 |
| zip | 3.0-12build2 |
| zsync | 0.6.2-3ubuntu1 |
+2
View File
@@ -0,0 +1,2 @@
# Name of pool supported by this image
POOL_NAME="Ubuntu 2004"
@@ -20,6 +20,7 @@ function Get-AndroidInstalledPackages {
return $androidSDKManagerList
}
function Build-AndroidTable {
$packageInfo = Get-AndroidInstalledPackages
return @(
@@ -43,6 +44,10 @@ function Build-AndroidTable {
"Package" = "Android SDK Platforms"
"Version" = Get-AndroidPlatformVersions -PackageInfo $packageInfo
},
@{
"Package" = "Android SDK Tools"
"Version" = Get-AndroidPackageVersions -PackageInfo $packageInfo -MatchedString "Android SDK Tools"
},
@{
"Package" = "Android Support Repository"
"Version" = Get-AndroidPackageVersions -PackageInfo $packageInfo -MatchedString "Android Support Repository"
@@ -74,7 +79,7 @@ function Build-AndroidTable {
) | Where-Object { $_.Version } | ForEach-Object {
[PSCustomObject] @{
"Package Name" = $_.Package
"Version" = $_.Version
"Version" = $_.Version
}
}
}
@@ -87,7 +92,7 @@ function Get-AndroidPackageVersions {
[object] $MatchedString
)
$versions = $PackageInfo | Where-Object { $_ -Match $MatchedString } | ForEach-Object {
$versions = $packageInfo | Where-Object { $_ -Match $MatchedString } | ForEach-Object {
$packageInfoParts = Split-TableRowByColumns $_
return $packageInfoParts[1]
}
@@ -100,7 +105,7 @@ function Get-AndroidPlatformVersions {
[object] $PackageInfo
)
$versions = $PackageInfo | Where-Object { $_ -Match "Android SDK Platform " } | ForEach-Object {
$versions = $packageInfo | Where-Object { $_ -Match "Android SDK Platform " } | ForEach-Object {
$packageInfoParts = Split-TableRowByColumns $_
$revision = $packageInfoParts[1]
$version = $packageInfoParts[0].split(";")[1]
@@ -123,7 +128,7 @@ function Get-AndroidBuildToolVersions {
[object] $PackageInfo
)
$versions = $PackageInfo | Where-Object { $_ -Match "Android SDK Build-Tools" } | ForEach-Object {
$versions = $packageInfo | Where-Object { $_ -Match "Android SDK Build-Tools" } | ForEach-Object {
$packageInfoParts = Split-TableRowByColumns $_
return $packageInfoParts[1]
}
@@ -141,7 +146,7 @@ function Get-AndroidGoogleAPIsVersions {
[object] $PackageInfo
)
$versions = $PackageInfo | Where-Object { $_ -Match "Google APIs" } | ForEach-Object {
$versions = $packageInfo | Where-Object { $_ -Match "Google APIs" } | ForEach-Object {
$packageInfoParts = Split-TableRowByColumns $_
return $packageInfoParts[0].split(";")[1]
}
@@ -151,7 +156,7 @@ function Get-AndroidGoogleAPIsVersions {
function Get-AndroidNDKVersions {
$ndkFolderPath = Join-Path (Get-AndroidSDKRoot) "ndk"
$versions = Get-ChildItem -Path $ndkFolderPath -Name
$ndkDefaultVersion = (Get-ToolsetContent).android.ndk.default
$ndkDefaultVersion = Get-ToolsetValue "android.ndk.default"
$ndkDefaultFullVersion = Get-ChildItem "$env:ANDROID_HOME/ndk/$ndkDefaultVersion.*" -Name | Select-Object -Last 1
return ($versions | ForEach-Object {
@@ -166,7 +171,7 @@ function Build-AndroidEnvironmentTable {
$shouldResolveLink = 'ANDROID_NDK', 'ANDROID_NDK_HOME', 'ANDROID_NDK_ROOT', 'ANDROID_NDK_LATEST_HOME'
return $androidVersions | Sort-Object -Property Name | ForEach-Object {
[PSCustomObject] @{
"Name" = $_.Name
"Name" = $_.Name
"Value" = if ($shouldResolveLink.Contains($_.Name )) { Get-PathWithLink($_.Value) } else {$_.Value}
}
}
@@ -1,59 +1,60 @@
function Get-ChromeVersion {
$googleChromeVersion = google-chrome --version | Get-StringPart -Part 2
$googleChromeVersion = google-chrome --version | Take-OutputPart -Part 2
return $googleChromeVersion
}
function Get-ChromeDriverVersion {
$chromeDriverVersion = chromedriver --version | Get-StringPart -Part 1
$chromeDriverVersion = chromedriver --version | Take-OutputPart -Part 1
return $chromeDriverVersion
}
function Get-FirefoxVersion {
$firefoxVersion = $(firefox --version) | Get-StringPart -Part 2
$firefoxVersion = $(firefox --version) | Take-OutputPart -Part 2
return $firefoxVersion
}
function Get-GeckodriverVersion {
$geckodriverVersion = geckodriver --version | Select-Object -First 1 | Get-StringPart -Part 1
$geckodriverVersion = geckodriver --version | Select-Object -First 1 | Take-OutputPart -Part 1
return $geckodriverVersion
}
function Get-ChromiumVersion {
$chromiumVersion = chromium-browser --version | Get-StringPart -Part 1
$chromiumVersion = chromium-browser --version | Take-OutputPart -Part 1
return $chromiumVersion
}
function Get-EdgeVersion {
$edgeVersion = (microsoft-edge --version).Trim() | Get-StringPart -Part 2
$edgeVersion = (microsoft-edge --version).Trim() | Take-OutputPart -Part 2
return $edgeVersion
}
function Get-EdgeDriverVersion {
$edgeDriverVersion = msedgedriver --version | Get-StringPart -Part 3
$edgeDriverVersion = msedgedriver --version | Take-OutputPart -Part 3
return $edgeDriverVersion
}
function Get-SeleniumVersion {
$fullSeleniumVersion = (Get-ChildItem "/usr/share/java/selenium-server-*").Name -replace "selenium-server-"
$seleniumBinaryName = Get-ToolsetValue "selenium.binary_name"
$fullSeleniumVersion = (Get-ChildItem "/usr/share/java/${seleniumBinaryName}-*").Name -replace "${seleniumBinaryName}-"
return $fullSeleniumVersion
}
function Build-BrowserWebdriversEnvironmentTable {
return @(
[PSCustomObject] @{
"Name" = "CHROMEWEBDRIVER"
"Name" = "CHROMEWEBDRIVER"
"Value" = $env:CHROMEWEBDRIVER
},
[PSCustomObject] @{
"Name" = "EDGEWEBDRIVER"
"Name" = "EDGEWEBDRIVER"
"Value" = $env:EDGEWEBDRIVER
},
[PSCustomObject] @{
"Name" = "GECKOWEBDRIVER"
"Name" = "GECKOWEBDRIVER"
"Value" = $env:GECKOWEBDRIVER
},
[PSCustomObject] @{
"Name" = "SELENIUM_JAR_PATH"
"Name" = "SELENIUM_JAR_PATH"
"Value" = $env:SELENIUM_JAR_PATH
}
)
@@ -1,18 +1,18 @@
function Get-ToolcacheRubyVersions {
$toolcachePath = Join-Path $env:AGENT_TOOLSDIRECTORY "Ruby"
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version] $_ }
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version]$_ }
}
function Get-ToolcachePythonVersions {
$toolcachePath = Join-Path $env:AGENT_TOOLSDIRECTORY "Python"
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version] $_ }
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version]$_ }
}
function Get-ToolcachePyPyVersions {
$toolcachePath = Join-Path $env:AGENT_TOOLSDIRECTORY "PyPy"
Get-ChildItem -Path $toolcachePath -Name | Sort-Object { [Version] $_ } | ForEach-Object {
$pypyRootPath = Join-Path $toolcachePath $_ "x64"
[string] $pypyVersionOutput = & "$pypyRootPath/bin/python" -c "import sys;print(sys.version)"
[string]$pypyVersionOutput = & "$pypyRootPath/bin/python" -c "import sys;print(sys.version)"
$pypyVersionOutput -match "^([\d\.]+) \(.+\) \[PyPy ([\d\.]+\S*) .+]$" | Out-Null
return "{0} [PyPy {1}]" -f $Matches[1], $Matches[2]
}
@@ -20,10 +20,10 @@ function Get-ToolcachePyPyVersions {
function Get-ToolcacheNodeVersions {
$toolcachePath = Join-Path $env:AGENT_TOOLSDIRECTORY "node"
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version] $_ }
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version]$_ }
}
function Get-ToolcacheGoVersions {
$toolcachePath = Join-Path $env:AGENT_TOOLSDIRECTORY "go"
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version] $_ }
return Get-ChildItem $toolcachePath -Name | Sort-Object { [Version]$_ }
}
@@ -10,17 +10,18 @@ function Get-DashVersion {
function Get-CPPVersions {
$result = Get-CommandResult "apt list --installed" -Multiline
$cppVersions = $result.Output | Where-Object { $_ -match "g\+\+-\d\d\/" } | ForEach-Object {
& $_.Split("/")[0] --version | Select-Object -First 1 | Get-StringPart -Part 3
} | Sort-Object {[Version] $_}
$cppVersions = $result.Output | Where-Object { $_ -match "g\+\+-\d+"} | ForEach-Object {
& $_.Split("/")[0] --version | Select-Object -First 1 | Take-OutputPart -Part 3
} | Sort-Object {[Version]$_}
return $cppVersions
}
function Get-FortranVersions {
$result = Get-CommandResult "apt list --installed" -Multiline
$fortranVersions = $result.Output | Where-Object { $_ -match "^gfortran-\d\d\/" } | ForEach-Object {
& $_.Split("/")[0] --version | Select-Object -First 1 | Get-StringPart -Part 4
} | Sort-Object {[Version] $_}
$fortranVersions = $result.Output | Where-Object { $_ -match "^gfortran-\d+"} | ForEach-Object {
$_ -match "now (?<version>\d+\.\d+\.\d+)-" | Out-Null
$Matches.version
} | Sort-Object {[Version]$_}
return $fortranVersions
}
@@ -29,25 +30,24 @@ function Get-ClangToolVersions {
[Parameter(Mandatory = $true)]
[string] $ToolName,
[string] $VersionLineMatcher = "${ToolName} version",
[string] $VersionPattern = "\d+\.\d+\.\d+)"
[string] $VersionPattern = "\d+\.\d+\.\d+)-"
)
$result = Get-CommandResult "apt list --installed" -Multiline
$toolVersions = $result.Output | Where-Object { $_ -match "^${ToolName}-\d+" } | ForEach-Object {
$toolVersions = $result.Output | Where-Object { $_ -match "^${ToolName}-\d+"} | ForEach-Object {
$clangCommand = ($_ -Split "/")[0]
Invoke-Expression "$clangCommand --version" | Where-Object { $_ -match "${VersionLineMatcher}" } | ForEach-Object {
$_ -match "${VersionLineMatcher} (?<version>${VersionPattern}" | Out-Null
$Matches.version
}
} | Sort-Object {[Version] $_}
} | Sort-Object {[Version]$_}
return $toolVersions
}
function Get-ClangTidyVersions {
$clangVersions = Get-ClangToolVersions -ToolName "clang-tidy" -VersionLineMatcher "LLVM version" -VersionPattern "\d+\.\d+\.\d+)"
return $clangVersions
return Get-ClangToolVersions -ToolName "clang-tidy" -VersionLineMatcher "LLVM version" -VersionPattern "\d+\.\d+\.\d+)"
}
@@ -64,7 +64,7 @@ function Get-ErlangRebar3Version {
}
function Get-MonoVersion {
$monoVersion = mono --version | Out-String | Get-StringPart -Part 4
$monoVersion = mono --version | Out-String | Take-OutputPart -Part 4
return $monoVersion
}
@@ -75,7 +75,7 @@ function Get-MsbuildVersion {
}
function Get-NuGetVersion {
$nugetVersion = nuget help | Select-Object -First 1 | Get-StringPart -Part 2
$nugetVersion = nuget help | Select-Object -First 1 | Take-OutputPart -Part 2
return $nugetVersion
}
@@ -85,8 +85,7 @@ function Get-NodeVersion {
}
function Get-OpensslVersion {
$opensslVersion = $(dpkg-query -W -f '${Version}' openssl)
return $opensslVersion
return $(dpkg-query -W -f '${Version}' openssl)
}
function Get-PerlVersion {
@@ -96,32 +95,31 @@ function Get-PerlVersion {
function Get-PythonVersion {
$result = Get-CommandResult "python --version"
$version = $result.Output | Get-StringPart -Part 1
$version = $result.Output | Take-OutputPart -Part 1
return $version
}
function Get-PowershellVersion {
$pwshVersion = $(pwsh --version) | Get-StringPart -Part 1
return $pwshVersion
return $(pwsh --version) | Take-OutputPart -Part 1
}
function Get-RubyVersion {
$rubyVersion = ruby --version | Out-String | Get-StringPart -Part 1
$rubyVersion = ruby --version | Out-String | Take-OutputPart -Part 1
return $rubyVersion
}
function Get-SwiftVersion {
$swiftVersion = swift --version | Out-String | Get-StringPart -Part 2
$swiftVersion = swift --version | Out-String | Take-OutputPart -Part 2
return $swiftVersion
}
function Get-KotlinVersion {
$kotlinVersion = kotlin -version | Out-String | Get-StringPart -Part 2
$kotlinVersion = kotlin -version | Out-String | Take-OutputPart -Part 2
return $kotlinVersion
}
function Get-JuliaVersion {
$juliaVersion = julia --version | Get-StringPart -Part 2
$juliaVersion = julia --version | Take-OutputPart -Part 2
return $juliaVersion
}
@@ -131,13 +129,13 @@ function Get-LernaVersion {
}
function Get-HomebrewVersion {
$result = Get-CommandResult "/home/linuxbrew/.linuxbrew/bin/brew --version"
$result = Get-CommandResult "/home/linuxbrew/.linuxbrew/bin/brew -v"
$result.Output -match "Homebrew (?<version>\d+\.\d+\.\d+)" | Out-Null
return $Matches.version
}
function Get-CpanVersion {
$result = Get-CommandResult "cpan --version" -ExpectedExitCode @(25, 255)
$result = Get-CommandResult "cpan --version" -ExpectExitCode @(25, 255)
$result.Output -match "version (?<version>\d+\.\d+) " | Out-Null
return $Matches.version
}
@@ -149,7 +147,7 @@ function Get-GemVersion {
}
function Get-MinicondaVersion {
$condaVersion = conda --version | Get-StringPart -Part 1
$condaVersion = conda --version | Take-OutputPart -Part 1
return $condaVersion
}
@@ -174,13 +172,15 @@ function Get-ParcelVersion {
}
function Get-PipVersion {
$pipVersion = pip --version | Get-StringPart -Part 1
return $pipVersion
$result = Get-CommandResult "pip --version"
$result.Output -match "pip (?<version>\d+\.\d+\.\d+)" | Out-Null
return $Matches.version
}
function Get-Pip3Version {
$pip3Version = pip3 --version | Get-StringPart -Part 1
return $pip3Version
$result = Get-CommandResult "pip3 --version"
$result.Output -match "pip (?<version>\d+\.\d+\.\d+)" | Out-Null
return $Matches.version
}
function Get-VcpkgVersion {
@@ -195,7 +195,7 @@ function Get-AntVersion {
}
function Get-GradleVersion {
$gradleVersion = (gradle -v) -match "^Gradle \d" | Get-StringPart -Part 1
$gradleVersion = (gradle -v) -match "^Gradle \d" | Take-OutputPart -Part 1
return $gradleVersion
}
@@ -207,20 +207,20 @@ function Get-MavenVersion {
function Get-SbtVersion {
$result = Get-CommandResult "sbt -version"
$result.Output -match "sbt runner version: (?<version>\d+\.\d+\.\d+)" | Out-Null
$result.Output -match "sbt script version: (?<version>\d+\.\d+\.\d+)" | Out-Null
return $Matches.version
}
function Get-PHPVersions {
$result = Get-CommandResult "apt list --installed" -Multiline
return $result.Output | Where-Object { $_ -match "^php\d+\.\d+/" } | ForEach-Object {
return $result.Output | Where-Object { $_ -match "^php\d+\.\d+/"} | ForEach-Object {
$_ -match "now (\d+:)?(?<version>\d+\.\d+\.\d+)" | Out-Null
$Matches.version
}
}
function Get-ComposerVersion {
$composerVersion = (composer --version) -replace " version" | Get-StringPart -Part 1
$composerVersion = (composer --version) -replace " version" | Take-OutputPart -Part 1
return $composerVersion
}
@@ -281,7 +281,7 @@ function Get-PowerShellModules {
}
function Get-DotNetCoreSdkVersions {
$dotNetCoreSdkVersion = dotnet --list-sdks list | ForEach-Object { $_ | Get-StringPart -Part 0 }
$dotNetCoreSdkVersion = dotnet --list-sdks list | ForEach-Object { $_ | Take-OutputPart -Part 0 }
return $dotNetCoreSdkVersion
}
@@ -307,8 +307,8 @@ function Get-CachedDockerImagesTableData {
$parts = $_.Split("|")
[PSCustomObject] @{
"Repository:Tag" = $parts[0]
"Digest" = $parts[1]
"Created" = $parts[2].split(' ')[0]
"Digest" = $parts[1]
"Created" = $parts[2].split(' ')[0]
}
} | Sort-Object -Property "Repository:Tag"
}
@@ -318,10 +318,10 @@ function Get-AptPackages {
$output = @()
ForEach ($pkg in ($apt.vital_packages + $apt.common_packages + $apt.cmd_packages)) {
$version = $(dpkg-query -W -f '${Version}' $pkg)
if ($null -eq $version) {
if ($Null -eq $version) {
$version = $(dpkg-query -W -f '${Version}' "$pkg*")
}
$version = $version -replace '~','\~'
$output += [PSCustomObject] @{
@@ -341,18 +341,18 @@ function Get-PipxVersion {
function Build-PackageManagementEnvironmentTable {
return @(
[PSCustomObject] @{
"Name" = "CONDA"
"Name" = "CONDA"
"Value" = $env:CONDA
},
[PSCustomObject] @{
"Name" = "VCPKG_INSTALLATION_ROOT"
"Name" = "VCPKG_INSTALLATION_ROOT"
"Value" = $env:VCPKG_INSTALLATION_ROOT
}
)
}
function Get-SystemdVersion {
$matchCollection = [regex]::Matches((systemctl --version | head -n 1), "\((.*?)\)")
$result = foreach ($match in $matchCollection) {$match.Groups[1].Value}
$matches = [regex]::Matches((systemctl --version | head -n 1), "\((.*?)\)")
$result = foreach ($match in $matches) {$match.Groups[1].Value}
return $result
}
}
@@ -1,25 +1,25 @@
function Get-PostgreSqlVersion {
$postgreSQLVersion = psql --version | Get-StringPart -Part 2
$postgreSQLVersion = psql --version | Take-OutputPart -Part 2
return $postgreSQLVersion
}
function Get-MongoDbVersion {
$mongoDBVersion = mongod --version | Select-Object -First 1 | Get-StringPart -Part 2 -Delimiter "v"
$mongoDBVersion = mongod --version | Select-Object -First 1 | Take-OutputPart -Part 2 -Delimiter "v"
return $mongoDBVersion
}
function Get-SqliteVersion {
$sqliteVersion = sqlite3 --version | Get-StringPart -Part 0
$sqliteVersion = sqlite3 --version | Take-OutputPart -Part 0
return $sqliteVersion
}
function Get-MySQLVersion {
$mySQLVersion = mysqld --version | Get-StringPart -Part 2
$mySQLVersion = mysqld --version | Take-OutputPart -Part 2
return $mySQLVersion
}
function Get-SQLCmdVersion {
$sqlcmdVersion = sqlcmd -? | Select-String -Pattern "Version" | Get-StringPart -Part 1
$sqlcmdVersion = sqlcmd -? | Select-String -Pattern "Version" | Take-OutputPart -Part 1
return $sqlcmdVersion
}
@@ -2,8 +2,8 @@ using module ./software-report-base/SoftwareReport.psm1
using module ./software-report-base/SoftwareReport.Nodes.psm1
param (
[Parameter(Mandatory)]
[string] $OutputDirectory
[Parameter(Mandatory)][string]
$OutputDirectory
)
$global:ErrorActionPreference = "Stop"
@@ -15,7 +15,7 @@ Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Browsers.psm1") -DisableN
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.CachedTools.psm1") -DisableNameChecking
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Common.psm1") -DisableNameChecking
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Databases.psm1") -DisableNameChecking
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Helpers.psm1") -DisableNameChecking
Import-Module "$PSScriptRoot/../helpers/SoftwareReport.Helpers.psm1" -DisableNameChecking
Import-Module "$PSScriptRoot/../helpers/Common.Helpers.psm1" -DisableNameChecking
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Java.psm1") -DisableNameChecking
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Rust.psm1") -DisableNameChecking
@@ -23,7 +23,7 @@ Import-Module (Join-Path $PSScriptRoot "SoftwareReport.Tools.psm1") -DisableName
Import-Module (Join-Path $PSScriptRoot "SoftwareReport.WebServers.psm1") -DisableNameChecking
# Restore file owner in user profile
sudo chown -R ${env:USER}: $env:HOME
Restore-UserOwner
# Software report
$softwareReport = [SoftwareReport]::new("Ubuntu $(Get-OSVersionShort)")
@@ -41,21 +41,22 @@ $languageAndRuntime.AddToolVersionsListInline("Clang", $(Get-ClangToolVersions -
$languageAndRuntime.AddToolVersionsListInline("Clang-format", $(Get-ClangToolVersions -ToolName "clang-format"), "^\d+")
$languageAndRuntime.AddToolVersionsListInline("Clang-tidy", $(Get-ClangTidyVersions), "^\d+")
$languageAndRuntime.AddToolVersion("Dash", $(Get-DashVersion))
if (Test-IsUbuntu20) {
$languageAndRuntime.AddToolVersion("Erlang", $(Get-ErlangVersion))
$languageAndRuntime.AddToolVersion("Erlang rebar3", $(Get-ErlangRebar3Version))
}
$languageAndRuntime.AddToolVersionsListInline("GNU C++", $(Get-CPPVersions), "^\d+")
$languageAndRuntime.AddToolVersionsListInline("GNU Fortran", $(Get-FortranVersions), "^\d+")
$languageAndRuntime.AddToolVersion("Julia", $(Get-JuliaVersion))
$languageAndRuntime.AddToolVersion("Kotlin", $(Get-KotlinVersion))
if (-not $(Test-IsUbuntu24)) {
$languageAndRuntime.AddToolVersion("Mono", $(Get-MonoVersion))
$languageAndRuntime.AddToolVersion("MSBuild", $(Get-MsbuildVersion))
}
$languageAndRuntime.AddToolVersion("Mono", $(Get-MonoVersion))
$languageAndRuntime.AddToolVersion("MSBuild", $(Get-MsbuildVersion))
$languageAndRuntime.AddToolVersion("Node.js", $(Get-NodeVersion))
$languageAndRuntime.AddToolVersion("Perl", $(Get-PerlVersion))
$languageAndRuntime.AddToolVersion("Python", $(Get-PythonVersion))
$languageAndRuntime.AddToolVersion("Ruby", $(Get-RubyVersion))
$languageAndRuntime.AddToolVersion("Swift", $(Get-SwiftVersion))
# Package Management
$packageManagement = $installedSoftware.AddHeader("Package Management")
$packageManagement.AddToolVersion("cpan", $(Get-CpanVersion))
@@ -63,9 +64,7 @@ $packageManagement.AddToolVersion("Helm", $(Get-HelmVersion))
$packageManagement.AddToolVersion("Homebrew", $(Get-HomebrewVersion))
$packageManagement.AddToolVersion("Miniconda", $(Get-MinicondaVersion))
$packageManagement.AddToolVersion("Npm", $(Get-NpmVersion))
if (-not $(Test-IsUbuntu24)) {
$packageManagement.AddToolVersion("NuGet", $(Get-NuGetVersion))
}
$packageManagement.AddToolVersion("NuGet", $(Get-NuGetVersion))
$packageManagement.AddToolVersion("Pip", $(Get-PipVersion))
$packageManagement.AddToolVersion("Pip3", $(Get-Pip3Version))
$packageManagement.AddToolVersion("Pipx", $(Get-PipxVersion))
@@ -82,47 +81,51 @@ to accomplish this.
# Project Management
$projectManagement = $installedSoftware.AddHeader("Project Management")
$projectManagement.AddToolVersion("Ant", $(Get-AntVersion))
$projectManagement.AddToolVersion("Gradle", $(Get-GradleVersion))
$projectManagement.AddToolVersion("Lerna", $(Get-LernaVersion))
if (Test-IsUbuntu20) {
$projectManagement.AddToolVersion("Ant", $(Get-AntVersion))
$projectManagement.AddToolVersion("Gradle", $(Get-GradleVersion))
}
if ((Test-IsUbuntu20) -or (Test-IsUbuntu22)) {
$projectManagement.AddToolVersion("Lerna", $(Get-LernaVersion))
}
$projectManagement.AddToolVersion("Maven", $(Get-MavenVersion))
if (Test-IsUbuntu22) {
if (Test-IsUbuntu20) {
$projectManagement.AddToolVersion("Sbt", $(Get-SbtVersion))
}
# Tools
$tools = $installedSoftware.AddHeader("Tools")
$tools.AddToolVersion("Ansible", $(Get-AnsibleVersion))
if (Test-IsUbuntu22) {
$tools.AddToolVersion("apt-fast", $(Get-AptFastVersion))
}
$tools.AddToolVersion("apt-fast", $(Get-AptFastVersion))
$tools.AddToolVersion("AzCopy", $(Get-AzCopyVersion))
$tools.AddToolVersion("Bazel", $(Get-BazelVersion))
$tools.AddToolVersion("Bazelisk", $(Get-BazeliskVersion))
$tools.AddToolVersion("Bicep", $(Get-BicepVersion))
$tools.AddToolVersion("Buildah", $(Get-BuildahVersion))
$tools.AddToolVersion("CMake", $(Get-CMakeVersion))
$tools.AddToolVersion("CodeQL Action Bundle", $(Get-CodeQLBundleVersion))
$tools.AddToolVersion("CodeQL Action Bundles", $(Get-CodeQLBundleVersions))
$tools.AddToolVersion("Docker Amazon ECR Credential Helper", $(Get-DockerAmazonECRCredHelperVersion))
$tools.AddToolVersion("Docker Compose v1", $(Get-DockerComposeV1Version))
$tools.AddToolVersion("Docker Compose v2", $(Get-DockerComposeV2Version))
$tools.AddToolVersion("Docker-Buildx", $(Get-DockerBuildxVersion))
$tools.AddToolVersion("Docker Client", $(Get-DockerClientVersion))
$tools.AddToolVersion("Docker Server", $(Get-DockerServerVersion))
$tools.AddToolVersion("Fastlane", $(Get-FastlaneVersion))
if ((Test-IsUbuntu20) -or (Test-IsUbuntu22)) {
$tools.AddToolVersion("Fastlane", $(Get-FastlaneVersion))
}
$tools.AddToolVersion("Git", $(Get-GitVersion))
$tools.AddToolVersion("Git LFS", $(Get-GitLFSVersion))
$tools.AddToolVersion("Git-ftp", $(Get-GitFTPVersion))
$tools.AddToolVersion("Haveged", $(Get-HavegedVersion))
if (Test-IsUbuntu22) {
$tools.AddToolVersion("Heroku", $(Get-HerokuVersion))
$tools.AddToolVersion("Heroku", $(Get-HerokuVersion))
if (Test-IsUbuntu20) {
$tools.AddToolVersion("HHVM (HipHop VM)", $(Get-HHVMVersion))
}
$tools.AddToolVersion("jq", $(Get-JqVersion))
$tools.AddToolVersion("Kind", $(Get-KindVersion))
$tools.AddToolVersion("Kubectl", $(Get-KubectlVersion))
$tools.AddToolVersion("Kustomize", $(Get-KustomizeVersion))
if (Test-IsUbuntu22) {
$tools.AddToolVersion("Leiningen", $(Get-LeiningenVersion))
}
$tools.AddToolVersion("Leiningen", $(Get-LeiningenVersion))
$tools.AddToolVersion("MediaInfo", $(Get-MediainfoVersion))
$tools.AddToolVersion("Mercurial", $(Get-HGVersion))
$tools.AddToolVersion("Minikube", $(Get-MinikubeVersion))
@@ -132,108 +135,89 @@ $tools.AddToolVersion("nvm", $(Get-NvmVersion))
$tools.AddToolVersion("OpenSSL", $(Get-OpensslVersion))
$tools.AddToolVersion("Packer", $(Get-PackerVersion))
$tools.AddToolVersion("Parcel", $(Get-ParcelVersion))
if (Test-IsUbuntu20) {
$tools.AddToolVersion("PhantomJS", $(Get-PhantomJSVersion))
}
$tools.AddToolVersion("Podman", $(Get-PodManVersion))
$tools.AddToolVersion("Pulumi", $(Get-PulumiVersion))
if (Test-IsUbuntu22) {
$tools.AddToolVersion("R", $(Get-RVersion))
}
$tools.AddToolVersion("R", $(Get-RVersion))
$tools.AddToolVersion("Skopeo", $(Get-SkopeoVersion))
$tools.AddToolVersion("Sphinx Open Source Search Server", $(Get-SphinxVersion))
if (Test-IsUbuntu22) {
$tools.AddToolVersion("SVN", $(Get-SVNVersion))
$tools.AddToolVersion("Terraform", $(Get-TerraformVersion))
}
$tools.AddToolVersion("SVN", $(Get-SVNVersion))
$tools.AddToolVersion("Terraform", $(Get-TerraformVersion))
$tools.AddToolVersion("yamllint", $(Get-YamllintVersion))
$tools.AddToolVersion("yq", $(Get-YqVersion))
$tools.AddToolVersion("zstd", $(Get-ZstdVersion))
$tools.AddToolVersion("Ninja", $(Get-NinjaVersion))
# CLI Tools
$cliTools = $installedSoftware.AddHeader("CLI Tools")
if (Test-IsUbuntu22) {
$cliTools.AddToolVersion("Alibaba Cloud CLI", $(Get-AlibabaCloudCliVersion))
}
$cliTools.AddToolVersion("Alibaba Cloud CLI", $(Get-AlibabaCloudCliVersion))
$cliTools.AddToolVersion("AWS CLI", $(Get-AWSCliVersion))
$cliTools.AddToolVersion("AWS CLI Session Manager Plugin", $(Get-AWSCliSessionManagerPluginVersion))
$cliTools.AddToolVersion("AWS SAM CLI", $(Get-AWSSAMVersion))
$cliTools.AddToolVersion("Azure CLI", $(Get-AzureCliVersion))
$cliTools.AddToolVersion("Azure CLI (azure-devops)", $(Get-AzureDevopsVersion))
$cliTools.AddToolVersion("GitHub CLI", $(Get-GitHubCliVersion))
$cliTools.AddToolVersion("Google Cloud CLI", $(Get-GoogleCloudCLIVersion))
if (Test-IsUbuntu22) {
$cliTools.AddToolVersion("Netlify CLI", $(Get-NetlifyCliVersion))
$cliTools.AddToolVersion("OpenShift CLI", $(Get-OCCliVersion))
$cliTools.AddToolVersion("ORAS CLI", $(Get-ORASCliVersion))
$cliTools.AddToolVersion("Vercel CLI", $(Get-VerselCliversion))
}
$cliTools.AddToolVersion("Google Cloud SDK", $(Get-GoogleCloudSDKVersion))
$cliTools.AddToolVersion("Hub CLI", $(Get-HubCliVersion))
$cliTools.AddToolVersion("Netlify CLI", $(Get-NetlifyCliVersion))
$cliTools.AddToolVersion("OpenShift CLI", $(Get-OCCliVersion))
$cliTools.AddToolVersion("ORAS CLI", $(Get-ORASCliVersion))
$cliTools.AddToolVersion("Vercel CLI", $(Get-VerselCliversion))
# Java
$installedSoftware.AddHeader("Java").AddTable($(Get-JavaVersionsTable))
# PHP Tools
$phpTools = $installedSoftware.AddHeader("PHP Tools")
$phpTools.AddToolVersionsListInline("PHP", $(Get-PHPVersions), "^\d+\.\d+")
$phpTools.AddToolVersion("Composer", $(Get-ComposerVersion))
$phpTools.AddToolVersion("PHPUnit", $(Get-PHPUnitVersion))
$phpTools.AddNote("Both Xdebug and PCOV extensions are installed, but only Xdebug is enabled.")
# Haskell Tools
$haskellTools = $installedSoftware.AddHeader("Haskell Tools")
$haskellTools.AddToolVersion("Cabal", $(Get-CabalVersion))
$haskellTools.AddToolVersion("GHC", $(Get-GHCVersion))
$haskellTools.AddToolVersion("GHCup", $(Get-GHCupVersion))
$haskellTools.AddToolVersion("Stack", $(Get-StackVersion))
# Rust Tools
Initialize-RustEnvironment
$rustTools = $installedSoftware.AddHeader("Rust Tools")
$rustTools.AddToolVersion("Cargo", $(Get-CargoVersion))
$rustTools.AddToolVersion("Rust", $(Get-RustVersion))
$rustTools.AddToolVersion("Rustdoc", $(Get-RustdocVersion))
$rustTools.AddToolVersion("Rustup", $(Get-RustupVersion))
# Packages
$rustToolsPackages = $rustTools.AddHeader("Packages")
if (Test-IsUbuntu22) {
$rustToolsPackages.AddToolVersion("Bindgen", $(Get-BindgenVersion))
$rustToolsPackages.AddToolVersion("Cargo audit", $(Get-CargoAuditVersion))
$rustToolsPackages.AddToolVersion("Cargo clippy", $(Get-CargoClippyVersion))
$rustToolsPackages.AddToolVersion("Cargo outdated", $(Get-CargoOutdatedVersion))
$rustToolsPackages.AddToolVersion("Cbindgen", $(Get-CbindgenVersion))
}
$rustToolsPackages.AddToolVersion("Bindgen", $(Get-BindgenVersion))
$rustToolsPackages.AddToolVersion("Cargo audit", $(Get-CargoAuditVersion))
$rustToolsPackages.AddToolVersion("Cargo clippy", $(Get-CargoClippyVersion))
$rustToolsPackages.AddToolVersion("Cargo outdated", $(Get-CargoOutdatedVersion))
$rustToolsPackages.AddToolVersion("Cbindgen", $(Get-CbindgenVersion))
$rustToolsPackages.AddToolVersion("Rustfmt", $(Get-RustfmtVersion))
# Browsers and Drivers
$browsersTools = $installedSoftware.AddHeader("Browsers and Drivers")
$browsersTools.AddToolVersion("Google Chrome", $(Get-ChromeVersion))
$browsersTools.AddToolVersion("ChromeDriver", $(Get-ChromeDriverVersion))
$browsersTools.AddToolVersion("Chromium", $(Get-ChromiumVersion))
$browsersTools.AddToolVersion("Microsoft Edge", $(Get-EdgeVersion))
$browsersTools.AddToolVersion("Microsoft Edge WebDriver", $(Get-EdgeDriverVersion))
$browsersTools.AddToolVersion("Selenium server", $(Get-SeleniumVersion))
$browsersTools.AddToolVersion("Mozilla Firefox", $(Get-FirefoxVersion))
$browsersTools.AddToolVersion("Geckodriver", $(Get-GeckodriverVersion))
# Environment variables
$browsersTools.AddHeader("Environment variables").AddTable($(Build-BrowserWebdriversEnvironmentTable))
# .NET Tools
$netCoreTools = $installedSoftware.AddHeader(".NET Tools")
$netCoreTools.AddToolVersionsListInline(".NET Core SDK", $(Get-DotNetCoreSdkVersions), "^\d+\.\d+\.\d")
$netCoreTools.AddNodes($(Get-DotnetTools))
# Databases
$databasesTools = $installedSoftware.AddHeader("Databases")
if (Test-IsUbuntu20) {
$databasesTools.AddToolVersion("MongoDB", $(Get-MongoDbVersion))
}
$databasesTools.AddToolVersion("sqlite3", $(Get-SqliteVersion))
$databasesTools.AddNode($(Build-PostgreSqlSection))
$databasesTools.AddNode($(Build-MySQLSection))
if (-not $(Test-IsUbuntu24)) {
$databasesTools.AddNode($(Build-MSSQLToolsSection))
}
$databasesTools.AddNode($(Build-MSSQLToolsSection))
# Cached Tools
$cachedTools = $installedSoftware.AddHeader("Cached Tools")
$cachedTools.AddToolVersionsList("Go", $(Get-ToolcacheGoVersions), "^\d+\.\d+")
$cachedTools.AddToolVersionsList("Node.js", $(Get-ToolcacheNodeVersions), "^\d+")
@@ -241,8 +225,6 @@ $cachedTools.AddToolVersionsList("Python", $(Get-ToolcachePythonVersions), "^\d+
$cachedTools.AddToolVersionsList("PyPy", $(Get-ToolcachePyPyVersions), "^\d+\.\d+")
$cachedTools.AddToolVersionsList("Ruby", $(Get-ToolcacheRubyVersions), "^\d+\.\d+")
# PowerShell Tools
$powerShellTools = $installedSoftware.AddHeader("PowerShell Tools")
$powerShellTools.AddToolVersion("PowerShell", $(Get-PowershellVersion))
$powerShellTools.AddHeader("PowerShell Modules").AddNodes($(Get-PowerShellModules))
@@ -251,12 +233,9 @@ $installedSoftware.AddHeader("Web Servers").AddTable($(Build-WebServersTable))
$androidTools = $installedSoftware.AddHeader("Android")
$androidTools.AddTable($(Build-AndroidTable))
$androidTools.AddHeader("Environment variables").AddTable($(Build-AndroidEnvironmentTable))
if (-not $(Test-IsUbuntu24)) {
$installedSoftware.AddHeader("Cached Docker images").AddTable($(Get-CachedDockerImagesTableData))
}
$installedSoftware.AddHeader("Cached Docker images").AddTable($(Get-CachedDockerImagesTableData))
$installedSoftware.AddHeader("Installed apt packages").AddTable($(Get-AptPackages))
$softwareReport.ToJson() | Out-File -FilePath "${OutputDirectory}/software-report.json" -Encoding UTF8NoBOM
@@ -1,5 +1,5 @@
function Get-JavaVersionsTable {
$javaToolcacheVersions = Get-ChildItem $env:AGENT_TOOLSDIRECTORY/Java*/* -Directory | Sort-Object { [int] $_.Name.Split(".")[0] }
$javaToolcacheVersions = Get-ChildItem $env:AGENT_TOOLSDIRECTORY/Java*/* -Directory | Sort-Object { [int]$_.Name.Split(".")[0] }
return $javaToolcacheVersions | ForEach-Object {
$majorVersion = $_.Name.split(".")[0]
@@ -5,51 +5,51 @@ function Initialize-RustEnvironment {
}
function Get-RustVersion {
$rustVersion = $(rustc --version) | Get-StringPart -Part 1
$rustVersion = $(rustc --version) | Take-OutputPart -Part 1
return $rustVersion
}
function Get-BindgenVersion {
$bindgenVersion = $(bindgen --version) | Get-StringPart -Part 1
$bindgenVersion = $(bindgen --version) | Take-OutputPart -Part 1
return $bindgenVersion
}
function Get-CargoVersion {
$cargoVersion = $(cargo --version) | Get-StringPart -Part 1
$cargoVersion = $(cargo --version) | Take-OutputPart -Part 1
return $cargoVersion
}
function Get-CargoAuditVersion {
$cargoAuditVersion = $(cargo-audit --version) | Get-StringPart -Part 1
$cargoAuditVersion = $(cargo-audit --version) | Take-OutputPart -Part 1
return $cargoAuditVersion
}
function Get-CargoOutdatedVersion {
$cargoOutdatedVersion = cargo outdated --version | Get-StringPart -Part 1
$cargoOutdatedVersion = cargo outdated --version | Take-OutputPart -Part 1
return $cargoOutdatedVersion
}
function Get-CargoClippyVersion {
$cargoClippyVersion = $(cargo-clippy --version) | Get-StringPart -Part 1
$cargoClippyVersion = $(cargo-clippy --version) | Take-OutputPart -Part 1
return $cargoClippyVersion
}
function Get-CbindgenVersion {
$cbindgenVersion = $(cbindgen --version) | Get-StringPart -Part 1
$cbindgenVersion = $(cbindgen --version) | Take-OutputPart -Part 1
return $cbindgenVersion
}
function Get-RustupVersion {
$rustupVersion = $(rustup --version) | Get-StringPart -Part 1
$rustupVersion = $(rustup --version) | Take-OutputPart -Part 1
return $rustupVersion
}
function Get-RustdocVersion {
$rustdocVersion = $(rustdoc --version) | Get-StringPart -Part 1
$rustdocVersion = $(rustdoc --version) | Take-OutputPart -Part 1
return $rustdocVersion
}
function Get-RustfmtVersion {
$rustfmtVersion = $(rustfmt --version) | Get-StringPart -Part 1 | Get-StringPart -Part 0 -Delimiter "-"
$rustfmtVersion = $(rustfmt --version) | Take-OutputPart -Part 1 | Take-OutputPart -Part 0 -Delimiter "-"
return $rustfmtVersion
}
@@ -10,18 +10,18 @@ function Get-AptFastVersion {
}
function Get-AzCopyVersion {
$azcopyVersion = [string]$(azcopy --version) | Get-StringPart -Part 2
$azcopyVersion = azcopy --version | Take-OutputPart -Part 2
return "$azcopyVersion - available by ``azcopy`` and ``azcopy10`` aliases"
}
function Get-BazelVersion {
$bazelVersion = bazel --version | Select-String "bazel" | Get-StringPart -Part 1
$bazelVersion = bazel --version | Select-String "bazel" | Take-OutputPart -Part 1
return $bazelVersion
}
function Get-BazeliskVersion {
$result = Get-CommandResult "bazelisk version" -Multiline
$bazeliskVersion = $result.Output | Select-String "Bazelisk version:" | Get-StringPart -Part 2 | Get-StringPart -Part 0 -Delimiter "v"
$bazeliskVersion = $result.Output | Select-String "Bazelisk version:" | Take-OutputPart -Part 2 | Take-OutputPart -Part 0 -Delimiter "v"
return $bazeliskVersion
}
@@ -30,38 +30,46 @@ function Get-BicepVersion {
return $Matches.Version
}
function Get-CodeQLBundleVersion {
function Get-CodeQLBundleVersions {
$CodeQLVersionsWildcard = Join-Path $Env:AGENT_TOOLSDIRECTORY -ChildPath "CodeQL" | Join-Path -ChildPath "*"
$CodeQLVersionPath = Get-ChildItem $CodeQLVersionsWildcard | Select-Object -First 1 -Expand FullName
$CodeQLPath = Join-Path $CodeQLVersionPath -ChildPath "x64" | Join-Path -ChildPath "codeql" | Join-Path -ChildPath "codeql"
$CodeQLVersion = & $CodeQLPath version --quiet
return $CodeQLVersion
$CodeQLVersionPaths = Get-ChildItem $CodeQLVersionsWildcard
$CodeQlVersions=@()
foreach ($CodeQLVersionPath in $CodeQLVersionPaths) {
$FullCodeQLVersionPath = $CodeQLVersionPath | Select-Object -Expand FullName
$CodeQLPath = Join-Path $FullCodeQLVersionPath -ChildPath "x64" | Join-Path -ChildPath "codeql" | Join-Path -ChildPath "codeql"
$CodeQLVersion = & $CodeQLPath version --quiet
$CodeQLVersions += $CodeQLVersion
}
return $CodeQLVersions
}
function Get-PodManVersion {
$podmanVersion = podman --version | Get-StringPart -Part 2
$podmanVersion = podman --version | Take-OutputPart -Part 2
return $podmanVersion
}
function Get-BuildahVersion {
$buildahVersion = buildah --version | Get-StringPart -Part 2
$buildahVersion = buildah --version | Take-OutputPart -Part 2
return $buildahVersion
}
function Get-SkopeoVersion {
$skopeoVersion = skopeo --version | Get-StringPart -Part 2
$skopeoVersion = skopeo --version | Take-OutputPart -Part 2
return $skopeoVersion
}
function Get-CMakeVersion {
$cmakeVersion = cmake --version | Select-Object -First 1 | Get-StringPart -Part 2
$cmakeVersion = cmake --version | Select-Object -First 1 | Take-OutputPart -Part 2
return $cmakeVersion
}
function Get-DockerComposeV1Version {
$composeVersion = docker-compose -v | Take-OutputPart -Part 2 | Take-OutputPart -Part 0 -Delimiter ","
return $composeVersion
}
function Get-DockerComposeV2Version {
$composeVersion = docker compose version | Get-StringPart -Part 3 | Get-StringPart -Part 0 -Delimiter "v"
$composeVersion = docker compose version | Take-OutputPart -Part 3 | Take-OutputPart -Part 0 -Delimiter "v"
return $composeVersion
}
@@ -76,62 +84,62 @@ function Get-DockerServerVersion {
}
function Get-DockerBuildxVersion {
$buildxVersion = docker buildx version | Get-StringPart -Part 1 | Get-StringPart -Part 0 -Delimiter "v"
$buildxVersion = docker buildx version | Take-OutputPart -Part 1 | Take-OutputPart -Part 0 -Delimiter "v"
return $buildxVersion
}
function Get-DockerAmazonECRCredHelperVersion {
$ecrVersion = docker-credential-ecr-login -v | Select-String "Version:" | Get-StringPart -Part 1
$ecrVersion = docker-credential-ecr-login -v | Select-String "Version:" | Take-OutputPart -Part 1
return $ecrVersion
}
function Get-GitVersion {
$gitVersion = git --version | Get-StringPart -Part -1
$gitVersion = git --version | Take-OutputPart -Part -1
return $gitVersion
}
function Get-GitLFSVersion {
$result = Get-CommandResult "git-lfs --version"
$gitlfsversion = $result.Output | Get-StringPart -Part 0 | Get-StringPart -Part 1 -Delimiter "/"
$gitlfsversion = $result.Output | Take-OutputPart -Part 0 | Take-OutputPart -Part 1 -Delimiter "/"
return $gitlfsversion
}
function Get-GitFTPVersion {
$gitftpVersion = git-ftp --version | Get-StringPart -Part 2
$gitftpVersion = git-ftp --version | Take-OutputPart -Part 2
return $gitftpVersion
}
function Get-GoogleCloudCLIVersion {
return (gcloud --version | Select-Object -First 1) | Get-StringPart -Part 3
function Get-GoogleCloudSDKVersion {
return (gcloud --version | Select-Object -First 1) | Take-OutputPart -Part 3
}
function Get-HavegedVersion {
$havegedVersion = dpkg-query --showformat='${Version}' --show haveged | Get-StringPart -Part 0 -Delimiter "-"
$havegedVersion = dpkg-query --showformat='${Version}' --show haveged | Take-OutputPart -Part 0 -Delimiter "-"
return $havegedVersion
}
function Get-HerokuVersion {
$herokuVersion = heroku version | Get-StringPart -Part 0 | Get-StringPart -Part 1 -Delimiter "/"
$herokuVersion = heroku version | Take-OutputPart -Part 0 | Take-OutputPart -Part 1 -Delimiter "/"
return $herokuVersion
}
function Get-HHVMVersion {
$hhvmVersion = hhvm --version | Select-Object -First 1 | Get-StringPart -Part 2
$hhvmVersion = hhvm --version | Select-Object -First 1 | Take-OutputPart -Part 2
return $hhvmVersion
}
function Get-SVNVersion {
$svnVersion = svn --version | Select-Object -First 1 | Get-StringPart -Part 2
$svnVersion = svn --version | Select-Object -First 1 | Take-OutputPart -Part 2
return $svnVersion
}
function Get-KustomizeVersion {
$kustomizeVersion = kustomize version --short | Get-StringPart -Part 0 | Get-StringPart -Part 1 -Delimiter "v"
$kustomizeVersion = kustomize version --short | Take-OutputPart -Part 0 | Take-OutputPart -Part 1 -Delimiter "v"
return $kustomizeVersion
}
function Get-KindVersion {
$kindVersion = kind version | Get-StringPart -Part 1 | Get-StringPart -Part 0 -Delimiter "v"
$kindVersion = kind version | Take-OutputPart -Part 1 | Take-OutputPart -Part 0 -Delimiter "v"
return $kindVersion
}
@@ -141,21 +149,21 @@ function Get-KubectlVersion {
}
function Get-MinikubeVersion {
$minikubeVersion = minikube version --short | Get-StringPart -Part 0 -Delimiter "v"
$minikubeVersion = minikube version --short | Take-OutputPart -Part 0 -Delimiter "v"
return $minikubeVersion
}
function Get-HGVersion {
$hgVersion = hg --version | Select-Object -First 1 | Get-StringPart -Part -1 | Get-StringPart -Part 0 -Delimiter ")"
$hgVersion = hg --version | Select-Object -First 1 | Take-OutputPart -Part -1 | Take-OutputPart -Part 0 -Delimiter ")"
return $hgVersion
}
function Get-LeiningenVersion {
return "$(lein -v | Get-StringPart -Part 1)"
return "$(lein -v | Take-OutputPart -Part 1)"
}
function Get-MediainfoVersion {
$mediainfoVersion = (mediainfo --version | Select-Object -Index 1 | Get-StringPart -Part 2).Replace('v', '')
$mediainfoVersion = (mediainfo --version | Select-Object -Index 1 | Take-OutputPart -Part 2).Replace('v', '')
return $mediainfoVersion
}
@@ -174,7 +182,7 @@ function Get-NvmVersion {
}
function Get-PackerVersion {
$packerVersion = (packer --version | Select-String "^Packer").Line.Replace('v','') | Get-StringPart -Part 1
$packerVersion = packer --version
return $packerVersion
}
@@ -184,11 +192,11 @@ function Get-PhantomJSVersion {
}
function Get-TerraformVersion {
return (terraform version | Select-String "^Terraform").Line.Replace('v','') | Get-StringPart -Part 1
return (terraform version | Select-String "^Terraform").Line.Replace('v','') | Take-OutputPart -Part 1
}
function Get-JqVersion {
$jqVersion = jq --version | Get-StringPart -Part 1 -Delimiter "-"
$jqVersion = jq --version | Take-OutputPart -Part 1 -Delimiter "-"
return $jqVersion
}
@@ -208,7 +216,7 @@ function Get-AlibabaCloudCliVersion {
function Get-AWSCliVersion {
$result = Get-CommandResult "aws --version"
$awsVersion = $result.Output | Get-StringPart -Part 0 | Get-StringPart -Part 1 -Delimiter "/"
$awsVersion = $result.Output | Take-OutputPart -Part 0 | Take-OutputPart -Part 1 -Delimiter "/"
return $awsVersion
}
@@ -218,21 +226,26 @@ function Get-AWSCliSessionManagerPluginVersion {
}
function Get-AWSSAMVersion {
return $(sam --version | Get-StringPart -Part -1)
return $(sam --version | Take-OutputPart -Part -1)
}
function Get-FastlaneVersion {
$fastlaneVersion = fastlane --version | Select-String "^fastlane [0-9]" | Get-StringPart -Part 1
$fastlaneVersion = fastlane --version | Select-String "^fastlane [0-9]" | Take-OutputPart -Part 1
return $fastlaneVersion
}
function Get-HubCliVersion {
$hubVersion = hub --version | Select-String "hub version" | Take-OutputPart -Part 2
return $hubVersion
}
function Get-GitHubCliVersion {
$ghVersion = gh --version | Select-String "gh version" | Get-StringPart -Part 2
$ghVersion = gh --version | Select-String "gh version" | Take-OutputPart -Part 2
return $ghVersion
}
function Get-NetlifyCliVersion {
$netlifyVersion = netlify --version | Get-StringPart -Part 0 | Get-StringPart -Part 1 -Delimiter "/"
$netlifyVersion = netlify --version | Take-OutputPart -Part 0 | Take-OutputPart -Part 1 -Delimiter "/"
return $netlifyVersion
}
@@ -242,7 +255,7 @@ function Get-OCCliVersion {
}
function Get-ORASCliVersion {
$orasVersion = oras version | Select-String "^Version:" | Get-StringPart -Part 1
$orasVersion = oras version | Select-String "^Version:" | Take-OutputPart -Part 1
return $orasVersion
}
@@ -252,34 +265,30 @@ function Get-VerselCliversion {
}
function Get-PulumiVersion {
$pulumiVersion = pulumi version | Get-StringPart -Part 0 -Delimiter "v"
$pulumiVersion = pulumi version | Take-OutputPart -Part 0 -Delimiter "v"
return $pulumiVersion
}
function Get-RVersion {
$rVersion = (Get-CommandResult "R --version | grep 'R version'").Output | Get-StringPart -Part 2
$rVersion = (Get-CommandResult "R --version | grep 'R version'").Output | Take-OutputPart -Part 2
return $rVersion
}
function Get-SphinxVersion {
$sphinxVersion = searchd -h | Select-Object -First 1 | Get-StringPart -Part 1 | Get-StringPart -Part 0 -Delimiter "-"
$sphinxVersion = searchd -h | Select-Object -First 1 | Take-OutputPart -Part 1 | Take-OutputPart -Part 0 -Delimiter "-"
return $sphinxVersion
}
function Get-YamllintVersion {
return $(yamllint --version) | Get-StringPart -Part 1
return $(yamllint --version) | Take-OutputPart -Part 1
}
function Get-ZstdVersion {
$zstdVersion = zstd --version | Get-StringPart -Part 1 -Delimiter "v" | Get-StringPart -Part 0 -Delimiter ","
$zstdVersion = zstd --version | Take-OutputPart -Part 1 -Delimiter "v" | Take-OutputPart -Part 0 -Delimiter ","
return "$zstdVersion"
}
function Get-NinjaVersion {
return $(ninja --version)
}
function Get-YqVersion {
$yqVersion = $(yq -V) | Get-StringPart -Part 3
$yqVersion = $(yq -V) | Take-OutputPart -Part 3
return $yqVersion.TrimStart("v").Trim()
}
}
@@ -0,0 +1,55 @@
function Get-ApacheVersion {
$name = "apache2"
$port = 80
$version = bash -c "apache2 -v | grep -Po 'Apache/(\d+.){2}\d+'" | Take-OutputPart -Part 1 -Delimiter "/"
$serviceStatus = systemctl status apache2 | grep "Active:" | Take-OutputPart -Part 1
$configFile = "/etc/apache2/apache2.conf"
return [PsCustomObject]@{
"Name" = $name
"Version" = $version
"ConfigFile" = $configFile
"ServiceStatus" = $serviceStatus
"ListenPort" = $port
}
}
function Get-NginxVersion {
$name = "nginx"
$port = 80
$version = (dpkg-query --showformat='${Version}' --show nginx).Split('-')[0]
$serviceStatus = systemctl status nginx | grep "Active:" | Take-OutputPart -Part 1
$configFile = "/etc/nginx/nginx.conf"
return [PsCustomObject]@{
"Name" = $name
"Version" = $version
"ConfigFile" = $configFile
"ServiceStatus" = $serviceStatus
"ListenPort" = $port
}
}
function Get-Xsp4Version {
$name = "mono-xsp4"
$port = (grep '^port=' /etc/default/mono-xsp4).Split('=')[1]
$version = (dpkg-query --showformat='${Version}' --show mono-xsp4).Split('-')[0]
$serviceStatus = systemctl show -p ActiveState --value mono-xsp4
$configFile = "/etc/default/mono-xsp4"
return [PsCustomObject]@{
"Name" = $name
"Version" = $version
"ConfigFile" = $configFile
"ServiceStatus" = $serviceStatus
"ListenPort" = $port
}
}
function Build-WebServersTable {
$servers = @()
$servers += (Get-ApacheVersion)
if (Test-IsUbuntu20) {
$servers += (Get-Xsp4Version)
}
$servers += (Get-NginxVersion)
return $servers
}
@@ -0,0 +1,7 @@
#!/bin/bash -e
prefix=/usr/local/bin
for tool in apt apt-get apt-fast apt-key;do
sudo rm -f $prefix/$tool
done
@@ -1,15 +1,12 @@
#!/bin/bash -e
################################################################################
## File: configure-apt-mock.sh
## Desc: A temporary workaround for https://github.com/Azure/azure-linux-extensions/issues/1238.
## Cleaned up during cleanup.sh.
################################################################################
# A temporary workaround for https://github.com/Azure/azure-linux-extensions/issues/1238
prefix=/usr/local/bin
for real_tool in /usr/bin/apt /usr/bin/apt-get /usr/bin/apt-key; do
tool=$(basename $real_tool)
cat >$prefix/$tool <<EOT
for real_tool in /usr/bin/apt /usr/bin/apt-get /usr/bin/apt-fast /usr/bin/apt-key;do
tool=`basename $real_tool`
cat >$prefix/$tool <<EOT
#!/bin/sh
i=1
@@ -50,5 +47,5 @@ while [ \$i -le 30 ];do
i=\$((i + 1))
done
EOT
chmod +x $prefix/$tool
chmod +x $prefix/$tool
done
@@ -0,0 +1,15 @@
#!/bin/bash -e
################################################################################
## File: apt-ubuntu-archive.sh
## Desc: Script for configuring apt sources. https://manpages.ubuntu.com/manpages/jammy/en/man1/apt-transport-mirror.1.html
################################################################################
touch /etc/apt/apt-mirrors.txt
printf "http://azure.archive.ubuntu.com/ubuntu/\tpriority:1\n" | tee -a /etc/apt/apt-mirrors.txt
printf "http://archive.ubuntu.com/ubuntu/\tpriority:2\n" | tee -a /etc/apt/apt-mirrors.txt
printf "http://security.ubuntu.com/ubuntu/\tpriority:3\n" | tee -a /etc/apt/apt-mirrors.txt
sed -i 's/http:\/\/azure.archive.ubuntu.com\/ubuntu\//mirror+file:\/etc\/apt\/apt-mirrors.txt/' /etc/apt/sources.list
cp -f /etc/apt/sources.list /etc/cloud/templates/sources.list.ubuntu.tmpl
@@ -1,10 +1,4 @@
#!/bin/bash -e
################################################################################
## File: configure-apt.sh
## Desc: Configure apt, install jq and apt-fast packages.
################################################################################
source $HELPER_SCRIPTS/os.sh
# Stop and disable apt-daily upgrade services;
systemctl stop apt-daily.timer
@@ -31,8 +25,6 @@ echo 'APT::Get::Always-Include-Phased-Updates "true";' > /etc/apt/apt.conf.d/99-
cat <<EOF >> /etc/apt/apt.conf.d/99bad_proxy
Acquire::http::Pipeline-Depth 0;
Acquire::http::No-Cache true;
Acquire::https::Pipeline-Depth 0;
Acquire::https::No-Cache true;
Acquire::BrokenProxy true;
EOF
@@ -40,18 +32,12 @@ EOF
apt-get purge unattended-upgrades
echo 'APT sources'
if ! is_ubuntu24; then
cat /etc/apt/sources.list
else
cat /etc/apt/sources.list.d/ubuntu.sources
fi
cat /etc/apt/sources.list
apt-get update
# Install jq
apt-get install jq
if ! is_ubuntu24; then
# Install apt-fast using quick-install.sh
# https://github.com/ilikenwf/apt-fast
bash -c "$(curl -fsSL https://raw.githubusercontent.com/ilikenwf/apt-fast/master/quick-install.sh)"
fi
# Install apt-fast using quick-install.sh
# https://github.com/ilikenwf/apt-fast
bash -c "$(curl -fsSL https://raw.githubusercontent.com/ilikenwf/apt-fast/master/quick-install.sh)"
@@ -1,8 +1,4 @@
#!/bin/bash -e
################################################################################
## File: configure-limits.sh
## Desc: Configure limits
################################################################################
echo 'session required pam_limits.so' >> /etc/pam.d/common-session
echo 'session required pam_limits.so' >> /etc/pam.d/common-session-noninteractive
@@ -1,6 +1,8 @@
#!/bin/bash -e
################################################################################
## File: Install-TortoiseSvn.ps1
## Desc: Install TortoiseSvn
## File: reboot.sh
## Desc: Reboot VM
################################################################################
Install-ChocoPackage tortoisesvn
echo "Reboot VM"
sudo reboot
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash -e
################################################################################
## File: repos.sh
## Desc: Installs official Microsoft package repos for the distribution
################################################################################
LSB_RELEASE=$(lsb_release -rs)
# Install Microsoft repository
wget https://packages.microsoft.com/config/ubuntu/$LSB_RELEASE/packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
# update
apt-get install -y apt-transport-https ca-certificates curl software-properties-common
apt-get -yq update
apt-get -yq dist-upgrade
@@ -1,16 +1,4 @@
#!/bin/bash -e
################################################################################
## File: configure-snap.sh
## Desc: Configure snap
################################################################################
# Source the helpers for use with the script
source $HELPER_SCRIPTS/etc-environment.sh
# Update /etc/environment to include /snap/bin in PATH
# because /etc/profile.d is ignored by `--norc` shell launch option
prepend_etc_environment_path "/snap/bin"
# Put snapd auto refresh on hold
# as it may generate too much traffic on Canonical's snap server
@@ -0,0 +1,93 @@
function Get-CommandResult {
param (
[Parameter(Mandatory=$true)]
[string] $Command,
[int[]] $ExpectExitCode = 0,
[switch] $Multiline,
[bool] $ValidateExitCode = $true
)
# Bash trick to suppress and show error output because some commands write to stderr (for example, "python --version")
$stdout = & bash -c "$Command 2>&1"
$exitCode = $LASTEXITCODE
if ($ValidateExitCode) {
if ($ExpectExitCode -notcontains $exitCode) {
try {
throw "StdOut: '$stdout' ExitCode: '$exitCode'"
} catch {
Write-Host $_.Exception.Message
Write-Host $_.ScriptStackTrace
exit $LASTEXITCODE
}
}
}
return @{
Output = If ($Multiline -eq $true) { $stdout } else { [string]$stdout }
ExitCode = $exitCode
}
}
function Get-OSVersionShort {
$(Get-OSVersionFull) | Take-OutputPart -Delimiter '.' -Part 0,1
}
function Get-OSVersionFull {
lsb_release -ds | Take-OutputPart -Part 1, 2
}
function Get-KernelVersion {
$kernelVersion = uname -r
return $kernelVersion
}
function Test-IsUbuntu20 {
return (lsb_release -rs) -eq "20.04"
}
function Test-IsUbuntu22 {
return (lsb_release -rs) -eq "22.04"
}
function Get-ToolsetContent {
$toolset = Join-Path $env:INSTALLER_SCRIPT_FOLDER "toolset.json"
Get-Content $toolset -Raw | ConvertFrom-Json
}
function Get-ToolsetValue {
param (
[Parameter(Mandatory = $true)]
[string] $KeyPath
)
$jsonNode = Get-ToolsetContent
$pathParts = $KeyPath.Split(".")
# try to walk through all arguments consequentially to resolve specific json node
$pathParts | ForEach-Object {
$jsonNode = $jsonNode.$_
}
return $jsonNode
}
function Get-AndroidPackages {
$packagesListFile = "/usr/local/lib/android/sdk/packages-list.txt"
if (-Not (Test-Path -Path $packagesListFile -PathType Leaf)) {
(/usr/local/lib/android/sdk/cmdline-tools/latest/bin/sdkmanager --list --verbose 2>&1) |
Where-Object { $_ -Match "^[^\s]" } |
Where-Object { $_ -NotMatch "^(Loading |Info: Parsing |---|\[=+|Installed |Available )" } |
Where-Object { $_ -NotMatch "^[^;]*$" } |
Out-File -FilePath $packagesListFile
Write-Host Android packages list:
Get-Content $packagesListFile
}
return Get-Content $packagesListFile
}
function Get-EnvironmentVariable($variable) {
return [System.Environment]::GetEnvironmentVariable($variable)
}
@@ -0,0 +1,41 @@
function Take-OutputPart {
param (
[Parameter(ValueFromPipeline)]
[string] $toolOutput,
[string] $Delimiter = " ",
[int[]] $Part
)
$parts = $toolOutput.Split($Delimiter, [System.StringSplitOptions]::RemoveEmptyEntries)
$selectedParts = $parts[$Part]
return [string]::Join($Delimiter, $selectedParts)
}
function Restore-UserOwner {
sudo chown -R ${env:USER}: $env:HOME
}
function Get-LinkTarget {
param (
[string] $inputPath
)
$link = Get-Item $inputPath | Select-Object -ExpandProperty Target
if ($link) {
return " -> $link"
}
return ""
}
function Get-PathWithLink {
param (
[string] $inputPath
)
$link = Get-LinkTarget($inputPath)
return "${inputPath}${link}"
}
function Get-AptSourceRepository {
param([String] $PackageName)
$sourceUrl = Get-Content "$PSScriptRoot/../helpers/apt-sources.txt" | Select-String -Pattern $PackageName | Take-OutputPart -Part (1..3)
return $sourceUrl
}
@@ -1,32 +1,13 @@
Import-Module "$PSScriptRoot/../helpers/Common.Helpers.psm1" -DisableNameChecking
# Validates that tool is installed and in PATH
function Validate-ToolExist($tool) {
Get-Command $tool -ErrorAction SilentlyContinue | Should -BeTrue
}
function Invoke-PesterTests {
<#
.SYNOPSIS
Runs Pester tests based on the provided test file and test name.
.DESCRIPTION
The Invoke-PesterTests function runs Pester tests based on the provided test file and test name.
.PARAMETER TestFile
The name of the test file to run. This should be the base name of the test file without the extension.
Using "*" will run all tests from all test files.
.PARAMETER TestName
The name of the specific test to run. If provided, only the test with the matching name will be executed.
.EXAMPLE
Invoke-PesterTests -TestFile "MyTests" -TestName "Test1"
Runs the test named "Test1" from the test file "MyTests.Tests.ps1".
.EXAMPLE
Invoke-PesterTests -TestFile "*"
Runs all tests from all test files
#>
Param(
[Parameter(Mandatory = $true)]
[string] $TestFile,
[Parameter(Mandatory)][string] $TestFile,
[string] $TestName
)
@@ -36,12 +17,12 @@ function Invoke-PesterTests {
}
# Check that Pester module is imported
if (-not (Get-Module "Pester")) {
if (!(Get-Module "Pester")) {
Import-Module Pester
}
$configuration = [PesterConfiguration] @{
Run = @{ Path = $testPath; PassThru = $true }
Run = @{ Path = $testPath; PassThru = $true }
Output = @{ Verbosity = "Detailed"; RenderMode = "Plaintext" }
}
if ($TestName) {
@@ -62,29 +43,6 @@ function Invoke-PesterTests {
}
function ShouldReturnZeroExitCode {
<#
.SYNOPSIS
Implements a custom Should-operator for the Pester framework.
.DESCRIPTION
This function is used to check if a command has returned a zero exit code.
It can be used by registering it using the Add-ShouldOperator function in Pester.
.PARAMETER ActualValue
The actual value to be checked.
.PARAMETER Negate
A switch parameter that, when specified, negates the result of the check.
.PARAMETER Because
An optional string that provides additional context or explanation for the check.
.NOTES
This function is designed to be used with the Pester framework.
.LINK
https://pester.dev/docs/assertions/custom-assertions
#>
Param(
[string] $ActualValue,
[switch] $Negate,
@@ -93,10 +51,11 @@ function ShouldReturnZeroExitCode {
$result = Get-CommandResult $ActualValue -ValidateExitCode $false
[bool] $succeeded = $result.ExitCode -eq 0
[bool]$succeeded = $result.ExitCode -eq 0
if ($Negate) { $succeeded = -not $succeeded }
if (-not $succeeded) {
if (-not $succeeded)
{
$commandOutputIndent = " " * 4
$commandOutput = ($result.Output | ForEach-Object { "${commandOutputIndent}${_}" }) -join "`n"
$failureMessage = "Command '${ActualValue}' has finished with exit code`n${commandOutput}"
@@ -108,30 +67,7 @@ function ShouldReturnZeroExitCode {
}
}
function ShouldOutputTextMatchingRegex {
<#
.SYNOPSIS
Implements a custom Should-operator for the Pester framework.
.DESCRIPTION
This function is used to check if a command outputs text that matches a regular expression.
It can be used by registering it using the Add-ShouldOperator function in Pester.
.PARAMETER ActualValue
The actual value to be checked.
.PARAMETER RegularExpression
The regular expression to be used for the check.
.PARAMETER Negate
A switch parameter that, when specified, negates the result of the check.
.NOTES
This function is designed to be used with the Pester framework.
.LINK
https://pester.dev/docs/assertions/custom-assertions
#>
function ShouldMatchCommandOutput {
Param(
[string] $ActualValue,
[string] $RegularExpression,
@@ -141,14 +77,17 @@ function ShouldOutputTextMatchingRegex {
$output = (Get-CommandResult $ActualValue -ValidateExitCode $false).Output | Out-String
[bool] $succeeded = $output -cmatch $RegularExpression
if ($Negate) { $succeeded = -not $succeeded }
if ($Negate) {
$succeeded = -not $succeeded
}
$failureMessage = ''
if (-not $succeeded) {
if ($Negate) {
$failureMessage = "Expected regular expression '$RegularExpression' for '$ActualValue' command to not match '$output', but it did match."
} else {
}
else {
$failureMessage = "Expected regular expression '$RegularExpression' for '$ActualValue' command to match '$output', but it did not match."
}
}
@@ -161,5 +100,5 @@ function ShouldOutputTextMatchingRegex {
If (Get-Command -Name Add-ShouldOperator -ErrorAction SilentlyContinue) {
Add-ShouldOperator -Name ReturnZeroExitCode -InternalName ShouldReturnZeroExitCode -Test ${function:ShouldReturnZeroExitCode}
Add-ShouldOperator -Name OutputTextMatchingRegex -InternalName ShouldOutputTextMatchingRegex -Test ${function:ShouldOutputTextMatchingRegex}
Add-ShouldOperator -Name MatchCommandOutput -InternalName ShouldMatchCommandOutput -Test ${function:ShouldMatchCommandOutput}
}
@@ -0,0 +1,85 @@
#!/bin/bash -e
################################################################################
## File: etc-environment.sh
## Desc: Helper functions for source and modify /etc/environment
################################################################################
# NB: sed expression use '%' as a delimiter in order to simplify handling
# values containg slashes (i.e. directory path)
# The values containing '%' will break the functions
function getEtcEnvironmentVariable {
variable_name="$1"
# remove `variable_name=` and possible quotes from the line
grep "^${variable_name}=" /etc/environment |sed -E "s%^${variable_name}=\"?([^\"]+)\"?.*$%\1%"
}
function addEtcEnvironmentVariable {
variable_name="$1"
variable_value="$2"
echo "$variable_name=$variable_value" | sudo tee -a /etc/environment
}
function replaceEtcEnvironmentVariable {
variable_name="$1"
variable_value="$2"
# modify /etc/environemnt in place by replacing a string that begins with variable_name
sudo sed -i -e "s%^${variable_name}=.*$%${variable_name}=\"${variable_value}\"%" /etc/environment
}
function setEtcEnvironmentVariable {
variable_name="$1"
variable_value="$2"
if grep "$variable_name" /etc/environment > /dev/null; then
replaceEtcEnvironmentVariable $variable_name $variable_value
else
addEtcEnvironmentVariable $variable_name $variable_value
fi
}
function prependEtcEnvironmentVariable {
variable_name="$1"
element="$2"
# TODO: handle the case if the variable does not exist
existing_value=$(getEtcEnvironmentVariable "${variable_name}")
setEtcEnvironmentVariable "${variable_name}" "${element}:${existing_value}"
}
function appendEtcEnvironmentVariable {
variable_name="$1"
element="$2"
# TODO: handle the case if the variable does not exist
existing_value=$(getEtcEnvironmentVariable "${variable_name}")
setEtcEnvironmentVariable "${variable_name}" "${existing_value}:${element}"
}
function prependEtcEnvironmentPath {
element="$1"
prependEtcEnvironmentVariable PATH "${element}"
}
function appendEtcEnvironmentPath {
element="$1"
appendEtcEnvironmentVariable PATH "${element}"
}
# Process /etc/environment as if it were shell script with `export VAR=...` expressions
# The PATH variable is handled specially in order to do not override the existing PATH
# variable. The value of PATH variable read from /etc/environment is added to the end
# of value of the exiting PATH variable exactly as it would happen with real PAM app read
# /etc/environment
#
# TODO: there might be the others variables to be processed in the same way as "PATH" variable
# ie MANPATH, INFOPATH, LD_*, etc. In the current implementation the values from /etc/evironments
# replace the values of the current environment
function reloadEtcEnvironment {
# add `export ` to every variable of /etc/environemnt except PATH and eval the result shell script
eval $(grep -v '^PATH=' /etc/environment | sed -e 's%^%export %')
# handle PATH specially
etc_path=$(getEtcEnvironmentVariable PATH)
export PATH="$PATH:$etc_path"
}
+89
View File
@@ -0,0 +1,89 @@
#!/bin/bash -e
################################################################################
## File: install.sh
## Desc: Helper functions for installing tools
################################################################################
download_with_retries() {
# Due to restrictions of bash functions, positional arguments are used here.
# In case if you using latest argument NAME, you should also set value to all previous parameters.
# Example: download_with_retries $ANDROID_SDK_URL "." "android_sdk.zip"
local URL="$1"
local DEST="${2:-.}"
local NAME="${3:-${URL##*/}}"
local COMPRESSED="$4"
if [[ $COMPRESSED == "compressed" ]]; then
local COMMAND="curl $URL -4 -sL --compressed -o '$DEST/$NAME' -w '%{http_code}'"
else
local COMMAND="curl $URL -4 -sL -o '$DEST/$NAME' -w '%{http_code}'"
fi
echo "Downloading '$URL' to '${DEST}/${NAME}'..."
retries=20
interval=30
while [ $retries -gt 0 ]; do
((retries--))
# Temporary disable exit on error to retry on non-zero exit code
set +e
http_code=$(eval $COMMAND)
exit_code=$?
set -e
if [ $http_code -eq 200 ] && [ $exit_code -eq 0 ]; then
echo "Download completed"
return 0
else
echo "Error — Either HTTP response code for '$URL' is wrong - '$http_code' or exit code is not 0 - '$exit_code'. Waiting $interval seconds before the next attempt, $retries attempts left"
sleep $interval
fi
done
echo "Could not download $URL"
return 1
}
## Use dpkg to figure out if a package has already been installed
## Example use:
## if ! IsPackageInstalled packageName; then
## echo "packageName is not installed!"
## fi
IsPackageInstalled() {
dpkg -S $1 &> /dev/null
}
verlte() {
sortedVersion=$(echo -e "$1\n$2" | sort -V | head -n1)
[ "$1" = "$sortedVersion" ]
}
get_toolset_path() {
echo "/imagegeneration/installers/toolset.json"
}
get_toolset_value() {
local toolset_path=$(get_toolset_path)
local query=$1
echo "$(jq -r "$query" $toolset_path)"
}
get_github_package_download_url() {
local REPO_ORG=$1
local FILTER=$2
local VERSION=$3
local SEARCH_IN_COUNT="100"
json=$(curl -fsSL "https://api.github.com/repos/${REPO_ORG}/releases?per_page=${SEARCH_IN_COUNT}")
if [ -n "$VERSION" ]; then
tagName=$(echo $json | jq -r '.[] | select(.prerelease==false).tag_name' | sort --unique --version-sort | egrep -v ".*-[a-z]|beta" | egrep "\w*${VERSION}" | tail -1)
else
tagName=$(echo $json | jq -r '.[] | select((.prerelease==false) and (.assets | length > 0)).tag_name' | sort --unique --version-sort | egrep -v ".*-[a-z]|beta" | tail -1)
fi
downloadUrl=$(echo $json | jq -r ".[] | select(.tag_name==\"${tagName}\").assets[].browser_download_url | select(${FILTER})" | head -n 1)
if [ -z "$downloadUrl" ]; then
echo "Failed to parse a download url for the '${tagName}' tag using '${FILTER}' filter"
exit 1
fi
echo $downloadUrl
}
@@ -4,5 +4,5 @@
## Desc: Helper function for invoking tests
################################################################################
pwsh -Command "Import-Module '$HELPER_SCRIPTS/../tests/Helpers.psm1' -DisableNameChecking
pwsh -Command "Import-Module '$HELPER_SCRIPTS/Tests.Helpers.psm1' -DisableNameChecking
Invoke-PesterTests -TestFile \"$1\" -TestName \"$2\""
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash -e
################################################################################
## File: install-helpers.sh
## Desc: Helper functions for installing tools
################################################################################
function isUbuntu20
{
lsb_release -d | grep -q 'Ubuntu 20'
}
function isUbuntu22
{
lsb_release -d | grep -q 'Ubuntu 22'
}
function getOSVersionLabel
{
lsb_release -cs
}
@@ -0,0 +1,95 @@
################################################################################
## File: Configure-Toolset.ps1
## Team: CI-Build
## Desc: Configure toolset
################################################################################
Import-Module "$env:HELPER_SCRIPTS/Tests.Helpers.psm1" -DisableNameChecking
function Get-ToolsetToolFullPath
{
param
(
[Parameter(Mandatory)] [string] $ToolName,
[Parameter(Mandatory)] [string] $ToolVersion,
[Parameter(Mandatory)] [string] $ToolArchitecture
)
$toolPath = Join-Path -Path $env:AGENT_TOOLSDIRECTORY -ChildPath $toolName
$toolPathVersion = Join-Path -Path $toolPath -ChildPath $toolVersion
$foundVersion = Get-Item $toolPathVersion | Sort-Object -Property {[version]$_.name} -Descending | Select-Object -First 1
$installationDir = Join-Path -Path $foundVersion -ChildPath $toolArchitecture
return $installationDir
}
function Add-EnvironmentVariable
{
param
(
[Parameter(Mandatory)] [string] $Name,
[Parameter(Mandatory)] [string] $Value,
[string] $FilePath = "/etc/environment"
)
$envVar = "{0}={1}" -f $name, $value
Tee-Object -InputObject $envVar -FilePath $filePath -Append
}
$ErrorActionPreference = "Stop"
Write-Host "Configure toolset tools environment..."
$toolsEnvironment = @{
go = @{
command = "ln -s {0}/bin/* /usr/bin/"
variableTemplate = "GOROOT_{0}_{1}_X64"
}
}
$toolset = Get-Content -Path "$env:INSTALLER_SCRIPT_FOLDER/toolset.json" -Raw | ConvertFrom-Json
foreach ($tool in $toolset.toolcache)
{
$toolName = $tool.name
$toolArch = $tool.arch
$toolEnvironment = $toolsEnvironment[$toolName]
if (-not $toolEnvironment)
{
continue
}
foreach ($toolVersion in $tool.versions)
{
Write-Host "Set $toolName $toolVersion environment variable..."
$toolPath = Get-ToolsetToolFullPath -ToolName $toolName -ToolVersion $toolVersion -ToolArchitecture $toolArch
$envName = $toolEnvironment.variableTemplate -f $toolVersion.split(".")
# Add environment variable name=value
Add-EnvironmentVariable -Name $envName -Value $toolPath
}
# Invoke command and add env variable for the default tool version
$toolDefVersion = $tool.default
if (-not $toolDefVersion)
{
continue
}
$envDefName = $toolEnvironment.defaultVariable
$toolPath = Get-ToolsetToolFullPath -ToolName $toolName -ToolVersion $toolDefVersion -ToolArchitecture $toolArch
if ($envDefName)
{
Write-Host "Set default $envDefName for $toolName $toolDefVersion environment variable..."
Add-EnvironmentVariable -Name $envDefName -Value $toolPath
}
if ($toolEnvironment.command)
{
$command = $toolEnvironment.command -f $toolPath
Write-Host "Invoke $command command for default $toolName $toolDefVersion..."
Invoke-Expression -Command $command
}
}
Invoke-PesterTests -TestFile "Toolset" -TestName "Toolset"
@@ -0,0 +1,40 @@
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Import-Module "$env:HELPER_SCRIPTS/Tests.Helpers.psm1" -DisableNameChecking
# Get modules content from toolset
$modules = (Get-ToolsetContent).azureModules
$installPSModulePath = "/usr/share"
foreach ($module in $modules)
{
$moduleName = $module.name
Write-Host "Installing ${moduleName} to the ${installPSModulePath} path..."
foreach ($version in $module.versions)
{
$modulePath = Join-Path -Path $installPSModulePath -ChildPath "${moduleName}_${version}"
Write-Host " - $version [$modulePath]"
Save-Module -Path $modulePath -Name $moduleName -RequiredVersion $version -Force
}
$assets = Invoke-RestMethod $module.url
# Get github release asset for each version
foreach ($toolVersion in $module.zip_versions) {
$asset = $assets | Where-Object version -eq $toolVersion `
| Select-Object -ExpandProperty files `
| Select-Object -First 1
Write-Host "Installing $($module.name) $toolVersion ..."
if ($null -ne $asset) {
Write-Host "Download $($asset.filename)"
wget $asset.download_url -nv --retry-connrefused --tries=10 -P $installPSModulePath
} else {
Write-Host "Asset was not found in versions manifest"
exit 1
}
}
}
Invoke-PesterTests -TestFile "PowerShellModules" -TestName "AzureModules"
@@ -1,35 +1,34 @@
################################################################################
## File: Install-PowerShellModules.ps1
## Desc: Install modules for PowerShell
################################################################################
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Import-Module "$env:HELPER_SCRIPTS/../tests/Helpers.psm1"
# Specifies the installation policy
Set-PSRepository -InstallationPolicy Trusted -Name PSGallery
# Try to update PowerShellGet before the actual installation
Install-Module -Name PowerShellGet -Force
Update-Module -Name PowerShellGet -Force
# Install PowerShell modules
$modules = (Get-ToolsetContent).powershellModules
foreach($module in $modules) {
$moduleName = $module.name
Write-Host "Installing ${moduleName} module"
if ($module.versions) {
foreach ($version in $module.versions) {
Write-Host " - $version"
Install-Module -Name $moduleName -RequiredVersion $version -Scope AllUsers -SkipPublisherCheck -Force
}
} else {
Install-Module -Name $moduleName -Scope AllUsers -SkipPublisherCheck -Force
}
}
Invoke-PesterTests -TestFile "PowerShellModules" -TestName "PowerShellModules"
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Import-Module "$env:HELPER_SCRIPTS/Tests.Helpers.psm1" -DisableNameChecking
# Specifies the installation policy
Set-PSRepository -InstallationPolicy Trusted -Name PSGallery
# Try to update PowerShellGet before the actual installation
Install-Module -Name PowerShellGet -Force
Update-Module -Name PowerShellGet -Force
# Install PowerShell modules
$modules = (Get-ToolsetContent).powershellModules
foreach($module in $modules)
{
$moduleName = $module.name
Write-Host "Installing ${moduleName} module"
if ($module.versions)
{
foreach ($version in $module.versions)
{
Write-Host " - $version"
Install-Module -Name $moduleName -RequiredVersion $version -Scope AllUsers -SkipPublisherCheck -Force
}
continue
}
Install-Module -Name $moduleName -Scope AllUsers -SkipPublisherCheck -Force
}
Invoke-PesterTests -TestFile "PowerShellModules" -TestName "PowerShellModules"
@@ -4,21 +4,19 @@
## Desc: Install toolset
################################################################################
Import-Module "$env:HELPER_SCRIPTS/../tests/Helpers.psm1"
function Install-Asset {
Function Install-Asset {
param(
[Parameter(Mandatory = $true)]
[object] $ReleaseAsset
)
Write-Host "Download $($ReleaseAsset.filename)"
$assetArchivePath = Invoke-DownloadWithRetry $ReleaseAsset.download_url
wget $ReleaseAsset.download_url -nv --retry-connrefused --tries=10
Write-Host "Extract $($ReleaseAsset.filename) content..."
$assetFolderPath = Join-Path "/tmp" "$($ReleaseAsset.filename)-temp-dir"
New-Item -ItemType Directory -Path $assetFolderPath | Out-Null
tar -xzf $assetArchivePath -C $assetFolderPath
$assetFolderPath = Join-Path $env:INSTALLER_SCRIPT_FOLDER $($ReleaseAsset.filename)
New-Item -ItemType Directory -Path $assetFolderPath
tar -xzf $ReleaseAsset.filename -C $assetFolderPath
Write-Host "Invoke installation script..."
Push-Location -Path $assetFolderPath
@@ -28,8 +26,10 @@ function Install-Asset {
$ErrorActionPreference = "Stop"
# Get toolcache content from toolset
$tools = (Get-ToolsetContent).toolcache | Where-Object { $_.url -ne $null }
# Get toolset content
$toolset = Get-Content -Path "$env:INSTALLER_SCRIPT_FOLDER/toolset.json" -Raw
$tools = ConvertFrom-Json -InputObject $toolset | Select-Object -ExpandProperty toolcache | Where-Object {$_.url -ne $null }
foreach ($tool in $tools) {
# Get versions manifest for current tool
@@ -38,18 +38,17 @@ foreach ($tool in $tools) {
# Get github release asset for each version
foreach ($toolVersion in $tool.versions) {
$asset = $assets | Where-Object version -like $toolVersion `
| Select-Object -ExpandProperty files `
| Where-Object { ($_.platform -eq $tool.platform) -and ($_.arch -eq $tool.arch) -and ($_.platform_version -eq $tool.platform_version)} `
| Select-Object -First 1
if (-not $asset) {
Write-Host "Asset for $($tool.name) $toolVersion $($tool.arch) not found in versions manifest"
exit 1
}
| Select-Object -ExpandProperty files `
| Where-Object { ($_.platform -eq $tool.platform) -and ($_.platform_version -eq $tool.platform_version)} `
| Select-Object -First 1
Write-Host "Installing $($tool.name) $toolVersion $($tool.arch)..."
Install-Asset -ReleaseAsset $asset
if ($null -ne $asset) {
Install-Asset -ReleaseAsset $asset
} else {
Write-Host "Asset was not found in versions manifest"
exit 1
}
}
chown -R "$($env:SUDO_USER):$($env:SUDO_USER)" "/opt/hostedtoolcache/$($tool.name)"
}
@@ -0,0 +1,24 @@
#!/bin/bash -e
################################################################################
## File: aliyun-cli.sh
## Desc: Installs Alibaba Cloud CLI
################################################################################
# Source the helpers for use with the script
source $HELPER_SCRIPTS/os.sh
source $HELPER_SCRIPTS/install.sh
# Install Alibaba Cloud CLI
# Pin tool version on ubuntu20 due to issues with GLIBC_2.32 not available
if isUbuntu20; then
toolsetVersion=$(get_toolset_value '.aliyunCli.version')
downloadUrl="https://github.com/aliyun/aliyun-cli/releases/download/v$toolsetVersion/aliyun-cli-linux-$toolsetVersion-amd64.tgz"
else
downloadUrl="https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz"
fi
download_with_retries $downloadUrl "/tmp"
tar xzf /tmp/aliyun-cli-linux-*-amd64.tgz
mv aliyun /usr/local/bin
invoke_tests "CLI.Tools" "Aliyun CLI"
+126
View File
@@ -0,0 +1,126 @@
#!/bin/bash -e
################################################################################
## File: android.sh
## Desc: Installs Android SDK
################################################################################
# Source the helpers for use with the script
source $HELPER_SCRIPTS/os.sh
source $HELPER_SCRIPTS/install.sh
source $HELPER_SCRIPTS/etc-environment.sh
function filter_components_by_version {
minimumVersion=$1
shift
toolsArr=("$@")
for item in ${toolsArr[@]}
do
# take the last argument after spliting string by ';'' and '-''
version=$(echo "${item##*[-;]}")
if verlte $minimumVersion $version
then
components+=($item)
fi
done
}
function get_full_ndk_version {
majorVersion=$1
ndkFullVersion=$($SDKMANAGER --list | grep "ndk;${majorVersion}.*" | awk '{gsub("ndk;", ""); print $1}' | sort -V | tail -n1)
echo "$ndkFullVersion"
}
# Set env variable for SDK Root (https://developer.android.com/studio/command-line/variables)
ANDROID_ROOT=/usr/local/lib/android
ANDROID_SDK_ROOT=${ANDROID_ROOT}/sdk
SDKMANAGER=${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin/sdkmanager
echo "ANDROID_SDK_ROOT=${ANDROID_SDK_ROOT}" | tee -a /etc/environment
# ANDROID_HOME is deprecated, but older versions of Gradle rely on it
echo "ANDROID_HOME=${ANDROID_SDK_ROOT}" | tee -a /etc/environment
# Create android sdk directory
mkdir -p ${ANDROID_SDK_ROOT}
cmdlineTools="android-cmdline-tools.zip"
# Download the latest command line tools so that we can accept all of the licenses.
# See https://developer.android.com/studio/#command-tools
cmdlineToolsVersion=$(get_toolset_value '.android."cmdline-tools"')
if [[ $cmdlineToolsVersion == "latest" ]]; then
repositoryXmlUrl="https://dl.google.com/android/repository/repository2-1.xml"
download_with_retries $repositoryXmlUrl "/tmp" "repository2-1.xml"
cmdlineToolsVersion=$(
yq -p=xml \
'.sdk-repository.remotePackage[] | select(."+@path" == "cmdline-tools;latest" and .channelRef."+@ref" == "channel-0").archives.archive[].complete.url | select(contains("commandlinetools-linux"))' \
/tmp/repository2-1.xml
)
if [[ -z $cmdlineToolsVersion ]]; then
echo "Failed to parse latest command-line tools version"
exit 1
fi
fi
download_with_retries "https://dl.google.com/android/repository/${cmdlineToolsVersion}" "." $cmdlineTools
unzip -qq $cmdlineTools -d ${ANDROID_SDK_ROOT}/cmdline-tools
# Command line tools need to be placed in ${ANDROID_SDK_ROOT}/sdk/cmdline-tools/latest to determine SDK root
mv ${ANDROID_SDK_ROOT}/cmdline-tools/cmdline-tools ${ANDROID_SDK_ROOT}/cmdline-tools/latest
rm -f $cmdlineTools
# Check sdk manager installation
${SDKMANAGER} --list 1>/dev/null
if [ $? -eq 0 ]
then
echo "Android SDK manager was installed"
else
echo "Android SDK manager was not installed"
exit 1
fi
minimumBuildToolVersion=$(get_toolset_value '.android.build_tools_min_version')
minimumPlatformVersion=$(get_toolset_value '.android.platform_min_version')
extras=$(get_toolset_value '.android.extra_list[]|"extras;" + .')
addons=$(get_toolset_value '.android.addon_list[]|"add-ons;" + .')
additional=$(get_toolset_value '.android.additional_tools[]')
ANDROID_NDK_MAJOR_VERSIONS=($(get_toolset_value '.android.ndk.versions[]'))
ANDROID_NDK_MAJOR_DEFAULT=$(get_toolset_value '.android.ndk.default')
components=("${extras[@]}" "${addons[@]}" "${additional[@]}")
for ndk_version in "${ANDROID_NDK_MAJOR_VERSIONS[@]}"
do
ndk_full_version=$(get_full_ndk_version $ndk_version)
components+=("ndk;$ndk_full_version")
done
ANDROID_NDK_MAJOR_LATEST=(${ANDROID_NDK_MAJOR_VERSIONS[-1]})
ndkDefaultFullVersion=$(get_full_ndk_version $ANDROID_NDK_MAJOR_DEFAULT)
ndkLatestFullVersion=$(get_full_ndk_version $ANDROID_NDK_MAJOR_LATEST)
ANDROID_NDK="$ANDROID_SDK_ROOT/ndk/$ndkDefaultFullVersion"
# ANDROID_NDK, ANDROID_NDK_HOME, and ANDROID_NDK_ROOT variables should be set as many customer builds depend on them https://github.com/actions/runner-images/issues/5879
echo "ANDROID_NDK=${ANDROID_NDK}" | tee -a /etc/environment
echo "ANDROID_NDK_HOME=${ANDROID_NDK}" | tee -a /etc/environment
echo "ANDROID_NDK_ROOT=${ANDROID_NDK}" | tee -a /etc/environment
echo "ANDROID_NDK_LATEST_HOME=$ANDROID_SDK_ROOT/ndk/$ndkLatestFullVersion" | tee -a /etc/environment
availablePlatforms=($($SDKMANAGER --list | sed -n '/Available Packages:/,/^$/p' | grep "platforms;android-[0-9]" | cut -d"|" -f 1))
allBuildTools=($($SDKMANAGER --list | grep "build-tools;" | cut -d"|" -f 1 | sort -u))
availableBuildTools=$(echo ${allBuildTools[@]//*rc[0-9]/})
filter_components_by_version $minimumPlatformVersion "${availablePlatforms[@]}"
filter_components_by_version $minimumBuildToolVersion "${availableBuildTools[@]}"
echo "y" | $SDKMANAGER ${components[@]}
# Old skdmanager from sdk tools doesn't work with Java > 8, set version 8 explicitly
if isUbuntu20 || isUbuntu22; then
sed -i "2i export JAVA_HOME=${JAVA_HOME_8_X64}" ${ANDROID_SDK_ROOT}/tools/bin/sdkmanager
fi
# Add required permissions
chmod -R a+rwx ${ANDROID_SDK_ROOT}
reloadEtcEnvironment
invoke_tests "Android"

Some files were not shown because too many files have changed in this diff Show More