* Add 'http://' to http(s)_proxy if there is no protocol
Before this commit, these URIs used to be ignored (nullproxy).
The new behaviour aligns with go style http(s)_proxy
* Update src/Runner.Sdk/RunnerWebProxy.cs
Co-authored-by: Nikola Jokic <[email protected]>
* Assert that original protocol is preserved
* Use startsWith for testing protocol
* Only modify proxy if useful
---------
Co-authored-by: Nikola Jokic <[email protected]>
* Bypass top level domain even if no_proxy specified it with leading '.'
E.g. no_proxy='.github.com' will now bypass github.com.
* Bypass proxy on all hosts when no_proxy is * (wildcard)
* Undo '.' stripping
* Simplify unit tests
* Respect wildcard even if it's one of many no_proxy items
* Sanitize OS Desc for UserAgents
* Only drop brackets if needed, refactoring
* Add missing ')'
* Readd missing brackets around '(header)'
* Add comments
* Use bracket solution from SDK
* Rename tests
* docker: escape key-value pair as -e KEY and VALUE being environment var
* removed code duplication, removed unused method and test
* add release notes
Co-authored-by: Nikola Jokic <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
Co-authored-by: Nikola Jokic <[email protected]>
* update.sh new logic with creating a file and waiting for it for 30 sec
* pr refactor
* removing file when it's detected bu run-helper
* Update src/Misc/layoutroot/run-helper.sh.template
Co-authored-by: Ferenc Hammerl <[email protected]>
* clean up update.finished in Runner.Listener right before it starts a process _update.sh
* pr fix IOUtil.DeleteFile
* self update widnows version - cmd
* same logic for returnCode 4 - ephemeral
* Init var so cmd doesn't run into syntax errors
* Use constants, setting a var messed up ERRORLEVEL
* Use var for updatefile
Co-authored-by: stefanruvceski <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
Co-authored-by: Ferenc Hammerl <[email protected]>
* Added ability to run Dockerfile.SUFFIX ContainerAction
* Extracted IsDockerFile method
* reformatted, moved from index to Last()
* extracted IsDockerfile to DockerUtil with L0
* added check for IsDockerfile to account for docker://
* updated test to clearly show path/dockerfile:tag
* fail if Data.Image is not Dockerfile or docker://[image]
* Setup noops for JobPrepare and JobCleanup hooks
* Add container jobstarted and jobcomplete hooks
* Run 'index.js' instead of specific command hooks
* Call jobprepare with command arg
* Use right command name (hardcoded)
Co-authored-by: Nikola Jokic <[email protected]>
* Invoke hooks with arguments
* Add PrepareJob hook to work with jobcontainers
Co-authored-by: Nikola Jokic <[email protected]>
* Rename methods
* Use new hookcontainer to run prep and clean hooks
* Get path from ENV
* Use enums
* Use IOUtils.cs
* Move container files to folder
* Move namespaces
* Store "state" between hooks
* Remove stdin stream in containerstephosts
* Update Constants.cs
* Throw if stdin fails
* Cleanup obvious nullrefs and unused vars
* Cleanup containerhook directory
* Call step exec hook
* Fix windows build
* Remove hook from hookContainer
* Rename file
* More renamings
* Add TODOs
* Fix env name
* Fix missing imports
* Fix imports
* Run script step on jobcontainer
* Enable feature if env is set
* Update ContainerHookManager.cs
* Update ContainerHookManager.cs
* Hooks allowed to work even when context isn't returned
* Custom hooks enabled flag and additional null checks
* New line at the end of the FeatureFlagManager.cs
* Code refactoring
* Supported just in time container building or pulling
* Try mock-build for osx
* Build all platforms
* Run mock on self-hosted
* Remove GITHUB prefix
* Use ContainerHooksPath instead of CustomHooksPath
* Null checks simplified
* Code refactoring
* Changing condition for image builing/pulling
* Code refactoring
* TODO comment removed
Co-authored-by: Ferenc Hammerl <[email protected]>
* Call container step if FF is on
* Rename run script function
* Use JToken instead of dynamic
* Add TODO
* Small refactoring + renames + TODOs
* Throw on DetermineNodeRuntimeVersion
* Fix formatting
* Add run-container-step
* Supported nodeJS in Alpine containers
* Renamed Alpine to IsAlpine in HookResponse
* Method for checking platform for alpine container
* Added container hooks feature flag check
* Update IsHookFeatureEnabled with new params
* Rename featureflag method
* Finish rename
* Set collection null values to empty arrays when JSON serialising them
* Disable FF until we merge
* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs
* Fix method name
* Change hookargs to superclass from interface
* Using only Path.Combine in GenerateResponsePath
* fix merge error
* EntryPointArgs changed to list of args instead of one args string
* Changed List to IEnumerable for EntryPointArgs and MountVolumes
* Get ContainerRuntimePath for JobContainers from hooks
* Read ContainerEnv from response file
* Port mappings saved after creating services
* Support case when responseFile doesn't exist
* Check if response file exists
* Logging in ExecuteHookScript
* Save hook state after all 4 hooks
* Code refactoring
* Remove TODO
Co-authored-by: Ferenc Hammerl <[email protected]>
* Remove second TODO
Co-authored-by: Ferenc Hammerl <[email protected]>
* Removing container env changes
* Removing containerEnv and dockerManager
* Delete mock-build.yml
* Update IOUtil.cs
* Add comment about containerhooks
* Fix merge mistake
* Remove solved todo
* Determine which shell to use for hooks scenario
* Overload for method ExecuteHookScript with prependPath as arg
* Adding HostContext to the GetDefaultShellForScript call
* prependPath as a mandatory parameter
* Improve logging for hooks
* Small changes in logging
* Allow null for ContainerEntryPointArgs
* Changed log messages
* Skip setting EntryPoint and EntryPointArgs if hooks are enabled
* Throw if IsAlpine is null in PrepareJob
* Code refactoring - added GetAndValidateResponse method
* Code refactoring
* Changes in exception message
* Only save hookState if returned
* Use FF from server
* Empty line
* Code refactoring
Co-authored-by: Ferenc Hammerl <[email protected]>
* Send null instead of string empty
* Remove TODO
* Code refactoring and some small changes
* Allow Globals to be null to pass L0
* Fix setup in StepHostL0
* Throw exception earlier if response file doesn't exist and prepare_job hook is running
* Refactoring GetResponse method
* Changing exception message if response file is not found
Co-authored-by: Ferenc Hammerl <[email protected]>
* Chaning exception message if isAlpine is null for prepare_job hook
* Rename hook folder
* Fail if compatible hookfile not found
* Use .Value instead of casting bool? to bool
* Format spacing
* Formatting
* User user and system mvs
* Use variables instead of entire context in featuremanager
* Update stepTelemetry if step uses containerhooks
* Restore import
* Remove unneccessary field from HookContainer
* Refactor response context and portmappings
* Force allow hooks if FF is on
* Code refactoring
* Revert deleting usings
* Better hookContainer defaults and use correct portmapping list
* Make GetDefaultShellForScript a HostContext extension method
* Generic hookresponse
* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs
* Remove empty line
* Code refactoring and better exception handling
* code refactor, removing unnecessary props
* Move hookstate to global ContainerHookState
* Trace exception before we throw it for not losing information
* Fix for null ref exception in GetResponse
* Adding additional check for null response in prepareJob hook
* Refactoring GetResponse with additional check
* Update error messages
* Ports in ResponseContainer changed from IList to IDictionary
* Fix port format
* Include dockerfile
* Send null Registry obj if there's nothing in it
* Minor formatting
* Check if hookIndexPath exists relocated to the ContainerHookManager
* Code refactoring - ValidateHookExecutable added to the ContainerHookManager
* check if ContainerHooksPath when AllowRunnerContainerHooks is on
* Submit JSON telemetry instead of boolean
* Prefix step hooks with "run"
* Rename FeatureManager
* Fix flipped condition
* Unify js shell path getter with ps1 and sh getter
* Validate on run, not on instantiation of manager
* Cleanup ExecuteAsync methods
* Handle exception in executeHookScript
* Better exception types
* Remove comment
* Simplify boolean
* Allow jobs without jobContainer to run
* Use JObject instead of JToken
* Use correct Response type
* Format class to move cleanupJobAsync to the end of public methods
* Rename HookIndexPath to HookScriptPath
* Refactor methods into expression bodies
* Fix args class hierarchy
* Fix argument order
* Formatting
* Fix nullref and don't swallow stacktrace
* Whilelist HookArgs
* Use FF in FeatureManager
* Update src/Runner.Worker/ContainerOperationProvider.cs
Co-authored-by: Tingluo Huang <[email protected]>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <[email protected]>
* Update src/Runner.Worker/ActionRunner.cs
Co-authored-by: Thomas Boop <[email protected]>
* Only mount well known dirs to job containers
* Get trace from hostcontext
* Use hook execution for setting telemetry
Co-authored-by: Nikola Jokic <[email protected]>
Co-authored-by: Nikola Jokic <[email protected]>
Co-authored-by: Nikola Jokic <[email protected]>
Co-authored-by: Stefan Ruvceski <[email protected]>
Co-authored-by: ruvceskistefan <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
Co-authored-by: stefanruvceski <[email protected]>
Co-authored-by: Tingluo Huang <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
* Allow admins to fail jobs without container
* Make method static
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <[email protected]>
* Update src/Runner.Worker/JobExtension.cs
Co-authored-by: Thomas Boop <[email protected]>
* Update src/Runner.Worker/JobExtension.cs
Co-authored-by: Thomas Boop <[email protected]>
* Rename env
* Add test for throwing when no container but required
* Update src/Runner.Worker/JobExtension.cs
* Update src/Test/L0/Worker/JobExtensionL0.cs
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
It generated invalid arguments for `Process()` when the `bash` command itself was an argument as well, for example:
```
_proc.StartInfo.FileName = "/usr/bin/docker";
_proc.StartInfo.Arguments = "exec -i --workdir /__w/container-hook-e2e/container-hook-e2e 47105c66144d8809d9fa2bce9a58ea0564cd14def0ae7952cd6231fba3576db1 sh -e '/__w/_temp/fd086560-cb92-4f3b-a99c-35a6b7b1bbdb.sh'";
```
* Add annotations if node 12 action is found
* Better placeholder
* Only warn if FF is on
* Move annotation logic
* Pass in the LTS Url
* Raise annotation right before executing the action
* Match server side FF name
* Change name back to features
* Better warning text
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <[email protected]>
Co-authored-by: Thomas Boop <[email protected]>
* Set GH actions Node version to 16 if FF is on unless user opted out
* Add L0s (WIP)
* Wrap tests into theory
* Only check for node12 actions
* Refactor node version picking
* Use 16 to run RunnerService.js
* Execute hashfiles using node16
* Run downloadCert.js using node16
* Run makeWebRequest.js using node16
* Run macos-run-invoker.js using node16
* Run hashFiles.js using node16
* Update tests to use node16
* Update documentation to recommend node16
* Duplicate macos service js fix for 16
* Add PR link
* Revert ADR node change
* Merge node12/16 retainment IFs
* Try both node12 and node16
* Close if
* Revert "Update tests to use node16"
This reverts commit bbca7b9f1c.
* Fix condition
* Unfurl if condition
* Allow user to force a node version
* Format update template
* Comment env var
* Rename vars
* Fix naming
* Fix rename
* Set node ver override if job message has it
* Format executionContext
* Can only receive 'forceNode12' or nothing from FF
No specific node version from server
Co-authored-by: Ferenc Hammerl <[email protected]>
* Kill node process to recover handle
So we can print to the console in Runner.Listener once again
* Revert testing changes
Co-authored-by: Ferenc Hammerl <[email protected]>
* Only execute post for actions that have one
* Working container runner update with run.sh
* Revert "Only execute post for actions that have one"
This reverts commit 9675941fdb.
* Relaunch the listener without quitting run.cmd
* Fix typo
* Extract most os run.sh logic so we can update it
* Add bash line endings
* Extract the logic from run.cmd
* Add EoF lines
* Add unexpected ERRORLEVEL messages to cmd
* Simplify contract between run and helper
* Remove unused exit
* WIP: run a copy of the helper so it's safe to update
* Throw NonRetryableException if not configured
* Log and format
* Fix typo
* Fix typo
* Use helper template system for bash as well
* Update run.sh
* Remove unnecessary comments
* Use ping instead of timeout
* Use localhost in ping-timeout (n times, w timeout)
Co-authored-by: Ferenc Hammerl <[email protected]>
* Only execute post for actions that have one
* Working container runner update with run.sh
* Revert "Only execute post for actions that have one"
This reverts commit 9675941fdb.
* Relaunch the listener without quitting run.cmd
* Fix typo
* Extract most os run.sh logic so we can update it
* Add bash line endings
* Extract the logic from run.cmd
* Add EoF lines
* Add unexpected ERRORLEVEL messages to cmd
* Simplify contract between run and helper
* Remove unused exit
* WIP: run a copy of the helper so it's safe to update
* Throw NonRetryableException if not configured
* Log and format
* Fix typo
* Fix typo
* Use helper template system for bash as well
* Update run.sh
* Remove unnecessary comments
* Use ping instead of timeout
* Use localhost in ping-timeout (n times, w timeout)
Co-authored-by: Ferenc Hammerl <[email protected]>
* conditional support for composite actions
* Fix Conditional function evaluation
* Push launch.json temporarily
* Revert "Push launch.json temporarily"
* rename context
* Cleanup comments
* fix success/failure functions to run based on pre/main steps
* idea of step_status
* change to use steps context, WIP
* add inputs to possible if condition expressions
* use action_status
* pr cleanup
* Added right stages
* Test on stage in conditional functions
* Fix naming and formatting
* Fix tests
* Add success and failure L0s
* Remove comment
* Remove whitespace
* Undo formatting
* Add L0 for step-if parsing
* Add ADR
Co-authored-by: Thomas Boop <[email protected]>
* Prevent stopTokens that are workflow commands
Co-authored-by: Thomas Boop <[email protected]>
* Check context for env var too
* Accept true, 1 and $true instead of just "true"
* Setup ExpressionValues in tests
* Update src/Runner.Common/Constants.cs
Co-authored-by: Thomas Boop <[email protected]>
* Separate success and fail tests for invalid token
* Fix envcontext for tests
Co-authored-by: Thomas Boop <[email protected]>
* Ignore agentpool when unconfiguring the runner
Runner names and IDs are unique within a ServiceHost
They don't need to be included when unconfiguring the runner.
* Use -1 instead of 0 to highlight how it is ignored
* Use overloads and 0 instead of -1
Using 0 seems to be the convention
* Fix typo calling the wrong method
* Add generateIdTokenUrl as an env var
* Add generateIdTokenUrl to env vars
* Add basic telemetry class and submit it on jobcompleted
* Use constructor overload
* Rename telemetry to jobTelemetry
* Rename telemetry file
* Make JobTelemetryType a string
* Collect telemetry
* Remove debugger
* Update src/Runner.Worker/ActionCommandManager.cs
Co-authored-by: Thomas Boop <[email protected]>
* Use same JobTelemetry for all contexts
* Mask telemetry data
* Mask in JobRunner instead
* Empty line
* Change method signature
Returning with a List suggests we clone it and that the
original doesn't change..
* Update launch.json
Co-authored-by: Thomas Boop <[email protected]>
* Add generateIdTokenUrl as an env var
* Add generateIdTokenUrl to env vars
* Add name and runner_group to context
* No longer add runner-group
* Update runner name if needed
* Get interface instead of concrete class
* Check for nulls on ReservedAgent
* Avoid loading setting file unnecesseraly
* Only check agentName once
* Use Trace.Error when can't update settings
* Better equals and exception handling
* Update JobDispatcher.cs
* Add tests and null check
* Trim pwsh special chars when masking secrets
* Add pwsh valueEncoder
* Explain regex
* Update ValueEncoders.cs
* Add tests for pwsh color codes in secrets
* Formatting
* Group tests into theories
* Split secret on PS chars and mask for them
* Clean up comments
* Remove unused unittest
* Rename escape methods
* Use flags in svc creation script
* Refactor regex and add comments
* Fix indentation and typo in user matching
* Consistency use flags in automation scripts
* Update documentation to reflect new usage
* Make example more readable
* Remove test echos from script
* Remove test echo
* Format scripts and remove test script
* Remove tar
* Use getopts and single letter flags
* Update docs to show flag usage
* Update usage of create svc
* Revert svc to not use flags
* Revert delete script
* Update docs
* Readd deleted comments
* Stop ignoring .vscode (launch scripts)
* Check in launch scripts for config and run
This can cause an issue with existing launch configuration on the machines of contributors.
* Improve error msg when runner is not configured
* Unignore .vscode/launch and tasks only
* Remove stopAtEntry and add eof newline
* Remove Runner.Listener from error message
* Rename tasks and run configs
* Ignore BuildConstants.cs
* Use better error msg
* Explain development steps in depth
* Add launch config to directly debug worker
* Update docs with VS Code tips
* Remove auto-generated comments
* Fix link to quickstart in vscode.md
* Remove ':' from link to quickstart
* Revert "Ignore BuildConstants.cs"
This reverts commit 0f13922a87.
* Replace `.sh` with `.(sh/cmd)` in docs
* Wait for debugger to attach in Worker
Only if GITHUB_ACTIONS_RUNNER_ATTACH_DEBUGGER is set
* Only wait if env variable parses to boolean 'true'
* Add 30s timeout to the wait
* Clean up leftover line
* Decrease wait to 20s
* Use ConvertToBoolean isntead of TryParse
Ferenc Hammerl