Commit Graph

36 Commits

Author SHA1 Message Date
Ferenc Hammerl 591f8c3510 Runner container hooks Beta (#1853)
* Added ability to run Dockerfile.SUFFIX ContainerAction

* Extracted IsDockerFile method

* reformatted, moved from index to Last()

* extracted IsDockerfile to DockerUtil with L0

* added check for IsDockerfile to account for docker://

* updated test to clearly show path/dockerfile:tag

* fail if Data.Image is not Dockerfile or docker://[image]

* Setup noops for JobPrepare and JobCleanup hooks

* Add container jobstarted and jobcomplete hooks

* Run 'index.js' instead of specific command hooks

* Call jobprepare with command arg

* Use right command name (hardcoded)

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Invoke hooks with arguments

* Add PrepareJob hook to work with jobcontainers

Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>

* Rename methods

* Use new hookcontainer to run prep and clean hooks

* Get path from ENV

* Use enums

* Use IOUtils.cs

* Move container files to folder

* Move namespaces

* Store "state" between hooks

* Remove stdin stream in containerstephosts

* Update Constants.cs

* Throw if stdin fails

* Cleanup obvious nullrefs and unused vars

* Cleanup containerhook directory

* Call step exec hook

* Fix windows build

* Remove hook from hookContainer

* Rename file

* More renamings

* Add TODOs

* Fix env name

* Fix missing imports

* Fix imports

* Run script step on jobcontainer

* Enable feature if env is set

* Update ContainerHookManager.cs

* Update ContainerHookManager.cs

* Hooks allowed to work even when context isn't returned

* Custom hooks enabled flag and additional null checks

* New line at the end of the FeatureFlagManager.cs

* Code refactoring

* Supported just in time container building or pulling

* Try mock-build for osx

* Build all platforms

* Run mock on self-hosted

* Remove GITHUB prefix

* Use ContainerHooksPath instead of CustomHooksPath

* Null checks simplified

* Code refactoring

* Changing condition for image builing/pulling

* Code refactoring

* TODO comment removed

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Call container step if FF is on

* Rename run script function

* Use JToken instead of dynamic

* Add TODO

* Small refactoring + renames + TODOs

* Throw on DetermineNodeRuntimeVersion

* Fix formatting

* Add run-container-step

* Supported nodeJS in Alpine containers

* Renamed Alpine to IsAlpine in HookResponse

* Method for checking platform for alpine container

* Added container hooks feature flag check

* Update IsHookFeatureEnabled with new params

* Rename featureflag method

* Finish rename

* Set collection null values to empty arrays when JSON serialising them

* Disable FF until we merge

* Update src/Runner.Worker/Container/ContainerHooks/HookContainer.cs

* Fix method name

* Change hookargs to superclass from interface

* Using only Path.Combine in GenerateResponsePath

* fix merge error

* EntryPointArgs changed to list of args instead of one args string

* Changed List to IEnumerable for EntryPointArgs and MountVolumes

* Get ContainerRuntimePath for JobContainers from hooks

* Read ContainerEnv from response file

* Port mappings saved after creating services

* Support case when responseFile doesn't exist

* Check if response file exists

* Logging in ExecuteHookScript

* Save hook state after all 4 hooks

* Code refactoring

* Remove TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Remove second TODO

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Removing container env changes

* Removing containerEnv and dockerManager

* Delete mock-build.yml

* Update IOUtil.cs

* Add comment about containerhooks

* Fix merge mistake

* Remove solved todo

* Determine which shell to use for hooks scenario

* Overload for method ExecuteHookScript with prependPath as arg

* Adding HostContext to the GetDefaultShellForScript call

* prependPath as a mandatory parameter

* Improve logging for hooks

* Small changes in logging

* Allow null for ContainerEntryPointArgs

* Changed log messages

* Skip setting EntryPoint and EntryPointArgs if hooks are enabled

* Throw if IsAlpine is null in PrepareJob

* Code refactoring - added GetAndValidateResponse method

* Code refactoring

* Changes in exception message

* Only save hookState if returned

* Use FF from server

* Empty line

* Code refactoring

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Send null instead of string empty

* Remove TODO

* Code refactoring and some small changes

* Allow Globals to be null to pass L0

* Fix setup in StepHostL0

* Throw exception earlier if response file doesn't exist and prepare_job hook is running

* Refactoring GetResponse method

* Changing exception message if response file is not found

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>

* Chaning exception message if isAlpine is null for prepare_job hook

* Rename hook folder

* Fail if compatible hookfile not found

* Use .Value instead of casting bool? to bool

* Format spacing

* Formatting

* User user and system mvs

* Use variables instead of entire context in featuremanager

* Update stepTelemetry if step uses containerhooks

* Restore  import

* Remove unneccessary field from HookContainer

* Refactor response context and portmappings

* Force allow hooks if FF is on

* Code refactoring

* Revert deleting usings

* Better hookContainer defaults and use correct portmapping list

* Make GetDefaultShellForScript a  HostContext extension method

* Generic hookresponse

* Code refactoring, unnecessary properties removed - HookContainer moved to the HookInput.cs

* Remove empty line

* Code refactoring and better exception handling

* code refactor, removing unnecessary props

* Move hookstate to global ContainerHookState

* Trace exception before we throw it for not losing information

* Fix for null ref exception in GetResponse

* Adding additional check for null response in prepareJob hook

* Refactoring GetResponse with additional check

* Update error messages

* Ports in ResponseContainer changed from IList to IDictionary

* Fix port format

* Include dockerfile

* Send null Registry obj if there's nothing in it

* Minor formatting

* Check if hookIndexPath exists relocated to the ContainerHookManager

* Code refactoring - ValidateHookExecutable added to the ContainerHookManager

* check if ContainerHooksPath when AllowRunnerContainerHooks is on

* Submit JSON telemetry instead of boolean

* Prefix step hooks with "run"

* Rename FeatureManager

* Fix flipped condition

* Unify js shell path getter with ps1 and sh getter

* Validate on run, not on instantiation of manager

* Cleanup ExecuteAsync methods

* Handle exception in executeHookScript

* Better exception types

* Remove comment

* Simplify boolean

* Allow jobs without jobContainer to run

* Use JObject instead of JToken

* Use correct Response type

* Format class to move cleanupJobAsync to the end of public methods

* Rename HookIndexPath to HookScriptPath

* Refactor methods into expression bodies

* Fix args class hierarchy

* Fix argument order

* Formatting

* Fix nullref and don't swallow stacktrace

* Whilelist HookArgs

* Use FF in FeatureManager

* Update src/Runner.Worker/ContainerOperationProvider.cs

Co-authored-by: Tingluo Huang <tingluohuang@github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/ActionRunner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Only mount well known dirs to job containers

* Get trace from hostcontext

* Use hook execution for setting telemetry

Co-authored-by: Nikola Jokic <nikola.jokic@akvelon.com>
Co-authored-by: Nikola Jokic <nikola-jokic@users.noreply.github.com>
Co-authored-by: Nikola Jokic <97525037+nikola-jokic@users.noreply.github.com>
Co-authored-by: Stefan Ruvceski <stefan.ruvceski@akvelon.com>
Co-authored-by: ruvceskistefan <96768603+ruvceskistefan@users.noreply.github.com>
Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: stefanruvceski <ruvceskistefan@github.com>
Co-authored-by: Tingluo Huang <tingluohuang@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-10 13:51:20 +00:00
Ferenc Hammerl 9623a44c2f Allow admins to fail jobs without container (#1895)
* Allow admins to fail jobs without container

* Make method static

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/JobExtension.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Worker/JobExtension.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Rename env

* Add test for throwing when no container but required

* Update src/Runner.Worker/JobExtension.cs

* Update src/Test/L0/Worker/JobExtensionL0.cs

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-06-09 17:17:11 -04:00
ruvceskistefan ead3509d5a Added warning in case of invalid combination of command and flags and/or arguments (#1781)
* Added warning in case of invalid combination of command and flags and/or arguments

* Deleting unnecessary comments

* Added separate list for generic options

* Added PAT to the valid remove options

* Added command name to the error message
2022-04-11 09:23:58 -04:00
Nikola Jokic c8cb600ac7 Use StepHost when evaluating inputs to actions (#1762)
* composite action github.action_path set based on the StepHost

* in progress on updating github context for input template

* Fixed updating the context data for evaluation

* refactored logic so it is a little cleaner

* removed resolving the action_path in CompositeActionHandler

* removed added DeepClone

* added feature flag and modified the dict in place

* refactored step host to change context data. Added L0

* repaired spaces

* moved logic from step host to execution context, added recursive translation

* removed empty lines

* moved to extension methods

* Update src/Test/L0/Worker/StepHostL0.cs

Co-authored-by: Ferenc Hammerl <31069338+fhammerl@users.noreply.github.com>
2022-04-11 12:43:24 +00:00
Ferenc Hammerl 408d6c579c Add annotations if Node 12 action is found and FF is on (#1735)
* Add annotations if node 12 action is found

* Better placeholder

* Only warn if FF is on

* Move annotation logic

* Pass in the LTS Url

* Raise annotation right before executing the action

* Match server side FF name

* Change name back to features

* Better warning text

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-03-22 10:43:25 +01:00
Ferenc Hammerl 7d5e9cd70f Runner Job Started/Completed Hooks (#1737)
* Prototype for pre job hook

* Remove debug log

* Enable hooks again

* Initialize with hostContext

* Add event_path, fix no-path bug

* Allow script post steps

* Call script handler with correct pre post stage

* Add job completed hook

* Make filecommand work and hardcode shell

* Conditionally print step details and no telemetry for hooks

* Figure out whih script to use

* Only check path for managed scripts

* Resture win dependency

* Nits

* Remove unused, add named params

* Telemetry + refactoring

* add message to job

* rename hooks remove stale comment

* cleanup

* Use .CreateService to create step

* Add L0s

* pr feedback

* update tests

* add disclaimer, clean up code

* spacing fix

* little more cleanup

* pr fix

* pr feedback

* Refactor to use JobExtension

* fix tests

* fix typo

* cleanup code

* more cleanup

* little more cleanup

* last bit of cleanup

* fix tests

* nit fix

* Update src/Runner.Worker/JobHookProvider.cs

Co-authored-by: Edward Thomson <ethomson@github.com>

* don't override runner telemtry

* pr feedback

* pr feedback

* pr feedback

Co-authored-by: Thomas Boop <thboop@github.com>
Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
Co-authored-by: Edward Thomson <ethomson@github.com>
2022-03-17 21:35:04 -04:00
Ferenc Hammerl 100c99f263 Force JS Actions Node version to 16 if FF is on unless user opted out (#1716)
* Set GH actions Node version to 16 if FF is on unless user opted out

* Add L0s (WIP)

* Wrap tests into theory

* Only check for node12 actions

* Refactor node version picking
2022-02-25 14:59:16 -05:00
Ferenc Hammerl e8ccafea63 Add internal to node version function and use better env var name (#1715) 2022-02-25 14:59:02 -05:00
Rob Herley 0cbf3351f4 Update summary max file size annotation error text (#1712)
* add doc link to summary file size err annotation

* Update src/Runner.Common/Constants.cs

Co-authored-by: Konrad Pabjan <konradpabjan@github.com>

* remove i18n from url

Co-authored-by: Konrad Pabjan <konradpabjan@github.com>
2022-02-24 21:01:49 -05:00
Ferenc Hammerl bd77ccf34e Prefer node16 over node12 when running internal scripts (#1621)
* Use 16 to run RunnerService.js

* Execute hashfiles using node16

* Run downloadCert.js using node16

* Run makeWebRequest.js using node16

* Run macos-run-invoker.js using node16

* Run hashFiles.js using node16

* Update tests to use node16

* Update documentation to recommend node16

* Duplicate macos service js fix for 16

* Add PR link

* Revert ADR node change

* Merge node12/16 retainment IFs

* Try both node12 and node16

* Close if

* Revert "Update tests to use node16"

This reverts commit bbca7b9f1c.

* Fix condition

* Unfurl if condition

* Allow user to force a node version

* Format update template

* Comment env var

* Rename vars

* Fix naming

* Fix rename

* Set node ver override if job message has it

* Format executionContext

* Can only receive 'forceNode12' or nothing from FF
No specific node version from server

Co-authored-by: Ferenc Hammerl <hammerl.ferenc@gmail.com>
2022-02-14 15:06:08 +01:00
Tingluo Huang 8eacbdc79f Runner config option to disable auto-update. (#1558)
* Runner config option to disable auto-update.

* Update src/Runner.Listener/Configuration/ConfigurationManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Listener/Configuration/ConfigurationManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Listener/Configuration/ConfigurationManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Update src/Runner.Listener/Configuration/ConfigurationManager.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* feedback.

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2022-01-26 13:23:24 -05:00
Thomas Boop 04269f7b1b Handle keeping previous OSX versions more smoothly on Mac (#1381)
* Handle macOS upgrade smoothly

* cleanup

* misc cleanup

* final updates

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Update src/Misc/layoutbin/update.sh.template

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>

* Upload telemetry and default to old method as needed

* minor fix

* add one more bit of logging

* some more telemetry

* quote variables to handle spaces

* tiny fix for ubuntu

* remove version and move telemetry to diag

* use full path

Co-authored-by: Patrick Ellis <319655+pje@users.noreply.github.com>
2021-09-29 15:49:31 -04:00
Ferenc Hammerl e89d2e84bd Stop-Commands: stopToken restrictions (#1371)
* Prevent stopTokens that are workflow commands

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Check context for env var too

* Accept true, 1 and $true instead of just "true"

* Setup ExpressionValues in tests

* Update src/Runner.Common/Constants.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

* Separate success and fail tests for invalid token

* Fix envcontext for tests

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-29 14:44:01 -04:00
Tingluo Huang 5afb52b272 Update the comment about the --once in Constants.cs (#1360)
* Update Constants.cs

* feedback.

* Update src/Runner.Listener/Runner.cs

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>

Co-authored-by: Thomas Boop <52323235+thboop@users.noreply.github.com>
2021-09-21 21:31:48 +00:00
Tingluo Huang 7ffd9af644 Support --ephemeral flag (#660)
This optional flag will configure the runner to only take one job, and let the service un-configure the runner after that job finishes.
2021-09-13 11:28:09 -04:00
Thomas Boop 7c90b2a929 Composite Actions (#1170)
Composite Actions Refactoring
2021-07-01 13:34:28 -04:00
Tingluo Huang 7ee333b5cd Adding --check to run a serials network test against GitHub or GHES. (#900)
* add --check.
2021-01-14 13:26:07 -05:00
Tingluo Huang e808190dd2 add warning when running out of disk. (#873) 2021-01-05 21:49:11 -05:00
Tingluo Huang 81a00fff3e config runner via PAT. (#874) 2020-12-25 21:54:23 -05:00
Thomas Boop a2e32170fd Disable set-env and add-pathcommands (#779)
* Disable Old Runner Commands set-env and add-path

* update dotnet install scripts

* update runner version and release notes
2020-11-16 08:20:43 -05:00
Thomas Boop 35dda19491 Add deprecation date and release 2.274.1 version (#796) 2020-11-09 09:01:47 -05:00
Thomas Boop 6332a52d76 Notify on unsecure commands (#731)
* notify on unsecure commands
2020-10-02 11:34:37 -04:00
Christopher Johnson 48ac96307c Add ability to register a runner to the non-default self-hosted runner group (#613)
Co-authored-by: Christopher Johnson <thchrisjohnson@github.com>
2020-07-23 17:46:48 -04:00
Tingluo Huang 70729fb3c4 Help trace worker crash in Kusto. (#450)
* Help trace worker crash in Kusto.

* more

* feedback.
2020-04-27 23:44:17 -04:00
Lokesh Gopu f9baec4b32 Added support for custom labels (#414)
* Added support for custom labels

* ignore case

* Added interactive config for labels

* Fixing L0s

* pr comments
2020-04-13 21:33:13 -04:00
Tingluo Huang f9b5d626c5 load and print machine setup info from .setup_info (#364) 2020-03-11 10:36:56 -04:00
Tingluo Huang 6bec1e3bb8 Switch to use token service instead of SPS for exchanging oauth token. (#325)
* Gracefully switch the runner to use Token Service instead of SPS.

* PR feedback.

* feedback2

* report error.
2020-03-04 21:40:58 -05:00
Josh Gross 94e7560ccd Add event type to credential call (#352)
* Add event type to credential call

* Move events to contants
2020-03-02 11:22:45 -05:00
Alberto Gimeno c59c0e2ded Support action.yaml file (#288)
* Support action.yaml file

* L0 tests.

* l0

Co-authored-by: Tingluo Huang <tingluohuang@github.com>
2020-01-20 12:22:59 -05:00
Julio Barba 50d979f1b2 Bring back tools folder fallback code (#232) 2019-12-17 18:21:13 -05:00
Tingluo Huang d0a4a41a63 delete un-used code. (#218) 2019-12-16 17:05:26 -05:00
Julio Barba a3c6a8c201 Introduce name config argument (#217) 2019-12-11 13:26:06 -05:00
Julio Barba 275ab753a1 Runner cleanup - continuation (#209)
* Agent/AgentCredential -> Runner/RunnerCredential
* Test trait rename: Agent -> Runner
* Enable remaining RunnerL0 tests
* Remove job message PII variable masking code
* Remove unused Agent.ToolsDirectory variable
* Misc test Agent -> Runner renaming
* Some more misc cleaning
2019-12-09 17:54:41 -05:00
Tingluo Huang d81a7656a4 Add Proxy Support for self-hosted runner. (#206) 2019-12-09 15:15:54 -05:00
Julio Barba cd2cec8282 Another runner code cleanup round (#197)
* Remove remaining non-SDK references of capabilities/demands
* Remove unused Runner.Common constants
* Remove more variables
* Clean up RU link, and named-pipe support
* Remove NotificationSocketAddress
* Re-add legacy OnPremises JobDispatcher code (commented out)
* More misc cleanup
2019-12-04 10:18:37 -05:00
Tingluo Huang c8afc84840 GitHub Actions Runner 2019-10-10 00:52:42 -04:00