Repository navigation
images: curated and full EC2 toolsets, from GitHub's runner images - #151
Merged
Merged
Conversation
The EC2 image had no compiler, make or unzip, so changing runs-on was not enough for a workflow written for ubuntu-latest. build-ec2-ami.sh --toolset runs the installers of a pinned actions/runner-images release, in the order its Packer template lists them. A full image took 44 GiB, and four public workloads read 3.5% of it. curated keeps the latest version of most tools and the defaults that system paths link to, in 18 GiB, and those workloads took the same time on both. Its trims follow runs-on/runner-images-for-aws. Jobs on these images run as "runner" with sudo, as on GitHub's images. The boot script now reads /etc/environment, after HOME, because PATH there holds entries such as $HOME/.cargo/bin. Closes #145 Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
There was a problem hiding this comment.
🟡 Changes recommended
ARM64 curated/full builds cannot complete with the pinned x64 tooling, and root-owned Miniconda prevents runner jobs from installing packages into the preinstalled base environment.
2 open issues
What changed in this PR
Adds curated and full EC2 toolsets using a pinned GitHub runner-images release, improving compatibility with workflows written for ubuntu-latest while retaining the minimal image as the default.
Changes:
- Adds toolset selection and ordered upstream installer execution, with trimming for curated images.
- Runs jobs as
runnerwith sudo on expanded images and loads tool environment settings. - Adds documentation, shell tests, VM checks, and CI integration.
| File | Description |
|---|---|
scripts/check.sh |
Adds toolset tests and shell checks. |
images/test-runner-images.sh |
Tests curated toolset trimming. |
images/test-ec2-vm.sh |
Adds toolset selection and job-environment checks. |
images/test-ec2-run.sh |
Tests loading tool environment settings. |
images/test-build-ec2-ami.sh |
Tests toolset rendering and validation. |
images/README.md |
Documents toolsets and build requirements. |
images/ec2/provision.sh |
Installs expanded toolsets and configures the runner user. |
images/ec2/install-runner-images.sh |
Runs pinned upstream installers and trims curated tools. |
images/build-ec2-ami.sh |
Adds toolset and upstream release options. |
images/assets/firerunner-ec2-run.sh |
Selects the job user and loads tool paths. |
.github/workflows/ci.yml |
Runs toolset tests in CI. |
Review effort: Balanced
Answers an independent review as well: - a job can write where setup actions and pipx install, as on GitHub's images; - /etc/environment reaches only the runner's subshell, because its PATH holds directories a job can write, and this script powers off as root; - zram's module follows the kernel the build upgraded to; - arm64 is refused, because the installers come from the x64 toolset; - a failed disk usage report no longer fails a finished build. Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
Signed-off-by: Yi-Ru Lin <deduce@gmail.com> # Conflicts: # images/test-ec2-vm.sh
The filter kept the first version and assumed the default was it, so a release with another default would leave it dangling and fail the build. Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
It exported RUNNER_ALLOW_RUNASROOT and root's home before the choice of user, so a job on a curated or full image ran as runner with root's settings around it. Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
The block separated "It launches" from the script it meant, and left a minimal build's time and egress reading as true of every toolset. Signed-off-by: Yi-Ru Lin <deduce@gmail.com>
Contributor
Author
|
babysit-pr · head |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
build-ec2-ami.sh --toolsetruns a pinned actions/runner-images release's installers in its Packer template's order.fulltakes 44 GiB;curatedtakes 18 GiB, and four public workloads took the same time on both. Jobs on these images run asrunnerwith sudo, as on GitHub's.Stacked on the boot tuning PR.
Closes #145