Skip to content

Add a generic aarch64 ISO and test it in CI - #247

Closed
emirb wants to merge 52 commits into
n1xfrom
arm64-generic
Closed

emirb wants to merge 52 commits into
n1xfrom
arm64-generic

Conversation

@emirb

@emirb emirb commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

A second aarch64 image next to the N1x one: a generic ISO on Arch Linux ARM's kernel, with nothing board-specific in it. It is the image a VM can boot, so CI builds it, boots it, installs it unattended and boots what it installed.

Builds on @ryanrhughes's and @spencerbull's N1x work on this branch. The N1x image is unchanged.

What it adds

bin/omarchy-iso-make --arch aarch64 --platform generic --edge
N1x Generic
Kernel linux-omarchy-n1x Arch Linux ARM's linux-aarch64
Board-specific parts Hardware probe, dock handling None
Boots in a VM No Yes
  • builder/linux-aarch64-live.sh makes linux-aarch64 the live kernel under the names mkarchiso and GRUB expect.
  • The installer recognises an installed kernel that records no pkgbase, and leaves out of the initramfs the modules that kernel does not have.
  • The offline mirror is indexed with one repo-add per CPU (builder/index-offline-mirror.sh), and its packages are kept between builds.
  • The live root is compressed with lz4 on the generic image.
  • The build runs without pacman's download sandbox where the kernel has no Landlock.

CI

.github/workflows/arm64-iso.yml, four jobs:

Job Runner What it does
Unit tests ubuntu-24.04 ./test/all in an Arch container, as quattro runs them
Build the generic ISO Namespace, arm64 16x32 Builds the ISO natively and hands it to the next jobs
Boot the live ISO under KVM Namespace, Apple Silicon 12x32 test/arm64-live-boot: UEFI, GRUB, live kernel, a login on the serial port
Install it unattended and boot the result Namespace, Apple Silicon 12x32 test/arm64-install: installs with nobody at the keyboard, reboots, checks the installed system and its desktop over SSH

The installed system takes Arch Linux ARM's packages from arm-mirror.omarchy.org.

Before this merges

  • The workflow runs on pushes to arm64-generic. That trigger moves to n1x, or to a manual run, when this lands.
  • The build takes Omarchy's own packages from omacom/omarchy's arm64-generic branch, which is n1x plus one mirrorlist line. Once that line is on n1x, the build can point there.

Tested

  • All four jobs passed on this branch before it was rebased onto today's n1x (run 37861676593).
  • After the rebase: unit tests pass, and the build, boot and install jobs are running on this pull request's head.

spencerbull and others added 30 commits October 9, 2026 02:00
Port of the N1x bring-up build path onto quattro, following the shape
proposed in plans/aarch64-support.md:

- bin/omarchy-iso-make gains --arch aarch64, --platform n1x, and
  --package-dir. aarch64 builds run in a digest-pinned Arch Linux ARM
  container, require --local-source (pkgs.omarchy.org serves no aarch64
  repo), verify the bundle's SHA256SUMS on the host, and never share the
  host pacman cache.
- builder/build-iso.sh indexes the bundle as the [omarchy] repo, uses Arch
  Linux ARM mirrors, runs the pinned archiso with the arm64-efi GRUB module
  patch, swaps the live kernel for linux-n1x, drops Plymouth and the x86
  hooks from the live initramfs, adds the I2C-HID keyboard modules, picks
  the arm64 Node.js tarball, and filters x86-only packages out of the
  Omarchy manifests through builder/aarch64-package-filter.sh.
- The live kernel command line pins console=tty0 acpi=nospcr: the N1x
  firmware publishes an SPCR serial console that otherwise swallows the
  console and the LUKS prompt.
- GRUB configs are templated for kernel and options; the N1x overlay adds
  the console+SSH recovery entry with key-only root SSH and a probe.
- profiledef.sh reads OMARCHY_ARCH and drops BIOS boot on aarch64.
- test/unit/aarch64-build-test.sh guards the contract statically.
The configurator reads the architecture and platform the ISO was built
for (/root/omarchy_arch, /root/omarchy_arm_platform) and, on aarch64,
selects the platform kernel (linux-n1x), Arch Linux ARM as the archinstall
mirror, and limine_aa64.efi as the Limine binary. The orchestrator derives
the Limine source and fallback names (BOOTAA64.EFI) from the running
machine architecture, which the live ISO and the target share.
Arch Linux ARM's makepkg.conf produces .pkg.tar.xz, so the zstd-only glob
found nothing on aarch64 and the mv aborted the build.
The installer pacstraps the target from the shipped omarchy-base.packages
copy, and the build's expected-package count re-reads it, so both saw the
x86 kernel, microcode, mise-bin, tensaku, and tzupdate that the offline
mirror had been filtered to exclude.
pacman resolves an explicit target from the first repository that lists
it. Arch Linux ARM's hyprland currently depends on a libaquamarine soname
its own aquamarine no longer provides, so the bundle's hyprland, built
against the current aquamarine, must be found first. This mirrors how
Omarchy pins its own hyprland on x86.
makepkg in the build container defaults to one make job and single-threaded
xz/zstd compression; on a 20-core aarch64 host that serializes the slowest
step of a --local-source build.
…te dev-only SSH script

pkgs.omarchy.org now publishes aarch64 packages on the edge channel, so the
package bundle shrinks to what no repository carries (the platform kernel
pair and the NVIDIA driver proven on the hardware) and becomes a [platform]
repo ahead of [omarchy] edge and Arch Linux ARM. aarch64 builds are pinned
to the edge channel; the manifest filter maps linux-omarchy to the platform
kernel and stops excluding packages edge now publishes.

- Autoinstall picks the platform kernel on aarch64, as the configurator does.
- The N1x recovery entry opens a shell with the hardware probe again instead
  of the installer.
- validate_boot checks the generated limine.conf against the N1x console
  contract: console=tty0 and acpi=nospcr on every kernel entry, and a rescue
  entry that still carries its own multi-user cmdline.
- DEV ONLY: builder/n1x-dev-ssh gives the development key root SSH on the
  live ISO and hands it to the installer's configure_ssh_access; a dev phase,
  listed only when the script is on the image, authorizes it for root on the
  installed system. Password login stays off. It replaces the recovery-key
  plumbing.
…wo unpublished packages

Arch Linux ARM still names Neovim neovim and replaces vi with ex-vi-compat.
obs-studio and yay-debug have aarch64 recipes in omarchy-pkgs but edge does
not publish them yet, so they are left out with a log line saying so.
…hat has no platform

The quattro orchestrator installs Limine itself, so the archinstall patch had
no caller. The configurator's :? guard only ended the detect_kernel subshell
and let an empty kernel into the generated config; it now exits up front. The
GRUB overlay no longer names a gb10 platform the build cannot select.
sshd keeps the first value it reads, and releng's 10-archiso.conf, which
enables root password login, sorted before 10-omarchy-n1x-dev-ssh.conf. The
drop-in is now 05-, so the live ISO is key-only as intended. Found by booting
the live ISO in a VM.
omarchy-pkgs now builds a separate Apple Silicon flavour of omarchy-settings-dev
and omarchy-dev on aarch64: no Limine template, initramfs hooks, Limine
drop-ins or Limine/Snapper dependency. The N1x boots like the x86_64 machines
and needs all of them; a VM install of the previous image stopped at the
missing Limine template. The local package build now passes the platform to
makepkg, and an N1x build fails if either package came out as the Apple
variant.
Provisioning staging looked only for node-v*-linux-x64.tar.gz, so an aarch64
install stopped at "no bundled Node tarball" although build-iso.sh had
bundled the linux-arm64 build. Found by a VM install of the N1x image.
omarchy-iso-make passes OMARCHY_RUNTIME_PACKAGE and OMARCHY_SETTINGS_PACKAGE
into the build container, where build-iso.sh already honours them, so an N1x
image can carry omarchy/omarchy-settings 4.0.4 built from a local release
checkout rather than the quattro -dev pair.
The aarch64 port moved the archinstall mirror lines out of the JSON heredocs
into single-quoted variables but kept the heredoc's \$ escapes, so the
generated user_configuration.json carried \$arch and the installer died on
"Invalid \\escape" before touching the disk, on x86_64 as well. Found by an
interactive install on the N1x.
Every N1x image carried the development key. It now takes
omarchy-iso-make --dev-ssh, which also names the image -devssh, so a normal
N1x image has no remote access and a dev image says what it is. The recovery
entry and its probe no longer promise SSH on an image that has none.
…access, updates

N1X.md covers both images, what differs from stock Omarchy, installing,
the USB and Limine boot menus, the dev image's SSH and how to remove it, the
NVIDIA hold, known limitations, and how to build both.
- Ship linux-omarchy-n1x: the build records the platform kernel in
  /root/omarchy_kernel, and the configurator and orchestrator install that
  rather than deriving linux-<platform>.
- Treat the kernel's fallback UKI as the rescue entry, which must boot to a
  text login with Plymouth off, and run the N1x boot checks only when the
  target's omarchy-hw-n1x finds N1x hardware, so the image also installs in
  a VM like any aarch64 machine.
- Add the Limine/Snapper stack in the installer on aarch64, where the
  runtime does not depend on it because Apple Silicon boots differently,
  instead of requiring an N1x variant of the runtime package.
- Boot the live image with Plymouth and quiet splash, and hide the GRUB
  menu as on every other image; the recovery entry is still a keypress away.
- Take the dev SSH key from the builder (--dev-ssh <file>) instead of a key
  baked into the sources.
- Carry every package in the platform bundle in the offline mirror, which
  brings the ProArt P14's amplifier firmware.
On an aarch64 host the VM is aarch64 too: a virt machine under KVM with
the edk2 aarch64 UEFI firmware, the ISO on a SCSI CD, virtio-gpu with GL,
and a USB keyboard and tablet. The vCPUs stay on the host's fastest cores,
since moving between big and little cores changes the CPU the guest sees.
arm64 firmware commonly publishes an ACPI serial console (SPCR), and the
kernel adopts it unless told otherwise. The live ISO boots aarch64 with
console=tty0 acpi=nospcr, but an installed system only got it from the
N1x hardware script, so an install elsewhere, in QEMU's virt machine for
one, asked for the disk passphrase on a serial port nobody was watching.

Give every aarch64 install the live ISO's console setting through a
limine-entry-tool drop-in.
Arch Linux ARM signs every package in its repositories, archlinux-keyring
included, with its build key, and only archlinuxarm-keyring carries that key.
The ISO bundles the package in its offline mirror and populates it in the
build container and the live image, but never installed it on the target. The
install itself runs with SigLevel = Never, so nothing noticed until the first
online pacman run on the installed system asked to import 77193F152BDBE6A6
and then rejected the package with "unknown trust" (omarchy-install-service-
tailscale on the first T7 install).

Install archlinuxarm-keyring with the early aarch64 packages. pacstrap has
initialized the target keyring by then, so its install script populates and
locally signs the keys. validate_boot now verifies the keyring package's
signature from the offline mirror against the target keyring and refuses a
keyring pacman would not accept, so this cannot regress silently.
Omarchy now enrolls a Thunderbolt or USB4 device plugged in while the session is unlocked, so the manual boltctl step is gone. Say what still needs a replug.
The USB4 PCIe ports now stay out of runtime suspend so a dock's Ethernet
appears on every plug, and each interface answers ARP only for its own
address so docked Ethernet on the Wi-Fi's network keeps its routes. Say so,
and what the root-port setting costs until the kernel wakes the port itself.
Less RAM is reclaimed now that the room next to the GPU firmware stays
reserved, both laptops get USB4, swap on zram and the x86_64 memory tuning,
and the XPS 16 gets its sound profile, battery, mic-mute key and OLED
brightness. Its embedded controller still sends no events.
MemTotal is about 119 GiB with the narrowed reclaim; 118 was free -g
rounding down.
ryanrhughes and others added 18 commits October 9, 2026 02:11
…i firmware fetcher, the XPS 13 speaker alias and Superwhisper
An aarch64 build had to be an N1x build: its own kernel, a package
bundle on disk and local source checkouts. None of that can be built or
booted by CI, and nothing else can boot the result.

--platform generic builds the same image without anything board-specific:
Arch Linux ARM's linux-aarch64, the published edge packages, no bundle
and no NVIDIA stack. It boots machines whose firmware describes their
hardware, which is what a VM is, so it is the image CI can install and
test, and the base a board's kernel and packages go on top of.

linux-aarch64 owns its mkinitcpio preset and names its image /boot/Image,
so the live kernel name, preset and initramfs are made after pacstrap,
inside the image (builder/linux-aarch64-live.sh). The approach is the
dragon branch's customize_airootfs.sh.

The generic image keeps both consoles, serial and panel. The N1x build is
unchanged.
pacman 7 sandboxes its downloads with Landlock and fails every database
sync on a kernel without it, which Namespace's AmpereOne arm64 machines
run: "restricting filesystem access failed because Landlock is not
supported by the kernel!". An ISO build there stops at its first
pacman -Sy.

Where /sys/kernel/security/lsm lists no landlock, or cannot be read, a
wrapper in /usr/local/bin runs every pacman in the build container,
pacstrap's included, with --disable-sandbox, the flag pacman provides for
such kernels. Not DisableSandbox in a config: pacman-offline.conf ships
on the ISO.
No aarch64 ISO has been built by CI. This builds the generic one natively
on every push to this branch, checks that it carries the boot loader, the
kernel, the live initramfs and the root image, and keeps it as an
artifact for three days. The unit tests run beside it.

The push trigger is for the branch: a manual run needs the workflow on
the default branch first.
The runner's Python ships a "test" package. Discovery from the
repository root, as test/all does it, then looks for test.unit inside
that package and finds nothing.
Thirteen of the installer's tests stage a Node tarball named linux-x64,
and the code they exercise looks for the host's architecture, so they
fail on any aarch64 host. quattro runs them on x86; so does this.
On plain Ubuntu four tests fail for reasons that are the runner's, not the
code's: the keyboard tests need Arch's keymaps behind localectl, and two
package-signing tests meet a Python whose Path.is_file raises on a
directory it may not enter. quattro's unit-test.yml runs ./test/all in an
Arch container with systemd for exactly this; this is that step, plus the
tools the aarch64 build test uses.
The aarch64 build test compares the dev key it wrote with cmp, which the
Arch base image does not carry.
repo-add takes one package at a time: it hashes the archive and lists its
files, which means decompressing all of it. Arch Linux ARM still ships
xz, so indexing the 1,118 packages of an aarch64 mirror took 278 s of a
769 s build on 16 cores.

A database is a tar of one directory per package. The packages are dealt
out to one repo-add per CPU, each writing its own database, and the
pieces are joined. On 160 packages here the result is identical to one
repo-add's, db and files, in 1.0 s against 6.3 s. Two versions of one
package name are an error: one repo-add would keep the last, the pieces
apart would keep both.
The build downloads 3.3 GB of packages into ~/.cache/omarchy on every run
and prunes what it no longer needs. On Namespace that directory can live
on a cache volume. The step is skipped, not failed, on a runner profile
that has none.
A built ISO says nothing about whether it starts. This boots it the way
a machine would, on one of Namespace's Apple-backed runners, the ARM ones
with KVM: UEFI firmware, GRUB from the ISO, the live kernel and
initramfs, the live root. It passes when the live system offers a login
on its serial port, and keeps the serial log and a picture of the screen
either way.

The machine is virt with gic-version=3, its=off and acpi=off: with the
interrupt translation service on, MSIs are lost under nested KVM there
and virtio disks hang.
The generic aarch64 ISO started its kernel and initramfs and stopped
there: mount: /run/archiso/airootfs: fsconfig() failed: Filesystem uses
"zstd" compression. This is not supported. Arch Linux ARM builds
linux-aarch64's squashfs without zstd.

Of what that kernel reads, lz4 keeps what zstd was chosen for, fast
decompression on the page-fault path. Repacked that way on the runner,
the same ISO offered a login 20 s after power-on. The root image grows
from 4,509 MB to 4,814 MB. Other kernels keep zstd.
An unattended install of the generic ISO stopped at its last step:
"Error: bundled Node.js tarball missing from /opt/packages". The ISO
carries Node for its own architecture; quattro's install/user/mise-work.sh
looks for the x64 one on every architecture. The n1x branch of omarchy
picks by uname -m, which is why the N1x image builds its packages from
--local-source. The generic image now does the same, from the n1x
branches of omarchy and omarchy-pkgs.

The ISO also goes to Namespace's artifact store for the jobs that boot
it. GitHub's store took 3 min 20 s to hand 4.5 GB back to a Namespace
runner; it stays as the fallback and for people.
The whole path with nobody at the keyboard, on a Namespace runner with
KVM: the installer takes its answers from a cidata drive, writes the
disk, and the machine restarts into the installed system, which is then
checked over SSH. The answers are the x86_64 integration harness's own
(build_cidata), changed where the architecture shows: Limine's binary,
the kernel, the mirror.

Checked on a runner by hand before this: the install runs to its last
step and writes a Limine entry and a unified kernel image for
linux-aarch64, and after a reset the firmware starts Limine from the
disk.
An unattended install of the generic aarch64 image did everything and
then stopped at its last check: "Validating boot setup failed: no
installed kernel found in target". The installer finds installed kernels
by the pkgbase file Arch's kernel packages leave next to their modules.
Arch Linux ARM's linux-aarch64 leaves none, though its modules, headers
and Limine's unified kernel image were all in place.

A kernel's module directory without a pkgbase is now named by the kernel
the image was built to install. modules.builtin marks a kernel's own
directory, so a leftover of out-of-tree modules is still not a kernel,
and kernels that do record a pkgbase are found exactly as before.
With the kernel recognised, the same install stopped one check later:
"/boot/EFI/Linux/omarchy_linux-aarch64.efi missing or empty". The
Omarchy settings add thunderbolt to the initramfs. linux-aarch64 has no
such module, mkinitcpio reports a name it cannot find as an error, and
limine-mkinitcpio-hook then builds the unified kernel image and does not
install it: "mkinitcpio failed for kernel 7.2.9-1-aarch64-ARCH,
skipping".

Before the boot image is built, the installer now asks modinfo which of
the listed modules the installed kernel has, and writes a drop-in that
sorts last and takes the others back out. With this on the runner, the
install finished, the machine restarted into the installed system, and
Hyprland was running with no failed unit.
The test waited out its whole timeout on an installer that had stopped
after two minutes, and said nothing about why. It now has a way in to
the live system (the live ISO runs cloud-init, so the cidata drive hands
root a key), fails as soon as the installer's log says it halted, and
prints the end of that log. It also checks that no unit failed, that
Hyprland is running, and that the installed system's packages come from
Omarchy's mirror and nowhere else.
The install test's last failing check is the installed system's
mirrorlist. The Omarchy runtime writes it from
default/pacman/mirrorlist-aarch64, which on n1x names Arch Linux ARM's
own mirror. omarchy's arm64-generic branch is n1x plus that one line
changed to arm-mirror.omarchy.org. The workflow builds from it until the
line lands in n1x.
@emirb
emirb requested a review from ryanrhughes October 9, 2026 20:54
emirb added 2 commits October 9, 2026 22:55
The aarch64 build test runs ssh-keygen, patch and cmp. quattro's
unit-test.yml installs none of them, so it stopped at 'ssh-keygen:
command not found' on every pull request into n1x. With that fixed,
arm64-iso.yml no longer needs its own copy of the unit job.
Docker Hub refused the anonymous pull of archlinux:latest three runs in
a row ('toomanyrequests'), failing the job in five seconds. The image
now comes from mirror.gcr.io, with Docker Hub as the fallback.
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Install test passed

This PR quattro Change
Install 185.2 s 82.5 s +102.7 s
First boot 21.0 s 17.4 s +3.6 s
Everyday boot 14.7 s 16.7 s −2.0 s
Install phases
Phase This PR quattro
Preparing live environment 0.3 s 0.2 s
Preparing install target 0 s 0 s
Installing Arch + Omarchy 159.7 s 65 s
Configuring hibernation 0.3 s 0.3 s
Configuring system 5.4 s 3.2 s
Staging provisioning 0.5 s 0.2 s
Finalizing Limine boot 11.3 s 10.1 s
Finalizing user 7.2 s 3.1 s
Configuring login 0.1 s 0.1 s
Configuring SSH access 0.3 s 0.3 s
Configuring Tailscale 0 s 0 s
Configuring DNS resolver 0 s 0 s
Validating boot setup 0 s 0 s
Creating factory snapshot 0.1 s 0 s

Run · 3a032ea

The install test built its ISO for seven minutes and then stopped at
Docker Hub's 'toomanyrequests' when pulling archlinux:latest. Same
fix as the unit tests: mirror.gcr.io first, Docker Hub as the fallback.
@emirb

emirb commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of #258

@emirb emirb closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants