From 9c73ea3d4d04b1687c56ab4977d3bcfb9bce439f Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 21 Aug 2026 12:57:56 +0200 Subject: [PATCH 01/67] Install the target from a btrfs root image instead of pacstrapping it The target install is the same ~940 packages for every machine; only the kernel, CPU microcode, audio firmware and Tailscale differ. pacman extracts that set single-threaded at ~40 packages/s, about 34s on any machine with four or more cores, and no amount of parallelism reaches it. So build-root-image.sh pacstraps the invariant set once at ISO build time into a btrfs subvolume mounted compress=zstd:3 (the level the installer mounts the target with) and ships it as a `btrfs send --compressed-data` stream. The orchestrator receives it at the target filesystem's top level right after archinstall mounts the layout, snapshots it writable in place of the empty @ subvolume, replays the mount table, and then lets archinstall finish with the per-machine delta (install_base_delta mirrors minimal_installation minus the bulk pacstrap), users and fstab. The application installers strap only what the target lacks, since the image carries their package sets and the mirror no longer does. Measured in the same 16-vCPU VM: the package phase drops from 35.7s to 24.5s and the whole install from 41.8s to 30.6s, with an identical set of 942 packages installed; the installed system boots. The receive itself is ~17s and independent of CPU count. The offline mirror keeps only what is still pacstrapped at install time: the live ISO's own packages, the per-machine packages, and the omarchy-other.packages extras omarchy-apply-system may pull in, resolved against the offline repo itself so the keep-set can only name files the mirror holds. The ISO grows from 6.2GB to 9.1GB, the live closure and the extras now sitting beside the 5.2GB image. Build details: the container needs loop device nodes made by hand (Docker fills /dev once, at start), pacman-key's gpg-agent must be stopped before the image unmounts, and stale copies of locally rebuilt omarchy packages are evicted from the shared pacman cache so mkarchiso's pacstrap does not hit a checksum mismatch. The dashboard gets a phase_progress signal from the unpack so the bar moves while the local pacman db is still empty, and the live ISO prefetches the leading bytes of the stream during the wizard. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01P9b7oP8j2GA6sZZ9e8aJYv --- README.md | 2 +- builder/build-iso.sh | 169 ++++++++--- builder/build-root-image.sh | 175 +++++++++++ builder/image.packages | 33 ++ configs/airootfs/root/.automated_script.sh | 30 +- .../usr/local/bin/omarchy-install-dashboard | 17 +- .../orchestrator/archinstall_adapter.py | 126 +++++++- .../share/omarchy-iso/orchestrator/main.py | 7 +- .../share/omarchy-iso/orchestrator/phases.py | 17 +- .../omarchy-iso/orchestrator/phases_impl.py | 287 ++++++++++++------ configs/profiledef.sh | 8 +- 11 files changed, 705 insertions(+), 166 deletions(-) create mode 100755 builder/build-root-image.sh create mode 100644 builder/image.packages diff --git a/README.md b/README.md index 70cc2c6be..4566b025e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Omarchy ISO -The Omarchy ISO is the only supported way to install Omarchy. It ships the Omarchy Configurator, installs Arch Linux, installs the Omarchy packages from the bundled mirror, runs target system setup in the chroot, creates the user, and runs `omarchy-setup-user` for that user. +The Omarchy ISO is the only supported way to install Omarchy. It ships the Omarchy Configurator, unpacks a pre-built Arch Linux + Omarchy root image onto the target with `btrfs receive`, installs the per-machine packages (kernel, microcode, audio firmware) from the bundled mirror, runs target system setup in the chroot, creates the user, and runs `omarchy-setup-user` for that user. ## Downloading the latest ISO diff --git a/builder/build-iso.sh b/builder/build-iso.sh index db8daa2c8..f11f8342f 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -30,7 +30,7 @@ pacman --noconfirm -Sy archlinux-keyring # so this container can be months behind the mirror it installs from. A plain # -Sy install is then a partial upgrade — new packages linked against a glibc # the container doesn't have yet. -pacman --noconfirm -Syu archiso git sudo base-devel jq grub imagemagick neovim nodejs npm tree-sitter-cli +pacman --noconfirm -Syu archiso git sudo base-devel jq grub imagemagick neovim nodejs npm tree-sitter-cli btrfs-progs # Pre-import the omarchy signing key (so pacman trusts our [omarchy] repo # during the build without keyserver lookups). @@ -51,7 +51,9 @@ fi # Build locations build_cache_dir=/var/cache offline_mirror_dir="$build_cache_dir/airootfs/var/cache/omarchy/mirror/offline" -mkdir -p "$build_cache_dir" "$offline_mirror_dir" +root_image_dir="$build_cache_dir/airootfs/var/cache/omarchy/rootfs" +root_image_stream="$root_image_dir/omarchy-root.btrfs" +mkdir -p "$build_cache_dir" "$offline_mirror_dir" "$root_image_dir" # Seed from the official Arch releng profile. cp -r /archiso/configs/releng/* "$build_cache_dir/" @@ -102,6 +104,14 @@ fi if [[ -d /omarchy-source && -d /omarchy-pkgs ]]; then bash /builder/build-omarchy-packages.sh "$offline_mirror_dir" LOCAL_OMARCHY_BUILD=1 + # A rebuild of the same revision produces the same filename with different + # bytes. pacman consults /var/cache/pacman/pkg (the host's cache, kept across + # --keep-pkg-cache builds) before the mirror, and a stale copy from an + # earlier build fails the checksum against the mirror's fresh db. + for local_package_name in \ + "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE"; do + rm -f /var/cache/pacman/pkg/"$local_package_name"-*.pkg.tar.zst{,.sig} + done fi # Node.js binary for offline mise install. @@ -228,22 +238,101 @@ if ! download_offline_packages; then download_offline_packages fi -# Resolve the exact filenames chosen by the same synced package databases used -# for the download. Pruning by this transaction (rather than merely keeping the -# newest version of every cached package name) removes packages that have left -# the lists or dependency closure, such as an old Electron major version. -if ! resolved_package_files="$( - pacman --config "/configs/pacman-online-${OMARCHY_MIRROR}.conf" --noconfirm \ - --dbpath /tmp/offlinedb -S --print --print-format '%f' "${all_packages[@]}" -)"; then - echo "ERROR: could not resolve the package files required by the offline mirror" >&2 +# Index the complete mirror so the root image can be pacstrapped from it. +# mkarchiso expects the mirror at /var/cache/omarchy/mirror/offline inside the +# container (the airootfs path); symlink rather than duplicate. +rebuild_offline_repo_db() { + rm -f "$offline_mirror_dir"/offline.db* "$offline_mirror_dir"/offline.files* + repo-add -q "$offline_mirror_dir/offline.db.tar.gz" "$offline_mirror_dir/"*.pkg.tar.zst +} +rebuild_offline_repo_db +mkdir -p /var/cache/omarchy/mirror +ln -sfn "$offline_mirror_dir" /var/cache/omarchy/mirror/offline + +# Packages that differ per machine and therefore stay out of the root image: +# the installer pacstraps them on the target after unpacking the image, and +# omarchy-apply-system installs more from omarchy-other.packages as the +# hardware dictates. Everything else the target gets is in the image. +hardware_packages=(linux linux-t2 amd-ucode intel-ucode sof-firmware alsa-firmware tailscale) + +mapfile -t image_packages < <( + { + grep -hv '^#\|^$' /builder/archinstall.packages + grep -hv '^#\|^$' /builder/image.packages + # The shipped copy, which is what the install reads too. + grep -hv '^#\|^$' "$build_cache_dir/airootfs/usr/share/omarchy-iso/omarchy-base.packages" + printf '%s\n' "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE" + } | sort -u | grep -Fxv "${hardware_packages[@]/#/-e}" +) + +# pacstrap resolves the image against the offline repo and, with CacheDir on +# the mirror itself, extracts the package files in place instead of copying +# several GiB into a cache first (the same trick the installer's bind mount +# plays on the target). +image_pacman_conf="$build_cache_dir/pacman-root-image.conf" +sed "/^\[options\]/a CacheDir = /var/cache/omarchy/mirror/offline/" \ + "$build_cache_dir/pacman-offline.conf" >"$image_pacman_conf" + +image_localdb=/tmp/omarchy-root-image-localdb +OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ + bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" + +# Resolve the exact filenames the mirror must keep now that the image carries +# the bulk of the target, against the complete offline repo just indexed so +# the answer can only name files the mirror holds (the local omarchy builds +# included): +# - the live ISO's own packages, against an empty root: mkarchiso pacstraps +# the live root from this mirror and needs their whole closure; +# - the per-machine packages and the hardware-dependent extras +# omarchy-apply-system may install from omarchy-other.packages, against +# the image's local db: only what the image does not already hold. +# Dependencies the live closure shares with the image cost ISO space twice, +# which is the price of keeping the mirror self-contained for the live +# environment. +resolve_mirror_files() { + local root="$1" + shift + mkdir -p "$root/var/lib/pacman" + pacman --config "$build_cache_dir/pacman-offline.conf" \ + --root "$root" --dbpath "$root/var/lib/pacman" --noconfirm -Sy >/dev/null || return 1 + pacman --config "$build_cache_dir/pacman-offline.conf" \ + --root "$root" --dbpath "$root/var/lib/pacman" --noconfirm \ + -S --print --print-format '%f' --needed "$@" +} + +live_root=/tmp/omarchy-mirror-live +rm -rf "$live_root" +mapfile -t live_targets < <(sort -u "$build_cache_dir/packages.x86_64") +if ! live_package_files="$(resolve_mirror_files "$live_root" "${live_targets[@]}")"; then + echo "ERROR: could not resolve the live ISO's packages from the offline mirror" >&2 + exit 1 +fi + +delta_root=/tmp/omarchy-mirror-delta +rm -rf "$delta_root" +mkdir -p "$delta_root/var/lib/pacman" +cp -a "$image_localdb/local" "$delta_root/var/lib/pacman/" +mapfile -t delta_targets < <( + { + grep -hv '^#\|^$' "${base_pkg_lists[1]}" + printf '%s\n' "${hardware_packages[@]}" + } | sort -u +) +if ! delta_package_files="$(resolve_mirror_files "$delta_root" "${delta_targets[@]}")"; then + echo "ERROR: could not resolve the per-machine packages from the offline mirror" >&2 exit 1 fi -mapfile -t required_package_files <<< "$resolved_package_files" + +mapfile -t required_package_files < <( + printf '%s\n%s\n' "$live_package_files" "$delta_package_files" | sort -u | grep . +) # The online transaction intentionally excludes packages built from the local # checkouts. Add those exact artifacts back to the keep-set after verifying # that the local build left exactly one file for each selected package name. +# The settings package is a live ISO package; the runtime and nvim packages +# are only in the image now, but keeping all three in the mirror leaves +# omarchy-apply-system able to resolve them like any published build. if [[ -n ${LOCAL_OMARCHY_BUILD:-} ]]; then for local_package_name in \ "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE"; do @@ -271,43 +360,22 @@ printf '%s\n' "${required_package_files[@]}" | # Rebuild the offline repo db from scratch so size/checksum/depends entries # always reflect only the package files selected for this build. -rm -f "$offline_mirror_dir"/offline.db* "$offline_mirror_dir"/offline.files* -repo-add "$offline_mirror_dir/offline.db.tar.gz" "$offline_mirror_dir/"*.pkg.tar.zst - -# mkarchiso expects the mirror at /var/cache/omarchy/mirror/offline inside the -# container (the airootfs path); symlink rather than duplicate. -mkdir -p /var/cache/omarchy/mirror -ln -sf "$offline_mirror_dir" /var/cache/omarchy/mirror/offline - -# Denominator for the install dashboard's progress bar. Resolving the mirror's -# own package lists against the mirror we just indexed, with an empty local db, -# is the question pacstrap asks at install time — same resolver, same repo, same -# lists — so no hand-kept constant can drift. -# -# It over-counts by ~1 in 925: archinstall.packages lists both amd-ucode and -# intel-ucode because the mirror must contain either. phases.py records expected -# and actual in the timing JSON, so growing drift shows up in acceptance runs. -# The early-bootstrap set is already inside this closure, so restating it would -# only add a second list to drift. +rebuild_offline_repo_db + +# Denominator for the install dashboard's progress bar: what the image holds +# plus the kernel's closure on top of it, resolved against the image's own +# local db with the pruned mirror, which is the question the installer's delta +# pacstrap asks. Plus one for the microcode package every real machine gets. +# phases.py records expected and actual in the timing JSON, so drift shows up +# in acceptance runs. resolve_expected_packages() { local resolve_root=/tmp/omarchy-expected-packages - local resolved - local -a targets + local image_count resolved rm -rf "$resolve_root" mkdir -p "$resolve_root/var/lib/pacman" - - mapfile -t targets < <( - { - grep -hv '^#\|^$' /builder/archinstall.packages - # Read the shipped copy, which is what _runtime_package_list reads at - # install time, not the build-time source it came from. - grep -hv '^#\|^$' \ - "$build_cache_dir/airootfs/usr/share/omarchy-iso/omarchy-base.packages" - printf '%s\n' "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" \ - "$OMARCHY_NVIM_PACKAGE" - } | sort -u - ) + cp -a "$image_localdb/local" "$resolve_root/var/lib/pacman/" + image_count=$(find "$resolve_root/var/lib/pacman/local" -mindepth 1 -maxdepth 1 -type d | wc -l) pacman --config "$build_cache_dir/pacman-offline.conf" \ --root "$resolve_root" --dbpath "$resolve_root/var/lib/pacman" \ @@ -318,19 +386,20 @@ resolve_expected_packages() { # dashboard's fallback. resolved="$(pacman --config "$build_cache_dir/pacman-offline.conf" \ --root "$resolve_root" --dbpath "$resolve_root/var/lib/pacman" \ - --noconfirm -S --print --print-format '%n' "${targets[@]}")" || return 1 + --noconfirm -S --print --print-format '%n' --needed linux)" || return 1 - printf '%s\n' "$resolved" | sort -u | grep -c . + echo $(( image_count + $(printf '%s\n' "$resolved" | sort -u | grep -c .) + 1 )) } # Worth failing the build over: -S --print only aborts when a target is missing -# from the offline repo, which would fail pacstrap the same way. A count that -# merely looks wrong is not — the dashboard falls back without the file. +# from the offline repo, which would fail the delta pacstrap the same way. A +# count that merely looks wrong is not — the dashboard falls back without the +# file. if ! expected_packages="$(resolve_expected_packages)"; then echo "ERROR: could not resolve the target package count from the offline mirror." >&2 echo " pacman -S --print aborts the whole transaction if any single target" >&2 - echo " is missing, so this almost certainly means pacstrap would fail the" >&2 - echo " same way at install time." >&2 + echo " is missing, so this almost certainly means the installer's delta" >&2 + echo " pacstrap would fail the same way." >&2 exit 1 fi if (( expected_packages < 600 || expected_packages > 2000 )); then diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh new file mode 100755 index 000000000..54e40fdb4 --- /dev/null +++ b/builder/build-root-image.sh @@ -0,0 +1,175 @@ +#!/bin/bash +# +# Build the root filesystem image the installer unpacks instead of pacstrapping +# every package on the target machine. +# +# The target install is the same ~940 packages for everyone; only the kernel, +# CPU microcode, audio firmware and Tailscale vary per machine. So pacstrap the +# invariant set once here, at build time, into a btrfs subvolume compressed at +# the same zstd level the installer mounts the target with, and ship it as a +# `btrfs send --compressed-data` stream. At install time `btrfs receive` writes +# the compressed extents straight to disk (no decompress/recompress), which +# measured at roughly half the time of extracting the same packages with pacman, +# independent of CPU count; the per-machine delta is a small pacstrap after. +# +# Usage: build-root-image.sh ... +# +# pacman.conf Config whose repositories resolve every package; its +# CacheDir must hold the package files so nothing is copied. +# output-stream Where to write the send stream. +# package... Packages to pacstrap into the image. +# +# Environment: +# OMARCHY_IMAGE_LOCALDB_COPY Directory to copy the image's pacman local db +# into, so the caller can resolve the per-machine +# delta against what the image already holds. +# OMARCHY_IMAGE_SIZE Sparse backing file size (default 24G). Only the +# written extents cost anything. +# +# Needs a privileged environment: loop devices and a btrfs mount. + +set -euo pipefail + +pacman_conf="${1:-}" +output="${2:-}" +shift 2 || true +packages=("$@") + +if [[ -z $pacman_conf || -z $output || ${#packages[@]} -eq 0 ]]; then + echo "Usage: build-root-image.sh ..." >&2 + exit 1 +fi +if [[ ! -r $pacman_conf ]]; then + echo "ERROR: pacman config not readable: $pacman_conf" >&2 + exit 1 +fi + +# Must match the compress= option the installer mounts the target with +# (configurator JSON mount_options / disk-partitioning.sh): the stream carries +# the extents as compressed here, and the installed system keeps writing new +# data at its own mount level either way. +IMAGE_COMPRESS="compress=zstd:3" +# Name of the subvolume inside the stream. The orchestrator looks for this name +# after `btrfs receive` (phases_impl.ROOT_IMAGE_SUBVOLUME). +IMAGE_SUBVOLUME="omarchy-root" + +# Boot-image pacman hooks that must not run while the image is built: there is +# no kernel in the image and no ESP to deploy to. pacstrap reads hooks from the +# building system's /etc/pacman.d/hooks (pacman.conf(5): HookDir is absolute, +# the target root is not prepended), so mask them here, the same way the +# orchestrator masks the live ISO's around its pacstrap. Keep in step with +# DEFERRED_BOOT_HOOKS in +# configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py. +deferred_boot_hooks=( + 60-mkinitcpio-remove.hook + 60-limine-mkinitcpio-remove-pre.hook + 80-limine-efi-deploy.hook + 90-limine-mkinitcpio-remove-post.hook + 90-mkinitcpio-install.hook +) + +work=$(mktemp -d /tmp/omarchy-root-image.XXXXXX) +backing="$work/image.btrfs" +mnt="$work/mnt" +loop="" +masked_hooks=() + +# pacstrap -K runs pacman-key --init against the image's keyring, which starts +# a gpg-agent for that homedir and leaves it running; until it is gone the +# image cannot be unmounted. +stop_image_gpg_agents() { + gpgconf --homedir "$mnt/$IMAGE_SUBVOLUME/etc/pacman.d/gnupg" --kill all 2>/dev/null || true + pkill -f "$mnt/$IMAGE_SUBVOLUME/etc/pacman.d/gnupg" 2>/dev/null || true +} + +cleanup() { + local status=$? + set +e + for hook in "${masked_hooks[@]}"; do + rm -f "/etc/pacman.d/hooks/$hook" + if [[ -e "/etc/pacman.d/hooks/$hook.omarchy-backup" ]]; then + mv -f "/etc/pacman.d/hooks/$hook.omarchy-backup" "/etc/pacman.d/hooks/$hook" + fi + done + if mountpoint -q "$mnt" 2>/dev/null; then + stop_image_gpg_agents + sleep 1 + if ! umount -R "$mnt"; then + echo "WARNING: $mnt is still mounted; leaving $work in place" >&2 + return $status + fi + fi + if [[ -n $loop ]]; then + losetup -d "$loop" || true + fi + rm -rf "$work" + return $status +} +trap cleanup EXIT + +mkdir -p /etc/pacman.d/hooks +for hook in "${deferred_boot_hooks[@]}"; do + path="/etc/pacman.d/hooks/$hook" + if [[ -e $path || -L $path ]]; then + mv -f "$path" "$path.omarchy-backup" + fi + ln -s /dev/null "$path" + masked_hooks+=("$hook") +done + +echo "Building root image with ${#packages[@]} packages" +truncate -s "${OMARCHY_IMAGE_SIZE:-24G}" "$backing" +# Docker fills the container's /dev once, at start. A loop device the kernel +# creates afterwards (loop-control hands one out on demand, loading the module +# on a host that had none) exists on the host but has no node in here, so make +# the nodes ourselves when they are missing. +[[ -e /dev/loop-control ]] || mknod /dev/loop-control c 10 237 +# losetup reports a device whose node is missing as "/dev/loopN (lost)". +loop=$(losetup --find | awk '{ print $1 }') +[[ $loop == /dev/loop[0-9]* ]] || { echo "ERROR: no free loop device: $loop" >&2; exit 1; } +[[ -b $loop ]] || mknod "$loop" b 7 "${loop#/dev/loop}" +losetup "$loop" "$backing" +mkfs.btrfs -q -L omarchy-root-image "$loop" +mkdir -p "$mnt" +mount -o "$IMAGE_COMPRESS" "$loop" "$mnt" +btrfs subvolume create "$mnt/$IMAGE_SUBVOLUME" >/dev/null +root="$mnt/$IMAGE_SUBVOLUME" + +# -c: packages come straight from the config's CacheDir (the offline mirror), +# so nothing is copied into the image's own cache first. +# -K: a keyring initialised for this image rather than the builder's copied in; +# the keyring packages populate it from their install scripts. +# -M: the builder's mirrorlist means nothing to an installed system; the +# installer writes the target's own. +pacstrap -C "$pacman_conf" -c -K -M "$root" "${packages[@]}" +stop_image_gpg_agents + +# Per-machine identity is the installer's job, never the image's: it runs +# systemd-machine-id-setup on the target after unpacking. Leave the id +# uninitialised rather than shipping one every install would share. +: >"$root/etc/machine-id" + +# /var/log is its own subvolume on installed systems and mounts over this +# directory, hiding anything left here. The orchestrator copies the image's +# pacman.log into that subvolume so the installed system keeps its history; +# everything else from the build is noise. +find "$root/var/log" -mindepth 1 ! -name pacman.log -delete + +# Nothing downloaded, nothing to keep; and no build-host resolv.conf. +rm -rf "$root/var/cache/pacman/pkg"/* "$root/etc/resolv.conf" + +if [[ -n ${OMARCHY_IMAGE_LOCALDB_COPY:-} ]]; then + rm -rf "$OMARCHY_IMAGE_LOCALDB_COPY" + mkdir -p "$OMARCHY_IMAGE_LOCALDB_COPY" + cp -a "$root/var/lib/pacman/local" "$OMARCHY_IMAGE_LOCALDB_COPY/" +fi + +installed=$(find "$root/var/lib/pacman/local" -mindepth 1 -maxdepth 1 -type d | wc -l) +echo "Root image holds $installed packages" + +sync +btrfs property set -ts "$root" ro true +rm -f "$output" +mkdir -p "$(dirname "$output")" +btrfs send -q --compressed-data -f "$output" "$root" +echo "Root image stream: $(du -h "$output" | cut -f1) at $output" diff --git a/builder/image.packages b/builder/image.packages new file mode 100644 index 000000000..a9c11c1c4 --- /dev/null +++ b/builder/image.packages @@ -0,0 +1,33 @@ +# Packages the installer used to add on top of the package lists at install +# time, now baked into the root image by build-root-image.sh. Hardware-specific +# packages (kernels, microcode, audio firmware, Tailscale) stay out of the image +# and are installed per machine; see hardware_packages in build-iso.sh. + +# archinstall's base set (Installer.__packages__) and what it adds for a btrfs +# root with zram swap. +base +sudo +linux-firmware +mkinitcpio +btrfs-progs +dosfstools +zram-generator + +# archinstall's PipeWire selection (AudioApp.pipewire_packages). +pipewire +pipewire-alsa +pipewire-jack +pipewire-pulse +gst-plugin-pipewire +libpulse +wireplumber + +# The orchestrator's early bootstrap set: Limine, the keyring, and what seeds +# /etc/skel before useradd. +base-devel +git +limine +efibootmgr +omarchy-keyring +lua51 +luarocks diff --git a/configs/airootfs/root/.automated_script.sh b/configs/airootfs/root/.automated_script.sh index 16904f879..ec19163a1 100644 --- a/configs/airootfs/root/.automated_script.sh +++ b/configs/airootfs/root/.automated_script.sh @@ -57,28 +57,40 @@ if [[ ${OMARCHY_INSTALL_DEBUG:-} == "1" ]]; then echo "================================" fi -# Warm the page cache for the bundled packages while the user works through the -# wizard. The install reads ~3GB out of the offline mirror, and pacman consumes -# it at only ~33MB/s, so on media slower than that the install is read-bound and -# every byte cached here is a byte it never waits for. On faster media this costs -# nothing but otherwise-idle bandwidth: the medium is untouched while the user -# types, and the target disk it writes to later is a different device. +# Warm the page cache for the root image and bundled packages while the user +# works through the wizard. The install streams ~5GB of root image into btrfs +# receive and then pacstraps a few packages out of the offline mirror; on media +# slower than the unpack the install is read-bound and every byte cached here +# is a byte it never waits for. On faster media this costs nothing but +# otherwise-idle bandwidth: the medium is untouched while the user types, and +# the target disk it writes to later is a different device. # # Clean page cache only, so the kernel reclaims it under pressure instead of # OOMing, and a budget so small machines never evict what was just warmed. # Set OMARCHY_NO_PREFETCH=1 to A/B the same ISO with this disabled. warm_offline_mirror() { local mirror=/var/cache/omarchy/mirror/offline + local image=/var/cache/omarchy/rootfs/omarchy-root.btrfs local budget_kb spent_kb=0 size_kb path [[ ${OMARCHY_NO_PREFETCH:-} == 1 ]] && return 0 - [[ -d $mirror ]] || return 0 budget_kb=$(($(awk '/^MemAvailable:/ { print $2 }' /proc/meminfo) / 2)) ((budget_kb > 262144)) || return 0 - # Largest first: the install reads most of the mirror, so when the budget - # cannot cover all of it this still front-loads the bytes that dominate. + # The image first, and only as much of it as fits: btrfs receive reads it + # front to back, so the leading bytes are the ones worth having cached. + if [[ -f $image ]]; then + size_kb=$(du -k -- "$image" | cut -f1) + ((size_kb > budget_kb)) && size_kb=$budget_kb + head -c "$((size_kb * 1024))" -- "$image" >/dev/null 2>&1 || true + spent_kb=$size_kb + fi + + [[ -d $mirror ]] || return 0 + + # Then the mirror, largest first: when the budget cannot cover all of it + # this still front-loads the bytes that dominate. while read -r size_kb path; do ((spent_kb + size_kb > budget_kb)) && continue cat -- "$path" >/dev/null 2>&1 || true diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 0048b4cba..50f9aec79 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -175,6 +175,9 @@ FINISH_TOP_ROW=2 # the install). libalpm makes one directory per package under # /var/lib/pacman/local; the denominator comes from # build-iso.sh, so there is no second package list to drift. +# The root image unpack that precedes the package delta publishes +# phase_progress (stream bytes consumed) in the state file; it drives +# the same band until the local db fills in when the unpack lands. # # Per-mille because a whole-percent asymptote over a 2-point band has two # reachable values and stops moving. No give-up predicate: two discontinuous @@ -293,7 +296,7 @@ count_packages() { # here would land on the TTY between the cursor escape and the bar. install_progress() { local phase finished index total target - local lo hi tau span t pos work gap n pkg_signal=0 + local lo hi tau span t pos work gap n pkg_signal=0 progress_pm local -a state PROGRESS_PM=$POS @@ -304,11 +307,14 @@ install_progress() { ((.finished_at // 0) | tonumber? // 0 | floor), ((.current_index // -1) | tonumber? // -1 | floor), ((.total_phases // 0) | tonumber? // 0 | floor), - (.target // "/mnt") + (.target // "/mnt"), + ((.phase_progress // -1) | tonumber? // -1 | . * 1000 | floor) ' "$STATE_FILE" 2>/dev/null) - (( ${#state[@]} == 5 )) || return 0 + (( ${#state[@]} == 6 )) || return 0 phase="${state[0]}" finished="${state[1]}" index="${state[2]}" total="${state[3]}" target="${state[4]}" + progress_pm="${state[5]}" + [[ $progress_pm =~ ^-?[0-9]+$ ]] || progress_pm=-1 [[ -n $target ]] || target="/mnt" if (( finished > 0 )) || [[ $phase == "Installation complete" ]]; then @@ -393,6 +399,11 @@ install_progress() { work=$(( lo + (span - 1 - PKG_TAIL_PM) * PKG_SEEN / PKG_TOTAL + PKG_BONUS )) (( work > pos )) && pos=$work fi + if (( progress_pm >= 0 )); then + (( progress_pm > 1000 )) && progress_pm=1000 + work=$(( lo + (span - 1 - PKG_TAIL_PM) * progress_pm / 1000 )) + (( work > pos )) && pos=$work + fi (( pos > hi - 1 )) && pos=$(( hi - 1 )) (( pos > POS )) && POS=$pos diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/archinstall_adapter.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/archinstall_adapter.py index c6a279bb5..623535293 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/archinstall_adapter.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/archinstall_adapter.py @@ -17,6 +17,9 @@ inst.generate_key_files() # encrypted only inst.set_mirrors(handler, mirror_config, on_target=False) inst.minimal_installation(...) # base + linux pacstrap + # (we unpack the root image + # and run install_base_delta + # instead; see below) inst.set_mirrors(handler, mirror_config, on_target=True) inst.setup_swap(algo=...) inst.create_users(users) @@ -50,6 +53,7 @@ from archinstall.lib.mirror.mirror_handler import MirrorListHandler from archinstall.lib.models import Bootloader from archinstall.lib.models.device import DiskLayoutType, EncryptionType +from archinstall.lib.pacman.config import PacmanConfig from archinstall.lib.models.users import User from .ui import info @@ -142,6 +146,100 @@ def open_installer( yield installer +def target_has_package(target: Path, name: str) -> bool: + """Whether pacman's local db in the target records `name` as installed.""" + local_db = target / "var" / "lib" / "pacman" / "local" + for entry in local_db.glob(f"{name}-*"): + desc = entry / "desc" + if not desc.is_file(): + continue + lines = desc.read_text(errors="ignore").splitlines() + try: + if lines[lines.index("%NAME%") + 1] == name: + return True + except (ValueError, IndexError): + continue + return False + + +def install_base_delta( + installer: Installer, + arch_config: ArchConfig, + *, + hostname: str | None, + locale_config, +) -> None: + """Installer.minimal_installation for a target that already holds the root + image: everything it does around its base pacstrap, with the pacstrap + reduced to the base packages the image does not carry (the kernel and the + CPU microcode). + + Mirrors archinstall 4.4's minimal_installation step for step so the target + ends up as that call would leave it: filesystem/encryption preparation + (which also decides the mkinitcpio hooks), microcode detection, pacman.conf + handling, vconsole, hostname, locale, and the helper flags later Installer + methods look at. LVM layouts are not supported by the image path.""" + disk_config = installer._disk_config + if disk_config.lvm_config: + raise RuntimeError("root image install does not support LVM layouts") + + for mod in disk_config.device_modifications: + for part in mod.partitions: + if part.fs_type is None: + continue + installer._prepare_fs_type(part.fs_type, part.mountpoint) + if part in installer._disk_encryption.partitions: + installer._prepare_encrypt() + + if ucode := installer._get_microcode(): + (installer.target / "boot" / ucode).unlink(missing_ok=True) + installer._base_packages.append(ucode.stem) + + mirror_config = arch_config.mirror_config + pacman_conf = PacmanConfig(installer.target) + pacman_conf.enable(mirror_config.optional_repositories if mirror_config else []) + pacman_conf.apply() + + if locale_config: + installer.set_vconsole(locale_config) + + delta = [pkg for pkg in installer._base_packages if not target_has_package(installer.target, pkg)] + if delta: + installer.pacman.strap(delta) + installer._helper_flags["base-strapped"] = True + + pacman_conf.persist() + if arch_config.pacman_config: + pacman_conf.configure(arch_config.pacman_config) + + if not installer._disable_fstrim: + installer.enable_periodic_trim() + + if hostname: + installer.set_hostname(hostname) + + if locale_config: + installer.set_locale(locale_config) + installer.set_keyboard_language(locale_config.kb_layout) + + installer._helper_flags["base"] = True + + for function in installer.post_base_install: + function(installer) + + +def setup_zram_swap(installer: Installer) -> None: + """Installer.setup_swap without its pacman.strap('zram-generator'): the + root image carries the package. Enables the zram unit and records that + zram is on, as archinstall 4.4 does. Its /etc zram-generator.conf is not + reproduced: omarchy-settings ships the tuning as a vendor drop-in and the + orchestrator removed archinstall's copy anyway.""" + if not target_has_package(installer.target, "zram-generator"): + installer.pacman.strap("zram-generator") + installer.enable_service("systemd-zram-setup@zram0.service") + installer._zram_enabled = True + + def is_encrypted(arch_config: ArchConfig) -> bool: disk = arch_config.disk_config if not disk or not disk.disk_encryption: @@ -233,10 +331,30 @@ def install_applications(installer: Installer, arch_config: ArchConfig) -> None: users = arch_config.auth_config.users if arch_config.auth_config else None handler = _application_handler() install_applications_method = handler.install_applications - if _method_accepts_users(install_applications_method): - install_applications_method(installer, app_config, users) - else: - install_applications_method(installer, app_config) + + # The application installers strap their package sets unconditionally. + # The root image already carries those (PipeWire and friends) and the + # offline mirror no longer does, and pacstrap --needed still has to + # resolve every target before it can skip it. Strap only what the target + # lacks (the hardware-detected firmware), through the installer instance + # the handlers call into. + original_strap = installer.add_additional_packages + + def strap_missing(packages: str | list[str]) -> None: + if isinstance(packages, str): + packages = [packages] + missing = [pkg for pkg in packages if not target_has_package(installer.target, pkg)] + if missing: + original_strap(missing) + + installer.add_additional_packages = strap_missing # type: ignore[method-assign] + try: + if _method_accepts_users(install_applications_method): + install_applications_method(installer, app_config, users) + else: + install_applications_method(installer, app_config) + finally: + del installer.add_additional_packages def root_user(arch_config: ArchConfig) -> User | None: diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py index 9b6ebf5b5..21808a940 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py @@ -19,9 +19,10 @@ def build_phases(ctx: InstallContext): """Phase order. Each entry is (display name, callable taking InstallContext). - The ordering is the whole point of this orchestrator: package-install - hooks (limine-mkinitcpio-hook, in particular) and useradd happen at - points where their prerequisites are guaranteed to be in place. + The ordering is the whole point of this orchestrator: the root image + unpack, package-install hooks (limine-mkinitcpio-hook, in particular) and + useradd happen at points where their prerequisites are guaranteed to be + in place. Full-disk and protected installs use the same phase sequence. The configurator only changes the JSON input: full-disk asks archinstall to diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases.py index cb8424470..46befebae 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases.py @@ -34,13 +34,16 @@ def run(ctx: InstallContext, phases: list[tuple[str, PhaseFn]]) -> None: "current_phase": "Starting installation", "phases": [], } - _write_state(state_path, state) + write_state(state_path, state) for index, (name, fn) in enumerate(phases): state["current_index"] = index state["current_phase"] = name state["phase_started_at"] = time.time() - _write_state(state_path, state) + # Phases that can measure themselves (the root image unpack) publish + # phase_progress for the dashboard; it means nothing across phases. + state.pop("phase_progress", None) + write_state(state_path, state) info(f"› {name}") started = time.time() @@ -54,7 +57,7 @@ def run(ctx: InstallContext, phases: list[tuple[str, PhaseFn]]) -> None: "elapsed": elapsed, "error": str(exc), }) - _write_state(state_path, state) + write_state(state_path, state) error(f"Phase '{name}' failed after {elapsed:.1f}s: {exc}") traceback.print_exc() @@ -62,7 +65,7 @@ def run(ctx: InstallContext, phases: list[tuple[str, PhaseFn]]) -> None: elapsed = time.time() - started state["phases"].append({"name": name, "status": "ok", "elapsed": elapsed}) - _write_state(state_path, state) + write_state(state_path, state) state["current_index"] = max(len(phases) - 1, 0) state["current_phase"] = "Installation complete" @@ -71,11 +74,11 @@ def run(ctx: InstallContext, phases: list[tuple[str, PhaseFn]]) -> None: # acceptance runs rather than only by watching a bar creep. state["installed_packages"] = _installed_package_count(ctx.target) state["expected_packages"] = _expected_package_count() - _write_state(state_path, state) + write_state(state_path, state) timing_path = ctx.target / "var" / "log" / "omarchy-install-timing.json" timing_path.parent.mkdir(parents=True, exist_ok=True) - _write_state(timing_path, state) + write_state(timing_path, state) def _installed_package_count(target: Path) -> int: @@ -97,7 +100,7 @@ def _expected_package_count() -> int: return 0 -def _write_state(path: Path, state: dict) -> None: +def write_state(path: Path, state: dict) -> None: # Dashboard polls this file while phases update it. Write atomically so the # reader never observes a truncated/partial JSON document and resets UI. tmp = path.with_name(f".{path.name}.tmp") diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 27451399d..0b5cf9eb1 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -8,8 +8,8 @@ prepare_install_target → verify pre-mounted target/ESP when the JSON uses pre_mounted_config; no-op for full-disk installs arch_install_system → one archinstall flow for partition/mount-or-use, - base install, early Omarchy packages, Limine setup, - useradd, runtime Omarchy packages, fstab + root image unpack (btrfs receive), per-machine + package delta, Limine setup, useradd, fstab configure_hibernation → root-owned swap/resume drop-ins run_system_finalizer → arch-chroot root omarchy-apply-system, including Snapper finalize_limine_boot → final Limine config/UKI build after hardware drop-ins @@ -23,6 +23,7 @@ from __future__ import annotations import hashlib +import json import os import re import shutil @@ -35,6 +36,7 @@ from . import archinstall_adapter as arch from .context import InstallContext from .keyboard import configure_keyboard +from .phases import write_state from .ui import error, info @@ -116,43 +118,26 @@ def _omarchy_nvim_package() -> str: return _package_targets()["nvim"] -# Packages installed BEFORE useradd. The selected omarchy-settings package and -# omarchy-nvim populate /etc/skel so the user's home gets seeded correctly, and -# omarchy-settings also ships the limine/snapper configs. Target-side setup -# commands are installed later by the selected Omarchy runtime package and -# executed in chroot. -EARLY_BOOTSTRAP_BASE_PACKAGES = [ - "base-devel", - "git", - "limine", - "efibootmgr", - "omarchy-keyring", -] +# The root image: a `btrfs send --compressed-data` stream of the invariant +# target system (build-root-image.sh), unpacked onto the target in place of +# pacstrapping it. The subvolume name is what build-root-image.sh sends. +ROOT_IMAGE_STREAM = Path("/var/cache/omarchy/rootfs/omarchy-root.btrfs") +ROOT_IMAGE_SUBVOLUME = "omarchy-root" -# Install LuaRocks before omarchy-nvim pulls in lua51-lpeg. Arch's lua-luarocks -# post_install script tries to rebuild manifests for existing rocks trees before -# the unversioned luarocks-admin command exists if both arrive in the wrong -# transaction order. Splitting this transaction avoids the harmless but noisy -# "luarocks-admin: command not found" line during ISO installs. -EARLY_LUAROCKS_PACKAGES = [ - "lua51", - "luarocks", -] +# Packages the image must carry for the rest of the install to work: Limine +# setup reads the settings package's limine config, useradd copies the skel the +# settings and nvim packages populate, and the target-side setup commands come +# from the runtime package. Checked right after unpacking so a mismatched +# image fails here with a clear message instead of three phases later. +ROOT_IMAGE_REQUIRED_PACKAGES = ("limine", "omarchy-keyring") -def _early_bootstrap_packages() -> list[str]: - return [*EARLY_BOOTSTRAP_BASE_PACKAGES, _omarchy_settings_package()] - - -def _early_user_seed_packages() -> list[str]: - return [_omarchy_nvim_package()] - - -def _early_packages() -> list[str]: +def _root_image_required_packages() -> list[str]: return [ - *_early_bootstrap_packages(), - *EARLY_LUAROCKS_PACKAGES, - *_early_user_seed_packages(), + *ROOT_IMAGE_REQUIRED_PACKAGES, + _omarchy_runtime_package(), + _omarchy_settings_package(), + _omarchy_nvim_package(), ] @@ -213,7 +198,9 @@ def prepare_install_target(ctx: InstallContext) -> None: def arch_install_system(ctx: InstallContext) -> None: - """Install the target system from the archinstall JSON. + """Install the target system: archinstall partitions and mounts per the + configurator JSON, the root image is unpacked onto the mounted layout, and + archinstall finishes with the per-machine package delta, users, and fstab. The phase sequence is the same for full-disk and protected installs. The JSON decides whether archinstall should create/mount a disk layout or use @@ -240,6 +227,11 @@ def arch_install_system(ctx: InstallContext) -> None: skip_wkd=True, ) + # Before anything writes into the target: the image replaces the + # (empty) root subvolume archinstall created, and everything written + # there first would go with it. + _install_root_image(ctx) + if not pre_mounted and arch.is_encrypted(config): installer.generate_key_files() @@ -249,23 +241,19 @@ def arch_install_system(ctx: InstallContext) -> None: _mount_offline_package_cache(ctx) _mask_mkinitcpio_pacman_hooks(ctx) try: - info("› installing base system (mkinitcpio deferred to final Limine UKI build)") + info("› installing per-machine packages (mkinitcpio deferred to final Limine UKI build)") # An empty kb_layout makes archinstall's set_keyboard_language skip # booting the target in a container just to run localectl; the # keymap is configured offline right after instead. kb_layout = config.locale_config.kb_layout if config.locale_config else "" - installer.minimal_installation( - optional_repositories=( - config.mirror_config.optional_repositories - if config.mirror_config else [] - ), - mkinitcpio=False, + arch.install_base_delta( + installer, + config, hostname=config.hostname, locale_config=( replace(config.locale_config, kb_layout="") if config.locale_config else None ), - pacman_config=config.pacman_config, ) if not configure_keyboard(installer.target, kb_layout): @@ -275,10 +263,9 @@ def arch_install_system(ctx: InstallContext) -> None: installer.set_mirrors(mirror_handler, config.mirror_config, on_target=True) if config.swap and config.swap.enabled: - installer.setup_swap(algo=config.swap.algorithm) + arch.setup_zram_swap(installer) _drop_archinstall_zram_conf(ctx) - _install_early_packages(installer) _configure_limine_boot(ctx, installer, config) info("› creating user (with /etc/skel populated)") @@ -286,12 +273,12 @@ def arch_install_system(ctx: InstallContext) -> None: installer.create_users(config.auth_config.users) if config.app_config: - info("› installing archinstall application selections") + # The image carries the PipeWire packages; this adds the audio + # firmware archinstall's hardware detection asks for and wires + # the per-user PipeWire units. + info("› applying archinstall application selections") arch.install_applications(installer, config) - info("› installing Omarchy runtime + omarchy-base.packages") - installer.add_additional_packages(_runtime_package_list(ctx)) - # Tailscale is bundled in the offline mirror but only installed # when an autoinstall drive staged an auth key; must happen here, # while the mirror is still bind-mounted, not in the phase that @@ -317,6 +304,170 @@ def arch_install_system(ctx: InstallContext) -> None: installer.genfstab() +# ───────────────────────────────────────────────────────────────────────────── +# Root image: btrfs receive the build-time system into the target filesystem +# and make it the @ subvolume. +# +# archinstall (or the configurator, for protected installs) has created and +# mounted the subvolume layout by the time this runs: @ at the target, with +# @home, @log, @pkg and the ESP mounted inside it. `btrfs receive` can only +# create a new subvolume, never fill an existing one, so the image is received +# at the filesystem's top level, snapshotted writable, and swapped in for the +# empty @ while the layout is unmounted; then the layout is mounted again +# exactly as it was. The mount table is replayed from findmnt rather than +# asking archinstall to mount a second time, which would also unlock LUKS a +# second time; the mapper stays open throughout. +# ───────────────────────────────────────────────────────────────────────────── + +def _findmnt_mounts(root: Path) -> list[dict]: + """Every mount at or below root, parents before children, as dicts with + target/source/fstype/options.""" + res = subprocess.run( + ["findmnt", "-R", "-J", "-o", "TARGET,SOURCE,FSTYPE,OPTIONS", str(root)], + capture_output=True, text=True, check=True, + ) + flat: list[dict] = [] + + def walk(nodes): + for node in nodes: + flat.append({k: node.get(k) for k in ("target", "source", "fstype", "options")}) + walk(node.get("children") or []) + + walk(json.loads(res.stdout).get("filesystems") or []) + return flat + + +def _remount_option_string(options: str) -> str: + # subvolid refers to the subvolume replaced here; subvol= names it. + return ",".join(opt for opt in options.split(",") if not opt.startswith("subvolid=")) + + +def _install_root_image(ctx: InstallContext) -> None: + target = ctx.target + if not ROOT_IMAGE_STREAM.is_file(): + raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") + + mounts = _findmnt_mounts(target) + if not mounts or Path(mounts[0]["target"]) != target: + raise RuntimeError(f"{target} is not a mountpoint") + root_mount = mounts[0] + if root_mount["fstype"] != "btrfs": + raise RuntimeError(f"root image install needs a btrfs target root, got {root_mount['fstype']}") + root_options = (root_mount["options"] or "").split(",") + if not any(opt in ("subvol=/@", "subvol=@") for opt in root_options): + raise RuntimeError(f"root image install needs the target root on the @ subvolume, got {root_mount['options']}") + device = (root_mount["source"] or "").split("[")[0] + if not device: + raise RuntimeError(f"could not determine the btrfs device backing {target}") + + top = ctx.state_dir / "image-top" + top.mkdir(parents=True, exist_ok=True) + subprocess.run(["mount", "-o", "subvolid=5", device, str(top)], check=True) + try: + received = top / ROOT_IMAGE_SUBVOLUME + if received.exists(): + subprocess.run(["btrfs", "subvolume", "delete", str(received)], check=True, capture_output=True) + + info(f"› unpacking root image ({ROOT_IMAGE_STREAM.stat().st_size >> 20} MiB stream)") + _receive_root_image(ctx, top) + + staged = top / "@.image" + if staged.exists(): + subprocess.run(["btrfs", "subvolume", "delete", str(staged)], check=True, capture_output=True) + subprocess.run(["btrfs", "subvolume", "snapshot", str(received), str(staged)], check=True, capture_output=True) + subprocess.run(["btrfs", "subvolume", "delete", str(received)], check=True, capture_output=True) + + # The image's pacman.log ends up under the @log mount; carry it over so + # the installed system's log starts with the packages it was built from. + image_log = staged / "var" / "log" / "pacman.log" + log_subvol = top / "@log" + if image_log.is_file() and log_subvol.is_dir(): + shutil.copy2(image_log, log_subvol / "pacman.log") + + info("› making the image the root subvolume") + _umount_tree(target) + try: + subprocess.run(["btrfs", "subvolume", "delete", str(top / "@")], check=True, capture_output=True) + staged.rename(top / "@") + finally: + for mount in mounts: + mountpoint = Path(mount["target"]) + mountpoint.mkdir(parents=True, exist_ok=True) + source = (mount["source"] or "").split("[")[0] + subprocess.run( + ["mount", "-t", mount["fstype"], "-o", _remount_option_string(mount["options"] or ""), + source, str(mountpoint)], + check=True, + ) + finally: + subprocess.run(["umount", str(top)], check=False, capture_output=True) + + missing = [pkg for pkg in _root_image_required_packages() if not arch.target_has_package(target, pkg)] + if missing: + raise RuntimeError(f"root image lacks required packages: {', '.join(missing)}") + + # Per-machine identity the image deliberately ships without. + subprocess.run(["systemd-machine-id-setup", f"--root={target}"], check=True, capture_output=True) + + +def _umount_tree(root: Path, attempts: int = 20) -> None: + """umount -R with a few retries: the dashboard polls the target's pacman + db from another process, and a poll landing mid-unmount is EBUSY.""" + for attempt in range(1, attempts + 1): + res = subprocess.run(["umount", "-R", str(root)], capture_output=True, text=True) + if res.returncode == 0: + return + if attempt == attempts: + raise RuntimeError(f"could not unmount {root}: {res.stderr.strip()}") + time.sleep(0.25) + + +def _receive_root_image(ctx: InstallContext, top: Path) -> None: + """Pipe the stream into btrfs receive, publishing progress for the + dashboard as a fraction of stream bytes consumed.""" + total = ROOT_IMAGE_STREAM.stat().st_size + errors = ctx.state_dir / "btrfs-receive.err" + with errors.open("w") as err: + proc = subprocess.Popen( + ["btrfs", "receive", "-q", str(top)], + stdin=subprocess.PIPE, + stderr=err, + ) + assert proc.stdin is not None + sent = 0 + last_report = 0.0 + try: + with ROOT_IMAGE_STREAM.open("rb") as stream: + while chunk := stream.read(8 << 20): + proc.stdin.write(chunk) + sent += len(chunk) + now = time.monotonic() + if now - last_report >= 0.5: + _write_phase_progress(ctx, sent / total if total else 1.0) + last_report = now + proc.stdin.close() + except BrokenPipeError: + pass + if proc.wait() != 0: + raise RuntimeError(f"btrfs receive failed: {errors.read_text(errors='replace').strip()}") + _write_phase_progress(ctx, 1.0) + + +def _write_phase_progress(ctx: InstallContext, fraction: float) -> None: + """Record how far the current phase has come (0..1) in the state file the + dashboard polls. Best effort: progress display must never fail an install.""" + state_path = ctx.state_dir / "state.json" + try: + state = json.loads(state_path.read_text()) + except (OSError, ValueError): + return + state["phase_progress"] = max(0.0, min(1.0, fraction)) + try: + write_state(state_path, state) + except OSError: + pass + + def _configure_limine_boot(ctx: InstallContext, installer, config) -> None: if not arch.bootloader_enabled(config): return @@ -624,20 +775,6 @@ def _drop_archinstall_zram_conf(ctx: InstallContext) -> None: zram_conf.unlink(missing_ok=True) -def _install_early_packages(installer) -> None: - bootstrap_packages = _early_bootstrap_packages() - user_seed_packages = _early_user_seed_packages() - - info(f"› installing early Omarchy packages: {', '.join(bootstrap_packages)}") - installer.add_additional_packages(bootstrap_packages) - - info(f"› installing LuaRocks prerequisites: {', '.join(EARLY_LUAROCKS_PACKAGES)}") - installer.add_additional_packages(EARLY_LUAROCKS_PACKAGES) - - info(f"› installing user seed packages: {', '.join(user_seed_packages)}") - installer.add_additional_packages(user_seed_packages) - - def _mount_offline_package_cache(ctx: InstallContext) -> None: """Let pacstrap consume bundled packages without copying them first. @@ -722,28 +859,6 @@ def _unmask_mkinitcpio_pacman_hooks( info(f"warning: failed to restore pacman hook mask for {name}: {exc}") -def _runtime_package_list(ctx: InstallContext) -> list[str]: - """Selected Omarchy runtime package + every package in the ISO-bundled - base package list that isn't already installed early.""" - base_pkgs_file = Path("/usr/share/omarchy-iso/omarchy-base.packages") - pkgs = [_omarchy_runtime_package()] - already_installed = set(_early_packages()) | { - _omarchy_runtime_package(), - _omarchy_settings_package(), - _omarchy_nvim_package(), - "omarchy", - "omarchy-settings", - "omarchy-nvim", - } - for raw in base_pkgs_file.read_text().splitlines(): - s = raw.strip() - if not s or s.startswith("#"): - continue - if s not in already_installed and s not in pkgs: - pkgs.append(s) - return pkgs - - # ───────────────────────────────────────────────────────────────────────────── # Install intent helpers: normalize the Omarchy-specific part of the # configurator JSON so full-disk and pre-mounted installs feed the same boot diff --git a/configs/profiledef.sh b/configs/profiledef.sh index 268a3cea6..0360ed95e 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -12,9 +12,10 @@ bootmodes=('bios.syslinux' 'uefi.grub') arch="x86_64" pacman_conf="pacman-offline.conf" airootfs_image_type="squashfs" -# Package archives in the offline mirror are already zstd-compressed. Storing -# them in an outer stream saves little space but makes pacman decompress the -# outer layer while hashing and extracting every package during installation. +# Package archives in the offline mirror are already zstd-compressed, and so is +# the root image stream (btrfs send --compressed-data). Storing them in an outer +# stream saves little space but makes pacman decompress the outer layer while +# hashing and extracting every package, and btrfs receive while unpacking. # # Everything else in the live root is zstd rather than xz. Squashfs decompresses # on the page-fault path through a single stream (CONFIG_SQUASHFS_DECOMP_SINGLE), @@ -27,6 +28,7 @@ airootfs_image_tool_options=( '-Xcompression-level' '19' '-b' '1M' '-action' 'uncompressed@subpathname(var/cache/omarchy/mirror/offline)' + '-action' 'uncompressed@subpathname(var/cache/omarchy/rootfs)' ) bootstrap_tarball_compression=('zstd' '-c' '-T0' '--auto-threads=logical' '--long' '-19') file_permissions=( From 0cefda38dcc41cf5081f5bd76d7587e023cddd4e Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 21 Aug 2026 13:25:29 +0200 Subject: [PATCH 02/67] Force zstd compression at level 15 when building the root image btrfs's incompressibility heuristic skips a lot of data in this tree that zstd compresses fine, and the level only costs build time: btrfs receive stores the extents as they arrive. Against plain compress=zstd:3 the send stream goes from 5.2GB to 3.3GB and the installed root from 5.4GB to 4.0GB, at the same ~17s receive time; level 15 adds under two minutes to the ISO build. btrfs clamps anything above 15. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01P9b7oP8j2GA6sZZ9e8aJYv --- bin/omarchy-iso-boot | 2 +- builder/build-root-image.sh | 11 ++++++----- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/bin/omarchy-iso-boot b/bin/omarchy-iso-boot index ac6e1d488..62fe04a4b 100755 --- a/bin/omarchy-iso-boot +++ b/bin/omarchy-iso-boot @@ -152,7 +152,7 @@ fi # Boot from ISO before we've installed, then boot from drive after qemu-system-x86_64 \ -cpu host -enable-kvm -machine q35,accel=kvm \ - -smp "$(($(nproc) < 8 ? $(nproc) : 8))" \ + -smp 16 \ -m 8192 \ -drive if=pflash,format=raw,readonly=on,file=/usr/share/edk2/x64/OVMF_CODE.4m.fd \ -drive if=pflash,format=raw,file="$OVMF_VARS" \ diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh index 54e40fdb4..c196e3556 100755 --- a/builder/build-root-image.sh +++ b/builder/build-root-image.sh @@ -44,11 +44,12 @@ if [[ ! -r $pacman_conf ]]; then exit 1 fi -# Must match the compress= option the installer mounts the target with -# (configurator JSON mount_options / disk-partitioning.sh): the stream carries -# the extents as compressed here, and the installed system keeps writing new -# data at its own mount level either way. -IMAGE_COMPRESS="compress=zstd:3" +# Forced zstd at a higher level than the installer's own compress=zstd (level +# 3): btrfs's incompressibility heuristic declines a lot of data in this tree +# that zstd handles fine, and the level only costs build time. btrfs receive +# stores the extents as they arrive, so neither choice affects install speed; +# the installed system writes new data at its own mount option either way. +IMAGE_COMPRESS="compress-force=zstd:15" # Name of the subvolume inside the stream. The orchestrator looks for this name # after `btrfs receive` (phases_impl.ROOT_IMAGE_SUBVOLUME). IMAGE_SUBVOLUME="omarchy-root" From 484cf7a1e11128638aa28cec3a309a80fa47a08a Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 21 Aug 2026 13:54:28 +0200 Subject: [PATCH 03/67] Drive the free-space install in omarchy-iso-test --free-space installs into the unallocated tail of a synthetic Windows-style disk: a FAT32 ESP and an ext4 data partition with a marker file, then ~76GiB of free space. The fixture is built without root (parted on the raw file, mkfs at the partition offsets), so unlike omarchy-iso-test-windows-disk it carries no EFI/Microsoft directory; the configurator's free-space mechanics are the same either way. The wizard is driven through the mode picker and the free-space confirm, and once the installed system is up the harness checks from inside it that both pre-existing partitions and the marker survived. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01P9b7oP8j2GA6sZZ9e8aJYv --- bin/omarchy-iso-test | 88 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 86 insertions(+), 2 deletions(-) diff --git a/bin/omarchy-iso-test b/bin/omarchy-iso-test index 281cf56ac..df402d45b 100755 --- a/bin/omarchy-iso-test +++ b/bin/omarchy-iso-test @@ -16,6 +16,9 @@ # --encrypt Drive the encrypted install flow (default: unencrypted) # --provision Drive the deferred-provisioning install flow: no user during install, the # first boot runs omarchy-provision-owner and creates it there +# --free-space Install into the free space of a synthetic Windows-style +# disk (existing ESP + data partition kept) and verify the +# existing partitions survive # --reuse-base Skip the install phase if a base image already exists # --install-only Stop after producing the installed base image # --sync-omarchy DIR Push DIR's test/acceptance into the guest before running @@ -39,6 +42,7 @@ INSTALL_TIMEOUT=2400 BOOT_TIMEOUT=600 ENCRYPT=false PROVISION=false +FREE_SPACE=false REUSE_BASE=false INSTALL_ONLY=false KEEP_RUNNING=false @@ -54,6 +58,7 @@ while (($#)); do case "$1" in --encrypt) ENCRYPT=true ;; --provision) PROVISION=true ;; + --free-space) FREE_SPACE=true ;; --reuse-base) REUSE_BASE=true ;; --install-only) INSTALL_ONLY=true ;; --keep-running) KEEP_RUNNING=true ;; @@ -91,6 +96,7 @@ OVMF_VARS_TEMPLATE="/usr/share/edk2/x64/OVMF_VARS.4m.fd" BASE_NAME="$(basename "$ISO" .iso)" $ENCRYPT && BASE_NAME+="-encrypted" $PROVISION && BASE_NAME+="-provision" +$FREE_SPACE && BASE_NAME+="-free-space" BASE_DIR="$REPO_DIR/test-runs/$BASE_NAME" RUN_DIR="$BASE_DIR/runs/$(date +%Y%m%d-%H%M%S)" BASE_DISK="$BASE_DIR/base.qcow2" @@ -685,6 +691,25 @@ drive_configurator() { capture_console "success-installer-10-install-disk" press ret # the single virtio disk + if $FREE_SPACE; then + # The picker only appears when the disk has usable free space on UEFI. + wait_for_screen "installation mode" 60 + capture_console "success-installer-11-install-mode" + press down # "Free space install (alongside existing data)" + press ret + # BitLocker/ESP checks and the free-space analysis run, then the confirm. + wait_for_screen "of free space" 120 + capture_console "success-installer-12-free-space-confirm-encrypted" + if ! $ENCRYPT; then + press ctrl-c # toggles the confirm to the unencrypted flow + wait_for_screen "without encryption" 30 + capture_console "success-installer-13-free-space-confirm-unencrypted" + fi + press ret + sleep 2 + capture_console "success-installer-14-install-started" + return 0 + fi wait_for_screen "installation mode\|recovery possible" 60 if ocr_screen | grep -qi "installation mode"; then capture_console "success-installer-11-install-mode" @@ -967,13 +992,71 @@ establish_session() { # -------------------------------------------------------------------- phases +# A synthetic Windows-style disk for the free-space install: a FAT32 ESP and +# an ext4 data partition carrying a marker file, followed by ~76GiB of +# unallocated space. Built without root: parted partitions the raw file, the +# filesystems are made straight into it at their offsets. Unlike +# omarchy-iso-test-windows-disk it has no EFI/Microsoft directory (that needs +# mtools or a mount), so the configurator reports no Windows ESP; the +# free-space mechanics it exercises are the same either way. +create_free_space_disk() { + local disk="$1" raw="$1.raw" populate="$RUN_DIR/.preserve" + local esp_start_mib=1 esp_size_mib=512 data_end_mib=$((20 * 1024)) + + rm -f "$disk" "$raw" + truncate -s 96G "$raw" + parted --script "$raw" \ + mklabel gpt \ + mkpart WINDOWS_ESP fat32 "${esp_start_mib}MiB" "$((esp_start_mib + esp_size_mib))MiB" \ + set 1 esp on \ + mkpart WINDOWS_DATA ext4 "$((esp_start_mib + esp_size_mib))MiB" "${data_end_mib}MiB" + + mkfs.fat -F32 -n WINDOWS_ESP -C "$RUN_DIR/.esp.img" $((esp_size_mib * 1024)) >/dev/null + dd if="$RUN_DIR/.esp.img" of="$raw" bs=1M seek="$esp_start_mib" conv=notrunc,sparse status=none + rm -f "$RUN_DIR/.esp.img" + + rm -rf "$populate" + mkdir -p "$populate" + echo "This file must survive the Omarchy install." >"$populate/OMARCHY-MUST-NOT-TOUCH.txt" + mkfs.ext4 -q -F -L WINDOWS_DATA -d "$populate" \ + -E offset=$(( (esp_start_mib + esp_size_mib) * 1024 * 1024 )) \ + "$raw" "$(( (data_end_mib - esp_start_mib - esp_size_mib) * 1024 ))k" + rm -rf "$populate" + + qemu-img convert -f raw -O qcow2 "$raw" "$disk" + rm -f "$raw" +} + +# The partitions the free-space install had to leave alone, checked from +# inside the installed system. +verify_preserved_partitions() { + log "Verifying the pre-existing partitions survived" + local report="$RUN_DIR/preserved-partitions.txt" + + ssh_guest "lsblk -o NAME,SIZE,FSTYPE,LABEL,MOUNTPOINTS" >"$report" 2>&1 || true + if ! ssh_guest "echo $GUEST_PASSWORD | sudo -S -p '' sh -c ' + blkid -L WINDOWS_ESP >/dev/null && + mkdir -p /run/preserve && mount -o ro LABEL=WINDOWS_DATA /run/preserve && + test -f /run/preserve/OMARCHY-MUST-NOT-TOUCH.txt; rc=\$?; umount /run/preserve 2>/dev/null; exit \$rc'"; then + capture_console "failure-free-space-partitions-lost" + echo "Pre-existing partitions were not preserved; see $report" >&2 + return 1 + fi + log "WINDOWS_ESP and WINDOWS_DATA (with its marker file) are intact" +} + install_phase() { log "Installing $(basename "$ISO") into a fresh VM (headless, driven via console)" [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-iso-test" -f "$SSH_KEY" - rm -f "$BASE_DISK" - qemu-img create -f qcow2 "$BASE_DISK" 40G >/dev/null + if $FREE_SPACE; then + log "Building the synthetic Windows-style disk" + create_free_space_disk "$BASE_DISK" + else + rm -f "$BASE_DISK" + qemu-img create -f qcow2 "$BASE_DISK" 40G >/dev/null + fi cp "$OVMF_VARS_TEMPLATE" "$BASE_OVMF" start_vm "$BASE_DISK" "$RUN_DIR/install-serial.log" \ @@ -989,6 +1072,7 @@ install_phase() { unlock_luks fi bootstrap_ssh + $FREE_SPACE && verify_preserved_partitions # SSH can come up (console bootstrap logs in on a spare TTY) while # omarchy-provision-owner is still finalizing the user and re-keying LUKS. Saving From fbbcdc174197cd5a7204dd655aaa72970900445a Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 21 Aug 2026 13:59:51 +0200 Subject: [PATCH 04/67] Ship only the mirror packages the root image does not provide mkarchiso pacstraps the live root from the complete mirror at build time, but at install time only packages the root image lacks can ever be downloaded from it. The live root's customize_airootfs.sh removes every package file the image already holds at the same version from its copy of the mirror, keeping the repo db complete so `pacman -S --needed` over the hardware scripts' mixed package lists still resolves every name. The build cache goes back to keeping the whole download closure, so a rebuild downloads nothing; the shipped selection is decided per build from the image's local db. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01P9b7oP8j2GA6sZZ9e8aJYv --- builder/build-iso.sh | 92 ++++++++++----------- configs/airootfs/root/customize_airootfs.sh | 41 +++++++++ configs/profiledef.sh | 1 + 3 files changed, 87 insertions(+), 47 deletions(-) create mode 100755 configs/airootfs/root/customize_airootfs.sh diff --git a/builder/build-iso.sh b/builder/build-iso.sh index f11f8342f..f827ead71 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -277,55 +277,20 @@ image_localdb=/tmp/omarchy-root-image-localdb OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" -# Resolve the exact filenames the mirror must keep now that the image carries -# the bulk of the target, against the complete offline repo just indexed so -# the answer can only name files the mirror holds (the local omarchy builds -# included): -# - the live ISO's own packages, against an empty root: mkarchiso pacstraps -# the live root from this mirror and needs their whole closure; -# - the per-machine packages and the hardware-dependent extras -# omarchy-apply-system may install from omarchy-other.packages, against -# the image's local db: only what the image does not already hold. -# Dependencies the live closure shares with the image cost ISO space twice, -# which is the price of keeping the mirror self-contained for the live -# environment. -resolve_mirror_files() { - local root="$1" - shift - mkdir -p "$root/var/lib/pacman" - pacman --config "$build_cache_dir/pacman-offline.conf" \ - --root "$root" --dbpath "$root/var/lib/pacman" --noconfirm -Sy >/dev/null || return 1 - pacman --config "$build_cache_dir/pacman-offline.conf" \ - --root "$root" --dbpath "$root/var/lib/pacman" --noconfirm \ - -S --print --print-format '%f' --needed "$@" -} - -live_root=/tmp/omarchy-mirror-live -rm -rf "$live_root" -mapfile -t live_targets < <(sort -u "$build_cache_dir/packages.x86_64") -if ! live_package_files="$(resolve_mirror_files "$live_root" "${live_targets[@]}")"; then - echo "ERROR: could not resolve the live ISO's packages from the offline mirror" >&2 - exit 1 -fi - -delta_root=/tmp/omarchy-mirror-delta -rm -rf "$delta_root" -mkdir -p "$delta_root/var/lib/pacman" -cp -a "$image_localdb/local" "$delta_root/var/lib/pacman/" -mapfile -t delta_targets < <( - { - grep -hv '^#\|^$' "${base_pkg_lists[1]}" - printf '%s\n' "${hardware_packages[@]}" - } | sort -u -) -if ! delta_package_files="$(resolve_mirror_files "$delta_root" "${delta_targets[@]}")"; then - echo "ERROR: could not resolve the per-machine packages from the offline mirror" >&2 +# Resolve the exact filenames chosen by the same synced package databases used +# for the download. Pruning by this transaction (rather than merely keeping the +# newest version of every cached package name) removes packages that have left +# the lists or dependency closure, such as an old Electron major version. This +# is the build cache: it keeps the whole closure so the next build downloads +# nothing; what the ISO ships is decided below. +if ! resolved_package_files="$( + pacman --config "/configs/pacman-online-${OMARCHY_MIRROR}.conf" --noconfirm \ + --dbpath /tmp/offlinedb -S --print --print-format '%f' "${all_packages[@]}" +)"; then + echo "ERROR: could not resolve the package files required by the offline mirror" >&2 exit 1 fi - -mapfile -t required_package_files < <( - printf '%s\n%s\n' "$live_package_files" "$delta_package_files" | sort -u | grep . -) +mapfile -t required_package_files <<< "$resolved_package_files" # The online transaction intentionally excludes packages built from the local # checkouts. Add those exact artifacts back to the keep-set after verifying @@ -362,6 +327,39 @@ printf '%s\n' "${required_package_files[@]}" | # always reflect only the package files selected for this build. rebuild_offline_repo_db +# What the ISO ships out of this mirror. mkarchiso pacstraps the live root from +# the complete mirror at build time, but at install time only packages the +# root image does not already hold can ever be downloaded from it: the +# per-machine packages, and whatever omarchy-apply-system adds from +# omarchy-other.packages. So the live root's customize_airootfs.sh removes +# every package file the image already provides, at the same version, from +# its copy of the mirror. The repo db stays complete on purpose: the hardware +# scripts run `pacman -S --needed` over lists that mix packages the image has +# with ones it lacks, and pacman must still resolve every name, even the ones +# it then skips as up to date. +mirror_package_index() { + bsdtar -xOf "$offline_mirror_dir/offline.db.tar.gz" --include='*/desc' | + awk '/^%FILENAME%$/ { getline f } /^%NAME%$/ { getline n } /^%VERSION%$/ { getline v; print n "\t" v "\t" f }' +} +image_package_index() { + local desc + for desc in "$image_localdb"/local/*/desc; do + awk '/^%NAME%$/ { getline n } /^%VERSION%$/ { getline v; print n "\t" v }' "$desc" + done +} +shipped_list="$build_cache_dir/airootfs/usr/share/omarchy-iso/offline-mirror.shipped" +awk -F'\t' ' + NR == FNR { image[$1 "\t" $2] = 1; next } + !(($1 "\t" $2) in image) { print $3 } +' <(image_package_index) <(mirror_package_index) | sort -u >"$shipped_list" +shipped_count=$(grep -c . "$shipped_list") +mirror_count=$(mirror_package_index | wc -l) +if (( shipped_count == 0 || shipped_count >= mirror_count )); then + echo "ERROR: the shipped-mirror selection looks wrong: $shipped_count of $mirror_count packages" >&2 + exit 1 +fi +echo "Shipping $shipped_count of $mirror_count mirror packages; the root image provides the rest." + # Denominator for the install dashboard's progress bar: what the image holds # plus the kernel's closure on top of it, resolved against the image's own # local db with the pruned mirror, which is the question the installer's delta diff --git a/configs/airootfs/root/customize_airootfs.sh b/configs/airootfs/root/customize_airootfs.sh new file mode 100755 index 000000000..2af20d6a3 --- /dev/null +++ b/configs/airootfs/root/customize_airootfs.sh @@ -0,0 +1,41 @@ +#!/bin/bash +# +# Runs inside the live root while mkarchiso builds it, after the live packages +# were pacstrapped from the bundled mirror. Drop the package files the root +# image already provides (build-iso.sh lists what to keep); nothing at install +# time can download them, since the same versions are installed from the +# image. The repo db is left complete so `pacman -S --needed` over package +# lists that mix image packages with new ones still resolves every name. + +set -euo pipefail + +mirror=/var/cache/omarchy/mirror/offline +shipped_list=/usr/share/omarchy-iso/offline-mirror.shipped + +[[ -f $shipped_list && -d $mirror ]] || exit 0 + +declare -A shipped=() +while IFS= read -r filename; do + [[ -n $filename ]] && shipped["$filename"]=1 +done <"$shipped_list" + +if (( ${#shipped[@]} == 0 )); then + echo "ERROR: refusing to prune the shipped mirror with an empty selection" >&2 + exit 1 +fi +for filename in "${!shipped[@]}"; do + if [[ ! -f $mirror/$filename ]]; then + echo "ERROR: shipped mirror selection names a missing package: $filename" >&2 + exit 1 + fi +done + +removed=0 +for path in "$mirror"/*.pkg.tar.*; do + filename=${path##*/} + [[ $filename == *.sig ]] && continue + [[ -n ${shipped[$filename]+x} ]] && continue + rm -f -- "$path" "$path.sig" + removed=$((removed + 1)) +done +echo "Removed $removed package files the root image already provides; ${#shipped[@]} remain." diff --git a/configs/profiledef.sh b/configs/profiledef.sh index 0360ed95e..bb1b8030e 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -37,6 +37,7 @@ file_permissions=( ["/root/.automated_script.sh"]="0:0:755" ["/root/.gnupg"]="0:0:700" ["/root/configurator"]="0:0:755" + ["/root/customize_airootfs.sh"]="0:0:755" ["/usr/local/bin/choose-mirror"]="0:0:755" ["/usr/local/bin/omarchy-cidata-load"]="0:0:755" ["/usr/local/bin/omarchy-iso-cleanup-disk"]="0:0:755" From 6406d4951c1202efe17fb6b980c79235a6b5abff Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:09:57 +0200 Subject: [PATCH 05/67] Validate the root image and the target layout before touching the disk arch_install_system partitions, formats and encrypts as its first step, and only then did _install_root_image check that the stream exists and that the layout puts the root on a btrfs @ subvolume, with the LVM guard later still. Any of those failing left a wiped disk (encrypted, on that path) with no system on it. All three are predicates on the ISO and the configurator JSON, so run them in prepare_install_target, the phase before anything destructive. The protected path checks the real mounts, which exist already. Existence does not cover a truncated stream, which on a badly flashed USB is the likelier failure: btrfs receive's per-command checksums catch that too, but after the disk is gone. build-iso.sh now writes a sha256 next to the stream and the same pre-flight verifies it, which also warms the page cache for the unpack. Unit tests cover the pre-flight checks and the subvolume swap in _install_root_image, asserted on the subprocess sequence the way create_factory_snapshot already is. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-iso.sh | 5 + .../omarchy-iso/orchestrator/phases_impl.py | 95 ++++- test/unit/test_root_image.py | 328 ++++++++++++++++++ 3 files changed, 421 insertions(+), 7 deletions(-) create mode 100644 test/unit/test_root_image.py diff --git a/builder/build-iso.sh b/builder/build-iso.sh index f827ead71..5dc23e30d 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -277,6 +277,11 @@ image_localdb=/tmp/omarchy-root-image-localdb OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" +# The installer verifies the stream against this before it touches the disk +# (orchestrator prepare_install_target), so a truncated copy on a badly +# flashed USB fails the install while it is still free to fail. +(cd "$root_image_dir" && sha256sum "${root_image_stream##*/}" >"$root_image_stream.sha256") + # Resolve the exact filenames chosen by the same synced package databases used # for the download. Pruning by this transaction (rather than merely keeping the # newest version of every cached package name) removes packages that have left diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 0b5cf9eb1..ec0f3ed47 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -5,8 +5,11 @@ prepare_live → disk cleanup when wiping, load configurator handlers (archinstall patch happens in the wrapper before Python imports it) - prepare_install_target → verify pre-mounted target/ESP when the JSON uses - pre_mounted_config; no-op for full-disk installs + prepare_install_target → everything that can fail before the disk is + touched: the pre-mounted target/ESP when the JSON + uses pre_mounted_config, the root image stream + and its checksum, and a disk layout the image can + land on arch_install_system → one archinstall flow for partition/mount-or-use, root image unpack (btrfs receive), per-machine package delta, Limine setup, useradd, fstab @@ -122,6 +125,8 @@ def _omarchy_nvim_package() -> str: # target system (build-root-image.sh), unpacked onto the target in place of # pacstrapping it. The subvolume name is what build-root-image.sh sends. ROOT_IMAGE_STREAM = Path("/var/cache/omarchy/rootfs/omarchy-root.btrfs") +# sha256sum output for the stream, written by build-iso.sh next to it. +ROOT_IMAGE_CHECKSUM = ROOT_IMAGE_STREAM.with_name(ROOT_IMAGE_STREAM.name + ".sha256") ROOT_IMAGE_SUBVOLUME = "omarchy-root" # Packages the image must carry for the rest of the install to work: Limine @@ -193,8 +198,77 @@ def _install_disk(ctx: InstallContext) -> str | None: # ───────────────────────────────────────────────────────────────────────────── def prepare_install_target(ctx: InstallContext) -> None: + """Everything that can fail before the disk is touched. The next phase + partitions, formats and encrypts as its first step, and a failure after + that leaves a wiped (or wiped and encrypted) disk with no system on it: + so the stream, its checksum and a layout the image can land on are all + checked here, where failing costs nothing.""" if ctx.is_protected: verify_protected_mounts(ctx) + # The protected layout exists already; check the real mounts. + _root_image_target_mounts(ctx.target) + else: + verify_root_image_layout(ctx.user_configuration.get("disk_config") or {}) + verify_root_image_stream(ctx) + + +def verify_root_image_layout(disk_config: dict) -> None: + """The root image replaces the target's @ subvolume, so the configurator + JSON must put the root on a btrfs @ subvolume, and the image path has no + LVM support (install_base_delta). Checked from the JSON so a layout the + image cannot land on fails before archinstall creates it.""" + if disk_config.get("lvm_config"): + raise RuntimeError("root image install does not support LVM layouts") + + for mod in disk_config.get("device_modifications") or []: + for part in mod.get("partitions") or []: + if part.get("fs_type") != "btrfs": + continue + for subvol in part.get("btrfs") or []: + if subvol.get("mountpoint") == "/" and subvol.get("name") in ("@", "/@"): + return + raise RuntimeError( + "root image install needs the target root on a btrfs @ subvolume; " + "disk_config mounts / from no such subvolume" + ) + + +def verify_root_image_stream(ctx: InstallContext) -> None: + """The stream is present and hashes to what the build recorded. A + truncated or corrupt copy (a badly flashed USB is the common case) would + also trip btrfs receive's per-command checksums, but only after the disk + is formatted. Reading the whole stream here also leaves as much of it as + fits in the page cache for the unpack that follows.""" + if not ROOT_IMAGE_STREAM.is_file(): + raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") + if not ROOT_IMAGE_CHECKSUM.is_file(): + raise RuntimeError(f"root image checksum missing: {ROOT_IMAGE_CHECKSUM}") + + # sha256sum output: " ". + fields = ROOT_IMAGE_CHECKSUM.read_text().split() + expected = fields[0].lower() if fields else "" + if not re.fullmatch(r"[0-9a-f]{64}", expected): + raise RuntimeError(f"root image checksum unreadable: {ROOT_IMAGE_CHECKSUM}") + + total = ROOT_IMAGE_STREAM.stat().st_size + info(f"› verifying root image ({total >> 20} MiB stream)") + digest = hashlib.sha256() + done = 0 + last_report = 0.0 + with ROOT_IMAGE_STREAM.open("rb") as stream: + while chunk := stream.read(8 << 20): + digest.update(chunk) + done += len(chunk) + now = time.monotonic() + if now - last_report >= 0.5: + _write_phase_progress(ctx, done / total if total else 1.0) + last_report = now + actual = digest.hexdigest() + if actual != expected: + raise RuntimeError( + f"root image stream is corrupt: sha256 {actual} != {expected} " + f"({ROOT_IMAGE_STREAM}, {total} bytes); re-flash the install medium" + ) def arch_install_system(ctx: InstallContext) -> None: @@ -342,11 +416,10 @@ def _remount_option_string(options: str) -> str: return ",".join(opt for opt in options.split(",") if not opt.startswith("subvolid=")) -def _install_root_image(ctx: InstallContext) -> None: - target = ctx.target - if not ROOT_IMAGE_STREAM.is_file(): - raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") - +def _root_image_target_mounts(target: Path) -> tuple[list[dict], str]: + """The mount table under target, checked for what the image swap needs: + target itself mounted, btrfs, on the @ subvolume. Returns the mounts and + the device backing the root.""" mounts = _findmnt_mounts(target) if not mounts or Path(mounts[0]["target"]) != target: raise RuntimeError(f"{target} is not a mountpoint") @@ -359,6 +432,14 @@ def _install_root_image(ctx: InstallContext) -> None: device = (root_mount["source"] or "").split("[")[0] if not device: raise RuntimeError(f"could not determine the btrfs device backing {target}") + return mounts, device + + +def _install_root_image(ctx: InstallContext) -> None: + target = ctx.target + if not ROOT_IMAGE_STREAM.is_file(): + raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") + mounts, device = _root_image_target_mounts(target) top = ctx.state_dir / "image-top" top.mkdir(parents=True, exist_ok=True) diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py new file mode 100644 index 000000000..9a521fd8e --- /dev/null +++ b/test/unit/test_root_image.py @@ -0,0 +1,328 @@ +"""Unit tests for the root image install path in the orchestrator. + +Covers the pre-flight checks prepare_install_target runs before the disk is +touched (stream present and matching its checksum, a disk layout the image +can land on) and the destructive subvolume dance in _install_root_image, +asserted on the subprocess calls against a temp target. +""" + +import hashlib +import shutil +import sys +import tempfile +import types +import unittest +from pathlib import Path +from subprocess import CompletedProcess +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "configs/airootfs/usr/share/omarchy-iso")) + +sys.modules.setdefault( + "orchestrator.archinstall_adapter", types.ModuleType("orchestrator.archinstall_adapter") +) + +from orchestrator import phases_impl # noqa: E402 + + +def btrfs_root_layout(name="@", mountpoint="/"): + return { + "config_type": "default_layout", + "device_modifications": [ + { + "device": "/dev/vda", + "wipe": True, + "partitions": [ + {"fs_type": "fat32", "mountpoint": "/boot", "btrfs": []}, + { + "fs_type": "btrfs", + "mountpoint": None, + "btrfs": [ + {"name": name, "mountpoint": mountpoint}, + {"name": "@home", "mountpoint": "/home"}, + ], + }, + ], + } + ], + } + + +class VerifyRootImageLayoutTest(unittest.TestCase): + def test_btrfs_root_subvolume_passes(self): + phases_impl.verify_root_image_layout(btrfs_root_layout()) + phases_impl.verify_root_image_layout(btrfs_root_layout(name="/@")) + + def test_lvm_rejected(self): + layout = btrfs_root_layout() + layout["lvm_config"] = {"config_type": "default", "vol_groups": []} + with self.assertRaisesRegex(RuntimeError, "LVM"): + phases_impl.verify_root_image_layout(layout) + + def test_root_not_on_at_subvolume_rejected(self): + with self.assertRaisesRegex(RuntimeError, "@ subvolume"): + phases_impl.verify_root_image_layout(btrfs_root_layout(name="root")) + + def test_ext4_root_rejected(self): + layout = { + "device_modifications": [ + {"partitions": [{"fs_type": "ext4", "mountpoint": "/", "btrfs": []}]} + ] + } + with self.assertRaisesRegex(RuntimeError, "@ subvolume"): + phases_impl.verify_root_image_layout(layout) + + def test_empty_config_rejected(self): + with self.assertRaises(RuntimeError): + phases_impl.verify_root_image_layout({}) + + +class VerifyRootImageStreamTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.dir = Path(self.tmp.name) + self.stream = self.dir / "omarchy-root.btrfs" + self.checksum = self.dir / "omarchy-root.btrfs.sha256" + self.ctx = types.SimpleNamespace(state_dir=self.dir / "state") + + for name, value in ( + ("ROOT_IMAGE_STREAM", self.stream), + ("ROOT_IMAGE_CHECKSUM", self.checksum), + ): + patch = mock.patch.object(phases_impl, name, value) + patch.start() + self.addCleanup(patch.stop) + for name in ("info", "_write_phase_progress"): + patch = mock.patch.object(phases_impl, name) + patch.start() + self.addCleanup(patch.stop) + + def write_stream(self, data=b"btrfs-stream" * 1000): + self.stream.write_bytes(data) + self.checksum.write_text(f"{hashlib.sha256(data).hexdigest()} {self.stream.name}\n") + + def test_matching_checksum_passes(self): + self.write_stream() + phases_impl.verify_root_image_stream(self.ctx) + + def test_missing_stream(self): + with self.assertRaisesRegex(RuntimeError, "stream missing"): + phases_impl.verify_root_image_stream(self.ctx) + + def test_missing_checksum(self): + self.stream.write_bytes(b"x") + with self.assertRaisesRegex(RuntimeError, "checksum missing"): + phases_impl.verify_root_image_stream(self.ctx) + + def test_unreadable_checksum(self): + self.stream.write_bytes(b"x") + self.checksum.write_text("not a digest\n") + with self.assertRaisesRegex(RuntimeError, "checksum unreadable"): + phases_impl.verify_root_image_stream(self.ctx) + + def test_truncated_stream_rejected(self): + self.write_stream() + data = self.stream.read_bytes() + self.stream.write_bytes(data[: len(data) // 2]) + with self.assertRaisesRegex(RuntimeError, "corrupt"): + phases_impl.verify_root_image_stream(self.ctx) + + +class PrepareInstallTargetTest(unittest.TestCase): + """prepare_install_target wires the checks together per install mode.""" + + def setUp(self): + self.calls = [] + for name in ("verify_protected_mounts", "verify_root_image_stream", + "verify_root_image_layout", "_root_image_target_mounts"): + patch = mock.patch.object( + phases_impl, name, side_effect=lambda *a, _n=name, **k: self.calls.append(_n) + ) + patch.start() + self.addCleanup(patch.stop) + + def test_full_disk_checks_json_layout_then_stream(self): + ctx = types.SimpleNamespace( + is_protected=False, target=Path("/mnt"), + user_configuration={"disk_config": btrfs_root_layout()}, + ) + phases_impl.prepare_install_target(ctx) + self.assertEqual(self.calls, ["verify_root_image_layout", "verify_root_image_stream"]) + + def test_protected_checks_real_mounts_then_stream(self): + ctx = types.SimpleNamespace( + is_protected=True, target=Path("/mnt"), + user_configuration={"disk_config": {"config_type": "pre_mounted_config"}}, + ) + phases_impl.prepare_install_target(ctx) + self.assertEqual( + self.calls, + ["verify_protected_mounts", "_root_image_target_mounts", "verify_root_image_stream"], + ) + + +class InstallRootImageTest(unittest.TestCase): + """The subvolume swap: receive at the top level, snapshot writable, drop + the empty @, rename the snapshot in, replay the mounts.""" + + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.target = Path(self.tmp.name) / "mnt" + self.target.mkdir() + self.state_dir = Path(self.tmp.name) / "state" + self.state_dir.mkdir() + self.top = self.state_dir / "image-top" + self.stream = Path(self.tmp.name) / "omarchy-root.btrfs" + self.stream.write_bytes(b"stream") + self.ctx = types.SimpleNamespace(target=self.target, state_dir=self.state_dir) + + self.mounts = [ + {"target": str(self.target), "source": "/dev/mapper/omarchy_root[/@]", + "fstype": "btrfs", "options": "rw,noatime,compress=zstd:3,subvolid=256,subvol=/@"}, + {"target": str(self.target / "boot"), "source": "/dev/vda1", + "fstype": "vfat", "options": "rw,relatime"}, + {"target": str(self.target / "home"), "source": "/dev/mapper/omarchy_root[/@home]", + "fstype": "btrfs", "options": "rw,noatime,compress=zstd:3,subvolid=257,subvol=/@home"}, + {"target": str(self.target / "var/log"), "source": "/dev/mapper/omarchy_root[/@log]", + "fstype": "btrfs", "options": "rw,noatime,compress=zstd:3,subvolid=258,subvol=/@log"}, + ] + self.calls = [] + self.received_packages = {"limine", "omarchy-keyring", "omarchy", "omarchy-settings", "omarchy-nvim"} + + def fake_run(cmd, **kwargs): + self.calls.append(cmd) + if cmd[0] == "mount" and cmd[1:3] == ["-o", "subvolid=5"]: + # The top level as archinstall left it: an empty @ and the + # other subvolumes of the layout. + top = Path(cmd[-1]) + for name in ("@", "@home", "@log"): + (top / name).mkdir(parents=True, exist_ok=True) + elif cmd[:3] == ["btrfs", "subvolume", "snapshot"]: + shutil.copytree(cmd[3], cmd[4]) + elif cmd[:3] == ["btrfs", "subvolume", "delete"]: + shutil.rmtree(cmd[3]) + return CompletedProcess(cmd, 0, stdout="", stderr="") + + def fake_receive(ctx, top): + self.calls.append(["btrfs", "receive", str(top)]) + received = top / phases_impl.ROOT_IMAGE_SUBVOLUME + (received / "var/log").mkdir(parents=True) + (received / "var/log/pacman.log").write_text("[image] installed base\n") + (received / "etc").mkdir() + + patches = [ + mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run), + mock.patch.object(phases_impl, "_findmnt_mounts", return_value=self.mounts), + mock.patch.object(phases_impl, "_receive_root_image", side_effect=fake_receive), + mock.patch.object(phases_impl, "_umount_tree", + side_effect=lambda root: self.calls.append(["umount", "-R", str(root)])), + mock.patch.object(phases_impl, "_root_image_required_packages", + return_value=["limine", "omarchy-keyring", "omarchy"]), + mock.patch.object(phases_impl.arch, "target_has_package", create=True, + side_effect=lambda target, pkg: pkg in self.received_packages), + mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), + mock.patch.object(phases_impl, "info"), + ] + for patch in patches: + patch.start() + self.addCleanup(patch.stop) + + def btrfs_calls(self): + return [cmd for cmd in self.calls if cmd[0] == "btrfs"] + + def test_swaps_received_image_in_for_the_empty_root(self): + phases_impl._install_root_image(self.ctx) + + top = str(self.top) + received = f"{top}/{phases_impl.ROOT_IMAGE_SUBVOLUME}" + self.assertEqual(self.calls[0], ["mount", "-o", "subvolid=5", "/dev/mapper/omarchy_root", top]) + self.assertEqual( + self.btrfs_calls(), + [ + ["btrfs", "receive", top], + ["btrfs", "subvolume", "snapshot", received, f"{top}/@.image"], + ["btrfs", "subvolume", "delete", received], + ["btrfs", "subvolume", "delete", f"{top}/@"], + ], + ) + # The old @ is deleted only once the layout is unmounted, and the + # snapshot takes its name. + unmount = self.calls.index(["umount", "-R", str(self.target)]) + delete_root = self.calls.index(["btrfs", "subvolume", "delete", f"{top}/@"]) + self.assertLess(unmount, delete_root) + self.assertTrue((self.top / "@" / "etc").is_dir()) + self.assertFalse((self.top / "@.image").exists()) + self.assertFalse((self.top / phases_impl.ROOT_IMAGE_SUBVOLUME).exists()) + + def test_replays_mounts_without_subvolid(self): + phases_impl._install_root_image(self.ctx) + + delete_root = self.calls.index(["btrfs", "subvolume", "delete", f"{self.top}/@"]) + remounts = [cmd for cmd in self.calls[delete_root:] if cmd[0] == "mount"] + self.assertEqual( + remounts, + [ + ["mount", "-t", "btrfs", "-o", "rw,noatime,compress=zstd:3,subvol=/@", + "/dev/mapper/omarchy_root", str(self.target)], + ["mount", "-t", "vfat", "-o", "rw,relatime", "/dev/vda1", str(self.target / "boot")], + ["mount", "-t", "btrfs", "-o", "rw,noatime,compress=zstd:3,subvol=/@home", + "/dev/mapper/omarchy_root", str(self.target / "home")], + ["mount", "-t", "btrfs", "-o", "rw,noatime,compress=zstd:3,subvol=/@log", + "/dev/mapper/omarchy_root", str(self.target / "var/log")], + ], + ) + # The top level is released last, then per-machine identity is set. + self.assertEqual(self.calls[-2], ["umount", str(self.top)]) + self.assertEqual(self.calls[-1], ["systemd-machine-id-setup", f"--root={self.target}"]) + + def test_carries_image_pacman_log_into_log_subvolume(self): + phases_impl._install_root_image(self.ctx) + self.assertEqual((self.top / "@log/pacman.log").read_text(), "[image] installed base\n") + + def test_stale_subvolumes_from_a_previous_attempt_are_removed_first(self): + original_run = phases_impl.subprocess.run.side_effect + + def run_with_leftovers(cmd, **kwargs): + result = original_run(cmd, **kwargs) + if cmd[0] == "mount" and cmd[1:3] == ["-o", "subvolid=5"]: + top = Path(cmd[-1]) + (top / phases_impl.ROOT_IMAGE_SUBVOLUME).mkdir() + (top / "@.image").mkdir() + return result + + phases_impl.subprocess.run.side_effect = run_with_leftovers + phases_impl._install_root_image(self.ctx) + + top = str(self.top) + received = f"{top}/{phases_impl.ROOT_IMAGE_SUBVOLUME}" + self.assertEqual( + self.btrfs_calls()[:4], + [ + ["btrfs", "subvolume", "delete", received], + ["btrfs", "receive", top], + ["btrfs", "subvolume", "delete", f"{top}/@.image"], + ["btrfs", "subvolume", "snapshot", received, f"{top}/@.image"], + ], + ) + + def test_missing_required_package_fails_after_the_swap(self): + self.received_packages.discard("omarchy") + with self.assertRaisesRegex(RuntimeError, "lacks required packages: omarchy"): + phases_impl._install_root_image(self.ctx) + # The layout is back in place and the top level released either way. + self.assertIn(["umount", str(self.top)], self.calls) + self.assertTrue((self.top / "@" / "etc").is_dir()) + + def test_receive_failure_releases_top_level_and_keeps_layout_mounted(self): + phases_impl._receive_root_image.side_effect = RuntimeError("btrfs receive failed") + with self.assertRaisesRegex(RuntimeError, "btrfs receive failed"): + phases_impl._install_root_image(self.ctx) + self.assertEqual(self.calls[-1], ["umount", str(self.top)]) + self.assertNotIn(["umount", "-R", str(self.target)], self.calls) + self.assertTrue((self.top / "@").is_dir()) + + +if __name__ == "__main__": + unittest.main() From 418c840206b626e2247532d4ec5acb3b5fad6528 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:11:07 +0200 Subject: [PATCH 06/67] Prune and index the offline mirror before building the root image The root image was pacstrapped from a repo db built over the unpruned cache. That cache persists across builds and can hold several versions of the same package; repo-add keeps whichever file it processes last (warning on downgrade, hidden by -q) and the glob orders by name, so foo-1.9 beats foo-1.10. The mirror db was rebuilt correctly after the prune, leaving an image that could carry an older package than the mirror beside it advertises. The resolve/prune/repo-add block does not depend on the image, so run it first and drop the early repo-add: the image now resolves against exactly the files this build ships. image.packages joins the download list so the pruned mirror always holds the image's own packages. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-iso.sh | 87 ++++++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 40 deletions(-) diff --git a/builder/build-iso.sh b/builder/build-iso.sh index 5dc23e30d..4dda6e66f 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -204,6 +204,9 @@ mapfile -t all_packages < <( cat "$build_cache_dir/packages.x86_64" grep -hv '^#\|^$' "${base_pkg_lists[@]}" grep -hv '^#\|^$' /builder/archinstall.packages + # The root image is pacstrapped from the pruned mirror, so its packages + # must be in the keep-set even when nothing in the lists depends on them. + grep -hv '^#\|^$' /builder/image.packages # Always include the selected Omarchy packages so the target install can # find the runtime and companion packages in the offline mirror. printf '%s\n' "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE" @@ -238,49 +241,16 @@ if ! download_offline_packages; then download_offline_packages fi -# Index the complete mirror so the root image can be pacstrapped from it. # mkarchiso expects the mirror at /var/cache/omarchy/mirror/offline inside the -# container (the airootfs path); symlink rather than duplicate. +# container (the airootfs path), and the root image is pacstrapped through the +# same path; symlink rather than duplicate. +mkdir -p /var/cache/omarchy/mirror +ln -sfn "$offline_mirror_dir" /var/cache/omarchy/mirror/offline + rebuild_offline_repo_db() { rm -f "$offline_mirror_dir"/offline.db* "$offline_mirror_dir"/offline.files* repo-add -q "$offline_mirror_dir/offline.db.tar.gz" "$offline_mirror_dir/"*.pkg.tar.zst } -rebuild_offline_repo_db -mkdir -p /var/cache/omarchy/mirror -ln -sfn "$offline_mirror_dir" /var/cache/omarchy/mirror/offline - -# Packages that differ per machine and therefore stay out of the root image: -# the installer pacstraps them on the target after unpacking the image, and -# omarchy-apply-system installs more from omarchy-other.packages as the -# hardware dictates. Everything else the target gets is in the image. -hardware_packages=(linux linux-t2 amd-ucode intel-ucode sof-firmware alsa-firmware tailscale) - -mapfile -t image_packages < <( - { - grep -hv '^#\|^$' /builder/archinstall.packages - grep -hv '^#\|^$' /builder/image.packages - # The shipped copy, which is what the install reads too. - grep -hv '^#\|^$' "$build_cache_dir/airootfs/usr/share/omarchy-iso/omarchy-base.packages" - printf '%s\n' "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE" - } | sort -u | grep -Fxv "${hardware_packages[@]/#/-e}" -) - -# pacstrap resolves the image against the offline repo and, with CacheDir on -# the mirror itself, extracts the package files in place instead of copying -# several GiB into a cache first (the same trick the installer's bind mount -# plays on the target). -image_pacman_conf="$build_cache_dir/pacman-root-image.conf" -sed "/^\[options\]/a CacheDir = /var/cache/omarchy/mirror/offline/" \ - "$build_cache_dir/pacman-offline.conf" >"$image_pacman_conf" - -image_localdb=/tmp/omarchy-root-image-localdb -OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ - bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" - -# The installer verifies the stream against this before it touches the disk -# (orchestrator prepare_install_target), so a truncated copy on a badly -# flashed USB fails the install while it is still free to fail. -(cd "$root_image_dir" && sha256sum "${root_image_stream##*/}" >"$root_image_stream.sha256") # Resolve the exact filenames chosen by the same synced package databases used # for the download. Pruning by this transaction (rather than merely keeping the @@ -328,10 +298,47 @@ fi printf '%s\n' "${required_package_files[@]}" | bash /builder/prune-offline-mirror.sh "$offline_mirror_dir" -# Rebuild the offline repo db from scratch so size/checksum/depends entries -# always reflect only the package files selected for this build. +# Index the mirror only now, from scratch, so size/checksum/depends entries +# reflect only the package files selected for this build. The cache persists +# across builds and can hold several versions of a package; repo-add keeps +# whichever it processes last (a warning on downgrade, hidden by -q), and the +# glob orders by name, not version. Indexing the unpruned cache could build the +# root image from an older package than the mirror beside it advertises. rebuild_offline_repo_db +# Packages that differ per machine and therefore stay out of the root image: +# the installer pacstraps them on the target after unpacking the image, and +# omarchy-apply-system installs more from omarchy-other.packages as the +# hardware dictates. Everything else the target gets is in the image. +hardware_packages=(linux linux-t2 amd-ucode intel-ucode sof-firmware alsa-firmware tailscale) + +mapfile -t image_packages < <( + { + grep -hv '^#\|^$' /builder/archinstall.packages + grep -hv '^#\|^$' /builder/image.packages + # The shipped copy, which is what the install reads too. + grep -hv '^#\|^$' "$build_cache_dir/airootfs/usr/share/omarchy-iso/omarchy-base.packages" + printf '%s\n' "$OMARCHY_RUNTIME_PACKAGE" "$OMARCHY_SETTINGS_PACKAGE" "$OMARCHY_NVIM_PACKAGE" + } | sort -u | grep -Fxv "${hardware_packages[@]/#/-e}" +) + +# pacstrap resolves the image against the offline repo and, with CacheDir on +# the mirror itself, extracts the package files in place instead of copying +# several GiB into a cache first (the same trick the installer's bind mount +# plays on the target). +image_pacman_conf="$build_cache_dir/pacman-root-image.conf" +sed "/^\[options\]/a CacheDir = /var/cache/omarchy/mirror/offline/" \ + "$build_cache_dir/pacman-offline.conf" >"$image_pacman_conf" + +image_localdb=/tmp/omarchy-root-image-localdb +OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ + bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" + +# The installer verifies the stream against this before it touches the disk +# (orchestrator prepare_install_target), so a truncated copy on a badly +# flashed USB fails the install while it is still free to fail. +(cd "$root_image_dir" && sha256sum "${root_image_stream##*/}" >"$root_image_stream.sha256") + # What the ISO ships out of this mirror. mkarchiso pacstraps the live root from # the complete mirror at build time, but at install time only packages the # root image does not already hold can ever be downloaded from it: the From 01ea166896552ce19fe99d2d0177748b5ca2ceba Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:11:07 +0200 Subject: [PATCH 07/67] Restore the nproc-capped -smp in omarchy-iso-boot 0cefda3 hardcoded -smp 16 while benchmarking the image receive and never mentioned it; it oversubscribes the VM on anything with fewer cores. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- bin/omarchy-iso-boot | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/omarchy-iso-boot b/bin/omarchy-iso-boot index 62fe04a4b..ac6e1d488 100755 --- a/bin/omarchy-iso-boot +++ b/bin/omarchy-iso-boot @@ -152,7 +152,7 @@ fi # Boot from ISO before we've installed, then boot from drive after qemu-system-x86_64 \ -cpu host -enable-kvm -machine q35,accel=kvm \ - -smp 16 \ + -smp "$(($(nproc) < 8 ? $(nproc) : 8))" \ -m 8192 \ -drive if=pflash,format=raw,readonly=on,file=/usr/share/edk2/x64/OVMF_CODE.4m.fd \ -drive if=pflash,format=raw,file="$OVMF_VARS" \ From 145b786ae30b572f56541485c53e8ea57bee11e8 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:11:33 +0200 Subject: [PATCH 08/67] Fail the build on an implausible target package count The resolved count is the image's package count plus the kernel closure, so since the image replaced the full pacstrap it is the one build-time signal that would catch a short root image; a WARNING that ships no denominator and lets the build continue is not enough for that. The shipped-mirror selection right above already exits on a bad count. Also let the zero case of that selection reach its error message: grep -c exits 1 on no match, which under set -e killed the script before the "looks wrong: 0 of N" line could print. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-iso.sh | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/builder/build-iso.sh b/builder/build-iso.sh index 4dda6e66f..39bc53d6b 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -364,7 +364,8 @@ awk -F'\t' ' NR == FNR { image[$1 "\t" $2] = 1; next } !(($1 "\t" $2) in image) { print $3 } ' <(image_package_index) <(mirror_package_index) | sort -u >"$shipped_list" -shipped_count=$(grep -c . "$shipped_list") +# grep -c exits 1 on no match; the count check below wants the 0. +shipped_count=$(grep -c . "$shipped_list" || true) mirror_count=$(mirror_package_index | wc -l) if (( shipped_count == 0 || shipped_count >= mirror_count )); then echo "ERROR: the shipped-mirror selection looks wrong: $shipped_count of $mirror_count packages" >&2 @@ -401,10 +402,12 @@ resolve_expected_packages() { echo $(( image_count + $(printf '%s\n' "$resolved" | sort -u | grep -c .) + 1 )) } -# Worth failing the build over: -S --print only aborts when a target is missing -# from the offline repo, which would fail the delta pacstrap the same way. A -# count that merely looks wrong is not — the dashboard falls back without the -# file. +# Both failures are worth the build: -S --print only aborts when a target is +# missing from the offline repo, which would fail the delta pacstrap the same +# way, and the count is image packages plus the kernel closure, so one outside +# the plausible range means the root image itself is short of packages. (The +# dashboard falls back to a time-based curve without the file, but nothing +# else would catch a short image before an install.) if ! expected_packages="$(resolve_expected_packages)"; then echo "ERROR: could not resolve the target package count from the offline mirror." >&2 echo " pacman -S --print aborts the whole transaction if any single target" >&2 @@ -413,14 +416,14 @@ if ! expected_packages="$(resolve_expected_packages)"; then exit 1 fi if (( expected_packages < 600 || expected_packages > 2000 )); then - echo "WARNING: resolved target package count $expected_packages is outside the" >&2 - echo " expected 600-2000 range; shipping no denominator so the install" >&2 - echo " dashboard falls back to its time-based curve." >&2 -else - printf '%s\n' "$expected_packages" \ - >"$build_cache_dir/airootfs/usr/share/omarchy-iso/expected-packages" - echo "Target install resolves to $expected_packages packages." + echo "ERROR: resolved target package count $expected_packages is outside the" >&2 + echo " expected 600-2000 range. The count is the image's package count" >&2 + echo " plus the kernel closure, so this means the root image is short." >&2 + exit 1 fi +printf '%s\n' "$expected_packages" \ + >"$build_cache_dir/airootfs/usr/share/omarchy-iso/expected-packages" +echo "Target install resolves to $expected_packages packages." # Live ISO uses the same offline pacman.conf. cp "$build_cache_dir/pacman-offline.conf" "$build_cache_dir/airootfs/etc/pacman.conf" From c9943695317b5bb8f624f51bd253567486816192 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:11:59 +0200 Subject: [PATCH 09/67] Leave already-masked hooks alone in build-root-image.sh The hook masking moved whatever was at the hook path to .omarchy-backup and put a /dev/null symlink in its place, with no check for a mask left behind by a run that died before its cleanup. A second run would then move the symlink over the real backup and mask the host's hook for good. Inert for the ISO build (fresh container every time), but the script documents itself as runnable standalone; skip such hooks the way the orchestrator's _is_devnull_symlink does. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-root-image.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh index c196e3556..c53d90ede 100755 --- a/builder/build-root-image.sh +++ b/builder/build-root-image.sh @@ -111,6 +111,14 @@ trap cleanup EXIT mkdir -p /etc/pacman.d/hooks for hook in "${deferred_boot_hooks[@]}"; do path="/etc/pacman.d/hooks/$hook" + # Already masked: an earlier run died before its cleanup. Moving the mask + # over that run's backup would lose the real hook for good, so leave both + # for it to be restored by hand (the orchestrator's _is_devnull_symlink + # makes the same call). + if [[ -L $path && $(readlink "$path") == /dev/null ]]; then + echo "WARNING: $path is already masked; leaving it and any backup alone" >&2 + continue + fi if [[ -e $path || -L $path ]]; then mv -f "$path" "$path.omarchy-backup" fi From 5e99a678ef244d548ee7ca76895092e5f98d803c Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 21 Aug 2026 23:03:57 +0200 Subject: [PATCH 10/67] Ship the root image as a plain file on the ISO MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Seed the send stream into mkarchiso's work/iso tree next to airootfs.sfs instead of the live squashfs: mkarchiso packs that directory as is, with the boot records intact, and the live system reads the stream straight off the boot medium at /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs. Measured against the squashfs location, same build cache, back-to-back: the ISO is byte-for-byte the same size, mkarchiso is 5s quicker (no 3GB copy into the squashfs), and the install's package phase drops from 18.7s to 16.2s — reading through squashfs costs a copy per 1MiB block even with no decompression. airootfs.sfs shrinks to the live root, and the image can be pulled out of the ISO with any ISO9660 tool. The orchestrator and the wizard-time prefetch look at the ISO path first and fall back to the squashfs path, so mixed old/new pieces still work. Builds before this left the stream in the persistent build cache, where it would ship a second time; the build removes it. The build also logs timestamps around the image step and mkarchiso. The pre-flight checksum follows the stream: build-iso.sh writes the sha256 next to it in the ISO tree, and the orchestrator derives the checksum path from whichever stream location it finds. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-iso.sh | 31 ++++++++-- configs/airootfs/root/.automated_script.sh | 7 ++- .../omarchy-iso/orchestrator/phases_impl.py | 56 ++++++++++++------- configs/profiledef.sh | 9 ++- test/unit/test_root_image.py | 35 +++++++++--- 5 files changed, 97 insertions(+), 41 deletions(-) diff --git a/builder/build-iso.sh b/builder/build-iso.sh index 39bc53d6b..6cda2094f 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -51,9 +51,7 @@ fi # Build locations build_cache_dir=/var/cache offline_mirror_dir="$build_cache_dir/airootfs/var/cache/omarchy/mirror/offline" -root_image_dir="$build_cache_dir/airootfs/var/cache/omarchy/rootfs" -root_image_stream="$root_image_dir/omarchy-root.btrfs" -mkdir -p "$build_cache_dir" "$offline_mirror_dir" "$root_image_dir" +mkdir -p "$build_cache_dir" "$offline_mirror_dir" # Seed from the official Arch releng profile. cp -r /archiso/configs/releng/* "$build_cache_dir/" @@ -330,13 +328,36 @@ image_pacman_conf="$build_cache_dir/pacman-root-image.conf" sed "/^\[options\]/a CacheDir = /var/cache/omarchy/mirror/offline/" \ "$build_cache_dir/pacman-offline.conf" >"$image_pacman_conf" +# The stream ships as a plain file in the ISO9660 tree next to airootfs.sfs, +# not inside the squashfs: mkarchiso packs its work/iso directory as is, so a +# file seeded there ends up on the ISO with the boot records intact. Read +# straight off the boot medium it skips squashfs's per-block copy (~10% off +# the unpack), mkarchiso no longer copies 3GB into the squashfs, and the +# image can be pulled out of the ISO with any ISO9660 tool. The live system +# finds it at /run/archiso/bootmnt///. +# +# profiledef.sh cannot be sourced standalone (its file_permissions needs +# mkarchiso's declare -A first); read the two values it sets instead. +iso_subdir="$(sed -n 's/^install_dir="\(.*\)"$/\1/p' "$build_cache_dir/profiledef.sh")/$(sed -n 's/^arch="\(.*\)"$/\1/p' "$build_cache_dir/profiledef.sh")" +[[ $iso_subdir == */x86_64 ]] || { echo "ERROR: could not read install_dir/arch from profiledef.sh: '$iso_subdir'" >&2; exit 1; } +root_image_dir="$build_cache_dir/work/iso/$iso_subdir" +root_image_stream="$root_image_dir/omarchy-root.btrfs" +mkdir -p "$root_image_dir" +# Builds before the move left the stream (and, briefly, its checksum) in the +# persistent build cache, where they would ship inside the squashfs alongside +# the new one. +rm -f "$build_cache_dir/airootfs/var/cache/omarchy/rootfs/omarchy-root.btrfs"* + image_localdb=/tmp/omarchy-root-image-localdb +echo "[timing] root image start $(date +%s)" OMARCHY_IMAGE_LOCALDB_COPY="$image_localdb" \ bash /builder/build-root-image.sh "$image_pacman_conf" "$root_image_stream" "${image_packages[@]}" +echo "[timing] root image end $(date +%s)" # The installer verifies the stream against this before it touches the disk # (orchestrator prepare_install_target), so a truncated copy on a badly -# flashed USB fails the install while it is still free to fail. +# flashed USB fails the install while it is still free to fail. Next to the +# stream, so it ships on the ISO beside it. (cd "$root_image_dir" && sha256sum "${root_image_stream##*/}" >"$root_image_stream.sha256") # What the ISO ships out of this mirror. mkarchiso pacstraps the live root from @@ -429,7 +450,9 @@ echo "Target install resolves to $expected_packages packages." cp "$build_cache_dir/pacman-offline.conf" "$build_cache_dir/airootfs/etc/pacman.conf" # Build the ISO. +echo "[timing] mkarchiso start $(date +%s)" mkarchiso -v -w "$build_cache_dir/work/" -o /out/ "$build_cache_dir/" +echo "[timing] mkarchiso end $(date +%s)" # Match host UID/GID on output. if [[ -n $HOST_UID && -n $HOST_GID ]]; then diff --git a/configs/airootfs/root/.automated_script.sh b/configs/airootfs/root/.automated_script.sh index ec19163a1..021b9e5b6 100644 --- a/configs/airootfs/root/.automated_script.sh +++ b/configs/airootfs/root/.automated_script.sh @@ -70,8 +70,11 @@ fi # Set OMARCHY_NO_PREFETCH=1 to A/B the same ISO with this disabled. warm_offline_mirror() { local mirror=/var/cache/omarchy/mirror/offline - local image=/var/cache/omarchy/rootfs/omarchy-root.btrfs - local budget_kb spent_kb=0 size_kb path + local image budget_kb spent_kb=0 size_kb path + # Same candidates, same order, as the orchestrator's ROOT_IMAGE_STREAM_CANDIDATES. + for image in /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs /var/cache/omarchy/rootfs/omarchy-root.btrfs; do + [[ -f $image ]] && break + done [[ ${OMARCHY_NO_PREFETCH:-} == 1 ]] && return 0 diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index ec0f3ed47..2dd904acb 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -123,12 +123,27 @@ def _omarchy_nvim_package() -> str: # The root image: a `btrfs send --compressed-data` stream of the invariant # target system (build-root-image.sh), unpacked onto the target in place of -# pacstrapping it. The subvolume name is what build-root-image.sh sends. -ROOT_IMAGE_STREAM = Path("/var/cache/omarchy/rootfs/omarchy-root.btrfs") -# sha256sum output for the stream, written by build-iso.sh next to it. -ROOT_IMAGE_CHECKSUM = ROOT_IMAGE_STREAM.with_name(ROOT_IMAGE_STREAM.name + ".sha256") +# pacstrapping it. build-iso.sh ships it as a plain file on the ISO, read +# straight off the boot medium; the squashfs location is where builds before +# that move put it, kept so an older live root and a newer orchestrator (or +# the reverse) still find the stream. The subvolume name is what +# build-root-image.sh sends. +ROOT_IMAGE_STREAM_CANDIDATES = ( + Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs"), + Path("/var/cache/omarchy/rootfs/omarchy-root.btrfs"), +) +# build-iso.sh writes sha256sum output for the stream next to it. +ROOT_IMAGE_CHECKSUM_SUFFIX = ".sha256" ROOT_IMAGE_SUBVOLUME = "omarchy-root" + +def _root_image_stream() -> Path: + for candidate in ROOT_IMAGE_STREAM_CANDIDATES: + if candidate.is_file(): + return candidate + searched = ", ".join(str(c) for c in ROOT_IMAGE_STREAM_CANDIDATES) + raise RuntimeError(f"root image stream missing; looked at {searched}") + # Packages the image must carry for the rest of the install to work: Limine # setup reads the settings package's limine config, useradd copies the skel the # settings and nvim packages populate, and the target-side setup commands come @@ -239,23 +254,23 @@ def verify_root_image_stream(ctx: InstallContext) -> None: also trip btrfs receive's per-command checksums, but only after the disk is formatted. Reading the whole stream here also leaves as much of it as fits in the page cache for the unpack that follows.""" - if not ROOT_IMAGE_STREAM.is_file(): - raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") - if not ROOT_IMAGE_CHECKSUM.is_file(): - raise RuntimeError(f"root image checksum missing: {ROOT_IMAGE_CHECKSUM}") + stream_path = _root_image_stream() + checksum = stream_path.with_name(stream_path.name + ROOT_IMAGE_CHECKSUM_SUFFIX) + if not checksum.is_file(): + raise RuntimeError(f"root image checksum missing: {checksum}") # sha256sum output: " ". - fields = ROOT_IMAGE_CHECKSUM.read_text().split() + fields = checksum.read_text().split() expected = fields[0].lower() if fields else "" if not re.fullmatch(r"[0-9a-f]{64}", expected): - raise RuntimeError(f"root image checksum unreadable: {ROOT_IMAGE_CHECKSUM}") + raise RuntimeError(f"root image checksum unreadable: {checksum}") - total = ROOT_IMAGE_STREAM.stat().st_size - info(f"› verifying root image ({total >> 20} MiB stream)") + total = stream_path.stat().st_size + info(f"› verifying root image ({total >> 20} MiB stream at {stream_path})") digest = hashlib.sha256() done = 0 last_report = 0.0 - with ROOT_IMAGE_STREAM.open("rb") as stream: + with stream_path.open("rb") as stream: while chunk := stream.read(8 << 20): digest.update(chunk) done += len(chunk) @@ -267,7 +282,7 @@ def verify_root_image_stream(ctx: InstallContext) -> None: if actual != expected: raise RuntimeError( f"root image stream is corrupt: sha256 {actual} != {expected} " - f"({ROOT_IMAGE_STREAM}, {total} bytes); re-flash the install medium" + f"({stream_path}, {total} bytes); re-flash the install medium" ) @@ -437,8 +452,7 @@ def _root_image_target_mounts(target: Path) -> tuple[list[dict], str]: def _install_root_image(ctx: InstallContext) -> None: target = ctx.target - if not ROOT_IMAGE_STREAM.is_file(): - raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") + stream = _root_image_stream() mounts, device = _root_image_target_mounts(target) top = ctx.state_dir / "image-top" @@ -449,8 +463,8 @@ def _install_root_image(ctx: InstallContext) -> None: if received.exists(): subprocess.run(["btrfs", "subvolume", "delete", str(received)], check=True, capture_output=True) - info(f"› unpacking root image ({ROOT_IMAGE_STREAM.stat().st_size >> 20} MiB stream)") - _receive_root_image(ctx, top) + info(f"› unpacking root image ({stream.stat().st_size >> 20} MiB stream from {stream})") + _receive_root_image(ctx, top, stream) staged = top / "@.image" if staged.exists(): @@ -503,10 +517,10 @@ def _umount_tree(root: Path, attempts: int = 20) -> None: time.sleep(0.25) -def _receive_root_image(ctx: InstallContext, top: Path) -> None: +def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> None: """Pipe the stream into btrfs receive, publishing progress for the dashboard as a fraction of stream bytes consumed.""" - total = ROOT_IMAGE_STREAM.stat().st_size + total = stream_path.stat().st_size errors = ctx.state_dir / "btrfs-receive.err" with errors.open("w") as err: proc = subprocess.Popen( @@ -518,7 +532,7 @@ def _receive_root_image(ctx: InstallContext, top: Path) -> None: sent = 0 last_report = 0.0 try: - with ROOT_IMAGE_STREAM.open("rb") as stream: + with stream_path.open("rb") as stream: while chunk := stream.read(8 << 20): proc.stdin.write(chunk) sent += len(chunk) diff --git a/configs/profiledef.sh b/configs/profiledef.sh index bb1b8030e..3458cef8b 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -12,10 +12,10 @@ bootmodes=('bios.syslinux' 'uefi.grub') arch="x86_64" pacman_conf="pacman-offline.conf" airootfs_image_type="squashfs" -# Package archives in the offline mirror are already zstd-compressed, and so is -# the root image stream (btrfs send --compressed-data). Storing them in an outer -# stream saves little space but makes pacman decompress the outer layer while -# hashing and extracting every package, and btrfs receive while unpacking. +# Package archives in the offline mirror are already zstd-compressed. Storing +# them in an outer stream saves little space but makes pacman decompress the +# outer layer while hashing and extracting every package during installation. +# (The root image stream is a plain file on the ISO, outside the squashfs.) # # Everything else in the live root is zstd rather than xz. Squashfs decompresses # on the page-fault path through a single stream (CONFIG_SQUASHFS_DECOMP_SINGLE), @@ -28,7 +28,6 @@ airootfs_image_tool_options=( '-Xcompression-level' '19' '-b' '1M' '-action' 'uncompressed@subpathname(var/cache/omarchy/mirror/offline)' - '-action' 'uncompressed@subpathname(var/cache/omarchy/rootfs)' ) bootstrap_tarball_compression=('zstd' '-c' '-T0' '--auto-threads=logical' '--long' '-19') file_permissions=( diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 9a521fd8e..977538c54 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -86,13 +86,9 @@ def setUp(self): self.checksum = self.dir / "omarchy-root.btrfs.sha256" self.ctx = types.SimpleNamespace(state_dir=self.dir / "state") - for name, value in ( - ("ROOT_IMAGE_STREAM", self.stream), - ("ROOT_IMAGE_CHECKSUM", self.checksum), - ): - patch = mock.patch.object(phases_impl, name, value) - patch.start() - self.addCleanup(patch.stop) + patch = mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (self.stream,)) + patch.start() + self.addCleanup(patch.stop) for name in ("info", "_write_phase_progress"): patch = mock.patch.object(phases_impl, name) patch.start() @@ -129,6 +125,26 @@ def test_truncated_stream_rejected(self): phases_impl.verify_root_image_stream(self.ctx) +class RootImageStreamLookupTest(unittest.TestCase): + def test_first_existing_candidate_wins(self): + with tempfile.TemporaryDirectory() as tmp: + iso = Path(tmp) / "bootmnt/omarchy-root.btrfs" + sfs = Path(tmp) / "rootfs/omarchy-root.btrfs" + sfs.parent.mkdir() + sfs.write_bytes(b"old") + with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (iso, sfs)): + self.assertEqual(phases_impl._root_image_stream(), sfs) + iso.parent.mkdir() + iso.write_bytes(b"new") + self.assertEqual(phases_impl._root_image_stream(), iso) + + def test_no_candidate_lists_every_path(self): + with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", + (Path("/nonexistent/a"), Path("/nonexistent/b"))): + with self.assertRaisesRegex(RuntimeError, "stream missing.*/nonexistent/a.*/nonexistent/b"): + phases_impl._root_image_stream() + + class PrepareInstallTargetTest(unittest.TestCase): """prepare_install_target wires the checks together per install mode.""" @@ -205,7 +221,8 @@ def fake_run(cmd, **kwargs): shutil.rmtree(cmd[3]) return CompletedProcess(cmd, 0, stdout="", stderr="") - def fake_receive(ctx, top): + def fake_receive(ctx, top, stream_path): + self.assertEqual(stream_path, self.stream) self.calls.append(["btrfs", "receive", str(top)]) received = top / phases_impl.ROOT_IMAGE_SUBVOLUME (received / "var/log").mkdir(parents=True) @@ -222,7 +239,7 @@ def fake_receive(ctx, top): return_value=["limine", "omarchy-keyring", "omarchy"]), mock.patch.object(phases_impl.arch, "target_has_package", create=True, side_effect=lambda target, pkg: pkg in self.received_packages), - mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), + mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (self.stream,)), mock.patch.object(phases_impl, "info"), ] for patch in patches: From 99e0a2e4af676420da8bb1fa12c9ab3caf143d6f Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:37:10 +0200 Subject: [PATCH 11/67] Keep the pacman keyring out of the root image; create it per machine The image was pacstrapped with -K, so it carried /etc/pacman.d/gnupg with a master key (secring.gpg) that every install would share. A signing key must never be distributed; pacstrap the image with -G and remove anything a scriptlet might have seeded regardless, as #108 does. On a target pacstrapped directly, as on quattro, pacstrap -K initialised a per-machine keyring and the keyring packages' scriptlets populated it in the same run. Here those packages come from the image, where their scriptlets ran with no keyring to populate, so the orchestrator does it: after the last pacstrap (each one runs its own pacman-key --init on the target), pacman-key --init, idempotent for the key the delta pacstrap already generated, then --populate archlinux omarchy from the target's own keyring files. Chroot-free via --gpgdir and --populate-from, and the gpg daemons are killed on every path so the target can be unmounted. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- builder/build-root-image.sh | 21 +++++--- .../omarchy-iso/orchestrator/phases_impl.py | 47 ++++++++++++++++- test/unit/test_root_image.py | 51 +++++++++++++++++++ 3 files changed, 112 insertions(+), 7 deletions(-) diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh index c53d90ede..081a3e26a 100755 --- a/builder/build-root-image.sh +++ b/builder/build-root-image.sh @@ -75,9 +75,10 @@ mnt="$work/mnt" loop="" masked_hooks=() -# pacstrap -K runs pacman-key --init against the image's keyring, which starts -# a gpg-agent for that homedir and leaves it running; until it is gone the -# image cannot be unmounted. +# Anything that touches a keyring under the image (pacman-key in a scriptlet, +# say) starts a gpg-agent for that homedir and leaves it running; until it is +# gone the image cannot be unmounted. Nothing should, as no keyring exists in +# the image, but kill whatever did. stop_image_gpg_agents() { gpgconf --homedir "$mnt/$IMAGE_SUBVOLUME/etc/pacman.d/gnupg" --kill all 2>/dev/null || true pkill -f "$mnt/$IMAGE_SUBVOLUME/etc/pacman.d/gnupg" 2>/dev/null || true @@ -146,12 +147,20 @@ root="$mnt/$IMAGE_SUBVOLUME" # -c: packages come straight from the config's CacheDir (the offline mirror), # so nothing is copied into the image's own cache first. -# -K: a keyring initialised for this image rather than the builder's copied in; -# the keyring packages populate it from their install scripts. +# -G: no pacman keyring in the image, neither the builder's copied in nor one +# initialised here (-K): its master key would be the same on every +# install, and a shared signing key must never be distributed. The +# installer initialises and populates a per-machine keyring on the target +# (phases_impl._init_target_keyring), as pacstrap -K plus the keyring +# packages' scriptlets did when the target was pacstrapped directly. # -M: the builder's mirrorlist means nothing to an installed system; the # installer writes the target's own. -pacstrap -C "$pacman_conf" -c -K -M "$root" "${packages[@]}" +pacstrap -C "$pacman_conf" -c -G -M "$root" "${packages[@]}" stop_image_gpg_agents +# The keyring packages' scriptlets only populate an existing keyring, so with +# -G they have nothing to do; remove whatever a scriptlet may have seeded +# anyway rather than trust that. +rm -rf "$root/etc/pacman.d/gnupg" # Per-machine identity is the installer's job, never the image's: it runs # systemd-machine-id-setup on the target after unpacking. Leave the id diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 2dd904acb..97bd62f6f 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -12,7 +12,8 @@ land on arch_install_system → one archinstall flow for partition/mount-or-use, root image unpack (btrfs receive), per-machine - package delta, Limine setup, useradd, fstab + package delta and pacman keyring, Limine setup, + useradd, fstab configure_hibernation → root-owned swap/resume drop-ins run_system_finalizer → arch-chroot root omarchy-apply-system, including Snapper finalize_limine_boot → final Limine config/UKI build after hardware drop-ins @@ -379,6 +380,10 @@ def arch_install_system(ctx: InstallContext) -> None: _unmask_mkinitcpio_pacman_hooks(ctx) _unmount_offline_package_cache(ctx) + # After the last pacstrap: each one runs its own pacman-key --init on + # the target's gnupg dir. + _init_target_keyring(ctx) + # Standard arch finishers. if config.timezone: installer.set_timezone(config.timezone) @@ -505,6 +510,46 @@ def _install_root_image(ctx: InstallContext) -> None: subprocess.run(["systemd-machine-id-setup", f"--root={target}"], check=True, capture_output=True) +def _init_target_keyring(ctx: InstallContext) -> None: + """Initialise and populate the target's per-machine pacman keyring. + + The image ships no /etc/pacman.d/gnupg: its master key would be the same + on every install, and a shared signing key must never be distributed. On + a target pacstrapped directly, pacstrap -K initialised the keyring and + the keyring packages' scriptlets populated it; here those packages come + from the image, where their scriptlets ran with no keyring to populate. + The delta pacstrap's -K still ran --init (generating the master key), and + --init is idempotent, so run it again for installs that pacstrapped + nothing, then populate from the target's own keyring files. + + Chroot-free: --gpgdir and --populate-from address the target directly, + so no API mounts are held. Nothing during the install reads this keyring + (the offline repo is SigLevel = Never); the installed system does. The + gpg-agent and dirmngr pacman-key starts hold sockets under the gnupg + dir and would keep the target busy at unmount, so kill them either way. + """ + gpgdir = ctx.target / "etc" / "pacman.d" / "gnupg" + keyrings = ctx.target / "usr" / "share" / "pacman" / "keyrings" + info("› initializing per-machine pacman keyring") + try: + for step, args in ( + ("--init", ["--init"]), + ("--populate", ["--populate-from", str(keyrings), "--populate", "archlinux", "omarchy"]), + ): + res = subprocess.run( + ["pacman-key", "--gpgdir", str(gpgdir), *args], + capture_output=True, text=True, + ) + if res.returncode != 0: + raise RuntimeError( + f"pacman-key {step} failed (exit {res.returncode}):\n" + f"{(res.stderr or res.stdout).strip()}" + ) + finally: + subprocess.run(["gpgconf", "--homedir", str(gpgdir), "--kill", "all"], + check=False, capture_output=True) + + def _umount_tree(root: Path, attempts: int = 20) -> None: """umount -R with a few retries: the dashboard polls the target's pacman db from another process, and a poll landing mid-unmount is EBUSY.""" diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 977538c54..947d64319 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -341,5 +341,56 @@ def test_receive_failure_releases_top_level_and_keeps_layout_mounted(self): self.assertTrue((self.top / "@").is_dir()) +class InitTargetKeyringTest(unittest.TestCase): + """The per-machine keyring the image deliberately ships without.""" + + def setUp(self): + self.target = Path("/mnt") + self.gpgdir = str(self.target / "etc/pacman.d/gnupg") + self.ctx = types.SimpleNamespace(target=self.target) + self.calls = [] + self.fail_on = None + + def fake_run(cmd, **kwargs): + self.calls.append(cmd) + if self.fail_on and self.fail_on in cmd: + return CompletedProcess(cmd, 1, stdout="", stderr="gpg: boom") + return CompletedProcess(cmd, 0, stdout="", stderr="") + + for patch in ( + mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run), + mock.patch.object(phases_impl, "info"), + ): + patch.start() + self.addCleanup(patch.stop) + + def test_inits_then_populates_from_the_target_chroot_free(self): + phases_impl._init_target_keyring(self.ctx) + self.assertEqual( + self.calls, + [ + ["pacman-key", "--gpgdir", self.gpgdir, "--init"], + ["pacman-key", "--gpgdir", self.gpgdir, + "--populate-from", str(self.target / "usr/share/pacman/keyrings"), + "--populate", "archlinux", "omarchy"], + ["gpgconf", "--homedir", self.gpgdir, "--kill", "all"], + ], + ) + self.assertFalse(any("arch-chroot" in cmd for cmd in self.calls)) + + def test_populate_failure_raises_and_still_kills_gpg_daemons(self): + self.fail_on = "--populate" + with self.assertRaisesRegex(RuntimeError, r"(?s)pacman-key --populate failed.*gpg: boom"): + phases_impl._init_target_keyring(self.ctx) + self.assertEqual(self.calls[-1], ["gpgconf", "--homedir", self.gpgdir, "--kill", "all"]) + + def test_init_failure_skips_populate(self): + self.fail_on = "--init" + with self.assertRaisesRegex(RuntimeError, "pacman-key --init failed"): + phases_impl._init_target_keyring(self.ctx) + self.assertFalse(any("--populate" in cmd for cmd in self.calls)) + self.assertEqual(self.calls[-1][0], "gpgconf") + + if __name__ == "__main__": unittest.main() From f58d9e2d443211b7ff79cfaace4494d440469dd5 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sat, 22 Aug 2026 16:44:14 +0200 Subject: [PATCH 12/67] Run the target keyring init as a transient systemd unit pacman-key --init/--populate took the critical path synchronously and left a gpg-agent and dirmngr to be killed by name afterwards. Start it with systemd-run --wait --pipe right after the last pacstrap instead, and join it in create_factory_snapshot: nothing in between reads the keyring (the offline repo is SigLevel = Never) or writes it, so the Limine, user and finalizer phases hide its few seconds, and the snapshot waits so @factory never captures it half-written. A unit rather than a detached child: systemd kills the gpg daemons with the rest of the cgroup the moment pacman-key exits, so no sockets under the target's gnupg dir survive to block the unmount; the dashboard's process-group kill does not reach it while systemctl stop still does, which main() runs on every exit path; and its output lands in the journal whatever happens to the orchestrator. --wait --pipe give a child to join with the unit's exit status and output; --collect releases the name after a failure. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01XdQdQ7YZmBdbvaZNQ2xjKN --- .../share/omarchy-iso/orchestrator/main.py | 4 + .../omarchy-iso/orchestrator/phases_impl.py | 106 +++++++++++------ test/unit/test_provisioning_state.py | 1 + test/unit/test_root_image.py | 110 ++++++++++++------ 4 files changed, 155 insertions(+), 66 deletions(-) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py index 21808a940..87f6b027e 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/main.py @@ -82,6 +82,7 @@ def main() -> int: cleanup_protected_state, cleanup_target_hook_masks, restore_cpu_governors, + stop_target_keyring_init, ) governors = boost_cpu_governor() @@ -102,6 +103,9 @@ def main() -> int: return 0 finally: restore_cpu_governors(governors) + # No-op after a completed install (create_factory_snapshot joined it); + # on a failure it ends the unit before the target is torn down. + stop_target_keyring_init(ctx) cleanup_bind_mounts(ctx) cleanup_target_hook_masks(ctx) if not success: diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 97bd62f6f..70c4e4e67 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -12,8 +12,9 @@ land on arch_install_system → one archinstall flow for partition/mount-or-use, root image unpack (btrfs receive), per-machine - package delta and pacman keyring, Limine setup, - useradd, fstab + package delta, Limine setup, useradd, fstab; the + per-machine pacman keyring starts as a transient + systemd unit after the last pacstrap configure_hibernation → root-owned swap/resume drop-ins run_system_finalizer → arch-chroot root omarchy-apply-system, including Snapper finalize_limine_boot → final Limine config/UKI build after hardware drop-ins @@ -22,6 +23,7 @@ configure_ssh_access → authorized_keys for autoinstall; no-op otherwise configure_tailscale → tailnet join staged for first boot; no-op otherwise validate_boot → assert UKI / limine.conf / kernel cmdline are sane + create_factory_snapshot→ joins the keyring unit, then snapshots @ as @factory """ from __future__ import annotations @@ -30,6 +32,7 @@ import json import os import re +import shlex import shutil import subprocess import textwrap @@ -381,8 +384,9 @@ def arch_install_system(ctx: InstallContext) -> None: _unmount_offline_package_cache(ctx) # After the last pacstrap: each one runs its own pacman-key --init on - # the target's gnupg dir. - _init_target_keyring(ctx) + # the target's gnupg dir. Runs on while the phases below configure the + # target; create_factory_snapshot joins it. + _start_target_keyring_init(ctx) # Standard arch finishers. if config.timezone: @@ -510,8 +514,13 @@ def _install_root_image(ctx: InstallContext) -> None: subprocess.run(["systemd-machine-id-setup", f"--root={target}"], check=True, capture_output=True) -def _init_target_keyring(ctx: InstallContext) -> None: - """Initialise and populate the target's per-machine pacman keyring. +# Transient systemd unit that initialises the target's pacman keyring. +TARGET_KEYRING_UNIT = "omarchy-target-keyring" + + +def _start_target_keyring_init(ctx: InstallContext) -> None: + """Initialise and populate the target's per-machine pacman keyring, as a + transient systemd unit that runs on while the install continues. The image ships no /etc/pacman.d/gnupg: its master key would be the same on every install, and a shared signing key must never be distributed. On @@ -520,34 +529,62 @@ def _init_target_keyring(ctx: InstallContext) -> None: from the image, where their scriptlets ran with no keyring to populate. The delta pacstrap's -K still ran --init (generating the master key), and --init is idempotent, so run it again for installs that pacstrapped - nothing, then populate from the target's own keyring files. - - Chroot-free: --gpgdir and --populate-from address the target directly, - so no API mounts are held. Nothing during the install reads this keyring - (the offline repo is SigLevel = Never); the installed system does. The - gpg-agent and dirmngr pacman-key starts hold sockets under the gnupg - dir and would keep the target busy at unmount, so kill them either way. + nothing, then populate from the target's own keyring files. Chroot-free: + --gpgdir and --populate-from address the target directly, so no API + mounts are held. + + Started after the last pacstrap (each runs its own --init on this dir) + and joined by create_factory_snapshot: nothing in between reads the + keyring (the offline repo is SigLevel = Never) or writes it, and the + snapshot must not capture it half-written. A unit rather than a plain + child process: pacman-key leaves a gpg-agent and dirmngr behind, which + systemd kills with the rest of the unit's cgroup the moment pacman-key + exits (sockets under the target's gnupg dir would otherwise block the + unmount); the dashboard's process-group kill does not reach it, while + `systemctl stop` still does (stop_target_keyring_init); and its output + is in the journal whatever happens to the orchestrator. --wait --pipe + give a child to join with the unit's exit status and output. """ - gpgdir = ctx.target / "etc" / "pacman.d" / "gnupg" - keyrings = ctx.target / "usr" / "share" / "pacman" / "keyrings" - info("› initializing per-machine pacman keyring") - try: - for step, args in ( - ("--init", ["--init"]), - ("--populate", ["--populate-from", str(keyrings), "--populate", "archlinux", "omarchy"]), - ): - res = subprocess.run( - ["pacman-key", "--gpgdir", str(gpgdir), *args], - capture_output=True, text=True, - ) - if res.returncode != 0: - raise RuntimeError( - f"pacman-key {step} failed (exit {res.returncode}):\n" - f"{(res.stderr or res.stdout).strip()}" - ) - finally: - subprocess.run(["gpgconf", "--homedir", str(gpgdir), "--kill", "all"], - check=False, capture_output=True) + gpgdir = shlex.quote(str(ctx.target / "etc" / "pacman.d" / "gnupg")) + keyrings = shlex.quote(str(ctx.target / "usr" / "share" / "pacman" / "keyrings")) + script = ( + f"pacman-key --gpgdir {gpgdir} --init && " + f"pacman-key --gpgdir {gpgdir} --populate-from {keyrings} --populate archlinux omarchy" + ) + info("› initializing per-machine pacman keyring (background unit)") + # A failed unit from an earlier attempt would hold the name (--collect + # releases it, but not for a unit started without it). + subprocess.run(["systemctl", "reset-failed", TARGET_KEYRING_UNIT], check=False, capture_output=True) + ctx.state["target_keyring_proc"] = subprocess.Popen( + ["systemd-run", "--wait", "--pipe", "--collect", "--quiet", + f"--unit={TARGET_KEYRING_UNIT}", "sh", "-c", script], + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + + +def _join_target_keyring_init(ctx: InstallContext, *, raise_on_error: bool = True) -> None: + """Wait for the keyring unit if one is running; no-op otherwise.""" + proc = ctx.state.pop("target_keyring_proc", None) + if proc is None: + return + out, _ = proc.communicate() + if raise_on_error and proc.returncode != 0: + raise RuntimeError( + f"per-machine pacman keyring init failed (exit {proc.returncode}):\n" + f"{(out or '').strip()}" + ) + + +def stop_target_keyring_init(ctx: InstallContext) -> None: + """Exit path: end the keyring unit if it is still running, so nothing + keeps writing into the target after the install has stopped. The + install's own error must win, so a keyring failure is not raised here.""" + if ctx.state.get("target_keyring_proc") is None: + return + subprocess.run(["systemctl", "stop", TARGET_KEYRING_UNIT], check=False, capture_output=True) + _join_target_keyring_init(ctx, raise_on_error=False) def _umount_tree(root: Path, attempts: int = 20) -> None: @@ -2050,6 +2087,9 @@ def _validate_pre_mounted_filesystems(ctx: InstallContext) -> None: # ───────────────────────────────────────────────────────────────────────────── def create_factory_snapshot(ctx: InstallContext) -> None: + # The keyring unit writes into @; the snapshot must not catch it midway. + _join_target_keyring_init(ctx) + fstype = _findmnt_value(ctx.target, "FSTYPE") if fstype != "btrfs": info(f"› target root is {fstype or 'unknown'}, not btrfs; skipping factory snapshot") diff --git a/test/unit/test_provisioning_state.py b/test/unit/test_provisioning_state.py index 7386f0012..e5377677d 100644 --- a/test/unit/test_provisioning_state.py +++ b/test/unit/test_provisioning_state.py @@ -168,6 +168,7 @@ def make_ctx(target, **overrides): user_credentials={"users": []}, state_dir=target / "state", authorized_keys_path=None, + state={}, ) defaults.update(overrides) return types.SimpleNamespace(**defaults) diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 947d64319..3d9d016e4 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -341,55 +341,99 @@ def test_receive_failure_releases_top_level_and_keeps_layout_mounted(self): self.assertTrue((self.top / "@").is_dir()) -class InitTargetKeyringTest(unittest.TestCase): - """The per-machine keyring the image deliberately ships without.""" +class FakeUnitProc: + """What Popen(systemd-run --wait --pipe ...) hands back.""" + + def __init__(self, returncode=0, output=""): + self.returncode = returncode + self.output = output + self.joined = 0 + + def communicate(self): + self.joined += 1 + return self.output, None + + +class TargetKeyringUnitTest(unittest.TestCase): + """The per-machine keyring the image deliberately ships without, run as + a transient unit and joined before the factory snapshot.""" def setUp(self): self.target = Path("/mnt") - self.gpgdir = str(self.target / "etc/pacman.d/gnupg") - self.ctx = types.SimpleNamespace(target=self.target) - self.calls = [] - self.fail_on = None + self.ctx = types.SimpleNamespace(target=self.target, state={}) + self.runs = [] + self.popens = [] def fake_run(cmd, **kwargs): - self.calls.append(cmd) - if self.fail_on and self.fail_on in cmd: - return CompletedProcess(cmd, 1, stdout="", stderr="gpg: boom") + self.runs.append(cmd) return CompletedProcess(cmd, 0, stdout="", stderr="") + def fake_popen(cmd, **kwargs): + self.popens.append((cmd, kwargs)) + return FakeUnitProc() + for patch in ( mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run), + mock.patch.object(phases_impl.subprocess, "Popen", side_effect=fake_popen), mock.patch.object(phases_impl, "info"), ): patch.start() self.addCleanup(patch.stop) - def test_inits_then_populates_from_the_target_chroot_free(self): - phases_impl._init_target_keyring(self.ctx) + def test_start_runs_init_then_populate_in_a_waited_unit(self): + phases_impl._start_target_keyring_init(self.ctx) + + self.assertEqual(self.runs, [["systemctl", "reset-failed", "omarchy-target-keyring"]]) + (cmd, kwargs), = self.popens self.assertEqual( - self.calls, - [ - ["pacman-key", "--gpgdir", self.gpgdir, "--init"], - ["pacman-key", "--gpgdir", self.gpgdir, - "--populate-from", str(self.target / "usr/share/pacman/keyrings"), - "--populate", "archlinux", "omarchy"], - ["gpgconf", "--homedir", self.gpgdir, "--kill", "all"], - ], + cmd[:6], + ["systemd-run", "--wait", "--pipe", "--collect", "--quiet", "--unit=omarchy-target-keyring"], + ) + self.assertEqual(cmd[6:8], ["sh", "-c"]) + self.assertEqual( + cmd[8], + "pacman-key --gpgdir /mnt/etc/pacman.d/gnupg --init && " + "pacman-key --gpgdir /mnt/etc/pacman.d/gnupg " + "--populate-from /mnt/usr/share/pacman/keyrings --populate archlinux omarchy", ) - self.assertFalse(any("arch-chroot" in cmd for cmd in self.calls)) - - def test_populate_failure_raises_and_still_kills_gpg_daemons(self): - self.fail_on = "--populate" - with self.assertRaisesRegex(RuntimeError, r"(?s)pacman-key --populate failed.*gpg: boom"): - phases_impl._init_target_keyring(self.ctx) - self.assertEqual(self.calls[-1], ["gpgconf", "--homedir", self.gpgdir, "--kill", "all"]) - - def test_init_failure_skips_populate(self): - self.fail_on = "--init" - with self.assertRaisesRegex(RuntimeError, "pacman-key --init failed"): - phases_impl._init_target_keyring(self.ctx) - self.assertFalse(any("--populate" in cmd for cmd in self.calls)) - self.assertEqual(self.calls[-1][0], "gpgconf") + self.assertNotIn("arch-chroot", cmd[8]) + self.assertIs(kwargs["stderr"], phases_impl.subprocess.STDOUT) + self.assertIsInstance(self.ctx.state["target_keyring_proc"], FakeUnitProc) + + def test_join_waits_once_and_clears_state(self): + proc = FakeUnitProc() + self.ctx.state["target_keyring_proc"] = proc + phases_impl._join_target_keyring_init(self.ctx) + phases_impl._join_target_keyring_init(self.ctx) # no-op: nothing pending + self.assertEqual(proc.joined, 1) + self.assertNotIn("target_keyring_proc", self.ctx.state) + + def test_join_raises_with_the_unit_output(self): + self.ctx.state["target_keyring_proc"] = FakeUnitProc(returncode=1, output="gpg: boom\n") + with self.assertRaisesRegex(RuntimeError, r"(?s)keyring init failed \(exit 1\).*gpg: boom"): + phases_impl._join_target_keyring_init(self.ctx) + self.assertNotIn("target_keyring_proc", self.ctx.state) + + def test_stop_ends_the_unit_and_never_raises(self): + proc = FakeUnitProc(returncode=1, output="killed") + self.ctx.state["target_keyring_proc"] = proc + phases_impl.stop_target_keyring_init(self.ctx) + self.assertEqual(self.runs, [["systemctl", "stop", "omarchy-target-keyring"]]) + self.assertEqual(proc.joined, 1) + self.assertNotIn("target_keyring_proc", self.ctx.state) + + def test_stop_without_a_unit_touches_nothing(self): + phases_impl.stop_target_keyring_init(self.ctx) + self.assertEqual(self.runs, []) + + def test_factory_snapshot_joins_the_unit_before_anything_else(self): + # Even when the snapshot itself is skipped, the join runs: a failed + # keyring must fail the install. + self.ctx.state["target_keyring_proc"] = FakeUnitProc(returncode=1, output="gpg: boom") + with mock.patch.object(phases_impl, "_findmnt_value", return_value="ext4"): + with self.assertRaisesRegex(RuntimeError, "keyring init failed"): + phases_impl.create_factory_snapshot(self.ctx) + self.assertEqual(self.runs, []) if __name__ == "__main__": From ce6b51f08bf23dc41765bf78c4e27cb0daf5ff79 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 05:27:30 +0200 Subject: [PATCH 13/67] Verify the root image in a systemd unit started at boot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pre-flight sha256 of the root image ran inline in the orchestrator's prepare_install_target phase: 1.6s on an NVMe-backed VM, tens of seconds from a USB stick, all of it after the user had pressed Install. The medium sits idle while the user works through the configurator, so move the read there: omarchy-root-image-verify.service runs `sha256sum -c` on the ISO copy of the stream as a oneshot at boot, niced and at idle I/O class, with RemainAfterExit so the verdict persists. prepare_install_target now only collects it: done → go on, failed → the corrupt-medium error with the unit's journal tail, still running → wait with progress read from the hasher's /proc/PID/io, never started → start it and wait. Measured in the install harness, the phase drops from 1.6s to 0.0s. The unit is the only verifier: the inline hashlib loop goes, and with it the squashfs fallback location for the stream, which only existed so a live root and an orchestrator from either side of the move to the plain ISO file could still pair up. Every ISO now ships the stream and its checksum at /run/archiso/bootmnt/arch/x86_64, which both conditions of the unit require; an orchestrator that finds no unit to ask fails the install instead of hashing quietly. The wizard-time prefetch in .automated_script.sh waits for the unit before reading the image: two sequential readers on one USB stick seek against each other, and the unit's pass is the warm-up anyway. Its head read afterwards is a cache hit where the image fit, and re-warms the leading bytes where a small budget let the kernel drop them behind the hash. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01DvydT2sYFgBvji2nk9QbDp --- .../omarchy-root-image-verify.service | 1 + .../system/omarchy-root-image-verify.service | 23 +++ configs/airootfs/root/.automated_script.sh | 17 ++- .../omarchy-iso/orchestrator/phases_impl.py | 139 ++++++++++++----- test/unit/test_root_image.py | 143 ++++++++++++------ 5 files changed, 231 insertions(+), 92 deletions(-) create mode 120000 configs/airootfs/etc/systemd/system/multi-user.target.wants/omarchy-root-image-verify.service create mode 100644 configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service diff --git a/configs/airootfs/etc/systemd/system/multi-user.target.wants/omarchy-root-image-verify.service b/configs/airootfs/etc/systemd/system/multi-user.target.wants/omarchy-root-image-verify.service new file mode 120000 index 000000000..3afc651ff --- /dev/null +++ b/configs/airootfs/etc/systemd/system/multi-user.target.wants/omarchy-root-image-verify.service @@ -0,0 +1 @@ +/etc/systemd/system/omarchy-root-image-verify.service \ No newline at end of file diff --git a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service new file mode 100644 index 000000000..04728ba54 --- /dev/null +++ b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service @@ -0,0 +1,23 @@ +[Unit] +Description=Verify the Omarchy root image on the install medium +# The root image and its checksum are plain files on the ISO, which the archiso +# initramfs mounts before pivoting, so they are there from the first unit on. +# Starting the hash here, rather than when the install begins, hides the read +# of a multi-GB stream behind the time the user spends in the configurator. +# This is the only verifier: the orchestrator's prepare_install_target phase +# collects the verdict, waits for the unit if it is still running, and fails +# the install if it could not run. +ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs +ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.sha256 + +[Service] +Type=oneshot +RemainAfterExit=yes +WorkingDirectory=/run/archiso/bootmnt/arch/x86_64 +ExecStart=/usr/bin/sha256sum --check --strict omarchy-root.btrfs.sha256 +# Background work: never compete with the console or anything interactive. +Nice=19 +IOSchedulingClass=idle + +[Install] +WantedBy=multi-user.target diff --git a/configs/airootfs/root/.automated_script.sh b/configs/airootfs/root/.automated_script.sh index 021b9e5b6..be0a947ee 100644 --- a/configs/airootfs/root/.automated_script.sh +++ b/configs/airootfs/root/.automated_script.sh @@ -70,17 +70,24 @@ fi # Set OMARCHY_NO_PREFETCH=1 to A/B the same ISO with this disabled. warm_offline_mirror() { local mirror=/var/cache/omarchy/mirror/offline - local image budget_kb spent_kb=0 size_kb path - # Same candidates, same order, as the orchestrator's ROOT_IMAGE_STREAM_CANDIDATES. - for image in /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs /var/cache/omarchy/rootfs/omarchy-root.btrfs; do - [[ -f $image ]] && break - done + local budget_kb spent_kb=0 size_kb path + # The orchestrator's ROOT_IMAGE_STREAM. + local image=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs [[ ${OMARCHY_NO_PREFETCH:-} == 1 ]] && return 0 budget_kb=$(($(awk '/^MemAvailable:/ { print $2 }' /proc/meminfo) / 2)) ((budget_kb > 262144)) || return 0 + # omarchy-root-image-verify.service (started at boot) reads the whole image + # front to back as it hashes it, and two readers on one USB stick seek + # against each other: let it finish first. Its pass is the warm-up for the + # image too; the head read below is then a cache hit where the image fit, + # and re-warms the front where a small budget made the kernel drop it. + while [[ $(systemctl is-active omarchy-root-image-verify.service 2>/dev/null) == activating ]]; do + sleep 1 + done + # The image first, and only as much of it as fits: btrfs receive reads it # front to back, so the leading bytes are the ones worth having cached. if [[ -f $image ]]; then diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 70c4e4e67..c1a4f88c5 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -128,25 +128,21 @@ def _omarchy_nvim_package() -> str: # The root image: a `btrfs send --compressed-data` stream of the invariant # target system (build-root-image.sh), unpacked onto the target in place of # pacstrapping it. build-iso.sh ships it as a plain file on the ISO, read -# straight off the boot medium; the squashfs location is where builds before -# that move put it, kept so an older live root and a newer orchestrator (or -# the reverse) still find the stream. The subvolume name is what -# build-root-image.sh sends. -ROOT_IMAGE_STREAM_CANDIDATES = ( - Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs"), - Path("/var/cache/omarchy/rootfs/omarchy-root.btrfs"), -) -# build-iso.sh writes sha256sum output for the stream next to it. -ROOT_IMAGE_CHECKSUM_SUFFIX = ".sha256" +# straight off the boot medium, with sha256sum output for it next to it. The +# subvolume name is what build-root-image.sh sends. +ROOT_IMAGE_STREAM = Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs") +ROOT_IMAGE_CHECKSUM = ROOT_IMAGE_STREAM.with_name(ROOT_IMAGE_STREAM.name + ".sha256") +# The live ISO starts this oneshot at boot: `sha256sum -c` of the stream, +# running while the user is in the configurator. It is the only verifier; +# verify_root_image_stream collects its verdict. +ROOT_IMAGE_VERIFY_UNIT = "omarchy-root-image-verify.service" ROOT_IMAGE_SUBVOLUME = "omarchy-root" def _root_image_stream() -> Path: - for candidate in ROOT_IMAGE_STREAM_CANDIDATES: - if candidate.is_file(): - return candidate - searched = ", ".join(str(c) for c in ROOT_IMAGE_STREAM_CANDIDATES) - raise RuntimeError(f"root image stream missing; looked at {searched}") + if not ROOT_IMAGE_STREAM.is_file(): + raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") + return ROOT_IMAGE_STREAM # Packages the image must carry for the rest of the install to work: Limine # setup reads the settings package's limine config, useradd copies the skel the @@ -256,38 +252,99 @@ def verify_root_image_stream(ctx: InstallContext) -> None: """The stream is present and hashes to what the build recorded. A truncated or corrupt copy (a badly flashed USB is the common case) would also trip btrfs receive's per-command checksums, but only after the disk - is formatted. Reading the whole stream here also leaves as much of it as - fits in the page cache for the unpack that follows.""" + is formatted. + + The hash is ROOT_IMAGE_VERIFY_UNIT's job: it starts at boot and reads + the stream while the user is in the configurator, so by now it is + usually long done and this only collects the verdict. If it is still + running, wait for it; if it never started, start it and wait. The read + also leaves as much of the stream as fits in the page cache for the + unpack that follows.""" stream_path = _root_image_stream() - checksum = stream_path.with_name(stream_path.name + ROOT_IMAGE_CHECKSUM_SUFFIX) - if not checksum.is_file(): - raise RuntimeError(f"root image checksum missing: {checksum}") + if not ROOT_IMAGE_CHECKSUM.is_file(): + raise RuntimeError(f"root image checksum missing: {ROOT_IMAGE_CHECKSUM}") + total = stream_path.stat().st_size - # sha256sum output: " ". - fields = checksum.read_text().split() - expected = fields[0].lower() if fields else "" - if not re.fullmatch(r"[0-9a-f]{64}", expected): - raise RuntimeError(f"root image checksum unreadable: {checksum}") + state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) + if state.get("LoadState") != "loaded": + raise RuntimeError(f"{ROOT_IMAGE_VERIFY_UNIT} is not on this live system; cannot verify the root image") + if state.get("ActiveState") == "inactive": + # Not started yet. Blocks until the oneshot exits. + subprocess.run(["systemctl", "start", ROOT_IMAGE_VERIFY_UNIT], check=False, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) - total = stream_path.stat().st_size - info(f"› verifying root image ({total >> 20} MiB stream at {stream_path})") - digest = hashlib.sha256() - done = 0 + reported = False last_report = 0.0 - with stream_path.open("rb") as stream: - while chunk := stream.read(8 << 20): - digest.update(chunk) - done += len(chunk) - now = time.monotonic() - if now - last_report >= 0.5: + while True: + active = state.get("ActiveState") + if active == "active": + info(f"› root image verified by {ROOT_IMAGE_VERIFY_UNIT} ({total >> 20} MiB stream at {stream_path})") + return + if active == "failed": + raise RuntimeError( + f"root image stream is corrupt: {ROOT_IMAGE_VERIFY_UNIT} failed " + f"({stream_path}, {total} bytes); re-flash the install medium" + + _journal_tail(ROOT_IMAGE_VERIFY_UNIT) + ) + if active != "activating": + raise RuntimeError( + f"{ROOT_IMAGE_VERIFY_UNIT} did not run (state {active!r}); cannot verify the root image" + + _journal_tail(ROOT_IMAGE_VERIFY_UNIT) + ) + + if not reported: + info(f"› waiting for {ROOT_IMAGE_VERIFY_UNIT} ({total >> 20} MiB stream at {stream_path})") + reported = True + now = time.monotonic() + if now - last_report >= 0.5: + done = _process_read_bytes(state.get("MainPID")) + if done is not None: _write_phase_progress(ctx, done / total if total else 1.0) - last_report = now - actual = digest.hexdigest() - if actual != expected: - raise RuntimeError( - f"root image stream is corrupt: sha256 {actual} != {expected} " - f"({stream_path}, {total} bytes); re-flash the install medium" + last_report = now + time.sleep(0.25) + state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) + + +def _systemctl_show(unit: str) -> dict[str, str]: + """`systemctl show` of the properties the verify handoff reads, as a + dict. Empty when systemctl is unavailable (unit tests off the ISO).""" + try: + result = subprocess.run( + ["systemctl", "show", "-p", "LoadState,ActiveState,MainPID", unit], + capture_output=True, text=True, check=False, + ) + except OSError: + return {} + if result.returncode != 0: + return {} + return dict(line.split("=", 1) for line in result.stdout.splitlines() if "=" in line) + + +def _process_read_bytes(pid: str | None) -> int | None: + """Bytes the process has read so far (rchar from /proc/PID/io): the + progress of a sequential hash. None when unknown.""" + if not pid or pid == "0": + return None + try: + for line in Path(f"/proc/{pid}/io").read_text().splitlines(): + if line.startswith("rchar:"): + return int(line.split()[1]) + except (OSError, ValueError, IndexError): + pass + return None + + +def _journal_tail(unit: str) -> str: + try: + result = subprocess.run( + ["journalctl", "-b", "-u", unit, "--no-pager", "-o", "cat", "-n", "5"], + capture_output=True, text=True, check=False, ) + except OSError: + return "" + output = result.stdout.strip() + return f"\n{output}" if output else "" def arch_install_system(ctx: InstallContext) -> None: diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 3d9d016e4..8e3c9c421 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -1,12 +1,12 @@ """Unit tests for the root image install path in the orchestrator. Covers the pre-flight checks prepare_install_target runs before the disk is -touched (stream present and matching its checksum, a disk layout the image -can land on) and the destructive subvolume dance in _install_root_image, +touched (stream present and verified by the boot-time unit, a disk layout +the image can land on) and the destructive subvolume dance in _install_root_image, asserted on the subprocess calls against a temp target. """ -import hashlib +import os import shutil import sys import tempfile @@ -78,6 +78,16 @@ def test_empty_config_rejected(self): class VerifyRootImageStreamTest(unittest.TestCase): + """The boot-time verify unit is the only hasher: this collects its + verdict, waits for it when it is still running, starts it when it never + ran, and fails the install when it cannot vouch for the stream.""" + + UNIT = phases_impl.ROOT_IMAGE_VERIFY_UNIT + ACTIVE = {"LoadState": "loaded", "ActiveState": "active", "MainPID": "0"} + FAILED = {"LoadState": "loaded", "ActiveState": "failed", "MainPID": "0"} + RUNNING = {"LoadState": "loaded", "ActiveState": "activating", "MainPID": "4242"} + IDLE = {"LoadState": "loaded", "ActiveState": "inactive", "MainPID": "0"} + def setUp(self): self.tmp = tempfile.TemporaryDirectory() self.addCleanup(self.tmp.cleanup) @@ -85,64 +95,105 @@ def setUp(self): self.stream = self.dir / "omarchy-root.btrfs" self.checksum = self.dir / "omarchy-root.btrfs.sha256" self.ctx = types.SimpleNamespace(state_dir=self.dir / "state") + self.runs = [] + self.progress = [] + + def fake_run(cmd, **kwargs): + self.runs.append(cmd) + return CompletedProcess(cmd, 0, stdout="", stderr="") - patch = mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (self.stream,)) - patch.start() - self.addCleanup(patch.stop) - for name in ("info", "_write_phase_progress"): - patch = mock.patch.object(phases_impl, name) + for patch in ( + mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), + mock.patch.object(phases_impl, "ROOT_IMAGE_CHECKSUM", self.checksum), + mock.patch.object(phases_impl, "info"), + mock.patch.object(phases_impl, "_write_phase_progress", + side_effect=lambda ctx, f: self.progress.append(f)), + mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run), + mock.patch.object(phases_impl.time, "sleep"), + mock.patch.object(phases_impl, "_journal_tail", + return_value="\nsha256sum: WARNING: 1 computed checksum did NOT match"), + ): patch.start() self.addCleanup(patch.stop) - def write_stream(self, data=b"btrfs-stream" * 1000): - self.stream.write_bytes(data) - self.checksum.write_text(f"{hashlib.sha256(data).hexdigest()} {self.stream.name}\n") + def write_stream(self): + # Contents are irrelevant: the orchestrator never hashes, the unit does. + self.stream.write_bytes(b"btrfs-stream" * 1000) + self.checksum.write_text(f"{'0' * 64} {self.stream.name}\n") - def test_matching_checksum_passes(self): - self.write_stream() - phases_impl.verify_root_image_stream(self.ctx) + def states(self, *seq): + it = iter(seq) + last = seq[-1] + return mock.patch.object(phases_impl, "_systemctl_show", side_effect=lambda unit: next(it, last)) - def test_missing_stream(self): - with self.assertRaisesRegex(RuntimeError, "stream missing"): + def test_unit_already_succeeded(self): + self.write_stream() + with self.states(self.ACTIVE): phases_impl.verify_root_image_stream(self.ctx) + self.assertEqual(self.runs, []) - def test_missing_checksum(self): - self.stream.write_bytes(b"x") - with self.assertRaisesRegex(RuntimeError, "checksum missing"): - phases_impl.verify_root_image_stream(self.ctx) + def test_unit_failure_is_a_corrupt_medium(self): + self.write_stream() + with self.states(self.FAILED): + with self.assertRaisesRegex(RuntimeError, r"(?s)corrupt.*re-flash.*did NOT match"): + phases_impl.verify_root_image_stream(self.ctx) - def test_unreadable_checksum(self): - self.stream.write_bytes(b"x") - self.checksum.write_text("not a digest\n") - with self.assertRaisesRegex(RuntimeError, "checksum unreadable"): + def test_waits_for_a_running_unit_and_reports_its_progress(self): + self.write_stream() + total = self.stream.stat().st_size + reads = iter([total // 4, total // 2]) + with self.states(self.RUNNING, self.RUNNING, self.ACTIVE), \ + mock.patch.object(phases_impl, "_process_read_bytes", side_effect=lambda pid: next(reads)), \ + mock.patch.object(phases_impl.time, "monotonic", side_effect=[1.0, 2.0]): phases_impl.verify_root_image_stream(self.ctx) + self.assertEqual(self.progress, [0.25, 0.5]) + self.assertEqual(self.runs, []) - def test_truncated_stream_rejected(self): + def test_unit_not_started_is_started_and_waited_for(self): self.write_stream() - data = self.stream.read_bytes() - self.stream.write_bytes(data[: len(data) // 2]) - with self.assertRaisesRegex(RuntimeError, "corrupt"): + with self.states(self.IDLE, self.ACTIVE): phases_impl.verify_root_image_stream(self.ctx) + self.assertEqual(self.runs, [["systemctl", "start", self.UNIT]]) + + def test_unit_that_will_not_run_fails_the_install(self): + self.write_stream() + with self.states(self.IDLE, self.IDLE): + with self.assertRaisesRegex(RuntimeError, "did not run"): + phases_impl.verify_root_image_stream(self.ctx) + self.assertEqual(self.runs, [["systemctl", "start", self.UNIT]]) + + def test_missing_unit_fails_the_install(self): + self.write_stream() + with self.states({"LoadState": "not-found"}): + with self.assertRaisesRegex(RuntimeError, "not on this live system"): + phases_impl.verify_root_image_stream(self.ctx) + + def test_missing_stream(self): + with self.states(self.ACTIVE): + with self.assertRaisesRegex(RuntimeError, "stream missing"): + phases_impl.verify_root_image_stream(self.ctx) + + def test_missing_checksum(self): + self.stream.write_bytes(b"x") + with self.states(self.ACTIVE): + with self.assertRaisesRegex(RuntimeError, "checksum missing"): + phases_impl.verify_root_image_stream(self.ctx) + def test_systemctl_show_parses_properties(self): + phases_impl.subprocess.run.side_effect = lambda cmd, **kw: CompletedProcess( + cmd, 0, stdout="LoadState=loaded\nActiveState=activating\nMainPID=7\n", stderr="") + self.assertEqual(phases_impl._systemctl_show("x.service"), + {"LoadState": "loaded", "ActiveState": "activating", "MainPID": "7"}) -class RootImageStreamLookupTest(unittest.TestCase): - def test_first_existing_candidate_wins(self): - with tempfile.TemporaryDirectory() as tmp: - iso = Path(tmp) / "bootmnt/omarchy-root.btrfs" - sfs = Path(tmp) / "rootfs/omarchy-root.btrfs" - sfs.parent.mkdir() - sfs.write_bytes(b"old") - with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (iso, sfs)): - self.assertEqual(phases_impl._root_image_stream(), sfs) - iso.parent.mkdir() - iso.write_bytes(b"new") - self.assertEqual(phases_impl._root_image_stream(), iso) + def test_systemctl_missing_means_no_unit(self): + phases_impl.subprocess.run.side_effect = FileNotFoundError("systemctl") + self.assertEqual(phases_impl._systemctl_show("x.service"), {}) - def test_no_candidate_lists_every_path(self): - with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", - (Path("/nonexistent/a"), Path("/nonexistent/b"))): - with self.assertRaisesRegex(RuntimeError, "stream missing.*/nonexistent/a.*/nonexistent/b"): - phases_impl._root_image_stream() + def test_process_read_bytes(self): + self.assertIsNone(phases_impl._process_read_bytes("0")) + self.assertIsNone(phases_impl._process_read_bytes(None)) + self.assertIsNone(phases_impl._process_read_bytes("99999999")) + self.assertIsInstance(phases_impl._process_read_bytes(str(os.getpid())), int) class PrepareInstallTargetTest(unittest.TestCase): @@ -239,7 +290,7 @@ def fake_receive(ctx, top, stream_path): return_value=["limine", "omarchy-keyring", "omarchy"]), mock.patch.object(phases_impl.arch, "target_has_package", create=True, side_effect=lambda target, pkg: pkg in self.received_packages), - mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM_CANDIDATES", (self.stream,)), + mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), mock.patch.object(phases_impl, "info"), ] for patch in patches: From 8ef3a32e548803b47548654233b87c248930a85a Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 07:05:39 +0200 Subject: [PATCH 14/67] Put the re-flash advice on screen, and test a corrupt medium end to end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A corrupt install medium failed the install with the right error and the disk untouched, but the dashboard centres each line of the failure in ~80 columns and clipped the one sentence that mattered: "root image stream is corrupt: omarchy-ro…" — the advice to re-flash never reached the screen, and four of the five "last log lines" were systemd's exit/failed/consumed boilerplate, which had pushed sha256sum's own "FAILED" line out of the tail. Seen on a throttled-cdrom run with one digit of the recorded sha256 flipped. Lead with the action on its own short line ("install medium is corrupt: re-flash it"), put the detail on the next, and append only what sha256sum wrote (journalctl -u _COMM=sha256sum) instead of the last five journal lines. The dashboard folds the failed phase's lines at word boundaries rather than truncating them. The corrupt-image integration scenario keeps it that way: copy the ISO, flip one hex digit of the checksum in place (ISO9660 has no per-file integrity data; reflink makes the copy free), autoinstall from it, and assert the verify unit failed, the install halted in the pre-flight phase with the re-flash advice and sha256sum's verdict, nothing later ran, the target disk still has no partition table, and both the stop screen and the advice are visible. It boots the ISO itself rather than the installed base and reaches the live root over SSH through a tty3 console login, now a base-test.sh helper (bootstrap_live_root_ssh / ssh_live_root). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01DvydT2sYFgBvji2nk9QbDp --- README.md | 2 + .../usr/local/bin/omarchy-install-dashboard | 7 +- .../omarchy-iso/orchestrator/phases_impl.py | 22 ++-- test/integration.d/base-test.sh | 73 +++++++++++ test/integration.d/corrupt-image-test.sh | 118 ++++++++++++++++++ test/unit/test_root_image.py | 31 ++++- 6 files changed, 243 insertions(+), 10 deletions(-) create mode 100755 test/integration.d/corrupt-image-test.sh diff --git a/README.md b/README.md index 4566b025e..e288ae010 100644 --- a/README.md +++ b/README.md @@ -122,6 +122,8 @@ Scenarios under `test/integration.d/` boot a real ISO install in QEMU and assert The first scenario is `factory-reset`: it proves `omarchy-system-factory-reset` hands a machine on without destroying a shared ESP. The installed ESP gets a Windows entry with payload plus a second Linux cloned under a foreign machine-id with its own boot directory and UKIs; a real factory reset is then driven through a guest pty, and the harness asserts the foreign entries survive both the staged reset and first-boot provisioning, that the old Omarchy identity is fully retired, and that the machine reaches first-boot setup unattended. +`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, flips one digit of the root image's recorded sha256, and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. + Artifacts — screenshots, the fixtured/staged/final `limine.conf`, the reset typescript, and the factory-reset log — land under `test-runs/-integration/runs/-/`, and `--no-preview` skips the `imv` review just like the acceptance harness. ## Signing the ISO diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 50f9aec79..1b9020397 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -665,9 +665,14 @@ render_failure() { if [[ -n $state_summary ]]; then blank_line + # The failed phase's error is the one thing the user must read, so wrap + # it at word boundaries rather than clip it: a multi-line error keeps its + # own line breaks, and an over-long line folds instead of losing its end. while IFS= read -r line; do [[ -n $line ]] || continue - center "${DIM}$(truncate_for_width "$line" "$CONTENT_WIDTH")${RESET}" "$CONTENT_WIDTH" + while IFS= read -r part; do + center "${DIM}${part}${RESET}" "$CONTENT_WIDTH" + done < <(fold -s -w "$CONTENT_WIDTH" <<<"$line") done <<<"$state_summary" fi diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index c1a4f88c5..5df9aec49 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -282,10 +282,12 @@ def verify_root_image_stream(ctx: InstallContext) -> None: info(f"› root image verified by {ROOT_IMAGE_VERIFY_UNIT} ({total >> 20} MiB stream at {stream_path})") return if active == "failed": + # The dashboard centres each line of this in ~80 columns, so the + # action comes first and on its own line; the detail follows. raise RuntimeError( - f"root image stream is corrupt: {ROOT_IMAGE_VERIFY_UNIT} failed " - f"({stream_path}, {total} bytes); re-flash the install medium" - + _journal_tail(ROOT_IMAGE_VERIFY_UNIT) + "install medium is corrupt: re-flash it\n" + f"sha256 mismatch on {stream_path.name} ({total} bytes, {ROOT_IMAGE_VERIFY_UNIT})" + + _journal_tail(ROOT_IMAGE_VERIFY_UNIT, comm="sha256sum") ) if active != "activating": raise RuntimeError( @@ -335,12 +337,16 @@ def _process_read_bytes(pid: str | None) -> int | None: return None -def _journal_tail(unit: str) -> str: +def _journal_tail(unit: str, comm: str | None = None) -> str: + """This boot's journal for the unit, one line per entry, prefixed with a + newline for appending to an error. With comm, only what that program + wrote: the unit's own verdict without systemd's exit/failed/consumed + boilerplate around it.""" + cmd = ["journalctl", "-b", "-u", unit, "--no-pager", "-o", "cat", "-n", "5"] + if comm: + cmd = ["journalctl", "-b", "-u", unit, f"_COMM={comm}", "--no-pager", "-o", "cat"] try: - result = subprocess.run( - ["journalctl", "-b", "-u", unit, "--no-pager", "-o", "cat", "-n", "5"], - capture_output=True, text=True, check=False, - ) + result = subprocess.run(cmd, capture_output=True, text=True, check=False) except OSError: return "" output = result.stdout.strip() diff --git a/test/integration.d/base-test.sh b/test/integration.d/base-test.sh index 10d4d2e2b..34dd2a6dd 100644 --- a/test/integration.d/base-test.sh +++ b/test/integration.d/base-test.sh @@ -353,6 +353,79 @@ EOF press ctrl-alt-f1 } +# Root SSH into the live ISO itself (not the installed system). The live root +# runs sshd and autologs in on tty1 with an empty password, but sshd refuses +# empty passwords, so authorize our key the same console-login way as above, +# on tty3 (tty1 is the installer's). +ssh_live_root() { + ssh -i "$SSH_KEY" -p "$SSH_PORT" \ + -o BatchMode=yes \ + -o IdentitiesOnly=yes \ + -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null \ + -o ConnectTimeout=5 \ + -o LogLevel=ERROR \ + "root@127.0.0.1" "$@" +} + +bootstrap_live_root_ssh() { + log "Authorizing root SSH on the live ISO via console login" + + mkdir -p "$BASE_DIR/www" + cat >"$BASE_DIR/www/bootstrap-live" <>/root/.ssh/authorized_keys +chmod 600 /root/.ssh/authorized_keys +EOF + + (cd "$BASE_DIR/www" && exec python3 -m http.server "$HTTP_PORT" --bind 127.0.0.1 >/dev/null 2>&1) & + HTTP_PID=$! + + local waited=0 + while true; do + press ctrl-alt-f3 + sleep 4 + ocr_screen | grep -qi "login:" && break + ((waited += 8)) + + if ((waited >= 300)); then + capture_console "failure-live-console-timeout" + echo "Timed out waiting for a live console login prompt" >&2 + return 1 + fi + sleep 4 + done + + type_text root + press ret + sleep 2 + # The password is empty; login still prompts for it. + ocr_screen | grep -qi "password" && { press ret; sleep 2; } + + type_text "curl -fsS http://10.0.2.2:$HTTP_PORT/bootstrap-live -o /tmp/bs && bash /tmp/bs" + press ret + + waited=0 + until ssh_live_root true 2>/dev/null; do + if ! vm_running; then + echo "VM exited while waiting for live root SSH" >&2 + return 1 + fi + if ((waited >= 180)); then + capture_console "failure-live-ssh-timeout" + echo "Timed out waiting for root SSH on the live ISO" >&2 + return 1 + fi + sleep 2 + ((waited += 2)) + done + capture_console "success-live-root-ssh" + + kill "$HTTP_PID" 2>/dev/null || true + HTTP_PID="" + press ctrl-alt-f1 +} + # ---------------------------------------------------------- cidata autoinstall # The configurator's own output files, synthesized for the 40G virtio disk the diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh new file mode 100755 index 000000000..ced26202e --- /dev/null +++ b/test/integration.d/corrupt-image-test.sh @@ -0,0 +1,118 @@ +#!/bin/bash +# +# A corrupt install medium is refused before the disk is touched. The root +# image ships on the ISO with its sha256 beside it; omarchy-root-image-verify +# checks it at boot and the installer's pre-flight phase takes that verdict. +# Flip one digit of the recorded checksum on a copy of the ISO, autoinstall +# from it, and assert: the unit fails, the install halts in "Preparing install +# target" telling the user to re-flash, nothing after that phase ran, and the +# target disk still has no partition table. +# +# Boots the ISO itself, not the installed base image, so it needs no base. + +source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" + +CORRUPT_ISO="$BASE_DIR/corrupt.iso" +STREAM_SUM=arch/x86_64/omarchy-root.btrfs.sha256 +VERIFY_UNIT=omarchy-root-image-verify.service +STATE=/run/omarchy-install/state.json + +# ------------------------------------------------------------------ fixture + +# A copy of the ISO with the first hex digit of the recorded checksum flipped. +# ISO9660 carries no per-file integrity data, so patching the byte in place +# leaves everything else on the medium intact; cp --reflink makes the copy +# free on btrfs. +corrupt_iso() { + local digest offset flipped patched + + digest=$(bsdtar -xOf "$ISO" "$STREAM_SUM" | cut -d' ' -f1) + [[ $digest =~ ^[0-9a-f]{64}$ ]] || { echo "no checksum on the ISO at $STREAM_SUM" >&2; return 1; } + + log "Copying the ISO and corrupting its root image checksum" + rm -f "$CORRUPT_ISO" + cp --reflink=auto "$ISO" "$CORRUPT_ISO" + offset=$(grep -boa -m1 "$digest" "$CORRUPT_ISO" | cut -d: -f1) + [[ -n $offset ]] || { echo "checksum not found in the ISO image" >&2; return 1; } + flipped=1 + [[ ${digest:0:1} == 1 ]] && flipped=0 + printf '%s' "$flipped" | dd of="$CORRUPT_ISO" bs=1 seek="$offset" conv=notrunc status=none + + patched=$(bsdtar -xOf "$CORRUPT_ISO" "$STREAM_SUM" | cut -d' ' -f1) + [[ $patched =~ ^[0-9a-f]{64}$ && $patched != "$digest" ]] || + { echo "patch did not take: $patched" >&2; return 1; } + log "Recorded $digest, now $patched" +} + +# -------------------------------------------------------------------- phases + +install_from_corrupt_medium() { + [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-integration" -f "$SSH_KEY" + detect_packages + build_cidata + + qemu-img create -f qcow2 "$RUN_DIR/disk.qcow2" 40G >/dev/null + cp "$OVMF_VARS_TEMPLATE" "$RUN_DIR/OVMF_VARS.4m.fd" + ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + + log "Autoinstalling from the corrupt medium (headless)" + start_vm "$RUN_DIR/disk.qcow2" "$RUN_DIR/serial.log" \ + -drive "file=$CORRUPT_ISO,media=cdrom,if=none,format=raw,id=cdrom0" \ + -device ide-cd,drive=cdrom0,bootindex=2 \ + -drive "file=$CIDATA_IMG,format=raw,if=none,id=cidata" \ + -device usb-storage,drive=cidata + + bootstrap_live_root_ssh + + log "Waiting for the installer to stop" + local waited=0 + until ssh_live_root 'grep -q "installer child exited" /var/log/omarchy-install.log' 2>/dev/null; do + if ! vm_running; then + echo "VM exited while waiting for the installer" >&2 + return 1 + fi + if ((waited >= 600)); then + capture_console "failure-installer-timeout" + echo "Timed out waiting for the installer to stop" >&2 + return 1 + fi + sleep 5 + ((waited += 5)) + done + sleep 2 + capture_console "success-installer-stopped" + ssh_live_root "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true + ssh_live_root "cat $STATE" >"$RUN_DIR/state.json" 2>/dev/null || true + ssh_live_root "journalctl -b -u $VERIFY_UNIT -o short-precise --no-pager" >"$RUN_DIR/verify-unit.journal" 2>/dev/null || true +} + +screen_shows() { + ocr_screen | grep -qi "$1" +} + +assert_refused() { + check "verify unit failed on the corrupt checksum" \ + ssh_live_root "systemctl show -p Result --value $VERIFY_UNIT | grep -qx exit-code" + check "install halted in the pre-flight phase" \ + ssh_live_root "jq -e '[.phases[] | select(.status == \"failed\") | .name] == [\"Preparing install target\"]' $STATE" + check "the error tells the user to re-flash the medium" \ + ssh_live_root "jq -r '.phases[] | select(.status == \"failed\") | .error' $STATE | grep -q 'install medium is corrupt: re-flash it'" + check "the error carries sha256sum's verdict" \ + ssh_live_root "jq -r '.phases[] | select(.status == \"failed\") | .error' $STATE | grep -q 'did NOT match'" + check "nothing ran after the pre-flight phase" \ + ssh_live_root "jq -e '[.phases[] | select(.status == \"ok\") | .name] == [\"Preparing live environment\"]' $STATE" + check "target disk has no partition table" \ + ssh_live_root "! lsblk -rno TYPE /dev/vda | grep -qx part && ! blkid /dev/vda" + check "dashboard shows the installation stopped" \ + screen_shows "installation stopped" + check "dashboard shows the re-flash advice" \ + screen_shows "re-flash" +} + +# ---------------------------------------------------------------------- main + +corrupt_iso +install_from_corrupt_medium +assert_refused +rm -f "$CORRUPT_ISO" +finish diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 8e3c9c421..d0bf81141 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -97,6 +97,7 @@ def setUp(self): self.ctx = types.SimpleNamespace(state_dir=self.dir / "state") self.runs = [] self.progress = [] + self.real_journal_tail = phases_impl._journal_tail def fake_run(cmd, **kwargs): self.runs.append(cmd) @@ -135,8 +136,36 @@ def test_unit_already_succeeded(self): def test_unit_failure_is_a_corrupt_medium(self): self.write_stream() with self.states(self.FAILED): - with self.assertRaisesRegex(RuntimeError, r"(?s)corrupt.*re-flash.*did NOT match"): + with self.assertRaises(RuntimeError) as raised: phases_impl.verify_root_image_stream(self.ctx) + lines = str(raised.exception).splitlines() + # The dashboard centres each line in ~80 columns: the action first, + # short enough to survive, then the detail, then sha256sum's verdict. + self.assertEqual(lines[0], "install medium is corrupt: re-flash it") + self.assertLess(len(lines[0]), 50) + self.assertRegex(lines[1], r"sha256 mismatch on omarchy-root\.btrfs \(\d+ bytes, omarchy-root-image-verify\.service\)") + self.assertEqual(lines[2], "sha256sum: WARNING: 1 computed checksum did NOT match") + phases_impl._journal_tail.assert_called_once_with(self.UNIT, comm="sha256sum") + + def test_journal_tail_filters_to_the_hasher(self): + # setUp patches _journal_tail for the handoff tests; exercise the real one. + real = self.real_journal_tail + def journal(cmd, **kw): + self.runs.append(cmd) + return CompletedProcess( + cmd, 0, stdout="omarchy-root.btrfs: FAILED\nsha256sum: WARNING: 1 computed checksum did NOT match\n", stderr="") + phases_impl.subprocess.run.side_effect = journal + self.assertEqual( + real(self.UNIT, comm="sha256sum"), + "\nomarchy-root.btrfs: FAILED\nsha256sum: WARNING: 1 computed checksum did NOT match", + ) + self.assertEqual( + self.runs[-1], + ["journalctl", "-b", "-u", self.UNIT, "_COMM=sha256sum", "--no-pager", "-o", "cat"], + ) + real(self.UNIT) + self.assertEqual(self.runs[-1][:5], ["journalctl", "-b", "-u", self.UNIT, "--no-pager"]) + self.assertNotIn("_COMM=sha256sum", self.runs[-1]) def test_waits_for_a_running_unit_and_reports_its_progress(self): self.write_stream() From 07fc5f085a702b14738470638d4f2229675aabba Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 09:01:55 +0200 Subject: [PATCH 15/67] Corrupt the root image itself in the corrupt-image scenario The scenario flipped a digit of the recorded checksum, which exercises the same failure but is not what a bad medium looks like: on a badly flashed stick the checksum file is fine and the bytes under it are not. Damage the image instead: find the stream on the ISO copy by the NUL-terminated magic every btrfs send stream starts with, and overwrite 16 bytes a third of the way in, deep in extent data. The checksum file is untouched. The fixture no longer re-hashes the damaged stream to check its own work; the assertions on the install's behaviour are the test. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01DvydT2sYFgBvji2nk9QbDp --- README.md | 2 +- test/integration.d/corrupt-image-test.sh | 48 ++++++++++++------------ 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index e288ae010..f1e7082d8 100644 --- a/README.md +++ b/README.md @@ -122,7 +122,7 @@ Scenarios under `test/integration.d/` boot a real ISO install in QEMU and assert The first scenario is `factory-reset`: it proves `omarchy-system-factory-reset` hands a machine on without destroying a shared ESP. The installed ESP gets a Windows entry with payload plus a second Linux cloned under a foreign machine-id with its own boot directory and UKIs; a real factory reset is then driven through a guest pty, and the harness asserts the foreign entries survive both the staged reset and first-boot provisioning, that the old Omarchy identity is fully retired, and that the machine reaches first-boot setup unattended. -`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, flips one digit of the root image's recorded sha256, and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. +`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, overwrites a few bytes deep inside the root image (the recorded sha256 stays right; the bytes under it don't, like a badly flashed stick), and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. Artifacts — screenshots, the fixtured/staged/final `limine.conf`, the reset typescript, and the factory-reset log — land under `test-runs/-integration/runs/-/`, and `--no-preview` skips the `imv` review just like the acceptance harness. diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh index ced26202e..34bfcb297 100755 --- a/test/integration.d/corrupt-image-test.sh +++ b/test/integration.d/corrupt-image-test.sh @@ -3,45 +3,45 @@ # A corrupt install medium is refused before the disk is touched. The root # image ships on the ISO with its sha256 beside it; omarchy-root-image-verify # checks it at boot and the installer's pre-flight phase takes that verdict. -# Flip one digit of the recorded checksum on a copy of the ISO, autoinstall -# from it, and assert: the unit fails, the install halts in "Preparing install -# target" telling the user to re-flash, nothing after that phase ran, and the -# target disk still has no partition table. +# Damage the image data on a copy of the ISO (the checksum stays right, the +# bytes under it don't: a badly flashed stick), autoinstall from it, and +# assert: the unit fails, the install halts in "Preparing install target" +# telling the user to re-flash, nothing after that phase ran, and the target +# disk still has no partition table. # # Boots the ISO itself, not the installed base image, so it needs no base. source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" CORRUPT_ISO="$BASE_DIR/corrupt.iso" -STREAM_SUM=arch/x86_64/omarchy-root.btrfs.sha256 +STREAM=arch/x86_64/omarchy-root.btrfs VERIFY_UNIT=omarchy-root-image-verify.service STATE=/run/omarchy-install/state.json # ------------------------------------------------------------------ fixture -# A copy of the ISO with the first hex digit of the recorded checksum flipped. -# ISO9660 carries no per-file integrity data, so patching the byte in place -# leaves everything else on the medium intact; cp --reflink makes the copy -# free on btrfs. +# A copy of the ISO with 16 bytes of the root image overwritten (0xFF) a +# third of the way in, deep in extent data. ISO9660 carries no per-file integrity +# data, so patching in place leaves everything else on the medium intact +# (cp --reflink makes the copy free on btrfs). The stream is found by the +# NUL-terminated magic every btrfs send stream starts with; its length comes +# from the ISO's directory. corrupt_iso() { - local digest offset flipped patched + local start size offset + local damage='\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff' - digest=$(bsdtar -xOf "$ISO" "$STREAM_SUM" | cut -d' ' -f1) - [[ $digest =~ ^[0-9a-f]{64}$ ]] || { echo "no checksum on the ISO at $STREAM_SUM" >&2; return 1; } + size=$(bsdtar -tvf "$ISO" "$STREAM" | awk '{print $5}') + [[ $size =~ ^[0-9]+$ ]] || { echo "no root image on the ISO at $STREAM" >&2; return 1; } - log "Copying the ISO and corrupting its root image checksum" + log "Copying the ISO and corrupting its root image" rm -f "$CORRUPT_ISO" cp --reflink=auto "$ISO" "$CORRUPT_ISO" - offset=$(grep -boa -m1 "$digest" "$CORRUPT_ISO" | cut -d: -f1) - [[ -n $offset ]] || { echo "checksum not found in the ISO image" >&2; return 1; } - flipped=1 - [[ ${digest:0:1} == 1 ]] && flipped=0 - printf '%s' "$flipped" | dd of="$CORRUPT_ISO" bs=1 seek="$offset" conv=notrunc status=none - - patched=$(bsdtar -xOf "$CORRUPT_ISO" "$STREAM_SUM" | cut -d' ' -f1) - [[ $patched =~ ^[0-9a-f]{64}$ && $patched != "$digest" ]] || - { echo "patch did not take: $patched" >&2; return 1; } - log "Recorded $digest, now $patched" + start=$(grep -Pboa -m1 'btrfs-stream\x00' "$CORRUPT_ISO" | cut -d: -f1) + [[ -n $start ]] || { echo "btrfs send stream magic not found in the ISO image" >&2; return 1; } + + offset=$((start + size / 3)) + echo -ne "$damage" | dd of="$CORRUPT_ISO" bs=1 seek="$offset" conv=notrunc status=none + log "Image damaged at stream offset $((size / 3))" } # -------------------------------------------------------------------- phases @@ -91,7 +91,7 @@ screen_shows() { } assert_refused() { - check "verify unit failed on the corrupt checksum" \ + check "verify unit failed on the corrupt image" \ ssh_live_root "systemctl show -p Result --value $VERIFY_UNIT | grep -qx exit-code" check "install halted in the pre-flight phase" \ ssh_live_root "jq -e '[.phases[] | select(.status == \"failed\") | .name] == [\"Preparing install target\"]' $STATE" From 8def994f8e17c922f7d40a9f76f3367bd9adf152 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 11:53:44 +0200 Subject: [PATCH 16/67] Pin copytoram=n: the root image is streamed from the boot medium The archiso hook defaults to copytoram=auto, which copies the airootfs to RAM and then unmounts /run/archiso/bootmnt when the boot device is not an optical drive, the airootfs is under 4 GiB (ours is 3.18 GB) and MemAvailable exceeds the image size plus 2 GiB. Every USB-booted laptop with 6 GB or more trips it. The installer then fails in prepare_install_target with "root image stream missing", because the root image is deliberately kept out of the airootfs and read straight off the medium. The QEMU integration tests attach the ISO as an IDE CD-ROM, which is the one case the auto rule excludes, so this only showed up on real hardware (ThinkPad X200s, 8 GB; X200, 6 GB). Add copytoram=n to every live-boot entry (BIOS syslinux, GRUB, systemd-boot), guard that with a unit test, and make the missing-stream error say when the medium was released by copytoram so a hand-edited cmdline fails with a useful message. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- .../omarchy-iso/orchestrator/phases_impl.py | 11 +++++++ .../entries/01-archiso-x86_64-linux.conf | 2 +- configs/grub/grub.cfg | 4 +-- configs/syslinux/archiso_sys-linux.cfg | 4 +-- test/unit/boot-cmdline-test.sh | 30 +++++++++++++++++++ test/unit/test_root_image.py | 9 ++++++ 6 files changed, 55 insertions(+), 5 deletions(-) create mode 100644 test/unit/boot-cmdline-test.sh diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 5df9aec49..1220df152 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -139,8 +139,19 @@ def _omarchy_nvim_package() -> str: ROOT_IMAGE_SUBVOLUME = "omarchy-root" +BOOT_MEDIUM_MOUNT = Path("/run/archiso/bootmnt") + + def _root_image_stream() -> Path: if not ROOT_IMAGE_STREAM.is_file(): + # The archiso hook unmounts the boot medium after copying the airootfs + # to RAM (copytoram). The boot entries pin copytoram=n, so this only + # happens when someone edits the kernel command line. + if not BOOT_MEDIUM_MOUNT.is_dir(): + raise RuntimeError( + f"boot medium is not mounted at {BOOT_MEDIUM_MOUNT}: the live system was " + "copied to RAM (copytoram) and the medium released; boot with copytoram=n" + ) raise RuntimeError(f"root image stream missing: {ROOT_IMAGE_STREAM}") return ROOT_IMAGE_STREAM diff --git a/configs/efiboot/loader/entries/01-archiso-x86_64-linux.conf b/configs/efiboot/loader/entries/01-archiso-x86_64-linux.conf index bf6d18c26..5219d101e 100644 --- a/configs/efiboot/loader/entries/01-archiso-x86_64-linux.conf +++ b/configs/efiboot/loader/entries/01-archiso-x86_64-linux.conf @@ -2,4 +2,4 @@ title Omarchy (x86_64, UEFI) sort-key 01 linux /%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 initrd /%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -options archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% quiet splash xe.enable_panel_replay=0 initramfs_async=0 +options archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% copytoram=n quiet splash xe.enable_panel_replay=0 initramfs_async=0 diff --git a/configs/grub/grub.cfg b/configs/grub/grub.cfg index a0f67ef68..1122ccbc0 100644 --- a/configs/grub/grub.cfg +++ b/configs/grub/grub.cfg @@ -54,13 +54,13 @@ timeout_style=hidden menuentry "Omarchy (%ARCH%, ${archiso_platform})" --class arch --class gnu-linux --class gnu --class os --id 'archlinux' { set gfxpayload=keep - linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% quiet splash xe.enable_panel_replay=0 initramfs_async=0 + linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% copytoram=n quiet splash xe.enable_panel_replay=0 initramfs_async=0 initrd /%INSTALL_DIR%/boot/%ARCH%/initramfs-linux-t2.img } menuentry "Omarchy with speakup screen reader (%ARCH%, ${archiso_platform})" --hotkey s --class arch --class gnu-linux --class gnu --class os --id 'archlinux-accessibility' { set gfxpayload=keep - linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% accessibility=on xe.enable_panel_replay=0 initramfs_async=0 + linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% copytoram=n accessibility=on xe.enable_panel_replay=0 initramfs_async=0 initrd /%INSTALL_DIR%/boot/%ARCH%/initramfs-linux-t2.img } diff --git a/configs/syslinux/archiso_sys-linux.cfg b/configs/syslinux/archiso_sys-linux.cfg index 805a5a732..309c7f344 100644 --- a/configs/syslinux/archiso_sys-linux.cfg +++ b/configs/syslinux/archiso_sys-linux.cfg @@ -6,7 +6,7 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, BIOS) LINUX /%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD /%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% quiet splash xe.enable_panel_replay=0 initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% copytoram=n quiet splash xe.enable_panel_replay=0 initramfs_async=0 # Accessibility boot option LABEL arch64speech @@ -17,4 +17,4 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, BIOS) with ^speech LINUX /%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD /%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% accessibility=on xe.enable_panel_replay=0 initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% copytoram=n accessibility=on xe.enable_panel_replay=0 initramfs_async=0 diff --git a/test/unit/boot-cmdline-test.sh b/test/unit/boot-cmdline-test.sh new file mode 100644 index 000000000..baba215cc --- /dev/null +++ b/test/unit/boot-cmdline-test.sh @@ -0,0 +1,30 @@ +#!/bin/bash +# +# Every boot entry that loads the live system must pin copytoram=n. The +# archiso hook defaults to copytoram=auto, which copies the airootfs to RAM +# and unmounts the boot medium whenever the image is under 4 GiB and the +# machine has RAM to spare -- the root image the installer streams from that +# medium then vanishes (seen on a ThinkPad X200s booting from USB; the QEMU +# tests never hit it because they boot the ISO as an optical drive). + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +cd "$ROOT" + +entries=$(grep -rhE '^\s*(APPEND|linux|options)\s.*archisosearchuuid=' \ + configs/syslinux/archiso_sys-linux.cfg configs/grub/grub.cfg configs/efiboot/loader/entries) + +count=$(printf '%s\n' "$entries" | wc -l) +[ "$count" -ge 5 ] || { echo "expected at least 5 boot entries, found $count"; exit 1; } + +fail=0 +while IFS= read -r line; do + if ! grep -qE '(^|[[:space:]])copytoram=n([[:space:]]|$)' <<<"$line"; then + echo "boot entry without copytoram=n: $line" + fail=1 + fi +done <<<"$entries" + +[ "$fail" -eq 0 ] && echo "ok: $count boot entries pin copytoram=n" +exit "$fail" diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index d0bf81141..981f77199 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -106,6 +106,7 @@ def fake_run(cmd, **kwargs): for patch in ( mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), mock.patch.object(phases_impl, "ROOT_IMAGE_CHECKSUM", self.checksum), + mock.patch.object(phases_impl, "BOOT_MEDIUM_MOUNT", self.dir), mock.patch.object(phases_impl, "info"), mock.patch.object(phases_impl, "_write_phase_progress", side_effect=lambda ctx, f: self.progress.append(f)), @@ -202,6 +203,14 @@ def test_missing_stream(self): with self.assertRaisesRegex(RuntimeError, "stream missing"): phases_impl.verify_root_image_stream(self.ctx) + def test_boot_medium_released_by_copytoram(self): + # What a ThinkPad X200s showed: archiso's copytoram=auto copied the + # airootfs to RAM and unmounted the USB stick, so bootmnt was gone. + with mock.patch.object(phases_impl, "BOOT_MEDIUM_MOUNT", self.dir / "bootmnt"): + with self.states(self.ACTIVE): + with self.assertRaisesRegex(RuntimeError, "copytoram"): + phases_impl.verify_root_image_stream(self.ctx) + def test_missing_checksum(self): self.stream.write_bytes(b"x") with self.states(self.ACTIVE): From 10e20734fad9f341d8677599673d4485608afd92 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 16:57:08 +0200 Subject: [PATCH 17/67] Put the boot medium on BFQ so the verify hash yields to the live system omarchy-root-image-verify.service hashes the multi-GB root image from the boot medium at boot with IOSchedulingClass=idle, while the live system pages its airootfs in lazily from the same medium (copytoram is off). The idle class only means anything under BFQ; the default mq-deadline ignores I/O priority, so on a slow USB stick the hash competes as an equal with every squashfs page-in and the boot crawls. A throttled QEMU boot (usb-storage capped at 33 MB/s, 6 GB RAM, the ISO booted as a real USB stick under SeaBIOS) confirms it: with a buffered sha256sum hog running idle-class, interactive random reads complete at ~50/s under mq-deadline versus ~130/s under BFQ -- roughly 2.5-3x more of the device handed to the live system. The configurator is interactive by ~45s either way while the hash runs to ~120s in the background. Add a udev rule that puts USB disks, SD cards and optical drives on BFQ (internal SATA/NVMe install targets keep their default), a boot-medium helper the verify unit runs as ExecStartPre to log the device and its scheduler next to the verify result, and a unit test over the shipped udev rules. The elevator= kernel parameter cannot do this: it was tied to the legacy single-queue block layer and became a no-op when blk-mq landed in 5.0. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- .../system/omarchy-root-image-verify.service | 3 ++ .../71-omarchy-iso-boot-medium-bfq.rules | 16 +++++++++++ .../usr/local/bin/omarchy-iso-boot-medium | 26 +++++++++++++++++ configs/profiledef.sh | 1 + test/unit/udev-rules-test.sh | 28 +++++++++++++++++++ 5 files changed, 74 insertions(+) create mode 100644 configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules create mode 100755 configs/airootfs/usr/local/bin/omarchy-iso-boot-medium create mode 100644 test/unit/udev-rules-test.sh diff --git a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service index 04728ba54..b19b064bf 100644 --- a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service +++ b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service @@ -14,6 +14,9 @@ ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.sha256 Type=oneshot RemainAfterExit=yes WorkingDirectory=/run/archiso/bootmnt/arch/x86_64 +# Logs the boot device and its I/O scheduler next to the verify result; +# the idle class below only takes effect under bfq. +ExecStartPre=/usr/local/bin/omarchy-iso-boot-medium ExecStart=/usr/bin/sha256sum --check --strict omarchy-root.btrfs.sha256 # Background work: never compete with the console or anything interactive. Nice=19 diff --git a/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules b/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules new file mode 100644 index 000000000..373b313d7 --- /dev/null +++ b/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules @@ -0,0 +1,16 @@ +# Put the live ISO's boot medium on the BFQ I/O scheduler. +# +# The live system pages its airootfs in lazily from the boot medium (copytoram +# is pinned off because the root image is streamed from the same medium), and +# omarchy-root-image-verify.service hashes that multi-GB image from boot with +# IOSchedulingClass=idle. Only BFQ implements I/O priority classes; under the +# default mq-deadline the hash's sequential reads compete as equals with every +# squashfs page-in, and on a USB 2.0 stick the boot crawls for the two minutes +# the hash takes. With BFQ the idle-class hash yields to the live system and +# fills the gaps instead. +# +# Scoped to the media an ISO realistically boots from: USB disks, SD cards and +# optical drives. Internal SATA/NVMe install targets keep their default. +# 71- so it runs after 60-persistent-storage.rules, which sets ID_BUS. +ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ENV{ID_BUS}=="usb", ATTR{queue/scheduler}="bfq" +ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", KERNEL=="sr[0-9]*|mmcblk[0-9]*", ATTR{queue/scheduler}="bfq" diff --git a/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium b/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium new file mode 100755 index 000000000..3662c1cf6 --- /dev/null +++ b/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium @@ -0,0 +1,26 @@ +#!/bin/bash +# +# Print which device the live ISO booted from and which I/O scheduler it is +# on. Run by omarchy-root-image-verify.service before the hash so the answer +# sits next to the verify result in `systemctl status` and the journal: the +# unit's IOSchedulingClass=idle only means something under bfq, which +# 71-omarchy-iso-boot-medium-bfq.rules is supposed to have set by now. + +set -euo pipefail + +mount=/run/archiso/bootmnt +source=$(findmnt -no SOURCE "$mount" 2>/dev/null || true) +if [[ -z $source ]]; then + echo "boot medium: $mount is not mounted" + exit 0 +fi + +disk=$(lsblk -no PKNAME "$source" 2>/dev/null | head -n1) +[[ -n $disk ]] || disk=${source#/dev/} + +scheduler="unknown" +if [[ -r /sys/block/$disk/queue/scheduler ]]; then + scheduler=$(cat "/sys/block/$disk/queue/scheduler") +fi + +echo "boot medium: $source on /dev/$disk, scheduler: $scheduler" diff --git a/configs/profiledef.sh b/configs/profiledef.sh index 3458cef8b..71b731370 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -39,6 +39,7 @@ file_permissions=( ["/root/customize_airootfs.sh"]="0:0:755" ["/usr/local/bin/choose-mirror"]="0:0:755" ["/usr/local/bin/omarchy-cidata-load"]="0:0:755" + ["/usr/local/bin/omarchy-iso-boot-medium"]="0:0:755" ["/usr/local/bin/omarchy-iso-cleanup-disk"]="0:0:755" ["/usr/local/bin/omarchy-install-dashboard"]="0:0:755" ["/usr/local/bin/omarchy-iso-install"]="0:0:755" diff --git a/test/unit/udev-rules-test.sh b/test/unit/udev-rules-test.sh new file mode 100644 index 000000000..fb5f9dccd --- /dev/null +++ b/test/unit/udev-rules-test.sh @@ -0,0 +1,28 @@ +#!/bin/bash +# +# The udev rules shipped in the live ISO must parse, and the boot-medium rule +# must keep putting USB disks and optical drives on BFQ: that is what makes +# the root-image verify unit's IOSchedulingClass=idle mean anything while the +# live system pages its airootfs in from the same stick. + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +rules_dir="$ROOT/configs/airootfs/etc/udev/rules.d" + +if ! command -v udevadm >/dev/null || ! udevadm verify --help >/dev/null 2>&1; then + echo "skip: udevadm verify unavailable" + exit 0 +fi + +udevadm verify --no-style "$rules_dir"/*.rules + +bfq_rule="$rules_dir/71-omarchy-iso-boot-medium-bfq.rules" +grep -qE '^ACTION=="add\|change".*ENV\{ID_BUS\}=="usb".*ATTR\{queue/scheduler\}="bfq"' "$bfq_rule" || + { echo "USB disks are no longer put on bfq in $bfq_rule"; exit 1; } +grep -qE '^ACTION=="add\|change".*KERNEL=="sr\[0-9\]\*.*ATTR\{queue/scheduler\}="bfq"' "$bfq_rule" || + { echo "optical drives are no longer put on bfq in $bfq_rule"; exit 1; } +grep -qE 'ENV\{DEVTYPE\}=="disk"' "$bfq_rule" || + { echo "bfq rule must be scoped to whole disks (partitions have no queue/)"; exit 1; } + +echo "ok: udev rules parse; boot media go on bfq" From 30596b0b3680a6b0c4c456d291e2f7ff18964507 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 18:19:31 +0200 Subject: [PATCH 18/67] Gate the free-space install on the root-image verify, via one shared helper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The full-disk path refuses a corrupt medium before archinstall formats (prepare_install_target runs the verify before phase 3). The free-space path formats in the configurator — parted, wipefs, luksFormat, mkfs — before the orchestrator ever starts, so a corrupt medium there halted only after two partitions had been created and LUKS-formatted with the user's passphrase, with no rollback from the orchestrator's failure path. Found by an automated review of #113; the cidata-based corrupt-image test never reached it because autoinstall skips the configurator. Fold the verdict collection and the boot-medium/scheduler logging into one script, omarchy-wait-root-image-verify (replacing omarchy-iso-boot-medium): it logs the boot device and its scheduler, then collects the boot-time hasher's verdict, waiting for the unit if it is still running and starting it if it never did. The configurator runs it before run_partition_execute on the free-space path; the orchestrator's verify_root_image_stream now shells out to the same script instead of reimplementing the systemd handoff in Python. One source of truth for both disk-touching paths; whoever reaches it first pays the wait. Drops the unit's ExecStartPre (the script logs the medium now) and the Python _systemctl_show/_process_read_bytes/_journal_tail helpers (the per-byte verify progress bar goes with them; the hash is almost always done before either caller reaches the gate). The copytoram-released-medium message moves into the script. New wait-root-image-verify-test.sh drives the gate with stubbed systemctl/findmnt; the Python verify tests now cover the shell-out. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- .../system/omarchy-root-image-verify.service | 3 - configs/airootfs/root/configurator | 14 ++ .../usr/local/bin/omarchy-iso-boot-medium | 26 --- .../local/bin/omarchy-wait-root-image-verify | 80 ++++++++ .../omarchy-iso/orchestrator/phases_impl.py | 120 +++--------- configs/profiledef.sh | 2 +- test/unit/test_root_image.py | 182 ++++-------------- test/unit/wait-root-image-verify-test.sh | 108 +++++++++++ 8 files changed, 267 insertions(+), 268 deletions(-) delete mode 100755 configs/airootfs/usr/local/bin/omarchy-iso-boot-medium create mode 100755 configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify create mode 100755 test/unit/wait-root-image-verify-test.sh diff --git a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service index b19b064bf..04728ba54 100644 --- a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service +++ b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service @@ -14,9 +14,6 @@ ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.sha256 Type=oneshot RemainAfterExit=yes WorkingDirectory=/run/archiso/bootmnt/arch/x86_64 -# Logs the boot device and its I/O scheduler next to the verify result; -# the idle class below only takes effect under bfq. -ExecStartPre=/usr/local/bin/omarchy-iso-boot-medium ExecStart=/usr/bin/sha256sum --check --strict omarchy-root.btrfs.sha256 # Background work: never compete with the console or anything interactive. Nice=19 diff --git a/configs/airootfs/root/configurator b/configs/airootfs/root/configurator index 4cd3db3da..35e1d6023 100644 --- a/configs/airootfs/root/configurator +++ b/configs/airootfs/root/configurator @@ -1083,6 +1083,20 @@ clear write_user_files if [[ $install_target == "free_space" ]]; then + # Refuse a corrupt or missing install medium before we partition or format + # anything. The full-disk path clears this in the orchestrator before + # archinstall formats; the free-space path formats here, in the configurator, + # so it runs the same gate first. omarchy-wait-root-image-verify collects the + # boot-time hasher's verdict, waiting for it if it is still running. (2>&1 + # 1>/dev/null captures stderr, where the failure message is, and drops the + # boot-medium/scheduler line on stdout — the orchestrator logs that later.) + if [[ ${1:-} != "dry" ]]; then + step "Verifying the install medium" + if ! verify_error=$(omarchy-wait-root-image-verify 2>&1 1>/dev/null); then + abort "$(head -1 <<<"$verify_error")" + fi + fi + run_partition_execute "$@" if [[ ${1:-} == "dry" ]]; then diff --git a/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium b/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium deleted file mode 100755 index 3662c1cf6..000000000 --- a/configs/airootfs/usr/local/bin/omarchy-iso-boot-medium +++ /dev/null @@ -1,26 +0,0 @@ -#!/bin/bash -# -# Print which device the live ISO booted from and which I/O scheduler it is -# on. Run by omarchy-root-image-verify.service before the hash so the answer -# sits next to the verify result in `systemctl status` and the journal: the -# unit's IOSchedulingClass=idle only means something under bfq, which -# 71-omarchy-iso-boot-medium-bfq.rules is supposed to have set by now. - -set -euo pipefail - -mount=/run/archiso/bootmnt -source=$(findmnt -no SOURCE "$mount" 2>/dev/null || true) -if [[ -z $source ]]; then - echo "boot medium: $mount is not mounted" - exit 0 -fi - -disk=$(lsblk -no PKNAME "$source" 2>/dev/null | head -n1) -[[ -n $disk ]] || disk=${source#/dev/} - -scheduler="unknown" -if [[ -r /sys/block/$disk/queue/scheduler ]]; then - scheduler=$(cat "/sys/block/$disk/queue/scheduler") -fi - -echo "boot medium: $source on /dev/$disk, scheduler: $scheduler" diff --git a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify new file mode 100755 index 000000000..b22c0b070 --- /dev/null +++ b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify @@ -0,0 +1,80 @@ +#!/bin/bash +# Single source of truth for validating the root image on the install medium +# before anything formats a disk. +# +# The root image is streamed from the boot medium (copytoram is off), so a bad +# medium must be caught before the full-disk orchestrator hands off to +# archinstall *and* before the free-space configurator partitions the target. +# omarchy-root-image-verify.service hashes the image in the background from boot +# (idle-class, so it yields to the live system under the boot-medium BFQ rule); +# this collects that verdict, waiting for the hash if it is still running and +# starting it if it never did. +# +# Also logs the boot device and its I/O scheduler to stdout (the idle class only +# takes effect under bfq) -- this folds in what used to be omarchy-iso-boot-medium. +# +# Exit 0: image verified (device/scheduler line on stdout). Non-zero: corrupt, +# missing, or unverifiable -- the message is on stderr with the first line +# written to headline the install dashboard. +set -uo pipefail + +UNIT=omarchy-root-image-verify.service +BOOT_MEDIUM_MOUNT=/run/archiso/bootmnt +STREAM="$BOOT_MEDIUM_MOUNT/arch/x86_64/omarchy-root.btrfs" +CHECKSUM="$STREAM.sha256" + +fail() { echo "$1" >&2; exit 1; } + +# --- boot medium + scheduler (absorbed from omarchy-iso-boot-medium) --- +src=$(findmnt -no SOURCE "$BOOT_MEDIUM_MOUNT" 2>/dev/null || true) +if [[ -n $src ]]; then + disk=$(lsblk -no PKNAME "$src" 2>/dev/null | head -n1) + disk=${disk:-${src#/dev/}} + sched=unknown + [[ -r /sys/block/$disk/queue/scheduler ]] && sched=$(cat "/sys/block/$disk/queue/scheduler") + echo "boot medium: $src on /dev/$disk, scheduler: $sched" +fi + +if [[ ! -f $STREAM ]]; then + # The archiso hook unmounts the boot medium after copying the airootfs to RAM + # (copytoram). The boot entries pin copytoram=n, so this only happens when + # someone edits the kernel command line. + [[ -d $BOOT_MEDIUM_MOUNT ]] || fail "boot medium is not mounted at $BOOT_MEDIUM_MOUNT: the live system was copied to RAM (copytoram) and the medium released; boot with copytoram=n" + fail "root image stream missing: $STREAM" +fi +[[ -f $CHECKSUM ]] || fail "root image checksum missing: $CHECKSUM" + +[[ "$(systemctl show "$UNIT" -p LoadState --value 2>/dev/null)" == loaded ]] || + fail "$UNIT is not on this live system; cannot verify the root image" + +# inactive -> never started; `systemctl start` blocks until the oneshot exits. +[[ "$(systemctl show "$UNIT" -p ActiveState --value 2>/dev/null)" == inactive ]] && + systemctl start "$UNIT" >/dev/null 2>&1 + +reported=false +while :; do + case "$(systemctl show "$UNIT" -p ActiveState --value 2>/dev/null)" in + active) + exit 0 + ;; + failed) + # The dashboard centres each line in ~80 columns, so the action comes + # first and on its own line; the journal tail (the hasher's own output) + # follows for debugging. + msg="install medium is corrupt: re-flash it"$'\n'"sha256 mismatch on the root image" + tail=$(journalctl -b -u "$UNIT" _COMM=sha256sum --no-pager -o cat 2>/dev/null) + [[ -n $tail ]] && msg+=$'\n'"$tail" + fail "$msg" + ;; + activating) + if [[ $reported == false ]]; then + echo "waiting for $UNIT to finish hashing the root image..." + reported=true + fi + sleep 0.5 + ;; + *) + fail "$UNIT did not run; cannot verify the root image" + ;; + esac +done diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 1220df152..e7155c4fa 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -131,12 +131,13 @@ def _omarchy_nvim_package() -> str: # straight off the boot medium, with sha256sum output for it next to it. The # subvolume name is what build-root-image.sh sends. ROOT_IMAGE_STREAM = Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs") -ROOT_IMAGE_CHECKSUM = ROOT_IMAGE_STREAM.with_name(ROOT_IMAGE_STREAM.name + ".sha256") -# The live ISO starts this oneshot at boot: `sha256sum -c` of the stream, -# running while the user is in the configurator. It is the only verifier; -# verify_root_image_stream collects its verdict. -ROOT_IMAGE_VERIFY_UNIT = "omarchy-root-image-verify.service" ROOT_IMAGE_SUBVOLUME = "omarchy-root" +# The live ISO starts omarchy-root-image-verify.service at boot: `sha256sum -c` +# of the stream, running while the user is in the configurator. It is the only +# verifier. Both this phase and the free-space configurator gate collect its +# verdict through this helper, which logs the boot medium and its I/O scheduler, +# waits for the unit if it is still hashing, and starts it if it never ran. +ROOT_IMAGE_VERIFY_HELPER = "/usr/local/bin/omarchy-wait-root-image-verify" BOOT_MEDIUM_MOUNT = Path("/run/archiso/bootmnt") @@ -265,103 +266,30 @@ def verify_root_image_stream(ctx: InstallContext) -> None: also trip btrfs receive's per-command checksums, but only after the disk is formatted. - The hash is ROOT_IMAGE_VERIFY_UNIT's job: it starts at boot and reads - the stream while the user is in the configurator, so by now it is - usually long done and this only collects the verdict. If it is still - running, wait for it; if it never started, start it and wait. The read - also leaves as much of the stream as fits in the page cache for the - unpack that follows.""" - stream_path = _root_image_stream() - if not ROOT_IMAGE_CHECKSUM.is_file(): - raise RuntimeError(f"root image checksum missing: {ROOT_IMAGE_CHECKSUM}") - total = stream_path.stat().st_size - - state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) - if state.get("LoadState") != "loaded": - raise RuntimeError(f"{ROOT_IMAGE_VERIFY_UNIT} is not on this live system; cannot verify the root image") - if state.get("ActiveState") == "inactive": - # Not started yet. Blocks until the oneshot exits. - subprocess.run(["systemctl", "start", ROOT_IMAGE_VERIFY_UNIT], check=False, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) - - reported = False - last_report = 0.0 - while True: - active = state.get("ActiveState") - if active == "active": - info(f"› root image verified by {ROOT_IMAGE_VERIFY_UNIT} ({total >> 20} MiB stream at {stream_path})") - return - if active == "failed": - # The dashboard centres each line of this in ~80 columns, so the - # action comes first and on its own line; the detail follows. - raise RuntimeError( - "install medium is corrupt: re-flash it\n" - f"sha256 mismatch on {stream_path.name} ({total} bytes, {ROOT_IMAGE_VERIFY_UNIT})" - + _journal_tail(ROOT_IMAGE_VERIFY_UNIT, comm="sha256sum") - ) - if active != "activating": - raise RuntimeError( - f"{ROOT_IMAGE_VERIFY_UNIT} did not run (state {active!r}); cannot verify the root image" - + _journal_tail(ROOT_IMAGE_VERIFY_UNIT) - ) - - if not reported: - info(f"› waiting for {ROOT_IMAGE_VERIFY_UNIT} ({total >> 20} MiB stream at {stream_path})") - reported = True - now = time.monotonic() - if now - last_report >= 0.5: - done = _process_read_bytes(state.get("MainPID")) - if done is not None: - _write_phase_progress(ctx, done / total if total else 1.0) - last_report = now - time.sleep(0.25) - state = _systemctl_show(ROOT_IMAGE_VERIFY_UNIT) - - -def _systemctl_show(unit: str) -> dict[str, str]: - """`systemctl show` of the properties the verify handoff reads, as a - dict. Empty when systemctl is unavailable (unit tests off the ISO).""" + ROOT_IMAGE_VERIFY_HELPER is the single source of truth: it collects the + boot-time hasher's verdict, waiting for the unit if it is still running + and starting it if it never did, and logs the boot medium and its I/O + scheduler. The free-space configurator gate runs the same helper before it + partitions, so both disk-touching paths clear the same check; whoever gets + there first pays the wait. The hasher's read also leaves as much of the + stream as fits in the page cache for the unpack that follows.""" try: result = subprocess.run( - ["systemctl", "show", "-p", "LoadState,ActiveState,MainPID", unit], + [ROOT_IMAGE_VERIFY_HELPER], capture_output=True, text=True, check=False, ) - except OSError: - return {} - if result.returncode != 0: - return {} - return dict(line.split("=", 1) for line in result.stdout.splitlines() if "=" in line) - - -def _process_read_bytes(pid: str | None) -> int | None: - """Bytes the process has read so far (rchar from /proc/PID/io): the - progress of a sequential hash. None when unknown.""" - if not pid or pid == "0": - return None - try: - for line in Path(f"/proc/{pid}/io").read_text().splitlines(): - if line.startswith("rchar:"): - return int(line.split()[1]) - except (OSError, ValueError, IndexError): - pass - return None + except OSError as exc: + raise RuntimeError(f"could not run {ROOT_IMAGE_VERIFY_HELPER}: {exc}") from exc + for line in result.stdout.splitlines(): + if line.strip(): + info(f"› {line.strip()}") -def _journal_tail(unit: str, comm: str | None = None) -> str: - """This boot's journal for the unit, one line per entry, prefixed with a - newline for appending to an error. With comm, only what that program - wrote: the unit's own verdict without systemd's exit/failed/consumed - boilerplate around it.""" - cmd = ["journalctl", "-b", "-u", unit, "--no-pager", "-o", "cat", "-n", "5"] - if comm: - cmd = ["journalctl", "-b", "-u", unit, f"_COMM={comm}", "--no-pager", "-o", "cat"] - try: - result = subprocess.run(cmd, capture_output=True, text=True, check=False) - except OSError: - return "" - output = result.stdout.strip() - return f"\n{output}" if output else "" + if result.returncode != 0: + raise RuntimeError( + result.stderr.strip() + or f"{ROOT_IMAGE_VERIFY_HELPER} failed with status {result.returncode}" + ) def arch_install_system(ctx: InstallContext) -> None: diff --git a/configs/profiledef.sh b/configs/profiledef.sh index 71b731370..136c66b3e 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -39,7 +39,7 @@ file_permissions=( ["/root/customize_airootfs.sh"]="0:0:755" ["/usr/local/bin/choose-mirror"]="0:0:755" ["/usr/local/bin/omarchy-cidata-load"]="0:0:755" - ["/usr/local/bin/omarchy-iso-boot-medium"]="0:0:755" + ["/usr/local/bin/omarchy-wait-root-image-verify"]="0:0:755" ["/usr/local/bin/omarchy-iso-cleanup-disk"]="0:0:755" ["/usr/local/bin/omarchy-install-dashboard"]="0:0:755" ["/usr/local/bin/omarchy-iso-install"]="0:0:755" diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 981f77199..26120b9a3 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -6,7 +6,6 @@ asserted on the subprocess calls against a temp target. """ -import os import shutil import sys import tempfile @@ -78,161 +77,60 @@ def test_empty_config_rejected(self): class VerifyRootImageStreamTest(unittest.TestCase): - """The boot-time verify unit is the only hasher: this collects its - verdict, waits for it when it is still running, starts it when it never - ran, and fails the install when it cannot vouch for the stream.""" + """verify_root_image_stream delegates to the shared shell helper + (omarchy-wait-root-image-verify): it passes when the helper exits 0 and + fails the install with the helper's message when it does not. The wait, + the systemd handoff and the corrupt-medium wording live in the helper and + are covered by test/unit/wait-root-image-verify-test.sh.""" - UNIT = phases_impl.ROOT_IMAGE_VERIFY_UNIT - ACTIVE = {"LoadState": "loaded", "ActiveState": "active", "MainPID": "0"} - FAILED = {"LoadState": "loaded", "ActiveState": "failed", "MainPID": "0"} - RUNNING = {"LoadState": "loaded", "ActiveState": "activating", "MainPID": "4242"} - IDLE = {"LoadState": "loaded", "ActiveState": "inactive", "MainPID": "0"} + HELPER = phases_impl.ROOT_IMAGE_VERIFY_HELPER def setUp(self): - self.tmp = tempfile.TemporaryDirectory() - self.addCleanup(self.tmp.cleanup) - self.dir = Path(self.tmp.name) - self.stream = self.dir / "omarchy-root.btrfs" - self.checksum = self.dir / "omarchy-root.btrfs.sha256" - self.ctx = types.SimpleNamespace(state_dir=self.dir / "state") - self.runs = [] - self.progress = [] - self.real_journal_tail = phases_impl._journal_tail - - def fake_run(cmd, **kwargs): - self.runs.append(cmd) - return CompletedProcess(cmd, 0, stdout="", stderr="") + self.ctx = types.SimpleNamespace(state_dir=Path("/nonexistent")) + self.infos = [] + patch = mock.patch.object(phases_impl, "info", side_effect=self.infos.append) + patch.start() + self.addCleanup(patch.stop) - for patch in ( - mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", self.stream), - mock.patch.object(phases_impl, "ROOT_IMAGE_CHECKSUM", self.checksum), - mock.patch.object(phases_impl, "BOOT_MEDIUM_MOUNT", self.dir), - mock.patch.object(phases_impl, "info"), - mock.patch.object(phases_impl, "_write_phase_progress", - side_effect=lambda ctx, f: self.progress.append(f)), - mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run), - mock.patch.object(phases_impl.time, "sleep"), - mock.patch.object(phases_impl, "_journal_tail", - return_value="\nsha256sum: WARNING: 1 computed checksum did NOT match"), - ): - patch.start() - self.addCleanup(patch.stop) + def helper_returns(self, returncode=0, stdout="", stderr=""): + self.captured = {} - def write_stream(self): - # Contents are irrelevant: the orchestrator never hashes, the unit does. - self.stream.write_bytes(b"btrfs-stream" * 1000) - self.checksum.write_text(f"{'0' * 64} {self.stream.name}\n") + def fake_run(cmd, **kwargs): + self.captured["cmd"] = cmd + return CompletedProcess(cmd, returncode, stdout=stdout, stderr=stderr) - def states(self, *seq): - it = iter(seq) - last = seq[-1] - return mock.patch.object(phases_impl, "_systemctl_show", side_effect=lambda unit: next(it, last)) + return mock.patch.object(phases_impl.subprocess, "run", side_effect=fake_run) - def test_unit_already_succeeded(self): - self.write_stream() - with self.states(self.ACTIVE): + def test_passes_when_helper_exits_zero(self): + with self.helper_returns(0, stdout="boot medium: /dev/sda1 on /dev/sda, scheduler: none mq-deadline kyber [bfq]\n"): phases_impl.verify_root_image_stream(self.ctx) - self.assertEqual(self.runs, []) - - def test_unit_failure_is_a_corrupt_medium(self): - self.write_stream() - with self.states(self.FAILED): + self.assertEqual(self.captured["cmd"], [self.HELPER]) + # The boot medium / scheduler line the helper prints reaches the log. + self.assertTrue(any("bfq" in m for m in self.infos)) + + def test_corrupt_medium_raises_with_the_helpers_message(self): + stderr = ("install medium is corrupt: re-flash it\n" + "sha256 mismatch on the root image\n" + "omarchy-root.btrfs: FAILED") + with self.helper_returns(1, stdout="boot medium: /dev/sda1 ...\n", stderr=stderr): with self.assertRaises(RuntimeError) as raised: phases_impl.verify_root_image_stream(self.ctx) - lines = str(raised.exception).splitlines() - # The dashboard centres each line in ~80 columns: the action first, - # short enough to survive, then the detail, then sha256sum's verdict. - self.assertEqual(lines[0], "install medium is corrupt: re-flash it") - self.assertLess(len(lines[0]), 50) - self.assertRegex(lines[1], r"sha256 mismatch on omarchy-root\.btrfs \(\d+ bytes, omarchy-root-image-verify\.service\)") - self.assertEqual(lines[2], "sha256sum: WARNING: 1 computed checksum did NOT match") - phases_impl._journal_tail.assert_called_once_with(self.UNIT, comm="sha256sum") - - def test_journal_tail_filters_to_the_hasher(self): - # setUp patches _journal_tail for the handoff tests; exercise the real one. - real = self.real_journal_tail - def journal(cmd, **kw): - self.runs.append(cmd) - return CompletedProcess( - cmd, 0, stdout="omarchy-root.btrfs: FAILED\nsha256sum: WARNING: 1 computed checksum did NOT match\n", stderr="") - phases_impl.subprocess.run.side_effect = journal - self.assertEqual( - real(self.UNIT, comm="sha256sum"), - "\nomarchy-root.btrfs: FAILED\nsha256sum: WARNING: 1 computed checksum did NOT match", - ) - self.assertEqual( - self.runs[-1], - ["journalctl", "-b", "-u", self.UNIT, "_COMM=sha256sum", "--no-pager", "-o", "cat"], - ) - real(self.UNIT) - self.assertEqual(self.runs[-1][:5], ["journalctl", "-b", "-u", self.UNIT, "--no-pager"]) - self.assertNotIn("_COMM=sha256sum", self.runs[-1]) - - def test_waits_for_a_running_unit_and_reports_its_progress(self): - self.write_stream() - total = self.stream.stat().st_size - reads = iter([total // 4, total // 2]) - with self.states(self.RUNNING, self.RUNNING, self.ACTIVE), \ - mock.patch.object(phases_impl, "_process_read_bytes", side_effect=lambda pid: next(reads)), \ - mock.patch.object(phases_impl.time, "monotonic", side_effect=[1.0, 2.0]): - phases_impl.verify_root_image_stream(self.ctx) - self.assertEqual(self.progress, [0.25, 0.5]) - self.assertEqual(self.runs, []) - - def test_unit_not_started_is_started_and_waited_for(self): - self.write_stream() - with self.states(self.IDLE, self.ACTIVE): - phases_impl.verify_root_image_stream(self.ctx) - self.assertEqual(self.runs, [["systemctl", "start", self.UNIT]]) - - def test_unit_that_will_not_run_fails_the_install(self): - self.write_stream() - with self.states(self.IDLE, self.IDLE): - with self.assertRaisesRegex(RuntimeError, "did not run"): - phases_impl.verify_root_image_stream(self.ctx) - self.assertEqual(self.runs, [["systemctl", "start", self.UNIT]]) - - def test_missing_unit_fails_the_install(self): - self.write_stream() - with self.states({"LoadState": "not-found"}): - with self.assertRaisesRegex(RuntimeError, "not on this live system"): - phases_impl.verify_root_image_stream(self.ctx) - - def test_missing_stream(self): - with self.states(self.ACTIVE): - with self.assertRaisesRegex(RuntimeError, "stream missing"): + # The dashboard headlines the first line, so it must lead and be short. + first = str(raised.exception).splitlines()[0] + self.assertEqual(first, "install medium is corrupt: re-flash it") + self.assertLess(len(first), 50) + + def test_nonzero_without_stderr_still_fails(self): + with self.helper_returns(3, stdout="", stderr=""): + with self.assertRaisesRegex(RuntimeError, "status 3"): phases_impl.verify_root_image_stream(self.ctx) - def test_boot_medium_released_by_copytoram(self): - # What a ThinkPad X200s showed: archiso's copytoram=auto copied the - # airootfs to RAM and unmounted the USB stick, so bootmnt was gone. - with mock.patch.object(phases_impl, "BOOT_MEDIUM_MOUNT", self.dir / "bootmnt"): - with self.states(self.ACTIVE): - with self.assertRaisesRegex(RuntimeError, "copytoram"): - phases_impl.verify_root_image_stream(self.ctx) - - def test_missing_checksum(self): - self.stream.write_bytes(b"x") - with self.states(self.ACTIVE): - with self.assertRaisesRegex(RuntimeError, "checksum missing"): + def test_missing_helper_fails_the_install(self): + with mock.patch.object(phases_impl.subprocess, "run", + side_effect=FileNotFoundError("no helper")): + with self.assertRaisesRegex(RuntimeError, "could not run"): phases_impl.verify_root_image_stream(self.ctx) - def test_systemctl_show_parses_properties(self): - phases_impl.subprocess.run.side_effect = lambda cmd, **kw: CompletedProcess( - cmd, 0, stdout="LoadState=loaded\nActiveState=activating\nMainPID=7\n", stderr="") - self.assertEqual(phases_impl._systemctl_show("x.service"), - {"LoadState": "loaded", "ActiveState": "activating", "MainPID": "7"}) - - def test_systemctl_missing_means_no_unit(self): - phases_impl.subprocess.run.side_effect = FileNotFoundError("systemctl") - self.assertEqual(phases_impl._systemctl_show("x.service"), {}) - - def test_process_read_bytes(self): - self.assertIsNone(phases_impl._process_read_bytes("0")) - self.assertIsNone(phases_impl._process_read_bytes(None)) - self.assertIsNone(phases_impl._process_read_bytes("99999999")) - self.assertIsInstance(phases_impl._process_read_bytes(str(os.getpid())), int) - class PrepareInstallTargetTest(unittest.TestCase): """prepare_install_target wires the checks together per install mode.""" diff --git a/test/unit/wait-root-image-verify-test.sh b/test/unit/wait-root-image-verify-test.sh new file mode 100755 index 000000000..776f9bdac --- /dev/null +++ b/test/unit/wait-root-image-verify-test.sh @@ -0,0 +1,108 @@ +#!/bin/bash +# +# omarchy-wait-root-image-verify is the single gate both disk-touching paths +# clear before formatting: the orchestrator (full-disk) and the configurator +# (free-space). It collects the boot-time hasher's verdict, waiting if it is +# still running and starting it if it never did. This drives it with a stubbed +# systemctl/findmnt/lsblk/journalctl and a fake boot medium on PATH, so the +# wait/verdict logic is checked without an ISO. + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +HELPER="$ROOT/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify" + +fails=0 +check() { # desc, expected_rc, actual_rc, [needle in output], [output] + local desc=$1 want=$2 got=$3 needle=${4:-} out=${5:-} + if [[ $got != "$want" ]]; then + echo "FAIL: $desc (rc want=$want got=$got)"; fails=1; return + fi + if [[ -n $needle && $out != *"$needle"* ]]; then + echo "FAIL: $desc (missing '$needle' in: $out)"; fails=1; return + fi + echo "ok: $desc" +} + +# A sandbox with a fake boot medium and stub tools on PATH. ACTIVE_SEQ is the +# newline-separated ActiveState values `systemctl show ... ActiveState` returns +# on successive calls (last one repeats); LOADSTATE and START_RC tune the rest. +run_helper() { # loadstate, active_seq, start_rc -> sets RC and OUT + local loadstate=$1 active_seq=$2 start_rc=$3 + local box; box=$(mktemp -d) + mkdir -p "$box/bin" "$box/medium/arch/x86_64" "$box/sys/block/sdz/queue" + : >"$box/medium/arch/x86_64/omarchy-root.btrfs" + : >"$box/medium/arch/x86_64/omarchy-root.btrfs.sha256" + echo "none mq-deadline kyber [bfq]" >"$box/sys/block/sdz/queue/scheduler" + printf '%s\n' "$active_seq" >"$box/active_seq" + + cat >"$box/bin/findmnt" <"$box/bin/lsblk" <"$box/bin/journalctl" <<'EOF' +#!/bin/bash +echo "omarchy-root.btrfs: FAILED" +EOF + # systemctl show -p LoadState|ActiveState --value ; systemctl start ... + cat >"$box/bin/systemctl" < 1)); then printf '%s\n' "\${seq[@]:1}" >"\$box/active_seq"; fi + ;; + esac + exit 0 +fi +if [[ \$1 == start ]]; then exit \$start_rc; fi +exit 0 +EOF + chmod +x "$box"/bin/* + + # Point the helper's hardcoded /run/archiso/bootmnt and /sys at the sandbox + # by running it through a tiny shim that rewrites those roots. + local shim="$box/run-helper" + sed -e "s#/run/archiso/bootmnt#$box/medium#g" \ + -e "s#/sys/block#$box/sys/block#g" \ + "$HELPER" >"$shim" + chmod +x "$shim" + + set +e + OUT=$(PATH="$box/bin:$PATH" bash "$shim" 2>&1) + RC=$? + set -e + rm -rf "$box" +} + +# Already verified before we look: pass, and the scheduler line is emitted. +run_helper loaded "active" 0 +check "active unit passes" 0 "$RC" "scheduler: none mq-deadline kyber [bfq]" "$OUT" + +# Still hashing, then completes: waits, then passes. +run_helper loaded $'activating\nactivating\nactive' 0 +check "waits for a running unit then passes" 0 "$RC" "waiting for" "$OUT" + +# Hash failed: corrupt medium, re-flash message leads. +run_helper loaded "failed" 0 +check "failed unit is a corrupt medium" 1 "$RC" "install medium is corrupt: re-flash it" "$OUT" + +# Never started and start leaves it inactive: cannot verify. +run_helper loaded "inactive" 0 +check "unit that will not run fails" 1 "$RC" "did not run" "$OUT" + +# Unit not on this system. +run_helper not-found "inactive" 0 +check "missing unit fails" 1 "$RC" "not on this live system" "$OUT" + +[[ $fails -eq 0 ]] && echo "ok: omarchy-wait-root-image-verify gate behaves" +exit "$fails" From 347c96cac1708f13c862a488fc97e88d0b5865f3 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 21:24:28 +0200 Subject: [PATCH 19/67] Run the integration suite on legacy BIOS, and assert the boot firmware MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The QEMU integration harness only ever booted UEFI (OVMF), so the legacy BIOS install path — archinstall's layout, Limine's MBR install, and booting that MBR — was covered by real hardware alone. Make the harness firmware-aware: OMARCHY_INTEGRATION_FIRMWARE=uefi|bios (or the runner's --bios flag) picks OVMF or QEMU's built-in SeaBIOS. Under BIOS the VMs run with no pflash, base images cache under a separate -bios dir, and the install boots the ISO from an ide-cd (SeaBIOS boots that reliably; it will not boot the isohybrid image off a fallback USB device). The install phase's wait-for-SSH-after-reboot then already proves the MBR boots. Add firmware-boot-test.sh, which boots the installed base and asserts it came up in the expected mode and carries the matching bootloader: under UEFI an EFI runtime, a Limine EFI binary on the ESP, and an efibootmgr entry; under BIOS no EFI runtime, Limine's BIOS stage under /boot, and Limine's boot code in the disk MBR. factory-reset is UEFI-only (shared-ESP dual boot) and skips under BIOS. Validated both ways: a full BIOS install reboots into the installed system and passes all four BIOS assertions; the UEFI assertions pass against the existing UEFI base. (Noted in passing: a BIOS install also drops inert EFI artifacts under /boot/EFI as a UEFI-machine fallback; harmless, not asserted either way.) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- test/integration | 6 +++ test/integration.d/base-test.sh | 42 +++++++++++++++--- test/integration.d/factory-reset-test.sh | 8 ++++ test/integration.d/firmware-boot-test.sh | 56 ++++++++++++++++++++++++ 4 files changed, 105 insertions(+), 7 deletions(-) create mode 100755 test/integration.d/firmware-boot-test.sh diff --git a/test/integration b/test/integration index 3954354cb..7dd77e6b8 100755 --- a/test/integration +++ b/test/integration @@ -10,6 +10,9 @@ # scenario Basename of a test/integration.d/-test.sh file; # all scenarios run when none are named # --reuse-base Skip the install if a saved base image exists +# --bios Boot the VMs on legacy BIOS (SeaBIOS) instead of UEFI; the +# install then exercises the Limine MBR path. Base images are +# cached separately per firmware. # --port PORT Host port forwarded to guest SSH (default 2322) # --memory MB VM memory (default 8192) # --timeout SECS Install timeout (default 2400) @@ -27,10 +30,13 @@ export OMARCHY_INTEGRATION_SSH_PORT=2322 export OMARCHY_INTEGRATION_MEMORY=8192 export OMARCHY_INTEGRATION_INSTALL_TIMEOUT=2400 export OMARCHY_INTEGRATION_NO_PREVIEW=false +export OMARCHY_INTEGRATION_FIRMWARE="${OMARCHY_INTEGRATION_FIRMWARE:-uefi}" while (($#)); do case "$1" in --reuse-base) REUSE_BASE=true ;; + --bios) OMARCHY_INTEGRATION_FIRMWARE=bios ;; + --uefi) OMARCHY_INTEGRATION_FIRMWARE=uefi ;; --no-preview) OMARCHY_INTEGRATION_NO_PREVIEW=true ;; --port) OMARCHY_INTEGRATION_SSH_PORT="$2"; shift ;; --memory) OMARCHY_INTEGRATION_MEMORY="$2"; shift ;; diff --git a/test/integration.d/base-test.sh b/test/integration.d/base-test.sh index 34dd2a6dd..cfa4f1327 100644 --- a/test/integration.d/base-test.sh +++ b/test/integration.d/base-test.sh @@ -23,10 +23,18 @@ GUEST_HOSTNAME="omarchy-test" SCENARIO="${SCENARIO:-$(basename "${0%-test.sh}")}" +# Firmware the VMs boot: uefi (OVMF, the default) or bios (SeaBIOS, legacy). +# BIOS exercises the Limine MBR boot path that only real hardware covered; set +# it with OMARCHY_INTEGRATION_FIRMWARE=bios or the runner's --bios flag. The +# base image and its caches are kept per-firmware so the two never collide. +FIRMWARE="${OMARCHY_INTEGRATION_FIRMWARE:-uefi}" OVMF_CODE="/usr/share/edk2/x64/OVMF_CODE.4m.fd" OVMF_VARS_TEMPLATE="/usr/share/edk2/x64/OVMF_VARS.4m.fd" BASE_DIR="$ROOT/test-runs/$(basename "$ISO" .iso)-integration" +if [[ $FIRMWARE == bios ]]; then + BASE_DIR+="-bios" +fi RUN_DIR="$BASE_DIR/runs/$(date +%Y%m%d-%H%M%S)-$SCENARIO" BASE_DISK="$BASE_DIR/base.qcow2" BASE_OVMF="$BASE_DIR/OVMF_VARS.4m.fd" @@ -69,7 +77,9 @@ finish() { } base_image_ready() { - [[ -f $BASE_DISK && -f $BASE_OVMF && -f $SSH_KEY ]] + [[ -f $BASE_DISK && -f $SSH_KEY ]] || return 1 + # UEFI keeps its NVRAM vars alongside the disk; BIOS has none. + [[ $FIRMWARE != uefi || -f $BASE_OVMF ]] } # ---------------------------------------------------------------- vm control @@ -160,12 +170,21 @@ start_vm() { local disk="$1" serial="$2" shift 2 + # UEFI needs the OVMF firmware pair; BIOS boots QEMU's built-in SeaBIOS with + # no pflash at all. + local firmware_args=() + if [[ $FIRMWARE == uefi ]]; then + firmware_args=( + -drive if=pflash,format=raw,readonly=on,file="$OVMF_CODE" + -drive if=pflash,format=raw,file="$ACTIVE_OVMF" + ) + fi + qemu-system-x86_64 \ -cpu host -enable-kvm -machine q35,accel=kvm \ -smp "$(nproc)" \ -m "$MEMORY" \ - -drive if=pflash,format=raw,readonly=on,file="$OVMF_CODE" \ - -drive if=pflash,format=raw,file="$ACTIVE_OVMF" \ + "${firmware_args[@]}" \ -drive file="$disk",format=qcow2,if=none,id=drive0 \ -device virtio-blk-pci,drive=drive0,bootindex=1 \ -device virtio-vga \ @@ -185,8 +204,10 @@ start_vm() { # leak between runs. start_vm_from_base() { qemu-img create -f qcow2 -b "$BASE_DISK" -F qcow2 "$RUN_DIR/run.qcow2" >/dev/null - cp "$BASE_OVMF" "$RUN_DIR/OVMF_VARS.4m.fd" - ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + if [[ $FIRMWARE == uefi ]]; then + cp "$BASE_OVMF" "$RUN_DIR/OVMF_VARS.4m.fd" + ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + fi start_vm "$RUN_DIR/run.qcow2" "$RUN_DIR/serial.log" } @@ -584,9 +605,16 @@ install_phase() { # clean shutdown, so a failed install can never pass for a reusable base. rm -f "$BASE_DISK" "$BASE_DISK.building" qemu-img create -f qcow2 "$BASE_DISK.building" 40G >/dev/null - cp "$OVMF_VARS_TEMPLATE" "$BASE_OVMF" - ACTIVE_OVMF="$BASE_OVMF" + if [[ $FIRMWARE == uefi ]]; then + cp "$OVMF_VARS_TEMPLATE" "$BASE_OVMF" + ACTIVE_OVMF="$BASE_OVMF" + fi + # Boot the ISO from an optical drive under both firmwares: OVMF and SeaBIOS + # both boot an ide-cd reliably, and the boot-medium type has no bearing on + # what the BIOS run exercises (Limine's MBR install and boot). SeaBIOS will + # not boot the isohybrid image off a fallback USB device here, so USB is not + # worth the flakiness. The copytoram-off / USB path is covered by unit tests. start_vm "$BASE_DISK.building" "$RUN_DIR/install-serial.log" \ -drive "file=$ISO,media=cdrom,if=none,format=raw,id=cdrom0" \ -device ide-cd,drive=cdrom0,bootindex=2 \ diff --git a/test/integration.d/factory-reset-test.sh b/test/integration.d/factory-reset-test.sh index ed6cc8789..ba42582fa 100755 --- a/test/integration.d/factory-reset-test.sh +++ b/test/integration.d/factory-reset-test.sh @@ -13,6 +13,14 @@ source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" +# This scenario is about a shared UEFI ESP (Windows + a foreign Linux under +# EFI/Linux, efibootmgr entries) — concepts that do not exist on a legacy BIOS +# install. Skip it there rather than fail its UEFI-specific assertions. +if [[ $FIRMWARE != uefi ]]; then + log "factory-reset is UEFI-only (shared-ESP dual boot); skipping under $FIRMWARE" + exit 0 +fi + FOREIGN_ID="fedcfedcfedcfedcfedcfedcfedcfedc" base_image_ready || { echo "No base image; run this through ./test/integration" >&2; exit 1; } diff --git a/test/integration.d/firmware-boot-test.sh b/test/integration.d/firmware-boot-test.sh new file mode 100755 index 000000000..d98fa1b31 --- /dev/null +++ b/test/integration.d/firmware-boot-test.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# +# The installed system booted, and it booted the way the firmware demands: +# under UEFI it comes up with an EFI runtime and a Limine EFI binary on the +# ESP; under legacy BIOS (--bios) it comes up with no EFI runtime and Limine's +# BIOS stage on the target disk's MBR path. The BIOS case is the one only real +# hardware covered before — the base install already proved the MBR boots (the +# install phase waits for SSH after the auto-reboot), and this pins down that +# the right bootloader was installed and the machine really is in BIOS mode. +# +# Runs against the base image, so it inherits whatever firmware that base was +# installed under: `./test/integration firmware-boot` (UEFI) or +# `./test/integration firmware-boot --bios`. + +source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" + +base_image_ready || { echo "No base image; run this through ./test/integration" >&2; exit 1; } + +verify_firmware_boot() { + log "Booting the installed base image ($FIRMWARE) to check its bootloader" + start_vm_from_base + wait_for_ssh "$BOOT_TIMEOUT" + + check "installed system is reachable (it booted, not the install medium)" \ + ssh_guest "test ! -e /run/archiso" + + ssh_sudo "find /boot -maxdepth 3 -print" >"$RUN_DIR/boot-tree.txt" 2>/dev/null || true + + # /boot is a root-only FAT mount, so its contents are checked with sudo. + if [[ $FIRMWARE == bios ]]; then + check "booted in legacy BIOS mode (no EFI runtime)" \ + ssh_guest "test ! -d /sys/firmware/efi" + check "Limine BIOS stage is installed under /boot" \ + ssh_sudo "test -f /boot/limine/limine-bios.sys" + check "Limine's boot code is in the disk's MBR" \ + ssh_sudo "dd if=/dev/vda bs=512 count=2048 2>/dev/null | grep -qa -i limine" + # Note: the install also drops EFI artifacts under /boot/EFI (a UEFI-machine + # fallback); they are inert under BIOS, so their presence is not asserted + # either way. What matters is that the machine booted via the BIOS/MBR path, + # which the checks above establish. + else + check "booted in UEFI mode (EFI runtime present)" \ + ssh_guest "test -d /sys/firmware/efi" + check "a Limine EFI binary is on the ESP" \ + ssh_sudo "test -e /boot/EFI/limine/limine_x64.efi -o -e /boot/EFI/BOOT/BOOTX64.EFI" + check "efibootmgr has a Limine entry" \ + ssh_sudo "efibootmgr | grep -qi limine" + fi + + capture_console "success-firmware-$FIRMWARE" +} + +# ---------------------------------------------------------------------- main + +verify_firmware_boot +finish From 059ee8459045409cf411e399168551179b8f4926 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 23 Aug 2026 21:38:26 +0200 Subject: [PATCH 20/67] Commit the boot-menu default before console-login on BIOS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The live-ISO console SSH bootstrap switches to a spare TTY with ctrl-alt-f3 before logging in. On BIOS that first keystroke lands on the ISOLINUX menu, which cancels its auto-boot countdown on any key — so the ISO never booted and the bootstrap timed out waiting for a login prompt (corrupt-image under --bios). Press Enter a few times first to commit the highlighted default (the install medium); it boots ISOLINUX, and once booted the presses are harmless newlines, well before the installer dashboard appears. BIOS-only, so the UEFI path is untouched. corrupt-image now passes 8/8 under --bios. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- test/integration.d/base-test.sh | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/test/integration.d/base-test.sh b/test/integration.d/base-test.sh index cfa4f1327..1c310ea51 100644 --- a/test/integration.d/base-test.sh +++ b/test/integration.d/base-test.sh @@ -402,6 +402,16 @@ EOF (cd "$BASE_DIR/www" && exec python3 -m http.server "$HTTP_PORT" --bind 127.0.0.1 >/dev/null 2>&1) & HTTP_PID=$! + if [[ $FIRMWARE == bios ]]; then + # ISOLINUX cancels its auto-boot countdown on any keystroke, so the + # ctrl-alt-f3 below would strand the VM at the boot menu. Commit the + # highlighted default (the install medium) with Enter first; a few presses + # catch the menu whenever it renders, and once booted they are harmless + # newlines (still well before the installer dashboard comes up). + local i + for i in 1 2 3; do press ret; sleep 2; done + fi + local waited=0 while true; do press ctrl-alt-f3 From bf1fc9898e0a6d87a926d576e33a9e687343d2ee Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 24 Aug 2026 10:07:12 +0200 Subject: [PATCH 21/67] Fix stale comment: the image compresses higher than the installer, not equal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The header said the image is compressed "at the same zstd level the installer mounts the target with," but IMAGE_COMPRESS is compress-force=zstd:15 while the installed system mounts compress=zstd (level 3) — a deliberately higher level, since it only costs build time and btrfs receive stores the extents as-is. Point the header at IMAGE_COMPRESS and its existing explanation instead of restating a wrong equivalence. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YJj2oLbxWnJn3Cvqdmz1HS --- builder/build-root-image.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh index 081a3e26a..94f1da631 100755 --- a/builder/build-root-image.sh +++ b/builder/build-root-image.sh @@ -6,7 +6,8 @@ # The target install is the same ~940 packages for everyone; only the kernel, # CPU microcode, audio firmware and Tailscale vary per machine. So pacstrap the # invariant set once here, at build time, into a btrfs subvolume compressed at -# the same zstd level the installer mounts the target with, and ship it as a +# a high zstd level (see IMAGE_COMPRESS below — higher than the installer's own +# compress=zstd, since the level only costs build time), and ship it as a # `btrfs send --compressed-data` stream. At install time `btrfs receive` writes # the compressed extents straight to disk (no decompress/recompress), which # measured at roughly half the time of extracting the same packages with pacman, From ec9a5c41f0816e9be7d1efd0d6f328f4c1d34549 Mon Sep 17 00:00:00 2001 From: Omabot Date: Mon, 24 Aug 2026 05:22:00 -0700 Subject: [PATCH 22/67] Pin copytoram=n on the loopback boot entries too mkarchiso ships configs/grub/loopback.cfg at /boot/grub/loopback.cfg on the ISO (_make_common_bootmode_grub_cfg copies every profile grub/*.cfg into isofs), and that is the file Ventoy and a hand-written GRUB entry use to boot the image as a file on a disk. archiso_loop_mnt sets archisodevice to the loop device it creates, so the auto rule's one exclusion -- an image on /dev/sr* -- never applies there, and copytoram=auto turns on for exactly the reasons it did on the ThinkPads. /run/archiso/bootmnt is then unmounted and the install aborts at the root-image gate. boot-cmdline-test.sh could not catch it: loopback.cfg was not in its file list, and it finds the medium by img_dev/img_loop rather than archisosearchuuid, so match on archisobasedir instead. Co-Authored-By: Claude Opus 5 (1M context) --- configs/grub/loopback.cfg | 4 ++-- test/unit/boot-cmdline-test.sh | 13 ++++++++++--- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/configs/grub/loopback.cfg b/configs/grub/loopback.cfg index 395a29113..f1d24328f 100644 --- a/configs/grub/loopback.cfg +++ b/configs/grub/loopback.cfg @@ -30,13 +30,13 @@ timeout_style=hidden menuentry "Omarchy (%ARCH%, ${archiso_platform})" --class arch --class gnu-linux --class gnu --class os --id 'archlinux' { set gfxpayload=keep - linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% img_dev=UUID=${archiso_img_dev_uuid} img_loop="${iso_path}" quiet splash xe.enable_panel_replay=0 initramfs_async=0 + linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% img_dev=UUID=${archiso_img_dev_uuid} img_loop="${iso_path}" copytoram=n quiet splash xe.enable_panel_replay=0 initramfs_async=0 initrd /%INSTALL_DIR%/boot/%ARCH%/initramfs-linux-t2.img } menuentry "Omarchy with speakup screen reader (%ARCH%, ${archiso_platform})" --hotkey s --class arch --class gnu-linux --class gnu --class os --id 'archlinux-accessibility' { set gfxpayload=keep - linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% img_dev=UUID=${archiso_img_dev_uuid} img_loop="${iso_path}" accessibility=on xe.enable_panel_replay=0 initramfs_async=0 + linux /%INSTALL_DIR%/boot/%ARCH%/vmlinuz-linux-t2 archisobasedir=%INSTALL_DIR% img_dev=UUID=${archiso_img_dev_uuid} img_loop="${iso_path}" copytoram=n accessibility=on xe.enable_panel_replay=0 initramfs_async=0 initrd /%INSTALL_DIR%/boot/%ARCH%/initramfs-linux-t2.img } diff --git a/test/unit/boot-cmdline-test.sh b/test/unit/boot-cmdline-test.sh index baba215cc..5ad04517e 100644 --- a/test/unit/boot-cmdline-test.sh +++ b/test/unit/boot-cmdline-test.sh @@ -6,17 +6,24 @@ # machine has RAM to spare -- the root image the installer streams from that # medium then vanishes (seen on a ThinkPad X200s booting from USB; the QEMU # tests never hit it because they boot the ISO as an optical drive). +# +# loopback.cfg counts: mkarchiso ships it at /boot/grub/loopback.cfg, which is +# how Ventoy and a hand-written GRUB entry boot the ISO as a file on a disk. +# That path loop-mounts the image, so archisodevice is /dev/loopN rather than +# /dev/sr*, and the auto rule fires there too. It is matched on archisobasedir +# rather than archisosearchuuid because it finds the medium by img_dev/img_loop. set -euo pipefail ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) cd "$ROOT" -entries=$(grep -rhE '^\s*(APPEND|linux|options)\s.*archisosearchuuid=' \ - configs/syslinux/archiso_sys-linux.cfg configs/grub/grub.cfg configs/efiboot/loader/entries) +entries=$(grep -rhE '^\s*(APPEND|linux|options)\s.*archisobasedir=' \ + configs/syslinux/archiso_sys-linux.cfg configs/grub/grub.cfg \ + configs/grub/loopback.cfg configs/efiboot/loader/entries) count=$(printf '%s\n' "$entries" | wc -l) -[ "$count" -ge 5 ] || { echo "expected at least 5 boot entries, found $count"; exit 1; } +[ "$count" -ge 7 ] || { echo "expected at least 7 boot entries, found $count"; exit 1; } fail=0 while IFS= read -r line; do From e8a793d8311823a29c019bad47ec719a0ec4583f Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 24 Aug 2026 15:30:42 +0200 Subject: [PATCH 23/67] Bound the verify hash with a build-time, size-based start timeout Type=oneshot units are exempt from systemd's default start timeout, so a stick that stalls reads instead of erroring would hang the boot-time hash -- and the install waiting on it -- forever, with nothing on screen. build-iso.sh now writes a drop-in next to the unit sized to the image it just built: a 2 MiB/s floor over the stream size plus ten minutes of slack. On timeout systemd sets Result=timeout, and the wait helper turns that into 'install medium is too slow: try another USB stick or port' instead of the corrupt-medium re-flash advice, which would not help. The 2 MiB/s floor is a first cut; old sticks on the X200's USB 2.0 ports will calibrate it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015y69myH5K7wPRNt6p21GGp --- builder/build-iso.sh | 17 +++++++++++++++++ .../system/omarchy-root-image-verify.service | 6 ++++++ .../local/bin/omarchy-wait-root-image-verify | 7 +++++++ test/unit/wait-root-image-verify-test.sh | 15 +++++++++++---- 4 files changed, 41 insertions(+), 4 deletions(-) diff --git a/builder/build-iso.sh b/builder/build-iso.sh index 6cda2094f..3c37e5cb6 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -360,6 +360,23 @@ echo "[timing] root image end $(date +%s)" # stream, so it ships on the ISO beside it. (cd "$root_image_dir" && sha256sum "${root_image_stream##*/}" >"$root_image_stream.sha256") +# Bound the boot-time hash: Type=oneshot units are exempt from systemd's +# default start timeout, so a stick that stalls reads instead of returning an +# error would hang omarchy-root-image-verify.service -- and with it the +# install -- forever. Budget a floor of 2 MiB/s over the image size (a USB 2.0 +# port sustains ~30 MB/s; the margin covers the idle-class hash yielding to +# the live system's page-ins) plus ten minutes of slack for boot. On timeout +# systemd sets Result=timeout and omarchy-wait-root-image-verify turns that +# into a "medium too slow" message instead of the corrupt-medium one. +root_image_bytes=$(stat -c %s "$root_image_stream") +verify_dropin_dir="$build_cache_dir/airootfs/etc/systemd/system/omarchy-root-image-verify.service.d" +mkdir -p "$verify_dropin_dir" +cat >"$verify_dropin_dir/50-size-timeout.conf" </dev/null)" == timeout ]]; then + fail "install medium is too slow: try another USB stick or port"$'\n'"reading the root image did not finish within its size-based timeout" + fi # The dashboard centres each line in ~80 columns, so the action comes # first and on its own line; the journal tail (the hasher's own output) # follows for debugging. diff --git a/test/unit/wait-root-image-verify-test.sh b/test/unit/wait-root-image-verify-test.sh index 776f9bdac..5dcc888cc 100755 --- a/test/unit/wait-root-image-verify-test.sh +++ b/test/unit/wait-root-image-verify-test.sh @@ -26,9 +26,10 @@ check() { # desc, expected_rc, actual_rc, [needle in output], [output] # A sandbox with a fake boot medium and stub tools on PATH. ACTIVE_SEQ is the # newline-separated ActiveState values `systemctl show ... ActiveState` returns -# on successive calls (last one repeats); LOADSTATE and START_RC tune the rest. -run_helper() { # loadstate, active_seq, start_rc -> sets RC and OUT - local loadstate=$1 active_seq=$2 start_rc=$3 +# on successive calls (last one repeats); LOADSTATE, START_RC and RESULT (the +# unit's Result property, exit-code unless overridden) tune the rest. +run_helper() { # loadstate, active_seq, start_rc, [result] -> sets RC and OUT + local loadstate=$1 active_seq=$2 start_rc=$3 result=${4:-exit-code} local box; box=$(mktemp -d) mkdir -p "$box/bin" "$box/medium/arch/x86_64" "$box/sys/block/sdz/queue" : >"$box/medium/arch/x86_64/omarchy-root.btrfs" @@ -51,10 +52,11 @@ EOF # systemctl show -p LoadState|ActiveState --value ; systemctl start ... cat >"$box/bin/systemctl" < Date: Mon, 24 Aug 2026 15:30:55 +0200 Subject: [PATCH 24/67] Pin copytoram=n on the PXE entries that can still install; drop HTTP The NBD and NFS hooks force copytoram=y unless the cmdline says exactly n, but both keep the server's image tree mounted, so pinned they can still stream the root image and LAN installs keep working. HTTP cannot be fixed by pinning: its hook downloads only the airootfs (plus optional checksums) into a tmpfs and never mounts an image tree, so the root image does not exist on that path and every install from it would die at the pre-flight gate. The entry is removed, and the boot-cmdline guard now covers the PXE file and refuses any reintroduced archiso_http_srv entry. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015y69myH5K7wPRNt6p21GGp --- configs/syslinux/archiso_pxe-linux.cfg | 26 +++++++++++++------------- test/unit/boot-cmdline-test.sh | 16 +++++++++++++--- 2 files changed, 26 insertions(+), 16 deletions(-) diff --git a/configs/syslinux/archiso_pxe-linux.cfg b/configs/syslinux/archiso_pxe-linux.cfg index 8cf909258..b0a80ba3e 100644 --- a/configs/syslinux/archiso_pxe-linux.cfg +++ b/configs/syslinux/archiso_pxe-linux.cfg @@ -1,3 +1,14 @@ +# PXE entries pin copytoram=n like every other boot entry: the root image the +# installer streams must stay reachable under /run/archiso/bootmnt. The NBD +# and NFS hooks would otherwise force copytoram=y (they only skip the copy +# when the cmdline says exactly n), but both keep the server's image tree +# mounted, so the stream is there. +# +# There is deliberately no HTTP entry: archiso's HTTP hook copies only the +# airootfs (plus optional checksums) into a tmpfs and never mounts the image +# tree, so the root image cannot exist on that path and an install from it +# would always fail at the pre-flight gate. + LABEL arch64_nbd TEXT HELP Boot the Omarchy install medium using NBD. @@ -6,7 +17,7 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, NBD) LINUX ::/%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD ::/%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% archiso_nbd_srv=${pxeserver} cms_verify=y initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% archiso_nbd_srv=${pxeserver} copytoram=n cms_verify=y initramfs_async=0 SYSAPPEND 3 LABEL arch64_nfs @@ -17,16 +28,5 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, NFS) LINUX ::/%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD ::/%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archiso_nfs_srv=${pxeserver}:/run/archiso/bootmnt cms_verify=y initramfs_async=0 -SYSAPPEND 3 - -LABEL arch64_http -TEXT HELP -Boot the Omarchy live medium using HTTP. -It allows you to install Omarchy or perform system maintenance. -ENDTEXT -MENU LABEL Omarchy install medium (x86_64, HTTP) -LINUX ::/%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 -INITRD ::/%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archiso_http_srv=http://${pxeserver}/ cms_verify=y initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archiso_nfs_srv=${pxeserver}:/run/archiso/bootmnt copytoram=n cms_verify=y initramfs_async=0 SYSAPPEND 3 diff --git a/test/unit/boot-cmdline-test.sh b/test/unit/boot-cmdline-test.sh index 5ad04517e..175357683 100644 --- a/test/unit/boot-cmdline-test.sh +++ b/test/unit/boot-cmdline-test.sh @@ -12,6 +12,11 @@ # That path loop-mounts the image, so archisodevice is /dev/loopN rather than # /dev/sr*, and the auto rule fires there too. It is matched on archisobasedir # rather than archisosearchuuid because it finds the medium by img_dev/img_loop. +# +# The PXE entries count too: the NBD and NFS hooks force copytoram=y unless +# the cmdline says exactly n, and both keep the server's image tree mounted, +# so pinned they can still stream the root image. HTTP cannot -- its hook only +# downloads the airootfs into a tmpfs -- so no entry may use archiso_http_srv. set -euo pipefail @@ -19,11 +24,16 @@ ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) cd "$ROOT" entries=$(grep -rhE '^\s*(APPEND|linux|options)\s.*archisobasedir=' \ - configs/syslinux/archiso_sys-linux.cfg configs/grub/grub.cfg \ - configs/grub/loopback.cfg configs/efiboot/loader/entries) + configs/syslinux/archiso_sys-linux.cfg configs/syslinux/archiso_pxe-linux.cfg \ + configs/grub/grub.cfg configs/grub/loopback.cfg configs/efiboot/loader/entries) count=$(printf '%s\n' "$entries" | wc -l) -[ "$count" -ge 7 ] || { echo "expected at least 7 boot entries, found $count"; exit 1; } +[ "$count" -ge 9 ] || { echo "expected at least 9 boot entries, found $count"; exit 1; } + +if grep -rq archiso_http_srv configs/syslinux configs/grub configs/efiboot; then + echo "HTTP PXE entry found: that path copies only the airootfs to RAM, so the root image is never on the medium and an install can never succeed" + exit 1 +fi fail=0 while IFS= read -r line; do From 50cdf157877f099a174c79f749ebeb3d85dd2388 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 24 Aug 2026 15:31:03 +0200 Subject: [PATCH 25/67] Track the verify hash's actual read position on the dashboard The refactor to the boot-time verifier dropped the byte progress the old in-phase hash reported, leaving 'Preparing install target' on the dashboard's time-driven band. While the unit is still activating, the orchestrator now mirrors the hasher's read offset (the unit's MainPID is sha256sum; /proc fdinfo pos says how far it has read) into phase_progress, then collects the verdict through the helper as before. Best effort throughout: a missed sample can never fail an install. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015y69myH5K7wPRNt6p21GGp --- .../omarchy-iso/orchestrator/phases_impl.py | 50 +++++++++++++++++++ test/unit/test_root_image.py | 39 +++++++++++++++ 2 files changed, 89 insertions(+) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index e7155c4fa..b00501b20 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -138,6 +138,7 @@ def _omarchy_nvim_package() -> str: # verdict through this helper, which logs the boot medium and its I/O scheduler, # waits for the unit if it is still hashing, and starts it if it never ran. ROOT_IMAGE_VERIFY_HELPER = "/usr/local/bin/omarchy-wait-root-image-verify" +ROOT_IMAGE_VERIFY_UNIT = "omarchy-root-image-verify.service" BOOT_MEDIUM_MOUNT = Path("/run/archiso/bootmnt") @@ -273,6 +274,7 @@ def verify_root_image_stream(ctx: InstallContext) -> None: partitions, so both disk-touching paths clear the same check; whoever gets there first pays the wait. The hasher's read also leaves as much of the stream as fits in the page cache for the unpack that follows.""" + _publish_verify_progress(ctx) try: result = subprocess.run( [ROOT_IMAGE_VERIFY_HELPER], @@ -292,6 +294,54 @@ def verify_root_image_stream(ctx: InstallContext) -> None: ) +def _publish_verify_progress(ctx: InstallContext) -> None: + """While the boot-time hasher is still reading the stream, mirror its read + position into phase_progress so the dashboard bar tracks the actual hash + instead of the phase's time-driven band. Best effort throughout: the + helper is the authority on the verdict, and any hiccup here (unit already + done, hasher between opens, /proc gone) just skips a sample.""" + try: + total = ROOT_IMAGE_STREAM.stat().st_size + except OSError: + return + while total and _verify_unit_property("ActiveState") == "activating": + pos = _hasher_read_pos() + if pos is not None: + _write_phase_progress(ctx, pos / total) + time.sleep(0.5) + + +def _verify_unit_property(prop: str) -> str: + res = subprocess.run( + ["systemctl", "show", ROOT_IMAGE_VERIFY_UNIT, "-p", prop, "--value"], + capture_output=True, text=True, check=False, + ) + return res.stdout.strip() + + +def _hasher_read_pos() -> int | None: + """Byte offset of the hasher's open fd on the stream: the unit's MainPID + is sha256sum while it runs, and fdinfo's pos is how far it has read.""" + pid = _verify_unit_property("MainPID") + if not pid.isdigit() or pid == "0": + return None + fd_dir = Path("/proc") / pid / "fd" + try: + for fd in fd_dir.iterdir(): + try: + if fd.resolve() != ROOT_IMAGE_STREAM: + continue + fdinfo = (fd_dir.parent / "fdinfo" / fd.name).read_text() + except OSError: + continue + for line in fdinfo.splitlines(): + if line.startswith("pos:"): + return int(line.split()[1]) + except OSError: + pass + return None + + def arch_install_system(ctx: InstallContext) -> None: """Install the target system: archinstall partitions and mounts per the configurator JSON, the root image is unpacked onto the mounted layout, and diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 26120b9a3..ef487b6e4 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -6,6 +6,7 @@ asserted on the subprocess calls against a temp target. """ +import os import shutil import sys import tempfile @@ -132,6 +133,44 @@ def test_missing_helper_fails_the_install(self): phases_impl.verify_root_image_stream(self.ctx) +class PublishVerifyProgressTest(unittest.TestCase): + """_publish_verify_progress mirrors the hasher's fdinfo read position into + phase_progress while the unit is activating. Our own unbuffered fd on a + temp stream stands in for sha256sum's, with MainPID pointed at this + process.""" + + def test_mirrors_hasher_read_position(self): + with tempfile.TemporaryDirectory() as tmp: + stream = (Path(tmp) / "omarchy-root.btrfs").resolve() + stream.write_bytes(b"\0" * 4096) + states = iter(["activating", "active"]) + + def unit_property(prop): + return next(states) if prop == "ActiveState" else str(os.getpid()) + + fractions = [] + fd = os.open(stream, os.O_RDONLY) + try: + os.read(fd, 1024) + with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", stream), \ + mock.patch.object(phases_impl, "_verify_unit_property", + side_effect=unit_property), \ + mock.patch.object(phases_impl, "_write_phase_progress", + side_effect=lambda ctx, f: fractions.append(f)), \ + mock.patch.object(phases_impl.time, "sleep"): + phases_impl._publish_verify_progress(ctx=None) + finally: + os.close(fd) + self.assertEqual(fractions, [0.25]) + + def test_missing_stream_is_a_no_op(self): + with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", + Path("/nonexistent/omarchy-root.btrfs")), \ + mock.patch.object(phases_impl, "_verify_unit_property") as show: + phases_impl._publish_verify_progress(ctx=None) + show.assert_not_called() + + class PrepareInstallTargetTest(unittest.TestCase): """prepare_install_target wires the checks together per install mode.""" From 20c37b7cba2b21b06ffc6c9df9128f13a4801770 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 24 Aug 2026 15:31:17 +0200 Subject: [PATCH 26/67] Skip OVMF vars on a BIOS corrupt-image run; own up to the BFQ scope corrupt-image-test.sh was the one file the firmware guard missed: it copied OVMF_VARS_TEMPLATE unconditionally, keeping an edk2 dependency on a --bios run even though start_vm ignores it there. Gated on uefi like base-test.sh. The BFQ rule's comment now says what the rule actually matches -- every USB whole disk (install targets and cidata drives included), optical drives, and every mmcblk node down to eMMC boot0/rpmb -- and why that wider-than-boot-medium scope is fine: the consequence is only the scheduler choice. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015y69myH5K7wPRNt6p21GGp --- .../etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules | 8 ++++++-- test/integration.d/corrupt-image-test.sh | 6 ++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules b/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules index 373b313d7..6c97b9efa 100644 --- a/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules +++ b/configs/airootfs/etc/udev/rules.d/71-omarchy-iso-boot-medium-bfq.rules @@ -9,8 +9,12 @@ # the hash takes. With BFQ the idle-class hash yields to the live system and # fills the gaps instead. # -# Scoped to the media an ISO realistically boots from: USB disks, SD cards and -# optical drives. Internal SATA/NVMe install targets keep their default. +# Matched wider than just the boot medium, on purpose (udev cannot know which +# disk we booted from): every USB whole disk, including a USB install target +# or a cidata drive; optical drives; and every mmcblk node, including internal +# eMMC and its boot0/rpmb partitions. The consequence is only the scheduler +# choice, and BFQ is a fine default on all of them. Internal SATA/NVMe keep +# their default. # 71- so it runs after 60-persistent-storage.rules, which sets ID_BUS. ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", ENV{ID_BUS}=="usb", ATTR{queue/scheduler}="bfq" ACTION=="add|change", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", KERNEL=="sr[0-9]*|mmcblk[0-9]*", ATTR{queue/scheduler}="bfq" diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh index 34bfcb297..d7199aa24 100755 --- a/test/integration.d/corrupt-image-test.sh +++ b/test/integration.d/corrupt-image-test.sh @@ -52,8 +52,10 @@ install_from_corrupt_medium() { build_cidata qemu-img create -f qcow2 "$RUN_DIR/disk.qcow2" 40G >/dev/null - cp "$OVMF_VARS_TEMPLATE" "$RUN_DIR/OVMF_VARS.4m.fd" - ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + if [[ $FIRMWARE == uefi ]]; then + cp "$OVMF_VARS_TEMPLATE" "$RUN_DIR/OVMF_VARS.4m.fd" + ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + fi log "Autoinstalling from the corrupt medium (headless)" start_vm "$RUN_DIR/disk.qcow2" "$RUN_DIR/serial.log" \ From f63e0e820da19b7e4b8bb9dea4bbdca9f477f201 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 10:45:38 +0200 Subject: [PATCH 27/67] Show live hash progress while the free-space gate waits The free-space configurator gate swallowed the wait helper's output on purpose (stdout carries the scheduler line, stderr the failure message), so a wait on a slow medium showed a frozen 'Verifying the install medium' header with nothing moving -- the one place the boot-time hash was still invisible. The helper now draws a \r-updating percent line when OMARCHY_VERIFY_PROGRESS names a sink (the configurator passes /dev/tty), computed from the hasher's fdinfo read position -- the same source the orchestrator mirrors into the dashboard, so no pv dependency and sha256sum -c stays exactly as it is. Success finishes the line at 100%; a failure, which includes a timed-out unfinished read, just ends it. Default behavior without the variable is byte-identical. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015y69myH5K7wPRNt6p21GGp --- configs/airootfs/root/configurator | 7 +++- .../local/bin/omarchy-wait-root-image-verify | 40 +++++++++++++++++++ test/unit/wait-root-image-verify-test.sh | 39 +++++++++++++++++- 3 files changed, 82 insertions(+), 4 deletions(-) diff --git a/configs/airootfs/root/configurator b/configs/airootfs/root/configurator index 35e1d6023..b5a3b16ad 100644 --- a/configs/airootfs/root/configurator +++ b/configs/airootfs/root/configurator @@ -1089,10 +1089,13 @@ if [[ $install_target == "free_space" ]]; then # so it runs the same gate first. omarchy-wait-root-image-verify collects the # boot-time hasher's verdict, waiting for it if it is still running. (2>&1 # 1>/dev/null captures stderr, where the failure message is, and drops the - # boot-medium/scheduler line on stdout — the orchestrator logs that later.) + # boot-medium/scheduler line on stdout — the orchestrator logs that later. + # OMARCHY_VERIFY_PROGRESS routes a live percent line straight to the + # terminal while the hash is still running, so a wait on a slow medium + # shows movement instead of a frozen step header.) if [[ ${1:-} != "dry" ]]; then step "Verifying the install medium" - if ! verify_error=$(omarchy-wait-root-image-verify 2>&1 1>/dev/null); then + if ! verify_error=$(OMARCHY_VERIFY_PROGRESS=/dev/tty omarchy-wait-root-image-verify 2>&1 1>/dev/null); then abort "$(head -1 <<<"$verify_error")" fi fi diff --git a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify index 7235a7564..8297a6663 100755 --- a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify +++ b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify @@ -25,6 +25,25 @@ CHECKSUM="$STREAM.sha256" fail() { echo "$1" >&2; exit 1; } +# How far the hasher has read, as an integer percent: the unit's MainPID is +# sha256sum while it runs, and its fdinfo pos on the stream is the byte +# offset (the same source the orchestrator mirrors into the dashboard). +# Fails silently between opens or once the unit is done; callers just skip +# that sample. +hash_pct() { + local pid fd pos size + pid=$(systemctl show "$UNIT" -p MainPID --value 2>/dev/null) + [[ $pid =~ ^[0-9]+$ && $pid -gt 0 ]] || return 1 + size=$(stat -c %s "$STREAM" 2>/dev/null) || return 1 + ((size > 0)) || return 1 + for fd in /proc/"$pid"/fd/*; do + [[ $(readlink "$fd" 2>/dev/null) == "$STREAM" ]] || continue + pos=$(awk '/^pos:/ {print $2; exit}' "/proc/$pid/fdinfo/${fd##*/}" 2>/dev/null) + [[ -n $pos ]] && { echo $((pos * 100 / size)); return 0; } + done + return 1 +} + # --- boot medium + scheduler (absorbed from omarchy-iso-boot-medium) --- src=$(findmnt -no SOURCE "$BOOT_MEDIUM_MOUNT" 2>/dev/null || true) if [[ -n $src ]]; then @@ -52,12 +71,24 @@ fi systemctl start "$UNIT" >/dev/null 2>&1 reported=false +progress_drawn=false +# The \r progress line needs terminating before anything else writes to the +# terminal. Only a verified image gets the cosmetic 100%; a failure (which +# includes a timed-out, unfinished read) just ends the line. +finish_progress() { + [[ $progress_drawn == true ]] || return 0 + [[ ${1:-} == full ]] && + printf '\rverifying the install medium: 100%%' >>"$OMARCHY_VERIFY_PROGRESS" 2>/dev/null + printf '\n' >>"$OMARCHY_VERIFY_PROGRESS" 2>/dev/null || true +} while :; do case "$(systemctl show "$UNIT" -p ActiveState --value 2>/dev/null)" in active) + finish_progress full exit 0 ;; failed) + finish_progress # Result=timeout: systemd killed the hash at the size-based # TimeoutStartSec the build wrote into the unit (2 MiB/s floor over the # image size). The medium is stalling reads, not returning bad data, so @@ -78,6 +109,15 @@ while :; do echo "waiting for $UNIT to finish hashing the root image..." reported=true fi + # Opt-in live progress for interactive callers (the free-space + # configurator gate): a \r-updating percent line written to the path in + # OMARCHY_VERIFY_PROGRESS (normally /dev/tty). Kept out of stdout, which + # the orchestrator logs line by line, and stderr, whose first line + # headlines the dashboard on failure. + if [[ -n ${OMARCHY_VERIFY_PROGRESS:-} ]] && pct=$(hash_pct); then + printf '\rverifying the install medium: %3d%%' "$pct" >>"$OMARCHY_VERIFY_PROGRESS" 2>/dev/null || true + progress_drawn=true + fi sleep 0.5 ;; *) diff --git a/test/unit/wait-root-image-verify-test.sh b/test/unit/wait-root-image-verify-test.sh index 5dcc888cc..aaff315b0 100755 --- a/test/unit/wait-root-image-verify-test.sh +++ b/test/unit/wait-root-image-verify-test.sh @@ -34,6 +34,18 @@ run_helper() { # loadstate, active_seq, start_rc, [result] -> sets RC and OUT mkdir -p "$box/bin" "$box/medium/arch/x86_64" "$box/sys/block/sdz/queue" : >"$box/medium/arch/x86_64/omarchy-root.btrfs" : >"$box/medium/arch/x86_64/omarchy-root.btrfs.sha256" + + # WITH_HASHER_PROC=1: a fake hasher (pid 4242) holds the stream at pos 512 + # of 1024 bytes, and the helper draws progress into $box/progress. The shim + # rewrites /proc to the sandbox, so hash_pct reads this fixture. + local mainpid=0 + if [[ -n ${WITH_HASHER_PROC:-} ]]; then + mainpid=4242 + head -c 1024 /dev/zero >"$box/medium/arch/x86_64/omarchy-root.btrfs" + mkdir -p "$box/proc/4242/fd" "$box/proc/4242/fdinfo" + ln -s "$box/medium/arch/x86_64/omarchy-root.btrfs" "$box/proc/4242/fd/7" + printf 'pos:\t512\nflags:\t0100000\n' >"$box/proc/4242/fdinfo/7" + fi echo "none mq-deadline kyber [bfq]" >"$box/sys/block/sdz/queue/scheduler" printf '%s\n' "$active_seq" >"$box/active_seq" @@ -52,11 +64,12 @@ EOF # systemctl show -p LoadState|ActiveState --value ; systemctl start ... cat >"$box/bin/systemctl" <"$shim" chmod +x "$shim" + local progress_env=() + [[ -n ${WITH_HASHER_PROC:-} ]] && progress_env=(OMARCHY_VERIFY_PROGRESS="$box/progress") + set +e - OUT=$(PATH="$box/bin:$PATH" bash "$shim" 2>&1) + OUT=$(PATH="$box/bin:$PATH" env "${progress_env[@]}" bash "$shim" 2>&1) RC=$? set -e + PROGRESS_OUT=$(cat "$box/progress" 2>/dev/null || true) rm -rf "$box" } @@ -94,6 +112,23 @@ check "active unit passes" 0 "$RC" "scheduler: none mq-deadline kyber [bfq]" "$O run_helper loaded $'activating\nactivating\nactive' 0 check "waits for a running unit then passes" 0 "$RC" "waiting for" "$OUT" +# With OMARCHY_VERIFY_PROGRESS set and a hasher pid to read, the wait draws a +# live percent (fixture: pos 512 of 1024 -> 50%) and finishes the line at 100% +# on success. Progress goes to the given path, never into stdout/stderr. +WITH_HASHER_PROC=1 run_helper loaded $'activating\nactivating\nactive' 0 +check "progress line drawn while hashing" 0 "$RC" "" "$OUT" +[[ $PROGRESS_OUT == *" 50%"* && $PROGRESS_OUT == *"100%"* ]] && + echo "ok: progress shows 50% then 100%" || + { echo "FAIL: progress output wrong: $(printf '%q' "$PROGRESS_OUT")"; fails=1; } +[[ $OUT == *"50%"* ]] && { echo "FAIL: percent leaked into stdout/stderr"; fails=1; } || true + +# A failed hash ends the progress line without the cosmetic 100%. (The first +# ActiveState answer is consumed by the inactive check before the wait loop.) +WITH_HASHER_PROC=1 run_helper loaded $'activating\nactivating\nfailed' 0 +[[ $PROGRESS_OUT == *" 50%"* && $PROGRESS_OUT != *"100%"* ]] && + echo "ok: failed hash ends progress without 100%" || + { echo "FAIL: failed-hash progress wrong: $(printf '%q' "$PROGRESS_OUT")"; fails=1; } + # Hash failed: corrupt medium, re-flash message leads. run_helper loaded "failed" 0 check "failed unit is a corrupt medium" 1 "$RC" "install medium is corrupt: re-flash it" "$OUT" From b99fb88ab23d2d1a57cfb4e95416c89e4e632554 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 11:26:24 +0200 Subject: [PATCH 28/67] Surface the verify percent on the dashboard's full-disk gate wait The full-disk path clears the install-medium verify inside the orchestrator's 'Preparing install target' phase, whose dashboard band is 15 per-mille wide -- sized for the subsecond case where the boot-time hash already finished. When install is pressed while the hash is still running, minutes of real progress move the bar about one cell and the UI reads as hung; the free-space path got a live percent line in f63e0e8, but the dashboard never did. The dashboard now maps the phase's progress into an honest percent and prints 'verifying the install medium: N%' on the row between the bar and the tip line -- blank in every other state, so the layout height never changes. The unpack band is wide enough that its progress stays in the bar itself. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 1b9020397..25fd714e7 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -199,6 +199,7 @@ PKG_FLOOR_TAU=900 # with a work signal, time is only a backstop until the POS=10 # position, per-mille, monotonic PROGRESS_PM=10 +VERIFY_PCT=-1 # honest percent of the pre-flight verify, -1 when idle NOW=0 # monotonic seconds, refreshed once per frame DASH_T0=0 PHASE_NAME="" # last phase this process observed @@ -300,6 +301,7 @@ install_progress() { local -a state PROGRESS_PM=$POS + VERIFY_PCT=-1 [[ -f $STATE_FILE ]] || return 0 mapfile -t state < <(jq -r ' @@ -401,6 +403,12 @@ install_progress() { fi if (( progress_pm >= 0 )); then (( progress_pm > 1000 )) && progress_pm=1000 + # The pre-flight band is 15 per-mille wide (the phase is subsecond once + # the boot-time hash has finished), so real progress mapped into it moves + # the bar about one cell over minutes and reads as a hang. Surface the + # honest percent as a status line instead (render_dynamic); the unpack's + # band is wide enough that its progress shows in the bar itself. + [[ $phase == "Preparing install target" ]] && VERIFY_PCT=$(( progress_pm / 10 )) work=$(( lo + (span - 1 - PKG_TAIL_PM) * progress_pm / 1000 )) (( work > pos )) && pos=$work fi @@ -468,7 +476,13 @@ render_dynamic() { line_at "$CONTENT_WIDTH" $(( (CONTENT_WIDTH - 34) / 2 )) "" progress_bar "$pm" 34 printf '\n' - blank_line + # Same row either way, so the layout height never changes: blank normally, + # the verify's honest percent while the pre-flight gate waits on the hash. + if (( VERIFY_PCT >= 0 )); then + center "${DIM}verifying the install medium: ${VERIFY_PCT}%${RESET}" "$CONTENT_WIDTH" + else + blank_line + fi center "${DIM}Tip:${RESET} ${GREEN}$(current_tip)${RESET}" "$CONTENT_WIDTH" printf '%s' "$CLEAR_TO_END" } From 6325a250e67e7cbb1c5ccd6e2298b45d8aa03cf9 Mon Sep 17 00:00:00 2001 From: Omarchybot Date: Tue, 25 Aug 2026 05:30:51 -0700 Subject: [PATCH 29/67] Guard the free-space path's verify gate against silent removal The free-space install formats inside the configurator, so the gate added in 30596b0 is the only thing standing between a corrupt install medium and parted, wipefs, luksFormat and mkfs on a disk that already holds someone else's OS. Nothing tests that it is still there: corrupt-image-test.sh autoinstalls from cidata, which skips the configurator entirely, and wait-root-image-verify-test.sh exercises the helper rather than its call site. Replacing the whole gate with `if false` leaves ./test/all green -- which is how the gate came to be missing in the first place. A static ordering check costs nothing and catches that: the verify call has to exist inside the free_space branch and precede run_partition_execute. Verified by mutation -- removing the gate and moving the format above it each fail the new test, and both passed without it. Co-Authored-By: Claude Opus 5 (1M context) --- test/unit/free-space-gate-test.sh | 39 +++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100755 test/unit/free-space-gate-test.sh diff --git a/test/unit/free-space-gate-test.sh b/test/unit/free-space-gate-test.sh new file mode 100755 index 000000000..45955c643 --- /dev/null +++ b/test/unit/free-space-gate-test.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# +# The free-space install formats inside the configurator, so the configurator +# is the only thing that can refuse a bad install medium before parted, wipefs, +# luksFormat and mkfs have already run. The full-disk path has an integration +# scenario for its gate (corrupt-image-test.sh, which autoinstalls from cidata +# and so never reaches the configurator at all); this path has none, and +# deleting its gate outright leaves ./test/all green. Assert the ordering +# statically instead: whatever else moves, omarchy-wait-root-image-verify has +# to run before run_partition_execute. + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +configurator="$ROOT/configs/airootfs/root/configurator" + +# Comment lines are dropped first: the branch below is explained in prose that +# names both the helper and the function, so matching them would find the +# explanation whether or not the code still does it. +line_of() { # what, pattern -> the one line number matching it + local what=$1 pattern=$2 hits + hits=$(grep -vnE '^\s*#' "$configurator" | grep -E ":$pattern" | cut -d: -f1) + if [[ $(grep -c . <<<"$hits") -ne 1 ]]; then + echo "expected exactly one $what in ${configurator#"$ROOT/"}, found: ${hits:-none}" >&2 + exit 1 + fi + printf '%s\n' "$hits" +} + +branch=$(line_of "free_space branch" 'if \[\[ \$install_target == "free_space" \]\]; then$') +gate=$(line_of "call to the verify gate" '.*omarchy-wait-root-image-verify') +format=$(line_of "call to run_partition_execute" '\s+run_partition_execute ') + +((branch < gate)) || + { echo "the verify gate is outside the free_space branch (branch $branch, gate $gate)"; exit 1; } +((gate < format)) || + { echo "run_partition_execute runs before the verify gate (gate $gate, format $format): a corrupt install medium would be found only after the disk was partitioned"; exit 1; } + +echo "ok: the free-space path verifies the install medium before it formats" From 36c68490836cd37dc4e919e509b5f2529f4a612d Mon Sep 17 00:00:00 2001 From: Omarchybot Date: Tue, 25 Aug 2026 05:35:41 -0700 Subject: [PATCH 30/67] Wait out the stop when the verify unit hits its start timeout The size-based TimeoutStartSec exists so a stick that stalls reads gets "install medium is too slow" instead of a hang, but the wait almost never said that. systemd answers a start timeout by stopping the unit, and the unit reports ActiveState=deactivating until that stop completes -- which is exactly what a dead medium makes slow, because SIGTERM does not land on a read stuck in the driver. The 0.5s poll therefore sampled deactivating long before failed, fell through to the wildcard branch, and told the user "omarchy-root-image-verify.service did not run; cannot verify the root image": the one message that describes neither what happened nor what to do. deactivating now waits for the unit to settle and reads the real Result, so the timeout headline lands. The wait is bounded past DefaultTimeoutStopSec rather than open-ended: a read on a dead medium may never be reaped, and an unbounded wait here is the hang the timeout was added to prevent. Both paths already aborted with the disk untouched; this is which error the user reads. Co-Authored-By: Claude Opus 5 (1M context) Co-Authored-By: Codex XHigh --- .../usr/local/bin/omarchy-wait-root-image-verify | 16 ++++++++++++++++ test/unit/wait-root-image-verify-test.sh | 7 +++++++ 2 files changed, 23 insertions(+) diff --git a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify index 8297a6663..b33deabd0 100755 --- a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify +++ b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify @@ -72,6 +72,7 @@ fi reported=false progress_drawn=false +settling=0 # The \r progress line needs terminating before anything else writes to the # terminal. Only a verified image gets the cosmetic 100%; a failure (which # includes a timed-out, unfinished read) just ends the line. @@ -120,6 +121,21 @@ while :; do fi sleep 0.5 ;; + deactivating) + # A start timeout stops the unit before it settles into failed, and the + # stop is what the medium makes slow: SIGTERM does not land on a read + # stuck in the driver. Classifying this state now would answer "did not + # run" for the one case the size-based TimeoutStartSec exists to name, + # so wait for the terminal state -- bounded past DefaultTimeoutStopSec, + # because a read on a dead medium may never be reaped at all. + if ((settling < 240)); then + settling=$((settling + 1)) + sleep 0.5 + continue + fi + finish_progress + fail "install medium is too slow: try another USB stick or port"$'\n'"the root image read could not be stopped after its size-based timeout" + ;; *) fail "$UNIT did not run; cannot verify the root image" ;; diff --git a/test/unit/wait-root-image-verify-test.sh b/test/unit/wait-root-image-verify-test.sh index aaff315b0..e4c19ccf0 100755 --- a/test/unit/wait-root-image-verify-test.sh +++ b/test/unit/wait-root-image-verify-test.sh @@ -138,6 +138,13 @@ check "failed unit is a corrupt medium" 1 "$RC" "install medium is corrupt: re-f run_helper loaded "failed" 0 timeout check "timed-out unit is a slow medium" 1 "$RC" "install medium is too slow" "$OUT" +# systemd stops a timed-out unit before it settles into failed, so the wait +# passes through deactivating. Waiting that out is what keeps the slow-medium +# advice: classifying it as terminal answered "did not run" instead. +run_helper loaded $'activating\ndeactivating\ndeactivating\nfailed' 0 timeout +check "a unit still stopping is waited out, not called unrunnable" 1 "$RC" \ + "install medium is too slow" "$OUT" + # Never started and start leaves it inactive: cannot verify. run_helper loaded "inactive" 0 check "unit that will not run fails" 1 "$RC" "did not run" "$OUT" From eec294f3ffd0b786a6971f82a9b31eee8396d103 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 15:51:42 +0200 Subject: [PATCH 31/67] Integration scenario: a timed-out verify names the slow medium The size-based TimeoutStartSec and its "install medium is too slow" message had no end-to-end coverage: the corrupt-image scenario proves the checksum path, but nothing boots an ISO whose medium stalls. This scenario forces the timeout on an unmodified ISO and asserts the user reads the slow-medium error with the disk untouched. Mechanics: a systemd credential injected over SMBIOS type 11 adds a drop-in shrinking TimeoutStartSec to 3 min -- named 99-* so it sorts after the build-time 50-size-timeout.conf and wins the merge. The medium boots throttled to 40 MB/s so the hash cannot finish early, then, once the installer is parked at the pre-flight gate, is choked to 64 KB/s over QMP: mid-read when the timeout's stop arrives, like a dying stick. The choke rate matters twice over -- slower turns reads into I/O errors past the guest kernel's 30 s command timeout (the unit then fails as corrupt, not slow), and the choke must come after the gate is reached or it stalls the installer's own climb there and the helper only samples the unit after the stop has settled. Validated against both sides of 95e477c: an ISO without it records "omarchy-root-image-verify.service did not run; cannot verify the root image" and fails the message assertions -- the exact regression that commit fixed -- while an ISO with it passes all nine. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- test/integration.d/slow-medium-test.sh | 176 +++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100755 test/integration.d/slow-medium-test.sh diff --git a/test/integration.d/slow-medium-test.sh b/test/integration.d/slow-medium-test.sh new file mode 100755 index 000000000..576cbdd30 --- /dev/null +++ b/test/integration.d/slow-medium-test.sh @@ -0,0 +1,176 @@ +#!/bin/bash +# +# A medium too slow to read is named as such, with the disk untouched. The +# boot-time verify unit carries a size-based TimeoutStartSec so a stalling +# stick fails the pre-flight gate instead of hanging the install forever; +# this scenario forces that timeout and asserts the user reads "install +# medium is too slow" — not a generic failure. The regression it guards: a +# start timeout makes systemd STOP the unit, and on a dying medium the stop +# itself drags (SIGTERM cannot land on a read stuck in the driver), so the +# unit reports 'deactivating' long before 'failed'; the wait helper used to +# classify that window as "did not run". +# +# Mechanics: the ISO is booted unmodified. A systemd credential injected via +# SMBIOS type 11 adds a drop-in shrinking TimeoutStartSec — named 99-* so it +# sorts after the build-time 50-size-timeout.conf and wins the merge. The +# medium is throttled at boot so the hash cannot finish early, then choked +# over QMP once the installer is waiting at the gate: the hash is mid-read +# when the timeout's stop arrives, exactly like a dying stick. +# +# Boots the ISO itself, not the installed base image, so it needs no base. + +source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" + +VERIFY_UNIT=omarchy-root-image-verify.service +STATE=/run/omarchy-install/state.json + +# Slow enough that hashing the multi-GB image cannot beat the timeout, fast +# enough that the live system boots promptly: the image needs 90+ s at this +# rate while the unit times out at 180 s only because the choke below stops +# the hash long before either. +BOOT_THROTTLE_BPS=$((40 * 1024 * 1024)) + +# From unit start (early boot): long enough that SSH is up and the choke has +# landed first, short enough to keep the scenario brisk. +TIMEOUT_DROPIN=$'[Service]\nTimeoutStartSec=180\n' + +# ------------------------------------------------------------------ fixture + +# QMP is line-delimited JSON: the whole command must be a single line, and +# qmp() swallows errors, so read the response — a failed choke would let the +# hash win the race against the timeout and pass the gate legitimately. +throttle_medium() { + local bps="$1" resp + resp=$(qmp "\"block_set_io_throttle\", \"arguments\": {\"device\": \"cdrom0\", \"bps\": $bps, \"bps_rd\": 0, \"bps_wr\": 0, \"iops\": 0, \"iops_rd\": 0, \"iops_wr\": 0}") + if [[ $resp == *'"error"'* || $resp != *'"return"'* ]]; then + echo "block_set_io_throttle failed: $resp" >&2 + return 1 + fi +} + +# -------------------------------------------------------------------- phases + +install_from_slow_medium() { + local cred + cred=$(base64 -w0 <<<"$TIMEOUT_DROPIN") + + [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-integration" -f "$SSH_KEY" + detect_packages + build_cidata + + qemu-img create -f qcow2 "$RUN_DIR/disk.qcow2" 40G >/dev/null + if [[ $FIRMWARE == uefi ]]; then + cp "$OVMF_VARS_TEMPLATE" "$RUN_DIR/OVMF_VARS.4m.fd" + ACTIVE_OVMF="$RUN_DIR/OVMF_VARS.4m.fd" + fi + + log "Autoinstalling from a throttled medium (headless)" + start_vm "$RUN_DIR/disk.qcow2" "$RUN_DIR/serial.log" \ + -drive "file=$ISO,media=cdrom,if=none,format=raw,id=cdrom0,throttling.bps-total=$BOOT_THROTTLE_BPS" \ + -device ide-cd,drive=cdrom0,bootindex=2 \ + -drive "file=$CIDATA_IMG,format=raw,if=none,id=cidata" \ + -device usb-storage,drive=cidata \ + -smbios "type=11,value=io.systemd.credential.binary:systemd.unit-dropin.${VERIFY_UNIT}~99-test-timeout=$cred" + + bootstrap_live_root_ssh + + check "shrunk timeout is in effect" \ + ssh_live_root "systemctl show -p TimeoutStartUSec --value $VERIFY_UNIT | grep -qx 3min" + + # The choke must land while the installer is already waiting at the gate: + # the field failure is a user parked on the pre-flight phase when the + # timeout hits, with the helper actively polling the unit through systemd's + # slow stop of it. Choking earlier stalls the installer's own climb to the + # gate, and it then only samples the unit after the stop has settled — + # which hides exactly the window this scenario exists to pin down. + log "Waiting for the installer to reach the verify gate" + local gated=0 + until ssh_live_root "jq -e '.current_phase == \"Preparing install target\"' $STATE" >/dev/null 2>&1; do + if ! vm_running; then + echo "VM exited before the installer reached the verify gate" >&2 + return 1 + fi + if ((gated >= 300)); then + capture_console "failure-gate-timeout" + echo "Timed out waiting for the installer to reach the verify gate" >&2 + return 1 + fi + sleep 2 + ((gated += 2)) + done + + # If the hash already finished, the gate passes legitimately and nothing + # below tests the timeout; the throttle above is sized to prevent this. + check "verify unit is still hashing when the medium dies" \ + ssh_live_root "systemctl show -p ActiveState --value $VERIFY_UNIT | grep -qx activating" + + log "Choking the medium to 64 KB/s" + # Slow enough that the hash cannot finish and the stop of a mid-read + # process takes visible seconds, fast enough that no single request + # outlives the guest kernel's 30 s command timeout — 1 KB/s turns reads + # into I/O errors and the unit fails as corrupt instead of slow. + throttle_medium 65536 + + log "Waiting for the installer to stop" + local waited=0 + until ssh_live_root 'grep -q "installer child exited" /var/log/omarchy-install.log' 2>/dev/null; do + if ! vm_running; then + echo "VM exited while waiting for the installer" >&2 + return 1 + fi + if ((waited >= 600)); then + capture_console "failure-installer-timeout" + echo "Timed out waiting for the installer to stop" >&2 + return 1 + fi + sleep 5 + ((waited += 5)) + done + + # Lift the choke so the stuck read drains and the unit can settle into its + # terminal state, and so the assertions below are not themselves throttled. + throttle_medium 0 + local settled=0 + until ssh_live_root "systemctl show -p ActiveState --value $VERIFY_UNIT | grep -qx failed" 2>/dev/null; do + if ((settled >= 120)); then + break + fi + sleep 2 + ((settled += 2)) + done + + sleep 2 + capture_console "success-installer-stopped" + ssh_live_root "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true + ssh_live_root "cat $STATE" >"$RUN_DIR/state.json" 2>/dev/null || true + ssh_live_root "journalctl -b -u $VERIFY_UNIT -o short-precise --no-pager" >"$RUN_DIR/verify-unit.journal" 2>/dev/null || true +} + +screen_shows() { + ocr_screen | grep -qi "$1" +} + +assert_named_slow() { + check "verify unit ended in a start timeout" \ + ssh_live_root "systemctl show -p Result --value $VERIFY_UNIT | grep -qx timeout" + check "install halted in the pre-flight phase" \ + ssh_live_root "jq -e '[.phases[] | select(.status == \"failed\") | .name] == [\"Preparing install target\"]' $STATE" + check "the error names the slow medium" \ + ssh_live_root "jq -r '.phases[] | select(.status == \"failed\") | .error' $STATE | grep -q 'install medium is too slow: try another USB stick or port'" + check "nothing ran after the pre-flight phase" \ + ssh_live_root "jq -e '[.phases[] | select(.status == \"ok\") | .name] == [\"Preparing live environment\"]' $STATE" + check "target disk has no partition table" \ + ssh_live_root "! lsblk -rno TYPE /dev/vda | grep -qx part && ! blkid /dev/vda" + # The stopped-banner renders red on dark, which OCR misreads; the log + # carries the dashboard's own record of the stop. + check "dashboard reported the installer stopped" \ + grep -q "installer child exited with status 1" "$RUN_DIR/omarchy-install.log" + check "dashboard shows the slow-medium advice" \ + screen_shows "too slow" +} + +# ---------------------------------------------------------------------- main + +install_from_slow_medium +assert_named_slow +finish From d99c555642a5735565d63847753d3798aef085c8 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 16:57:46 +0200 Subject: [PATCH 32/67] Let the install medium be removed at the Reboot Now screen With copytoram=n the live root pages from the install medium on demand, so pulling the stick before reboot completes crashes the live system -- the caveat the hardware A/B run recorded. The medium's real work ends with the install; what kept it captive was the graceful reboot, whose stop jobs page unit files and ExecStop binaries off it. The finish screen now says 'You can now remove the install medium', and makes it true: before the prompt, everything the button press needs is pre-faulted while the medium is still present; at the press, the target -- intentionally left mounted until now so the other ttys can inspect it -- is unmounted recursively, synced, and the reboot taken with systemctl reboot -ff. The medium is read-only and the live logs are tmpfs, so after the umount nothing anywhere is dirty and the immediate path is clean; skipping the stop jobs is precisely what removes the last reads. Failure and deferred-provisioning paths keep the graceful reboot: the medium is still wanted for diagnosis, and unattended installs have nobody holding a stick. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 49 ++++++++++++++++++- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index d005aee37..e435b5ca0 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -161,7 +161,8 @@ LAST_PM=10 # one button — so the last thing the installer shows matches the first. It runs # one row taller: `gum confirm` always paints prompt/blank/button and clears all # three, so the button cannot sit closer than two rows under a title we drew. -FINISH_HEIGHT=$((LOGO_HEIGHT + 5)) +# Two more rows under the button: a blank and the medium-removal note. +FINISH_HEIGHT=$((LOGO_HEIGHT + 7)) FINISH_TOP_ROW=2 # ── Install progress ───────────────────────────────────────────────────────── @@ -561,6 +562,46 @@ render_finish() { >"$TTY_PATH" 2>/dev/null || dashboard_log "logo effect skipped (ttfx exited $?)" printf '%s%d;1H' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 5))" >"$TTY_PATH" fi + + # True because of warm_reboot_path + release_target_and_reboot: from here to + # the reboot syscall nothing reads the install medium again. + { + printf '%s%d;1H' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 6))" + center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" + } >"$TTY_PATH" +} + +# ── Medium-free reboot ────────────────────────────────────────────────────── +# +# The live root pages from the install medium on demand (copytoram=n keeps +# the root image reachable during the install), so pulling the stick is only +# safe once nothing will fault a cold read anymore. A graceful reboot is the +# opposite of that: its stop jobs page unit files and ExecStop binaries. So: +# while the medium is still present, pre-fault everything the button press +# needs; at the press, release the target — held mounted until now so the +# other ttys can inspect it — sync, and take the immediate reboot path. The +# medium itself is read-only and the live logs are tmpfs: after umount -R of +# the target nothing anywhere is dirty, which is what makes -ff clean here. + +warm_reboot_path() { + local bin path dep + sync + for bin in umount systemctl sync jq; do + path=$(command -v "$bin" 2>/dev/null) || continue + cat "$path" >/dev/null 2>&1 || true + while IFS= read -r dep; do + [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 + done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') + done +} + +release_target_and_reboot() { + local target + target=$(jq -r '.target // "/mnt"' "$STATE_FILE" 2>/dev/null) || target=/mnt + sync + umount -R "$target" 2>/dev/null || true + sync + systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true } reboot_prompt() { @@ -907,12 +948,16 @@ if (( finish_render_status != 0 )); then } >"$TTY_PATH" 2>/dev/null || true fi +# Warm before the prompt: the removal note the finish screen shows is only +# honest once the button press can run without the medium. +warm_reboot_path 2>/dev/null || true + if reboot_prompt; then if [[ ${OMARCHY_UI_AUTO_REBOOT:-} == "no" ]]; then exit 0 fi printf '%s' "$CLEAR" >"$TTY_PATH" 2>/dev/null || true - reboot 2>/dev/null || systemctl reboot 2>/dev/null || true + release_target_and_reboot fi exit 0 From 6fd69657b2930b427fde283e844ef29f9dcf2da3 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 17:09:54 +0200 Subject: [PATCH 33/67] Draw the removal note above the button, where gum cannot erase it gum repaints its confirm view every frame and clears everything under it, so the note below the button never survived the first paint. It now sits between the install-time line and the prompt, and the button anchors two rows lower to make room. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 21 +++++++++---------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index e435b5ca0..36586d904 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -161,7 +161,8 @@ LAST_PM=10 # one button — so the last thing the installer shows matches the first. It runs # one row taller: `gum confirm` always paints prompt/blank/button and clears all # three, so the button cannot sit closer than two rows under a title we drew. -# Two more rows under the button: a blank and the medium-removal note. +# Two more rows between title and button: a blank and the medium-removal +# note — above the prompt, because gum clears everything under its view. FINISH_HEIGHT=$((LOGO_HEIGHT + 7)) FINISH_TOP_ROW=2 @@ -539,6 +540,11 @@ render_finish() { blank_line center "Installed Omarchy in ${duration}" "$CONTENT_WIDTH" blank_line + # Above the button, not below: gum clears everything under its view on + # each repaint, so a line below the prompt would be erased. True because + # of warm_reboot_path + release_target_and_reboot: from here to the + # reboot syscall nothing reads the install medium again. + center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" } >"$TTY_PATH" if [[ -f $LOGO_PATH ]]; then @@ -562,13 +568,6 @@ render_finish() { >"$TTY_PATH" 2>/dev/null || dashboard_log "logo effect skipped (ttfx exited $?)" printf '%s%d;1H' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 5))" >"$TTY_PATH" fi - - # True because of warm_reboot_path + release_target_and_reboot: from here to - # the reboot syscall nothing reads the install medium again. - { - printf '%s%d;1H' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 6))" - center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" - } >"$TTY_PATH" } # ── Medium-free reboot ────────────────────────────────────────────────────── @@ -612,9 +611,9 @@ reboot_prompt() { interactive || return 0 # gum paints its prompt row at the cursor and the button two rows under it, so - # anchor on the row right below the title to put the button as close to it as - # gum allows. FINISH_HEIGHT budgets for the same three rows. - printf '%s%d;1H%s' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 2))" "$CLEAR_LINE" >"$TTY_PATH" + # anchor on the row right below the removal note to put the button as close + # to it as gum allows. FINISH_HEIGHT budgets for the same three rows. + printf '%s%d;1H%s' "$CSI" "$((FINISH_TOP_ROW + LOGO_HEIGHT + 4))" "$CLEAR_LINE" >"$TTY_PATH" prompt_pad=$(( ( $(term_cols) - 18 ) / 2 )) (( prompt_pad < 0 )) && prompt_pad=0 From 8559a8420eeb8044c25588dac9e60e6f0da3bbef Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 17:58:34 +0200 Subject: [PATCH 34/67] Release the target before offering the medium's removal The umount moved from the button press to before the finish screen: the install is fully on its own disk by the time the user can act on the removal note, so a pulled stick followed by anything -- power loss included -- leaves no mounted-rw target behind. The press itself is now just the immediate reboot, and the warm shrinks to the one binary it still runs. The cost, accepted deliberately: the installed tree can no longer be inspected from another tty between install and reboot. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 54 +++++++++---------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 36586d904..03a224e69 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -575,32 +575,30 @@ render_finish() { # The live root pages from the install medium on demand (copytoram=n keeps # the root image reachable during the install), so pulling the stick is only # safe once nothing will fault a cold read anymore. A graceful reboot is the -# opposite of that: its stop jobs page unit files and ExecStop binaries. So: -# while the medium is still present, pre-fault everything the button press -# needs; at the press, release the target — held mounted until now so the -# other ttys can inspect it — sync, and take the immediate reboot path. The -# medium itself is read-only and the live logs are tmpfs: after umount -R of -# the target nothing anywhere is dirty, which is what makes -ff clean here. - -warm_reboot_path() { - local bin path dep - sync - for bin in umount systemctl sync jq; do - path=$(command -v "$bin" 2>/dev/null) || continue - cat "$path" >/dev/null 2>&1 || true - while IFS= read -r dep; do - [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 - done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') - done -} - -release_target_and_reboot() { +# opposite of that: its stop jobs page unit files and ExecStop binaries. So, +# before the finish screen offers the medium's removal: release the target — +# fully written to its own disk from that moment, whatever happens after — +# and pre-fault the one binary the button press still runs. The medium is +# read-only and the live logs are tmpfs, so past the umount nothing anywhere +# is dirty, which is what makes the immediate reboot at the press clean. +# The cost, accepted deliberately: the installed tree can no longer be +# inspected from another tty between install and reboot. + +release_target() { local target target=$(jq -r '.target // "/mnt"' "$STATE_FILE" 2>/dev/null) || target=/mnt sync - umount -R "$target" 2>/dev/null || true + umount -R "$target" 2>/dev/null || dashboard_log "release: umount -R $target failed" sync - systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true +} + +warm_reboot_path() { + local path dep + path=$(command -v systemctl 2>/dev/null) || return 0 + cat "$path" >/dev/null 2>&1 || true + while IFS= read -r dep; do + [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 + done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') } reboot_prompt() { @@ -935,6 +933,12 @@ if [[ ${OMARCHY_UI_DEFER_PROVISIONING:-} == "yes" ]]; then exit 0 fi +# Before the screen that offers the medium's removal: the target is released +# first, so the install is fully on its own disk before the user can act on +# the note; the warm makes the button press itself medium-free. +release_target 2>/dev/null || true +warm_reboot_path 2>/dev/null || true + set +e render_finish finish_render_status=$? @@ -947,16 +951,12 @@ if (( finish_render_status != 0 )); then } >"$TTY_PATH" 2>/dev/null || true fi -# Warm before the prompt: the removal note the finish screen shows is only -# honest once the button press can run without the medium. -warm_reboot_path 2>/dev/null || true - if reboot_prompt; then if [[ ${OMARCHY_UI_AUTO_REBOOT:-} == "no" ]]; then exit 0 fi printf '%s' "$CLEAR" >"$TTY_PATH" 2>/dev/null || true - release_target_and_reboot + systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true fi exit 0 From caf8fe5fa3072ddbe3f1addf6ee52d69bc7abf6f Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 17:59:02 +0200 Subject: [PATCH 35/67] Say what debugging actually loses: a pre-mount, not access Jonny's point: the target can always be mounted by hand from a tty, so the release costs a convenience, not the ability to inspect. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- configs/airootfs/usr/local/bin/omarchy-install-dashboard | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 03a224e69..df854e55e 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -581,8 +581,8 @@ render_finish() { # and pre-fault the one binary the button press still runs. The medium is # read-only and the live logs are tmpfs, so past the umount nothing anywhere # is dirty, which is what makes the immediate reboot at the press clean. -# The cost, accepted deliberately: the installed tree can no longer be -# inspected from another tty between install and reboot. +# Debugging keeps its access: the installed tree is no longer pre-mounted on +# the other ttys, but one mount command away for whoever wants it. release_target() { local target From 7affaa75da9ab01f4dcd4635e9f9224a217035e0 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 18:08:14 +0200 Subject: [PATCH 36/67] Swapoff the target's swapfile before the release umount The hibernation setup leaves the target's swapfile active in the live system, and swap is an invisible holder: fuser shows no process, the submounts unmount fine, and the btrfs still reports busy. The release now swapoffs everything under the target first -- verified live: with the swapfile active, umount -R fails exactly as the dashboard log recorded; after swapoff it releases cleanly. This also means the earlier press-time umount had been failing silently and the -ff reboot left a mounted target for btrfs to recover; the log line that exposed it stays. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- configs/airootfs/usr/local/bin/omarchy-install-dashboard | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index df854e55e..883853a5f 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -585,9 +585,15 @@ render_finish() { # the other ttys, but one mount command away for whoever wants it. release_target() { - local target + local target swapfile target=$(jq -r '.target // "/mnt"' "$STATE_FILE" 2>/dev/null) || target=/mnt sync + # The hibernation setup leaves the target's swapfile active, and swap is an + # invisible holder: no process shows in fuser, submounts unmount fine, and + # the btrfs still refuses to release until the swapoff. + while IFS= read -r swapfile; do + [[ $swapfile == "$target"/* ]] && swapoff "$swapfile" 2>/dev/null + done < <(awk 'NR > 1 { print $1 }' /proc/swaps) umount -R "$target" 2>/dev/null || dashboard_log "release: umount -R $target failed" sync } From 03479feff25a84bc12ab0fa14408fd23db51845b Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 18:55:02 +0200 Subject: [PATCH 37/67] Swapoff the target's swap in the protected failure cleanup too Same invisible holder as the release path fixed in 7affaa7: the hibernation setup's swapon in the chroot lands in the shared kernel, so a protected install failing after that phase left cleanup's umount -R silently busy and the LUKS mapper unclosable -- a retry then found the stale mapper. Deactivate the target's swap first, exactly as the dashboard release does. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../omarchy-iso/orchestrator/phases_impl.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index e68b416c0..7b3ff89e9 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -2258,6 +2258,21 @@ def cleanup_target_hook_masks(ctx: InstallContext) -> None: _unmask_mkinitcpio_pacman_hooks(ctx, ctx.target, TARGET_DEFERRED_BOOT_HOOKS) +def _swapoff_under(target: Path) -> None: + """Deactivate every swap area on the target. The hibernation setup's + swapon in the chroot lands in the shared kernel, and active swap is an + invisible mount holder: fuser shows no process, yet umount -R reports + busy until the swapoff.""" + try: + entries = Path("/proc/swaps").read_text().splitlines()[1:] + except OSError: + return + for entry in entries: + swap_path = entry.split()[0] + if swap_path.startswith(f"{target}/"): + subprocess.run(["swapoff", swap_path], check=False, capture_output=True) + + def cleanup_protected_state(ctx: InstallContext) -> None: """Tear down protected-mode mounts and LUKS mapper after a failed install. @@ -2267,6 +2282,7 @@ def cleanup_protected_state(ctx: InstallContext) -> None: if not ctx.is_protected: return + _swapoff_under(ctx.target) subprocess.run(["umount", "-R", str(ctx.target)], check=False, capture_output=True) if Path("/dev/mapper/omarchy_root").exists(): subprocess.run( From e4ecf47c50f22b59a1d1ed5dacfd45cb7a21e017 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:13:40 +0200 Subject: [PATCH 38/67] One release-the-target implementation, in bash, for both callers The swapoff-then-umount dance existed twice: bash in the dashboard's pre-reboot release, Python in the protected-failure cleanup. It is now one script, omarchy-release-install-target, carrying the explanation of the invisible swap holder once; the dashboard shells out to it and the orchestrator subprocesses it. Idempotent, exits nonzero only when the tree is still mounted after umount -R -- the dashboard logs that, the cleanup ignores it as before. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 14 +++----- .../local/bin/omarchy-release-install-target | 32 +++++++++++++++++++ .../omarchy-iso/orchestrator/phases_impl.py | 23 ++++--------- configs/profiledef.sh | 1 + 4 files changed, 43 insertions(+), 27 deletions(-) create mode 100755 configs/airootfs/usr/local/bin/omarchy-release-install-target diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 883853a5f..e14cc0e4f 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -585,17 +585,11 @@ render_finish() { # the other ttys, but one mount command away for whoever wants it. release_target() { - local target swapfile + local target target=$(jq -r '.target // "/mnt"' "$STATE_FILE" 2>/dev/null) || target=/mnt - sync - # The hibernation setup leaves the target's swapfile active, and swap is an - # invisible holder: no process shows in fuser, submounts unmount fine, and - # the btrfs still refuses to release until the swapoff. - while IFS= read -r swapfile; do - [[ $swapfile == "$target"/* ]] && swapoff "$swapfile" 2>/dev/null - done < <(awk 'NR > 1 { print $1 }' /proc/swaps) - umount -R "$target" 2>/dev/null || dashboard_log "release: umount -R $target failed" - sync + # Swapoff-then-umount, shared with the orchestrator's failure cleanup. + omarchy-release-install-target "$target" 2>/dev/null || + dashboard_log "release: could not release $target" } warm_reboot_path() { diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target new file mode 100755 index 000000000..bc21abd2f --- /dev/null +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -0,0 +1,32 @@ +#!/bin/bash +# +# Release the install target: deactivate every swap area under it, then +# recursively unmount it. The one source of truth for letting go of the +# target — the dashboard runs it before offering the install medium's +# removal, and the orchestrator's protected-failure cleanup runs it before +# closing the LUKS mapper. +# +# The swapoff comes first because active swap is an invisible holder: the +# hibernation setup's swapon in the chroot lands in the shared kernel, fuser +# then shows no process and the submounts unmount fine, yet the filesystem +# reports busy until the swap is off. +# +# Idempotent: a target with nothing mounted exits 0. Exits nonzero only when +# the tree is (still) mounted and umount -R failed. + +set -uo pipefail + +target="${1:?usage: omarchy-release-install-target }" + +sync + +while IFS=' ' read -r swap_path _; do + [[ $swap_path == "$target"/* ]] && swapoff "$swap_path" 2>/dev/null +done < <(tail -n +2 /proc/swaps 2>/dev/null) + +if mountpoint -q "$target"; then + umount -R "$target" || exit 1 +fi + +sync +exit 0 diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 7b3ff89e9..edb3b864e 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -2258,21 +2258,6 @@ def cleanup_target_hook_masks(ctx: InstallContext) -> None: _unmask_mkinitcpio_pacman_hooks(ctx, ctx.target, TARGET_DEFERRED_BOOT_HOOKS) -def _swapoff_under(target: Path) -> None: - """Deactivate every swap area on the target. The hibernation setup's - swapon in the chroot lands in the shared kernel, and active swap is an - invisible mount holder: fuser shows no process, yet umount -R reports - busy until the swapoff.""" - try: - entries = Path("/proc/swaps").read_text().splitlines()[1:] - except OSError: - return - for entry in entries: - swap_path = entry.split()[0] - if swap_path.startswith(f"{target}/"): - subprocess.run(["swapoff", swap_path], check=False, capture_output=True) - - def cleanup_protected_state(ctx: InstallContext) -> None: """Tear down protected-mode mounts and LUKS mapper after a failed install. @@ -2282,8 +2267,12 @@ def cleanup_protected_state(ctx: InstallContext) -> None: if not ctx.is_protected: return - _swapoff_under(ctx.target) - subprocess.run(["umount", "-R", str(ctx.target)], check=False, capture_output=True) + # Swapoff-then-umount; shared with the dashboard's pre-reboot release. + subprocess.run( + ["omarchy-release-install-target", str(ctx.target)], + check=False, + capture_output=True, + ) if Path("/dev/mapper/omarchy_root").exists(): subprocess.run( ["cryptsetup", "close", "omarchy_root"], diff --git a/configs/profiledef.sh b/configs/profiledef.sh index 1c9bf18b6..69567f9d7 100644 --- a/configs/profiledef.sh +++ b/configs/profiledef.sh @@ -41,6 +41,7 @@ file_permissions=( ["/usr/local/bin/omarchy-cidata-load"]="0:0:755" ["/usr/local/bin/omarchy-wait-root-image-verify"]="0:0:755" ["/usr/local/bin/omarchy-iso-cleanup-disk"]="0:0:755" + ["/usr/local/bin/omarchy-release-install-target"]="0:0:755" ["/usr/local/bin/omarchy-install-dashboard"]="0:0:755" ["/usr/local/bin/omarchy-install-diagnose-media"]="0:0:755" ["/usr/local/bin/omarchy-iso-install"]="0:0:755" From 544691af96c1a46ef50bfa3ed7375e09fb1d6dff Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:20:59 +0200 Subject: [PATCH 39/67] Close the LUKS mappers in the target release too The release script now reads the mappers off the mount table before the unmount empties it and closes them after: an encrypted install's Reboot Now screen no longer leaves /dev/mapper/root open, and the protected cleanup's mappers close through the shared path. cryptsetup refuses anything not crypt or still in use, so the sweep cannot take down more than the target held. The cleanup keeps one specific close as fallback: a failure between luksOpen and mount leaves omarchy_root open with nothing in the mount table for the release to see. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../local/bin/omarchy-release-install-target | 25 ++++++++++++++----- .../omarchy-iso/orchestrator/phases_impl.py | 6 ++++- 2 files changed, 24 insertions(+), 7 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index bc21abd2f..b652bafbc 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -1,15 +1,18 @@ #!/bin/bash # -# Release the install target: deactivate every swap area under it, then -# recursively unmount it. The one source of truth for letting go of the -# target — the dashboard runs it before offering the install medium's -# removal, and the orchestrator's protected-failure cleanup runs it before -# closing the LUKS mapper. +# Release the install target: deactivate every swap area under it, +# recursively unmount it, and close the LUKS mappers that backed it. The one +# source of truth for letting go of the target — the dashboard runs it +# before offering the install medium's removal, and the orchestrator's +# protected-failure cleanup runs it before its own stale-mapper sweep. # # The swapoff comes first because active swap is an invisible holder: the # hibernation setup's swapon in the chroot lands in the shared kernel, fuser # then shows no process and the submounts unmount fine, yet the filesystem -# reports busy until the swap is off. +# reports busy until the swap is off. The mappers to close are read off the +# mount table before the unmount empties it; cryptsetup itself refuses +# anything that is not crypt or still in use, so the close attempts cannot +# take down more than what the target held. # # Idempotent: a target with nothing mounted exits 0. Exits nonzero only when # the tree is (still) mounted and umount -R failed. @@ -24,9 +27,19 @@ while IFS=' ' read -r swap_path _; do [[ $swap_path == "$target"/* ]] && swapoff "$swap_path" 2>/dev/null done < <(tail -n +2 /proc/swaps 2>/dev/null) +mappers=() +while IFS= read -r source; do + source="${source%%\[*}" + [[ $source == /dev/mapper/* ]] && mappers+=("${source#/dev/mapper/}") +done < <(findmnt -rn -o SOURCE -R "$target" 2>/dev/null | sort -u) + if mountpoint -q "$target"; then umount -R "$target" || exit 1 fi +for mapper in "${mappers[@]}"; do + cryptsetup close "$mapper" 2>/dev/null +done + sync exit 0 diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index edb3b864e..aa890cc50 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -2267,12 +2267,16 @@ def cleanup_protected_state(ctx: InstallContext) -> None: if not ctx.is_protected: return - # Swapoff-then-umount; shared with the dashboard's pre-reboot release. + # Swapoff, umount -R, and close of the mappers the mounts were backed by; + # shared with the dashboard's pre-reboot release. subprocess.run( ["omarchy-release-install-target", str(ctx.target)], check=False, capture_output=True, ) + # The release only closes mappers it saw mounted: a failure between + # luksOpen and mount leaves omarchy_root open with nothing in the mount + # table, so this specific close stays as the fallback. if Path("/dev/mapper/omarchy_root").exists(): subprocess.run( ["cryptsetup", "close", "omarchy_root"], From 2fc43220140c9a2f63fb08ec7d5ecbed248155ad Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:32:26 +0200 Subject: [PATCH 40/67] Fold the stale-mapper fallback into the release script The protected cleanup kept its own cryptsetup close in Python next to the script that closes mappers -- the duplication Jonny flagged. The script now takes extra mapper names for what the mount-table sweep cannot see (a failure between luksOpen and mount leaves the mapper open with nothing mounted from it), and the cleanup just names omarchy_root. A mapper both named and seen mounted gets a second close attempt that fails quietly; closing is now written exactly once. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-release-install-target | 9 +++++++-- .../share/omarchy-iso/orchestrator/phases_impl.py | 15 ++++----------- 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index b652bafbc..8db35a9dc 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -14,12 +14,18 @@ # anything that is not crypt or still in use, so the close attempts cannot # take down more than what the target held. # +# Extra arguments name mappers to close beyond what the mount table shows: +# a failure between luksOpen and mount leaves a mapper open with nothing +# mounted from it, which the table sweep cannot see. +# # Idempotent: a target with nothing mounted exits 0. Exits nonzero only when # the tree is (still) mounted and umount -R failed. set -uo pipefail -target="${1:?usage: omarchy-release-install-target }" +target="${1:?usage: omarchy-release-install-target [mapper ...]}" +shift +mappers=("$@") sync @@ -27,7 +33,6 @@ while IFS=' ' read -r swap_path _; do [[ $swap_path == "$target"/* ]] && swapoff "$swap_path" 2>/dev/null done < <(tail -n +2 /proc/swaps 2>/dev/null) -mappers=() while IFS= read -r source; do source="${source%%\[*}" [[ $source == /dev/mapper/* ]] && mappers+=("${source#/dev/mapper/}") diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index aa890cc50..c2e10bbbe 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -2268,18 +2268,11 @@ def cleanup_protected_state(ctx: InstallContext) -> None: return # Swapoff, umount -R, and close of the mappers the mounts were backed by; - # shared with the dashboard's pre-reboot release. + # shared with the dashboard's pre-reboot release. omarchy_root is named + # explicitly because a failure between luksOpen and mount leaves it open + # with nothing in the mount table for the release to see. subprocess.run( - ["omarchy-release-install-target", str(ctx.target)], + ["omarchy-release-install-target", str(ctx.target), "omarchy_root"], check=False, capture_output=True, ) - # The release only closes mappers it saw mounted: a failure between - # luksOpen and mount leaves omarchy_root open with nothing in the mount - # table, so this specific close stays as the fallback. - if Path("/dev/mapper/omarchy_root").exists(): - subprocess.run( - ["cryptsetup", "close", "omarchy_root"], - check=False, - capture_output=True, - ) From 54831acbf6631e38fe3cbb4ba440c36ed1a016b7 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:32:48 +0200 Subject: [PATCH 41/67] Drop a stale phrase: the cleanup's own sweep is gone Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- configs/airootfs/usr/local/bin/omarchy-release-install-target | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index 8db35a9dc..1def995b9 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -4,7 +4,7 @@ # recursively unmount it, and close the LUKS mappers that backed it. The one # source of truth for letting go of the target — the dashboard runs it # before offering the install medium's removal, and the orchestrator's -# protected-failure cleanup runs it before its own stale-mapper sweep. +# protected-failure cleanup runs it naming the mapper it may have opened. # # The swapoff comes first because active swap is an invisible holder: the # hibernation setup's swapon in the chroot lands in the shared kernel, fuser From ad4df0a83abe00f9fb44f0a27e3b8f6cc10e45aa Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:42:27 +0200 Subject: [PATCH 42/67] Unmount a mapper's stray mountpoints before closing it The release unmounted only the target tree, but the orchestrator stage-mounts the same filesystem under /run/omarchy-install during some phases; a mid-phase failure can leave one behind, and the close then fails silently on a device the target-tree unmount never saw. Each mapper's remaining mountpoints -- wherever they are -- now unmount right before its close. The rest of the candidates were checked and deliberately left out: pre-existing disk state (old swap, LVM, stale crypt) is omarchy-iso-cleanup-disk's job at install start, these installs create no LVM or loop devices, and a blanket losetup sweep would detach the airootfs loop under the live system. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../airootfs/usr/local/bin/omarchy-release-install-target | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index 1def995b9..d08f39372 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -42,7 +42,14 @@ if mountpoint -q "$target"; then umount -R "$target" || exit 1 fi +# A mapper can be mounted outside the target tree too: the orchestrator +# stage-mounts the same filesystem under /run/omarchy-install during some +# phases, and a mid-phase failure can leave one behind — the close would +# then fail silently on a device the target-tree unmount never saw. for mapper in "${mappers[@]}"; do + while IFS= read -r mountpoint; do + umount -R "$mountpoint" 2>/dev/null + done < <(findmnt -rn -o TARGET -S "/dev/mapper/$mapper" 2>/dev/null) cryptsetup close "$mapper" 2>/dev/null done From 70fb70488f059f626e1e13ccc3d6b9b781099de8 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Tue, 25 Aug 2026 21:47:50 +0200 Subject: [PATCH 43/67] Name the holders when the target release fails A failed release said nothing about why -- the active-swapfile holder took a live debugging session to find, and fuser was the tool that found it. The script now does that diagnosis itself on stderr: on a failed umount, the fuser holder table plus what is still mounted and what swap is active; on a mapper that will not close, its holders and dmsetup info. Both callers forward the output into the install log, which bug reports already carry. fuser over lsof because the live ISO ships psmisc and not lsof, and the holder table is what gets acted on. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_016C4YbdRMV55n8FxMxwu8Bn --- .../usr/local/bin/omarchy-install-dashboard | 12 +++++++--- .../local/bin/omarchy-release-install-target | 23 +++++++++++++++++-- .../omarchy-iso/orchestrator/phases_impl.py | 8 +++++-- 3 files changed, 36 insertions(+), 7 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index e14cc0e4f..408510ec1 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -585,11 +585,17 @@ render_finish() { # the other ttys, but one mount command away for whoever wants it. release_target() { - local target + local target out line target=$(jq -r '.target // "/mnt"' "$STATE_FILE" 2>/dev/null) || target=/mnt - # Swapoff-then-umount, shared with the orchestrator's failure cleanup. - omarchy-release-install-target "$target" 2>/dev/null || + # Swapoff, umount, mapper close — shared with the orchestrator's failure + # cleanup. On failure the script names the holders on stderr; forward that + # into the install log, where the diagnosis is worth more than a shrug. + if ! out=$(omarchy-release-install-target "$target" 2>&1); then dashboard_log "release: could not release $target" + fi + while IFS= read -r line; do + [[ -n $line ]] && dashboard_log "release: $line" + done <<<"${out:-}" } warm_reboot_path() { diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index d08f39372..bd73607c4 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -38,8 +38,21 @@ while IFS= read -r source; do [[ $source == /dev/mapper/* ]] && mappers+=("${source#/dev/mapper/}") done < <(findmnt -rn -o SOURCE -R "$target" 2>/dev/null | sort -u) +# On failure, name the holders on stderr: this is the diagnosis the swapfile +# hunt needed once, done by the machine. fuser over lsof because the live +# ISO ships psmisc and not lsof, and the holder table is what gets acted on. if mountpoint -q "$target"; then - umount -R "$target" || exit 1 + if ! umount -R "$target"; then + { + echo "could not unmount $target; holders:" + fuser -vm "$target" 2>&1 + echo "still mounted under it:" + findmnt -rn -R "$target" + echo "active swap:" + tail -n +2 /proc/swaps + } >&2 + exit 1 + fi fi # A mapper can be mounted outside the target tree too: the orchestrator @@ -50,7 +63,13 @@ for mapper in "${mappers[@]}"; do while IFS= read -r mountpoint; do umount -R "$mountpoint" 2>/dev/null done < <(findmnt -rn -o TARGET -S "/dev/mapper/$mapper" 2>/dev/null) - cryptsetup close "$mapper" 2>/dev/null + if ! cryptsetup close "$mapper" 2>/dev/null && [[ -e /dev/mapper/$mapper ]]; then + { + echo "could not close mapper $mapper; holders:" + fuser -vm "/dev/mapper/$mapper" 2>&1 + dmsetup info "$mapper" 2>&1 + } >&2 + fi done sync diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index c2e10bbbe..c1a63971f 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -2270,9 +2270,13 @@ def cleanup_protected_state(ctx: InstallContext) -> None: # Swapoff, umount -R, and close of the mappers the mounts were backed by; # shared with the dashboard's pre-reboot release. omarchy_root is named # explicitly because a failure between luksOpen and mount leaves it open - # with nothing in the mount table for the release to see. - subprocess.run( + # with nothing in the mount table for the release to see. On failure the + # script names the holders on stderr; surface that in the install log. + result = subprocess.run( ["omarchy-release-install-target", str(ctx.target), "omarchy_root"], check=False, capture_output=True, + text=True, ) + for line in (result.stderr or "").splitlines(): + info(f"release: {line}") From 1974a0b4975663ae05377e0d6f64769e432b3135 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 12:28:30 +0200 Subject: [PATCH 44/67] Drop cms_verify from the PXE entries: unsigned ISOs cannot pass it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The flag was inherited from releng, whose official ISOs are codesigned. This build never hands mkarchiso a certificate, so no .cms.sig exists beside the airootfs — and the archiso hook treats a missing signature under cms_verify=y as fatal, aborting every PXE boot into an emergency shell before the live system comes up. The boot-cmdline guard now refuses any non-comment cms_verify in the boot configs, since a releng config resync would quietly bring it back. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- configs/syslinux/archiso_pxe-linux.cfg | 9 +++++++-- test/unit/boot-cmdline-test.sh | 11 +++++++++++ 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/configs/syslinux/archiso_pxe-linux.cfg b/configs/syslinux/archiso_pxe-linux.cfg index b0a80ba3e..b0cc919f8 100644 --- a/configs/syslinux/archiso_pxe-linux.cfg +++ b/configs/syslinux/archiso_pxe-linux.cfg @@ -8,6 +8,11 @@ # airootfs (plus optional checksums) into a tmpfs and never mounts the image # tree, so the root image cannot exist on that path and an install from it # would always fail at the pre-flight gate. +# +# cms_verify is deliberately absent too: these ISOs are not codesigned +# (mkarchiso is never given a certificate), and with the flag set but no +# .cms.sig beside the airootfs the archiso hook aborts into an emergency +# shell — every PXE boot would die there. LABEL arch64_nbd TEXT HELP @@ -17,7 +22,7 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, NBD) LINUX ::/%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD ::/%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% archiso_nbd_srv=${pxeserver} copytoram=n cms_verify=y initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archisosearchuuid=%ARCHISO_UUID% archiso_nbd_srv=${pxeserver} copytoram=n initramfs_async=0 SYSAPPEND 3 LABEL arch64_nfs @@ -28,5 +33,5 @@ ENDTEXT MENU LABEL Omarchy install medium (x86_64, NFS) LINUX ::/%INSTALL_DIR%/boot/x86_64/vmlinuz-linux-t2 INITRD ::/%INSTALL_DIR%/boot/x86_64/initramfs-linux-t2.img -APPEND archisobasedir=%INSTALL_DIR% archiso_nfs_srv=${pxeserver}:/run/archiso/bootmnt copytoram=n cms_verify=y initramfs_async=0 +APPEND archisobasedir=%INSTALL_DIR% archiso_nfs_srv=${pxeserver}:/run/archiso/bootmnt copytoram=n initramfs_async=0 SYSAPPEND 3 diff --git a/test/unit/boot-cmdline-test.sh b/test/unit/boot-cmdline-test.sh index 175357683..096330a80 100644 --- a/test/unit/boot-cmdline-test.sh +++ b/test/unit/boot-cmdline-test.sh @@ -17,6 +17,12 @@ # the cmdline says exactly n, and both keep the server's image tree mounted, # so pinned they can still stream the root image. HTTP cannot -- its hook only # downloads the airootfs into a tmpfs -- so no entry may use archiso_http_srv. +# +# No entry may carry cms_verify either: these ISOs are not codesigned +# (mkarchiso is never given a certificate), and the archiso hook aborts into +# an emergency shell when the flag is set with no .cms.sig on the medium. The +# releng profile the build seeds from ships it on the PXE entries, so a +# config resync would quietly reintroduce it. set -euo pipefail @@ -35,6 +41,11 @@ if grep -rq archiso_http_srv configs/syslinux configs/grub configs/efiboot; then exit 1 fi +if grep -rhE '^[^#]*cms_verify' configs/syslinux configs/grub configs/efiboot | grep -q .; then + echo "cms_verify found: the ISO is not codesigned, so any boot entry carrying it aborts into an emergency shell" + exit 1 +fi + fail=0 while IFS= read -r line; do if ! grep -qE '(^|[[:space:]])copytoram=n([[:space:]]|$)' <<<"$line"; then From c65a7be27030a4d9f4b90ac9705ae0a5e8ebf249 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 12:28:30 +0200 Subject: [PATCH 45/67] Add a PXE integration scenario: netboot, NBD medium, full install MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The PXE entries had no coverage at all: nothing ever booted the ISO's netboot path, and the pinned copytoram=n's whole claim — that an NBD boot keeps the root image streamable — was argued from the hooks' source, not demonstrated. The scenario fakes the LAN inside QEMU's user-mode networking: slirp's built-in TFTP server serves the ISO's own /boot/syslinux and /arch/boot trees to iPXE (plus the one-line pxelinux.cfg/default every real deployment writes), and qemu-nbd exports the ISO as 'archiso' on the host loopback, which is where the guest's 10.0.2.2 — and therefore the hook's ${pxeserver} — lands. No install medium is attached: the VM netboots, the initramfs dials the export, and the standard cidata autoinstall runs to the installed system, proving the verify gate and the root-image stream both work across the network. BIOS-only by nature (the PXE config is syslinux; the ISO has no UEFI netboot path), so the scenario forces SeaBIOS itself. The NBD port is fixed at 10809 because archiso_nbd_srv carries an address only. The host-side gate on the NBD dial (ss on the export port) covers the slow TFTP transfer of kernel+initramfs, which shows nothing OCR-able. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- test/integration.d/base-test.sh | 10 +- test/integration.d/pxe-test.sh | 209 ++++++++++++++++++++++++++++++++ 2 files changed, 217 insertions(+), 2 deletions(-) create mode 100644 test/integration.d/pxe-test.sh diff --git a/test/integration.d/base-test.sh b/test/integration.d/base-test.sh index 1c310ea51..5f012f136 100644 --- a/test/integration.d/base-test.sh +++ b/test/integration.d/base-test.sh @@ -44,6 +44,12 @@ CIDATA_IMG="$BASE_DIR/cidata.img" HTTP_PORT=$((SSH_PORT + 1)) HTTP_PID="" +# Scenario extensions to the network stack, appended verbatim (leading comma +# included) to the user netdev and the NIC device: the PXE scenario adds the +# built-in TFTP server to the former and a boot order to the latter. +NETDEV_EXTRA="" +NIC_EXTRA="" + mkdir -p "$BASE_DIR" "$RUN_DIR" QMP_SOCK=$(mktemp -u "${TMPDIR:-/tmp}/omarchy-integration-qmp.XXXXXX.sock") @@ -190,8 +196,8 @@ start_vm() { -device virtio-vga \ -display none \ -usb -device usb-tablet \ - -netdev user,id=net0,hostfwd=tcp:127.0.0.1:$SSH_PORT-:22 \ - -device virtio-net-pci,netdev=net0 \ + -netdev "user,id=net0,hostfwd=tcp:127.0.0.1:$SSH_PORT-:22$NETDEV_EXTRA" \ + -device "virtio-net-pci,netdev=net0$NIC_EXTRA" \ -qmp "unix:$QMP_SOCK,server,nowait" \ -serial "file:$serial" \ -pidfile "$PIDFILE" \ diff --git a/test/integration.d/pxe-test.sh b/test/integration.d/pxe-test.sh new file mode 100644 index 000000000..3ce979d06 --- /dev/null +++ b/test/integration.d/pxe-test.sh @@ -0,0 +1,209 @@ +#!/bin/bash +# +# The ISO's PXE path really boots and really installs: PXELINUX routed into +# the PXE menu (whichsys.c32) brings up a live environment whose medium is an +# NBD export, the pre-flight gate hashes the multi-GB root image across the +# network, and the unattended cidata install streams it to disk — the LAN +# install the pinned copytoram=n on the PXE entries exists to keep working. +# +# Mechanics: no boot medium is attached to the VM. QEMU's user-mode network +# stack plays the LAN — its built-in TFTP server hands iPXE lpxelinux.0 and +# the boot files straight out of the ISO's own /boot/syslinux and /arch/boot +# trees, and qemu-nbd serves the ISO itself as the export the initramfs dials +# back to (the guest reaches the host at 10.0.2.2, which is also what the +# archiso_pxe_common hook resolves ${pxeserver} to from the DHCP handshake). +# The one file the ISO does not carry is pxelinux.cfg/default — PXELINUX's +# config entry point, which every real deployment writes too; ours is a +# one-line INCLUDE of the ISO's own syslinux.cfg. +# +# Two fixed conditions, both inherent to the boot path under test: +# - BIOS only: the PXE config is syslinux and the ISO has no UEFI netboot +# path, so the scenario forces SeaBIOS regardless of the runner's flag. +# - Port 10809: archiso_nbd_srv carries an address only (${pxeserver}), so +# nbd-client always dials the well-known NBD port; qemu-nbd must own it +# on the host loopback for the duration of the run. +# +# Boots the ISO itself (over the wire), so it needs no base image. + +export OMARCHY_INTEGRATION_FIRMWARE=bios + +source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" + +NBD_PORT=10809 +NBD_PID="" +TFTP_ROOT="$RUN_DIR/tftp" +STATE=/run/omarchy-install/state.json + +pxe_cleanup() { + local status=$? + [[ -n $NBD_PID ]] && kill "$NBD_PID" 2>/dev/null || true + cleanup + return $status +} +trap pxe_cleanup EXIT + +# ------------------------------------------------------------------ fixture + +prepare_netboot() { + log "Assembling the TFTP root from the ISO's own netboot files" + rm -rf "$TFTP_ROOT" + mkdir -p "$TFTP_ROOT" + # Only the boot trees; the airootfs and the root image stay on the ISO and + # travel over NBD. The ISO's read-only modes would block the glue file below. + bsdtar -xf "$ISO" -C "$TFTP_ROOT" boot arch/boot + chmod -R u+w "$TFTP_ROOT" + + # PXELINUX looks for its configuration at pxelinux.cfg/default next to + # lpxelinux.0. Route it into the ISO's own config: its whichsys.c32 line + # detects PXELINUX and switches to the PXE menu. + mkdir -p "$TFTP_ROOT/boot/syslinux/pxelinux.cfg" + echo "INCLUDE syslinux.cfg" >"$TFTP_ROOT/boot/syslinux/pxelinux.cfg/default" +} + +start_nbd() { + log "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" + qemu-nbd --read-only --persistent --export-name=archiso \ + --bind=127.0.0.1 --port="$NBD_PORT" "$ISO" & + NBD_PID=$! + + sleep 1 + if ! kill -0 "$NBD_PID" 2>/dev/null; then + NBD_PID="" + echo "qemu-nbd could not serve 127.0.0.1:$NBD_PORT (the port is fixed by nbd-client); is it already taken?" >&2 + return 1 + fi +} + +nbd_dialed() { + [[ -n $(ss -Htn "sport = :$NBD_PORT") ]] +} + +# -------------------------------------------------------------------- phases + +netboot_into_installer() { + [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-integration" -f "$SSH_KEY" + detect_packages + build_cidata + + qemu-img create -f qcow2 "$RUN_DIR/disk.qcow2" 40G >/dev/null + + log "Netbooting with no attached install medium (headless)" + NETDEV_EXTRA=",tftp=$TFTP_ROOT,bootfile=/boot/syslinux/lpxelinux.0" + NIC_EXTRA=",bootindex=2" + start_vm "$RUN_DIR/disk.qcow2" "$RUN_DIR/serial.log" \ + -drive "file=$CIDATA_IMG,format=raw,if=none,id=cidata" \ + -device usb-storage,drive=cidata + + # Unlike the ISO's boot menu, the PXE menu has no auto-boot timeout, so + # commit the highlighted default — the NBD entry leads the list. + wait_for_screen "NBD" 300 + capture_console "success-pxe-01-menu" + press ret + + # The kernel and initramfs now cross the TFTP link, which is slow, and the + # boot shows no OCR-able text until much later. The first hard evidence the + # initramfs is up is it dialing the NBD export — gate on that from the host + # side before driving the console. + log "Waiting for the initramfs to dial the NBD export" + local waited=0 + until nbd_dialed; do + if ! vm_running; then + echo "VM exited before connecting to the NBD export" >&2 + return 1 + fi + if ((waited >= 900)); then + capture_console "failure-nbd-connect-timeout" + echo "Timed out waiting for the guest to dial the NBD export" >&2 + return 1 + fi + sleep 5 + ((waited += 5)) + done + log "NBD connection established" + + bootstrap_live_root_ssh +} + +assert_nbd_medium() { + # The NBD hook hands /dev/nbd0 to the mount handler, but the entry's + # archisosearchuuid makes the archiso hook re-resolve by UUID — and on an + # isohybrid image that lands on the partition (nbd0p1), the same way a USB + # stick boots from sdX1. Either is the NBD transport. + check "live medium is NBD-backed" \ + ssh_live_root "findmnt -no SOURCE /run/archiso/bootmnt | grep -qxE '/dev/nbd0(p[0-9]+)?'" + check "kernel cmdline pinned copytoram=n" \ + ssh_live_root "grep -q 'copytoram=n' /proc/cmdline" + check "root image stream is reachable on the netboot medium" \ + ssh_live_root "test -f /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs" +} + +wait_for_netboot_install() { + log "Waiting for the unattended install to finish (timeout ${INSTALL_TIMEOUT}s)" + + local waited=0 text progress_name + while true; do + # An unattended install reboots on its own; the guest user's SSH + # answering means the installed system is up (the live environment has no + # such user, so no false positive from the installer phase). + if ssh_guest true 2>/dev/null; then + log "Install finished and rebooted into the installed system." + capture_console "success-pxe-03-first-boot" + return 0 + fi + + text=$(ocr_screen) + + if grep -qi "Reboot Now" <<<"$text"; then + log "Install finished. Confirming the reboot prompt." + capture_console "success-pxe-02-reboot" + press ret + fi + + if grep -qi "installation stopped" <<<"$text"; then + capture_console "failure-install-stopped" + ssh_live_root "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true + ssh_live_root "cat $STATE" >"$RUN_DIR/state.json" 2>/dev/null || true + echo "Install failed — logs and screenshot saved to $RUN_DIR" >&2 + return 1 + fi + + if ! vm_running; then + echo "VM exited during install" >&2 + return 1 + fi + + if ((waited >= INSTALL_TIMEOUT)); then + capture_console "failure-install-timeout" + echo "Timed out after ${INSTALL_TIMEOUT}s waiting for install" >&2 + return 1 + fi + + if ((waited % 120 == 0)); then + printf -v progress_name 'success-pxe-install-progress-%04ds' "$waited" + capture_console "$progress_name" + echo " ... installing (${waited}s)" + fi + + sleep 10 + ((waited += 10)) + done +} + +assert_installed_system() { + check "installed system is reachable (not the live medium)" \ + ssh_guest "test ! -e /run/archiso" + check "booted in legacy BIOS mode" \ + ssh_guest "test ! -d /sys/firmware/efi" + + ssh_sudo "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true +} + +# ---------------------------------------------------------------------- main + +prepare_netboot +start_nbd +netboot_into_installer +assert_nbd_medium +wait_for_netboot_install +assert_installed_system +finish From 10bdf4238b17d351b270d75ad5d522636c79e90a Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 13:55:50 +0200 Subject: [PATCH 46/67] Add omarchy-iso-boot-pxe: hands-on netboot of an ISO in a visible VM MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The interactive twin of test/integration.d/pxe-test.sh's rig: slirp's TFTP server hands iPXE the ISO's own boot trees, qemu-nbd serves the ISO as the export the initramfs dials, and the VM has a display and your keyboard instead of OCR and QMP keystrokes — no install medium attached. Same conventions as omarchy-iso-boot (gum picker, --reuse, --ssh-port, -- passthrough); SeaBIOS only, since BIOS is the ISO's netboot path. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- bin/omarchy-iso-boot-pxe | 155 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100755 bin/omarchy-iso-boot-pxe diff --git a/bin/omarchy-iso-boot-pxe b/bin/omarchy-iso-boot-pxe new file mode 100755 index 000000000..43e0e7745 --- /dev/null +++ b/bin/omarchy-iso-boot-pxe @@ -0,0 +1,155 @@ +#!/bin/bash +# +# Boot an Omarchy ISO over PXE in a visible QEMU VM: real display, real +# keyboard, no install medium attached. The LAN is faked the same way +# test/integration.d/pxe-test.sh does it — QEMU's built-in TFTP server hands +# iPXE lpxelinux.0 and the boot files out of the ISO's own trees, and +# qemu-nbd serves the ISO as the NBD export the initramfs dials back to (the +# guest reaches the host at 10.0.2.2). BIOS/SeaBIOS only: that is the ISO's +# netboot path. Drive the install by hand; after "Reboot Now" the VM boots +# the installed disk (the NBD server outlives the reboot). +# +# The PXE menu has no auto-boot timeout — press Enter on the NBD entry. + +set -e + +ISO="" +REUSE="" +SSH_PORT="${OMARCHY_VM_SSH_PORT:-2222}" +WORK="${OMARCHY_VM_PXE_DIR:-/tmp/omarchy-pxe-boot}" +NBD_PORT=10809 # not configurable: archiso_nbd_srv carries an address only +EXTRA_QEMU_ARGS=() + +usage() { + cat < 0)); do + case "$1" in + --reuse | reuse) + REUSE="reuse" + shift + ;; + --ssh-port) + SSH_PORT="${2:-}" + shift 2 + ;; + --work) + WORK="${2:-}" + shift 2 + ;; + -h | --help) + usage + exit 0 + ;; + --) + shift + EXTRA_QEMU_ARGS+=("$@") + break + ;; + *) + if [[ -z $ISO ]]; then + ISO="$1" + else + echo "Unexpected argument: $1" >&2 + usage >&2 + exit 1 + fi + shift + ;; + esac +done + +if [[ -z $ISO && -t 0 ]]; then + ISO=$(ls -t release/*.iso 2>/dev/null | gum choose --header "Select ISO to netboot") + if gum confirm --default=false "Reuse existing disk?"; then + REUSE="reuse" + fi +fi + +if [[ ! -f ${ISO:-} ]]; then + usage >&2 + exit 1 +fi +ISO=$(realpath "$ISO") + +omarchy-pkg-add qemu-full + +# QEMU needs the desktop session for its SDL window (same dance as +# omarchy-iso-boot). +if [[ -z ${WAYLAND_DISPLAY:-} && -z ${DISPLAY:-} ]]; then + while IFS='=' read -r name value; do + case "$name" in + WAYLAND_DISPLAY | DISPLAY | XAUTHORITY | XDG_SESSION_TYPE) + export "$name=$value" + ;; + esac + done < <(systemctl --user show-environment 2>/dev/null || true) +fi + +if [[ -z ${WAYLAND_DISPLAY:-} && -z ${DISPLAY:-} ]]; then + echo "Unable to find a graphical session for QEMU." >&2 + echo "Run omarchy-iso-boot-pxe from the Omarchy desktop session." >&2 + exit 1 +fi + +TFTP_ROOT="$WORK/tftp" +DISK="$WORK/disk.qcow2" +mkdir -p "$WORK" + +# TFTP root: the ISO's own boot trees plus the one-line PXELINUX config +# entry point every real deployment writes. Rebuilt when the ISO changes. +if [[ ! -f "$TFTP_ROOT/.from-iso" || $(cat "$TFTP_ROOT/.from-iso") != "$ISO" ]]; then + echo "Assembling the TFTP root from $ISO" + rm -rf "$TFTP_ROOT" + mkdir -p "$TFTP_ROOT" + bsdtar -xf "$ISO" -C "$TFTP_ROOT" boot arch/boot + chmod -R u+w "$TFTP_ROOT" + mkdir -p "$TFTP_ROOT/boot/syslinux/pxelinux.cfg" + echo "INCLUDE syslinux.cfg" >"$TFTP_ROOT/boot/syslinux/pxelinux.cfg/default" + echo "$ISO" >"$TFTP_ROOT/.from-iso" +fi + +if [[ -n $(ss -Htln "sport = :$NBD_PORT") ]]; then + echo "Port $NBD_PORT is already taken; stop whatever serves it first." >&2 + exit 1 +fi + +echo "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" +qemu-nbd --read-only --persistent --export-name=archiso \ + --bind=127.0.0.1 --port="$NBD_PORT" "$ISO" & +NBD_PID=$! +trap 'kill $NBD_PID 2>/dev/null || true' EXIT + +if [[ ! -f $DISK || $REUSE != "reuse" ]]; then + qemu-img create -f qcow2 "$DISK" 40G +fi + +echo "SSH forwarding enabled: ssh -p $SSH_PORT @localhost" + +# SeaBIOS, no pflash: BIOS is the ISO's only netboot path. The empty disk is +# first in the boot order so the post-install reboot lands on it; SeaBIOS +# falls through to the NIC while the disk has no boot sector. +qemu-system-x86_64 \ + -cpu host -enable-kvm -machine q35,accel=kvm \ + -smp "$(($(nproc) < 8 ? $(nproc) : 8))" \ + -m 8192 \ + -drive file="$DISK",format=qcow2,if=none,id=drive0 \ + -device virtio-blk-pci,drive=drive0,bootindex=1 \ + -device virtio-vga-gl \ + -display sdl,gl=on \ + -usb -device usb-tablet \ + -serial none \ + -netdev "user,id=net0,hostfwd=tcp:127.0.0.1:$SSH_PORT-:22,tftp=$TFTP_ROOT,bootfile=/boot/syslinux/lpxelinux.0" \ + -device virtio-net-pci,netdev=net0,bootindex=2 \ + -boot menu=on \ + "${EXTRA_QEMU_ARGS[@]}" From bc14beb57d4a02f28bfb898d8fd0cb1993ba774e Mon Sep 17 00:00:00 2001 From: Omabot Date: Wed, 26 Aug 2026 05:44:40 -0700 Subject: [PATCH 47/67] Warm ttfx and gum too, not just systemctl The removal note is drawn inside render_finish, and render_finish is not finished when it appears: the laser effect execs ttfx for up to eight seconds afterwards, and reboot_prompt then execs gum to paint the button. Both live on the squashfs the live root pages from, so both are cold reads the note has already invited the user to take away. Warming only systemctl covered the press and not the two execs in front of it. The configurator runs ttfx and gum earlier in the same boot, so on an interactive install their pages are usually still resident -- usually, on a machine that has just written several GB through the page cache. This makes the invariant the comment claims hold by construction instead of by luck. Co-Authored-By: Claude Opus 5 (1M context) Co-Authored-By: Codex XHigh --- .../usr/local/bin/omarchy-install-dashboard | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 408510ec1..66785af14 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -578,7 +578,7 @@ render_finish() { # opposite of that: its stop jobs page unit files and ExecStop binaries. So, # before the finish screen offers the medium's removal: release the target — # fully written to its own disk from that moment, whatever happens after — -# and pre-fault the one binary the button press still runs. The medium is +# and pre-fault the binaries the finish screen still runs. The medium is # read-only and the live logs are tmpfs, so past the umount nothing anywhere # is dirty, which is what makes the immediate reboot at the press clean. # Debugging keeps its access: the installed tree is no longer pre-mounted on @@ -599,12 +599,17 @@ release_target() { } warm_reboot_path() { - local path dep - path=$(command -v systemctl 2>/dev/null) || return 0 - cat "$path" >/dev/null 2>&1 || true - while IFS= read -r dep; do - [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 - done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') + local binary path dep + # ttfx and gum, not just systemctl: the note goes up inside render_finish, + # so the logo effect (up to 8s of it) and the button gum paints are both + # still ahead of the moment the user is invited to pull the stick. + for binary in systemctl ttfx gum; do + path=$(command -v "$binary" 2>/dev/null) || continue + cat "$path" >/dev/null 2>&1 || true + while IFS= read -r dep; do + [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 + done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') + done } reboot_prompt() { From e10840f6aa2eb44b15b5fbc271ff484e01eae15d Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 15:09:06 +0200 Subject: [PATCH 48/67] Close only crypt mappers: bound the sweep by dm UUID, not by trust MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comment rested the unconditional closes on cryptsetup refusing anything that is not crypt. It refuses no such thing — round 5 removed a dm-linear device with an LVM- UUID through cryptsetup close, exit 0. No installer path puts a foreign mapper under the target today, so nothing bites; but the sweep should be bounded by a check, not a claim. Each candidate's device-mapper UUID must now carry the CRYPT- prefix before the close is attempted. A missing mapper falls out of the same gate, keeping the no-op run silent. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- .../usr/local/bin/omarchy-release-install-target | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index bd73607c4..4c5334011 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -10,9 +10,11 @@ # hibernation setup's swapon in the chroot lands in the shared kernel, fuser # then shows no process and the submounts unmount fine, yet the filesystem # reports busy until the swap is off. The mappers to close are read off the -# mount table before the unmount empties it; cryptsetup itself refuses -# anything that is not crypt or still in use, so the close attempts cannot -# take down more than what the target held. +# mount table before the unmount empties it, and each one's device-mapper +# UUID must carry the CRYPT- prefix before the close is attempted — +# cryptsetup close happily removes a non-crypt mapping (LVM, linear), so the +# sweep is bounded by that check, not by trust in a refusal. A busy crypt +# mapper still fails its close, and the failure is reported. # # Extra arguments name mappers to close beyond what the mount table shows: # a failure between luksOpen and mount leaves a mapper open with nothing @@ -63,6 +65,12 @@ for mapper in "${mappers[@]}"; do while IFS= read -r mountpoint; do umount -R "$mountpoint" 2>/dev/null done < <(findmnt -rn -o TARGET -S "/dev/mapper/$mapper" 2>/dev/null) + # Crypt mappings only — a foreign mapper under the target (or a stale name + # passed as an argument) is not ours to tear down, and cryptsetup would + # remove it without complaint. A missing mapper falls out here too, which + # keeps the no-op run silent the way idempotency wants. + uuid=$(dmsetup info -c --noheadings -o uuid "$mapper" 2>/dev/null) + [[ $uuid == CRYPT-* ]] || continue if ! cryptsetup close "$mapper" 2>/dev/null && [[ -e /dev/mapper/$mapper ]]; then { echo "could not close mapper $mapper; holders:" From 938f238c218b1765972a7beab6acf449a42fceb2 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 15:09:06 +0200 Subject: [PATCH 49/67] Rebuild the netboot TFTP cache when the ISO changes, not its name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cache marker held the ISO's pathname, so rebuilding to the same release/*.iso name kept serving the previous build's kernel and initramfs over TFTP while qemu-nbd exported the new image — a netboot of stale boot files against the wrong medium (round 5, Codex's find). The marker now carries size+mtime alongside the path. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- bin/omarchy-iso-boot-pxe | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/bin/omarchy-iso-boot-pxe b/bin/omarchy-iso-boot-pxe index 43e0e7745..fd064f731 100755 --- a/bin/omarchy-iso-boot-pxe +++ b/bin/omarchy-iso-boot-pxe @@ -107,8 +107,12 @@ DISK="$WORK/disk.qcow2" mkdir -p "$WORK" # TFTP root: the ISO's own boot trees plus the one-line PXELINUX config -# entry point every real deployment writes. Rebuilt when the ISO changes. -if [[ ! -f "$TFTP_ROOT/.from-iso" || $(cat "$TFTP_ROOT/.from-iso") != "$ISO" ]]; then +# entry point every real deployment writes. Rebuilt when the ISO changes — +# keyed on size+mtime, not the pathname, or rebuilding to the same +# release/*.iso name would netboot the previous build's kernel and +# initramfs against the new build's NBD medium. +ISO_STAMP="$ISO $(stat -c '%s %Y' "$ISO")" +if [[ ! -f "$TFTP_ROOT/.from-iso" || $(cat "$TFTP_ROOT/.from-iso") != "$ISO_STAMP" ]]; then echo "Assembling the TFTP root from $ISO" rm -rf "$TFTP_ROOT" mkdir -p "$TFTP_ROOT" @@ -116,7 +120,7 @@ if [[ ! -f "$TFTP_ROOT/.from-iso" || $(cat "$TFTP_ROOT/.from-iso") != "$ISO" ]]; chmod -R u+w "$TFTP_ROOT" mkdir -p "$TFTP_ROOT/boot/syslinux/pxelinux.cfg" echo "INCLUDE syslinux.cfg" >"$TFTP_ROOT/boot/syslinux/pxelinux.cfg/default" - echo "$ISO" >"$TFTP_ROOT/.from-iso" + echo "$ISO_STAMP" >"$TFTP_ROOT/.from-iso" fi if [[ -n $(ss -Htln "sport = :$NBD_PORT") ]]; then From f82c6d0cf4af3dfa979750957f04e246b7005296 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 15:09:06 +0200 Subject: [PATCH 50/67] Round-5 tidyups: credit a real function, normalize pxe-test's mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The removal note's comment credited release_target_and_reboot, which stopped existing when the release moved ahead of the finish screen — release_target and warm_reboot_path are what make the line true now. pxe-test.sh gets the 755 its sibling scenarios have (the runner invokes via bash, so this is consistency, not repair). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- configs/airootfs/usr/local/bin/omarchy-install-dashboard | 4 ++-- test/integration.d/pxe-test.sh | 0 2 files changed, 2 insertions(+), 2 deletions(-) mode change 100644 => 100755 test/integration.d/pxe-test.sh diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 66785af14..c5efe9ebc 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -542,8 +542,8 @@ render_finish() { blank_line # Above the button, not below: gum clears everything under its view on # each repaint, so a line below the prompt would be erased. True because - # of warm_reboot_path + release_target_and_reboot: from here to the - # reboot syscall nothing reads the install medium again. + # of release_target + warm_reboot_path: from here to the reboot syscall + # nothing reads the install medium again. center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" } >"$TTY_PATH" diff --git a/test/integration.d/pxe-test.sh b/test/integration.d/pxe-test.sh old mode 100644 new mode 100755 From 6394e38d9cf49a0d1770062357857ccb8ad65942 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 15:12:17 +0200 Subject: [PATCH 51/67] Withhold the removal offer when the release fails; reboot gracefully MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 5's open item: release_target logged a failure and the finish screen promised medium removal anyway, with -ff then skipping the unmount a graceful shutdown would have done. The screen now keeps its promise honest — a failed release swaps the note for 'Leave the install medium in until the reboot completes' and the button press falls back to a graceful systemctl reboot. That fallback is safe precisely because the offer was withheld: the medium is still present for the stop jobs' cold reads (the #125 wedge needs a pulled stick), and systemd gets its own chance at the unmount the release could not do. The layout is untouched: one dim line swaps for another inside the same FINISH_HEIGHT budget. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- .../usr/local/bin/omarchy-install-dashboard | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index c5efe9ebc..0113e826c 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -543,8 +543,14 @@ render_finish() { # Above the button, not below: gum clears everything under its view on # each repaint, so a line below the prompt would be erased. True because # of release_target + warm_reboot_path: from here to the reboot syscall - # nothing reads the install medium again. - center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" + # nothing reads the install medium again — but only when the release + # actually happened, so a failed one withdraws the offer instead of + # printing a promise the machine is not keeping. + if [[ $TARGET_RELEASED == yes ]]; then + center "${DIM}You can now remove the install medium${RESET}" "$CONTENT_WIDTH" + else + center "${DIM}Leave the install medium in until the reboot completes${RESET}" "$CONTENT_WIDTH" + fi } >"$TTY_PATH" if [[ -f $LOGO_PATH ]]; then @@ -583,6 +589,14 @@ render_finish() { # is dirty, which is what makes the immediate reboot at the press clean. # Debugging keeps its access: the installed tree is no longer pre-mounted on # the other ttys, but one mount command away for whoever wants it. +# +# If the release fails, the offer is withheld: the finish screen says to +# leave the medium in instead, and the button press falls back to a graceful +# reboot — the medium was never offered for removal, so its stop jobs can +# still page from it, and systemd gets its own chance at the unmount the +# release could not do. + +TARGET_RELEASED=yes release_target() { local target out line @@ -592,6 +606,7 @@ release_target() { # into the install log, where the diagnosis is worth more than a shrug. if ! out=$(omarchy-release-install-target "$target" 2>&1); then dashboard_log "release: could not release $target" + TARGET_RELEASED=no fi while IFS= read -r line; do [[ -n $line ]] && dashboard_log "release: $line" @@ -967,7 +982,14 @@ if reboot_prompt; then exit 0 fi printf '%s' "$CLEAR" >"$TTY_PATH" 2>/dev/null || true - systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true + if [[ $TARGET_RELEASED == yes ]]; then + systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true + else + # The target is still (partly) held, so let systemd unmount what the + # release could not: the finish screen told the user to leave the medium + # in, which keeps the graceful path's cold reads safe. + systemctl reboot 2>/dev/null || reboot 2>/dev/null || true + fi fi exit 0 From a253017897810fbabb5db7d49bac7894526e7ca2 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 16:06:00 +0200 Subject: [PATCH 52/67] A mapper that stays open fails the release, and the offer with it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Greptile's P1 on the PR: cryptsetup close failing on a busy mapper only logged the holders and the helper still exited 0, so the dashboard printed the removal note and took the -ff path with part of the target still active. Historically that was deliberate — an open-but-unmounted mapper holds nothing dirty, which is why the pre-close design shipped — but since the removal offer keys on this exit code, 'released' has to mean fully released. An unclosed crypt mapper now exits 1; the dashboard then withholds the offer and falls back to the graceful reboot, and the orchestrator's cleanup caller is unaffected (check=False, it only forwards stderr). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- .../usr/local/bin/omarchy-release-install-target | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index 4c5334011..dd5ef045b 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -20,8 +20,10 @@ # a failure between luksOpen and mount leaves a mapper open with nothing # mounted from it, which the table sweep cannot see. # -# Idempotent: a target with nothing mounted exits 0. Exits nonzero only when -# the tree is (still) mounted and umount -R failed. +# Idempotent: a target with nothing mounted and no mapper left open exits 0. +# Exits nonzero when the tree stayed mounted (umount -R failed) or a crypt +# mapper stayed open — either way the target is not fully let go, and the +# dashboard's removal offer and immediate-reboot choice key on exactly that. set -uo pipefail @@ -61,6 +63,7 @@ fi # stage-mounts the same filesystem under /run/omarchy-install during some # phases, and a mid-phase failure can leave one behind — the close would # then fail silently on a device the target-tree unmount never saw. +close_failed=0 for mapper in "${mappers[@]}"; do while IFS= read -r mountpoint; do umount -R "$mountpoint" 2>/dev/null @@ -77,8 +80,9 @@ for mapper in "${mappers[@]}"; do fuser -vm "/dev/mapper/$mapper" 2>&1 dmsetup info "$mapper" 2>&1 } >&2 + close_failed=1 fi done sync -exit 0 +exit $close_failed From 49995e84774f563cc2dace4ce8af9cf5420c3be0 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 16:35:43 +0200 Subject: [PATCH 53/67] Warm the finish window by dry-running it, measured to zero MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The invariant above the removal note was argued from ldd output; a probe measured it instead. Method: drive a real interactive install in QEMU, park at the finish screen by not answering the reboot prompt, then over root SSH drop the page cache (worse than any real memory pressure), re-run the warm, and replay each post-note step with the medium's sectors-read counter sampled around it. After the cat+ldd warm of systemctl/ttfx/gum, the window still cold- read ~1.4 MiB the moment the note was already up: the timeout+ttfx exec 880 KiB, stty (which term_cols runs to size the prompt) 320 KiB, gum's runtime 188 KiB — exec-time reads ldd cannot list. timeout and stty were missing from the warm entirely; and warming by cat alone cannot reach what only an exec faults in. So the warm now also dry-runs the real commands — the effect's frames to /dev/null, stty against the dashboard tty, gum --version — after which every step of the replayed window measures zero. systemctl needs no dry run: cat+ldd covers the reboot-shaped invocation (is-system-running reads nothing after it; the 1.1 MiB a 'status' proxy pulls is journal machinery reboot -ff never touches). Measurement notes for posterity: squashfs 1 MiB blocks make any small cold file cost hundreds of KiB, and a first probe wrongly blamed a ~2 MiB tty-gated read on ttfx — that was the probe's own script(1) wrapper execing root's zsh; ttfx is a single Rust binary and reads the same set either way. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- .../usr/local/bin/omarchy-install-dashboard | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 0113e826c..0c3850191 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -615,16 +615,28 @@ release_target() { warm_reboot_path() { local binary path dep - # ttfx and gum, not just systemctl: the note goes up inside render_finish, - # so the logo effect (up to 8s of it) and the button gum paints are both + # Everything the post-note window still execs: the note goes up inside + # render_finish, so the logo effect (up to 8s of it), the timeout in front + # of it, the stty that sizes the prompt, and the button gum paints are all # still ahead of the moment the user is invited to pull the stick. - for binary in systemctl ttfx gum; do + for binary in systemctl ttfx gum timeout stty; do path=$(command -v "$binary" 2>/dev/null) || continue cat "$path" >/dev/null 2>&1 || true while IFS= read -r dep; do [[ -f $dep ]] && cat "$dep" >/dev/null 2>&1 done < <(ldd "$path" 2>/dev/null | awk '{ for (i = 1; i <= NF; i++) if ($i ~ /^\//) print $i }') done + + # cat+ldd alone still leaves ~1.4 MiB of exec-time reads ldd cannot list + # (loader path, locale, runtime data) — measured on a dropped cache in the + # live VM: timeout+ttfx 880 KiB, stty 320 KiB, gum 188 KiB, each a + # one-time fault. Dry-running the real commands takes those reads now, + # while the medium is still welcome; the effect's frames go to /dev/null, + # so nothing shows. systemctl needs no dry run: cat+ldd covers the + # reboot-shaped invocation (verified via is-system-running: zero reads). + timeout 1s ttfx -i "$LOGO_PATH" --xterm-colors laseretch >/dev/null 2>&1 || true + stty size <"$TTY_PATH" >/dev/null 2>&1 || true + gum --version >/dev/null 2>&1 || true } reboot_prompt() { From 96a19e203da0a1ed8f295c664131458e619d45cd Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 16:47:43 +0200 Subject: [PATCH 54/67] Trim the warm's ttfx dry run to --version, and cat the logo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Jonny's call on the latency trade: --version returns in milliseconds where the effect dry run burned its full one-second cap, and it was measured to fault in the effect's read set to within ~12 KiB (a single first-frame fault). The logo file joins the warm as an explicit cat — render_finish reads it before the note anyway, but the warm should not depend on that ordering to stay true. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- .../airootfs/usr/local/bin/omarchy-install-dashboard | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index 0c3850191..fa449d89f 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -631,10 +631,13 @@ warm_reboot_path() { # (loader path, locale, runtime data) — measured on a dropped cache in the # live VM: timeout+ttfx 880 KiB, stty 320 KiB, gum 188 KiB, each a # one-time fault. Dry-running the real commands takes those reads now, - # while the medium is still welcome; the effect's frames go to /dev/null, - # so nothing shows. systemctl needs no dry run: cat+ldd covers the - # reboot-shaped invocation (verified via is-system-running: zero reads). - timeout 1s ttfx -i "$LOGO_PATH" --xterm-colors laseretch >/dev/null 2>&1 || true + # while the medium is still welcome. ttfx --version rather than a hidden + # effect run: measured to cover the effect's read set to within ~12 KiB, + # and it returns in milliseconds instead of burning a capped second. + # systemctl needs no dry run: cat+ldd covers the reboot-shaped invocation + # (verified via is-system-running: zero reads). + cat "$LOGO_PATH" >/dev/null 2>&1 || true + ttfx --version >/dev/null 2>&1 || true stty size <"$TTY_PATH" >/dev/null 2>&1 || true gum --version >/dev/null 2>&1 || true } From e9bff6440b7466c0d46171b4c0afa8227031886a Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 17:51:38 +0200 Subject: [PATCH 55/67] Share the netboot rig between the PXE scenario and its hands-on twin MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The TFTP-root assembly (with the pxelinux.cfg/default glue and the size+mtime cache stamp) and the qemu-nbd export existed twice, one copy in the scenario and one in omarchy-iso-boot-pxe, already starting to drift (the cache stamp lived only in the tool). Both now source bin/lib/netboot.sh, so what the test proves is exactly what the tool boots — by construction, not by discipline. The scenario re-verified green end to end through the shared rig (netboot → NBD → live → unattended install → installed system), incidentally the first PXE run of an ISO carrying the finish-window work. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- bin/lib/netboot.sh | 53 ++++++++++++++++++++++++++++++++++ bin/omarchy-iso-boot-pxe | 30 ++++--------------- test/integration.d/pxe-test.sh | 43 +++++---------------------- 3 files changed, 65 insertions(+), 61 deletions(-) create mode 100644 bin/lib/netboot.sh diff --git a/bin/lib/netboot.sh b/bin/lib/netboot.sh new file mode 100644 index 000000000..43b921d6c --- /dev/null +++ b/bin/lib/netboot.sh @@ -0,0 +1,53 @@ +# Shared netboot rig: the TFTP root and NBD export that fake a PXE LAN +# inside QEMU's user-mode networking. Sourced by test/integration.d/ +# pxe-test.sh (the scenario) and bin/omarchy-iso-boot-pxe (its hands-on +# twin) so the two can never drift apart — what the test proves is exactly +# what the tool boots. + +NBD_PORT=10809 # not configurable: archiso_nbd_srv carries an address only +NBD_PID="" + +# The ISO's own boot trees plus the one-line PXELINUX config entry point +# every real deployment writes: PXELINUX looks for pxelinux.cfg/default +# next to lpxelinux.0, and the INCLUDE routes it into the ISO's own config, +# whose whichsys.c32 line detects PXELINUX and switches to the PXE menu. +# Only the boot trees are extracted — the airootfs and the root image stay +# on the ISO and travel over NBD. Rebuilt when the ISO changes, keyed on +# size+mtime: a pathname key would keep serving a previous build's kernel +# and initramfs after a rebuild to the same release/*.iso name. +assemble_tftp_root() { + local iso="$1" root="$2" stamp + stamp="$iso $(stat -c '%s %Y' "$iso")" + if [[ -f "$root/.from-iso" && $(cat "$root/.from-iso") == "$stamp" ]]; then + return 0 + fi + rm -rf "$root" + mkdir -p "$root" + bsdtar -xf "$iso" -C "$root" boot arch/boot + # The ISO's read-only modes would block the glue file below. + chmod -R u+w "$root" + mkdir -p "$root/boot/syslinux/pxelinux.cfg" + echo "INCLUDE syslinux.cfg" >"$root/boot/syslinux/pxelinux.cfg/default" + echo "$stamp" >"$root/.from-iso" +} + +# Serve the ISO as the NBD export the initramfs dials back to: name +# 'archiso' on the well-known port, bound to the host loopback — which is +# where the guest's 10.0.2.2 (and therefore the hook's ${pxeserver}) lands. +# Sets NBD_PID; the caller's exit trap owns the kill. +serve_nbd() { + local iso="$1" + if [[ -n $(ss -Htln "sport = :$NBD_PORT") ]]; then + echo "Port $NBD_PORT is already taken (the port nbd-client dials is fixed); stop whatever serves it first." >&2 + return 1 + fi + qemu-nbd --read-only --persistent --export-name=archiso \ + --bind=127.0.0.1 --port="$NBD_PORT" "$iso" & + NBD_PID=$! + sleep 1 + if ! kill -0 "$NBD_PID" 2>/dev/null; then + NBD_PID="" + echo "qemu-nbd could not serve 127.0.0.1:$NBD_PORT." >&2 + return 1 + fi +} diff --git a/bin/omarchy-iso-boot-pxe b/bin/omarchy-iso-boot-pxe index fd064f731..884a6c10c 100755 --- a/bin/omarchy-iso-boot-pxe +++ b/bin/omarchy-iso-boot-pxe @@ -13,11 +13,12 @@ set -e +source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/lib/netboot.sh" + ISO="" REUSE="" SSH_PORT="${OMARCHY_VM_SSH_PORT:-2222}" WORK="${OMARCHY_VM_PXE_DIR:-/tmp/omarchy-pxe-boot}" -NBD_PORT=10809 # not configurable: archiso_nbd_srv carries an address only EXTRA_QEMU_ARGS=() usage() { @@ -106,32 +107,11 @@ TFTP_ROOT="$WORK/tftp" DISK="$WORK/disk.qcow2" mkdir -p "$WORK" -# TFTP root: the ISO's own boot trees plus the one-line PXELINUX config -# entry point every real deployment writes. Rebuilt when the ISO changes — -# keyed on size+mtime, not the pathname, or rebuilding to the same -# release/*.iso name would netboot the previous build's kernel and -# initramfs against the new build's NBD medium. -ISO_STAMP="$ISO $(stat -c '%s %Y' "$ISO")" -if [[ ! -f "$TFTP_ROOT/.from-iso" || $(cat "$TFTP_ROOT/.from-iso") != "$ISO_STAMP" ]]; then - echo "Assembling the TFTP root from $ISO" - rm -rf "$TFTP_ROOT" - mkdir -p "$TFTP_ROOT" - bsdtar -xf "$ISO" -C "$TFTP_ROOT" boot arch/boot - chmod -R u+w "$TFTP_ROOT" - mkdir -p "$TFTP_ROOT/boot/syslinux/pxelinux.cfg" - echo "INCLUDE syslinux.cfg" >"$TFTP_ROOT/boot/syslinux/pxelinux.cfg/default" - echo "$ISO_STAMP" >"$TFTP_ROOT/.from-iso" -fi - -if [[ -n $(ss -Htln "sport = :$NBD_PORT") ]]; then - echo "Port $NBD_PORT is already taken; stop whatever serves it first." >&2 - exit 1 -fi +echo "Assembling the TFTP root from $ISO (cached until the ISO changes)" +assemble_tftp_root "$ISO" "$TFTP_ROOT" echo "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" -qemu-nbd --read-only --persistent --export-name=archiso \ - --bind=127.0.0.1 --port="$NBD_PORT" "$ISO" & -NBD_PID=$! +serve_nbd "$ISO" trap 'kill $NBD_PID 2>/dev/null || true' EXIT if [[ ! -f $DISK || $REUSE != "reuse" ]]; then diff --git a/test/integration.d/pxe-test.sh b/test/integration.d/pxe-test.sh index 3ce979d06..792631aae 100755 --- a/test/integration.d/pxe-test.sh +++ b/test/integration.d/pxe-test.sh @@ -28,9 +28,10 @@ export OMARCHY_INTEGRATION_FIRMWARE=bios source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" +# The TFTP root and NBD export come from the same rig bin/omarchy-iso-boot-pxe +# uses, so what this scenario proves is exactly what the hands-on tool boots. +source "$ROOT/bin/lib/netboot.sh" -NBD_PORT=10809 -NBD_PID="" TFTP_ROOT="$RUN_DIR/tftp" STATE=/run/omarchy-install/state.json @@ -42,38 +43,6 @@ pxe_cleanup() { } trap pxe_cleanup EXIT -# ------------------------------------------------------------------ fixture - -prepare_netboot() { - log "Assembling the TFTP root from the ISO's own netboot files" - rm -rf "$TFTP_ROOT" - mkdir -p "$TFTP_ROOT" - # Only the boot trees; the airootfs and the root image stay on the ISO and - # travel over NBD. The ISO's read-only modes would block the glue file below. - bsdtar -xf "$ISO" -C "$TFTP_ROOT" boot arch/boot - chmod -R u+w "$TFTP_ROOT" - - # PXELINUX looks for its configuration at pxelinux.cfg/default next to - # lpxelinux.0. Route it into the ISO's own config: its whichsys.c32 line - # detects PXELINUX and switches to the PXE menu. - mkdir -p "$TFTP_ROOT/boot/syslinux/pxelinux.cfg" - echo "INCLUDE syslinux.cfg" >"$TFTP_ROOT/boot/syslinux/pxelinux.cfg/default" -} - -start_nbd() { - log "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" - qemu-nbd --read-only --persistent --export-name=archiso \ - --bind=127.0.0.1 --port="$NBD_PORT" "$ISO" & - NBD_PID=$! - - sleep 1 - if ! kill -0 "$NBD_PID" 2>/dev/null; then - NBD_PID="" - echo "qemu-nbd could not serve 127.0.0.1:$NBD_PORT (the port is fixed by nbd-client); is it already taken?" >&2 - return 1 - fi -} - nbd_dialed() { [[ -n $(ss -Htn "sport = :$NBD_PORT") ]] } @@ -200,8 +169,10 @@ assert_installed_system() { # ---------------------------------------------------------------------- main -prepare_netboot -start_nbd +log "Assembling the TFTP root from the ISO's own netboot files" +assemble_tftp_root "$ISO" "$TFTP_ROOT" +log "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" +serve_nbd "$ISO" netboot_into_installer assert_nbd_medium wait_for_netboot_install From 4cf2fa888405f4a2469d5bb946d8616cdbeaf3b1 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Wed, 26 Aug 2026 18:00:52 +0200 Subject: [PATCH 56/67] Share the unattended-install wait between the base install and PXE MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The scenario's wait_for_netboot_install was a near-verbatim copy of the loop inside install_phase — the same SSH-as-guest success signal, Reboot Now press, stopped-install branch, timeout and progress screenshots — differing only in artifact names. Both now use one wait_for_unattended_install with a name prefix, which also gives the base install the failure-time collection of the live system's install log and state the copy had grown (best-effort: root SSH may not be authorized at that point). Verified with one run exercising both callers: a fresh cidata base build, then the PXE scenario, both green. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01UT9vNJDEqYjJGLS9ARphwR --- test/integration.d/base-test.sh | 85 +++++++++++++++++++-------------- test/integration.d/pxe-test.sh | 55 +-------------------- 2 files changed, 50 insertions(+), 90 deletions(-) diff --git a/test/integration.d/base-test.sh b/test/integration.d/base-test.sh index 5f012f136..4a24b4065 100644 --- a/test/integration.d/base-test.sh +++ b/test/integration.d/base-test.sh @@ -610,55 +610,38 @@ detect_packages() { fi } -install_phase() { - log "Installing $(basename "$ISO") unattended via cidata (headless)" - - [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-integration" -f "$SSH_KEY" - detect_packages - build_cidata - - # Build under a staging name: the finished base is promoted only after a - # clean shutdown, so a failed install can never pass for a reusable base. - rm -f "$BASE_DISK" "$BASE_DISK.building" - qemu-img create -f qcow2 "$BASE_DISK.building" 40G >/dev/null - if [[ $FIRMWARE == uefi ]]; then - cp "$OVMF_VARS_TEMPLATE" "$BASE_OVMF" - ACTIVE_OVMF="$BASE_OVMF" - fi - - # Boot the ISO from an optical drive under both firmwares: OVMF and SeaBIOS - # both boot an ide-cd reliably, and the boot-medium type has no bearing on - # what the BIOS run exercises (Limine's MBR install and boot). SeaBIOS will - # not boot the isohybrid image off a fallback USB device here, so USB is not - # worth the flakiness. The copytoram-off / USB path is covered by unit tests. - start_vm "$BASE_DISK.building" "$RUN_DIR/install-serial.log" \ - -drive "file=$ISO,media=cdrom,if=none,format=raw,id=cdrom0" \ - -device ide-cd,drive=cdrom0,bootindex=2 \ - -drive "file=$CIDATA_IMG,format=raw,if=none,id=cidata" \ - -device usb-storage,drive=cidata +# Wait out an unattended cidata install to its reboot into the installed +# system: SSH answering as the guest user is the success signal (cidata's +# authorized_keys enables sshd there, and the live environment has no such +# user, so there is no false positive from the installer phase). Presses the +# Reboot Now prompt if one appears. Screenshot names take the given prefix so +# each caller's artifacts stay apart; on a stopped install the live system's +# log and state are saved best-effort (root SSH may not be authorized yet). +wait_for_unattended_install() { + local prefix="$1" + local waited=0 text progress_name log "Waiting for the unattended install to finish (timeout ${INSTALL_TIMEOUT}s)" - local waited=0 text progress_name while true; do - # An unattended install reboots on its own; SSH answering means the - # installed system is up (cidata's authorized_keys enables sshd). if ssh_guest true 2>/dev/null; then log "Install finished and rebooted into the installed system." - capture_console "success-install-first-boot" - break + capture_console "success-$prefix-first-boot" + return 0 fi text=$(ocr_screen) if grep -qi "Reboot Now" <<<"$text"; then log "Install finished. Confirming the reboot prompt." - capture_console "success-install-reboot" + capture_console "success-$prefix-reboot" press ret fi if grep -qi "installation stopped" <<<"$text"; then - capture_console "failure-install-stopped" - echo "Install failed — screenshot saved to $RUN_DIR" >&2 + capture_console "failure-$prefix-stopped" + ssh_live_root "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true + ssh_live_root "cat /run/omarchy-install/state.json" >"$RUN_DIR/state.json" 2>/dev/null || true + echo "Install failed — artifacts saved to $RUN_DIR" >&2 return 1 fi @@ -668,13 +651,13 @@ install_phase() { fi if ((waited >= INSTALL_TIMEOUT)); then - capture_console "failure-install-timeout" + capture_console "failure-$prefix-timeout" echo "Timed out after ${INSTALL_TIMEOUT}s waiting for install" >&2 return 1 fi if ((waited % 120 == 0)); then - printf -v progress_name 'success-install-progress-%04ds' "$waited" + printf -v progress_name 'success-%s-progress-%04ds' "$prefix" "$waited" capture_console "$progress_name" echo " ... installing (${waited}s)" fi @@ -682,6 +665,36 @@ install_phase() { sleep 10 ((waited += 10)) done +} + +install_phase() { + log "Installing $(basename "$ISO") unattended via cidata (headless)" + + [[ -f $SSH_KEY ]] || ssh-keygen -t ed25519 -N "" -q -C "omarchy-integration" -f "$SSH_KEY" + detect_packages + build_cidata + + # Build under a staging name: the finished base is promoted only after a + # clean shutdown, so a failed install can never pass for a reusable base. + rm -f "$BASE_DISK" "$BASE_DISK.building" + qemu-img create -f qcow2 "$BASE_DISK.building" 40G >/dev/null + if [[ $FIRMWARE == uefi ]]; then + cp "$OVMF_VARS_TEMPLATE" "$BASE_OVMF" + ACTIVE_OVMF="$BASE_OVMF" + fi + + # Boot the ISO from an optical drive under both firmwares: OVMF and SeaBIOS + # both boot an ide-cd reliably, and the boot-medium type has no bearing on + # what the BIOS run exercises (Limine's MBR install and boot). SeaBIOS will + # not boot the isohybrid image off a fallback USB device here, so USB is not + # worth the flakiness. The copytoram-off / USB path is covered by unit tests. + start_vm "$BASE_DISK.building" "$RUN_DIR/install-serial.log" \ + -drive "file=$ISO,media=cdrom,if=none,format=raw,id=cdrom0" \ + -device ide-cd,drive=cdrom0,bootindex=2 \ + -drive "file=$CIDATA_IMG,format=raw,if=none,id=cidata" \ + -device usb-storage,drive=cidata + + wait_for_unattended_install install || return 1 log "Installed system is up. Saving base image." stop_vm diff --git a/test/integration.d/pxe-test.sh b/test/integration.d/pxe-test.sh index 792631aae..af56baf26 100755 --- a/test/integration.d/pxe-test.sh +++ b/test/integration.d/pxe-test.sh @@ -33,7 +33,6 @@ source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" source "$ROOT/bin/lib/netboot.sh" TFTP_ROOT="$RUN_DIR/tftp" -STATE=/run/omarchy-install/state.json pxe_cleanup() { local status=$? @@ -106,58 +105,6 @@ assert_nbd_medium() { ssh_live_root "test -f /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs" } -wait_for_netboot_install() { - log "Waiting for the unattended install to finish (timeout ${INSTALL_TIMEOUT}s)" - - local waited=0 text progress_name - while true; do - # An unattended install reboots on its own; the guest user's SSH - # answering means the installed system is up (the live environment has no - # such user, so no false positive from the installer phase). - if ssh_guest true 2>/dev/null; then - log "Install finished and rebooted into the installed system." - capture_console "success-pxe-03-first-boot" - return 0 - fi - - text=$(ocr_screen) - - if grep -qi "Reboot Now" <<<"$text"; then - log "Install finished. Confirming the reboot prompt." - capture_console "success-pxe-02-reboot" - press ret - fi - - if grep -qi "installation stopped" <<<"$text"; then - capture_console "failure-install-stopped" - ssh_live_root "cat /var/log/omarchy-install.log" >"$RUN_DIR/omarchy-install.log" 2>/dev/null || true - ssh_live_root "cat $STATE" >"$RUN_DIR/state.json" 2>/dev/null || true - echo "Install failed — logs and screenshot saved to $RUN_DIR" >&2 - return 1 - fi - - if ! vm_running; then - echo "VM exited during install" >&2 - return 1 - fi - - if ((waited >= INSTALL_TIMEOUT)); then - capture_console "failure-install-timeout" - echo "Timed out after ${INSTALL_TIMEOUT}s waiting for install" >&2 - return 1 - fi - - if ((waited % 120 == 0)); then - printf -v progress_name 'success-pxe-install-progress-%04ds' "$waited" - capture_console "$progress_name" - echo " ... installing (${waited}s)" - fi - - sleep 10 - ((waited += 10)) - done -} - assert_installed_system() { check "installed system is reachable (not the live medium)" \ ssh_guest "test ! -e /run/archiso" @@ -175,6 +122,6 @@ log "Serving the ISO as NBD export 'archiso' on 127.0.0.1:$NBD_PORT" serve_nbd "$ISO" netboot_into_installer assert_nbd_medium -wait_for_netboot_install +wait_for_unattended_install pxe assert_installed_system finish From d7aba7b3c177bfdbddcb61449c80cc85a4230cf1 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Thu, 27 Aug 2026 16:39:33 +0200 Subject: [PATCH 57/67] A read error off the medium must not leave btrfs receive blocked MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit omarchybot's round-6 find. _receive_root_image closed stdin and reaped on its normal and BrokenPipeError paths only, so an OSError out of stream.read() — the EIO from a dying USB, which is the failure the whole verify machinery exists for — propagated with proc.stdin still open. btrfs receive then sits blocked on that pipe forever, which makes the caller's staging mount busy, and that umount at the end of _install_root_image is check=False: /run/omarchy-install/image-top would stay mounted on the target filesystem with nothing said about it, plus a process left in the live session. A finally that closes and waits covers every way out. The wait stays unbounded on purpose: past EOF btrfs receive is committing the subvolume, which on slow media is legitimately slow and must not be killed on a timer. _receive_root_image was mocked out in every test_root_image.py case, so it now has its own, driven against a real child on a real pipe (a shell draining stdin in place of btrfs receive, so the pipe blocks the way the real one does): the stream goes through and reports done, a failing receive raises with its error output, and a stream that dies mid-read still leaves the child closed and reaped. That last one hangs on the pre-fix code — confirmed by reverting the fix under it. Co-Authored-By: Claude Opus 5 (1M context) --- .../omarchy-iso/orchestrator/phases_impl.py | 27 ++++- test/unit/test_root_image.py | 99 +++++++++++++++++++ 2 files changed, 124 insertions(+), 2 deletions(-) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index c1a63971f..8893897c6 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -675,14 +675,37 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No if now - last_report >= 0.5: _write_phase_progress(ctx, sent / total if total else 1.0) last_report = now - proc.stdin.close() except BrokenPipeError: pass - if proc.wait() != 0: + finally: + # Whatever happened above — including the EIO off a dying medium that + # the whole verify machinery exists to catch — btrfs receive must not + # be left blocked on an open stdin. It would hold the caller's staging + # mount busy, and that umount is check=False: the mount would leak onto + # the target filesystem with nothing said about it. + code = _close_receive(proc) + if code != 0: raise RuntimeError(f"btrfs receive failed: {errors.read_text(errors='replace').strip()}") _write_phase_progress(ctx, 1.0) +def _close_receive(proc: subprocess.Popen) -> int: + """Close btrfs receive's stdin and reap it, returning its exit status. + + Never raises: it runs on the way out of a failing read, where the original + exception is the one worth keeping. Closing flushes, so a receive that has + already died answers with a BrokenPipeError of its own; the wait is + unbounded because a receive that has read EOF is committing the subvolume, + which on slow media is legitimately slow and must not be killed. + """ + if proc.stdin is not None: + try: + proc.stdin.close() + except OSError: + pass + return proc.wait() + + def _write_phase_progress(ctx: InstallContext, fraction: float) -> None: """Record how far the current phase has come (0..1) in the state file the dashboard polls. Best effort: progress display must never fail an install.""" diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index ef487b6e4..51dae5f4b 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -6,8 +6,12 @@ asserted on the subprocess calls against a temp target. """ +import errno +import json import os +import shlex import shutil +import subprocess import sys import tempfile import types @@ -462,5 +466,100 @@ def test_factory_snapshot_joins_the_unit_before_anything_else(self): self.assertEqual(self.runs, []) +class ReceiveRootImageTest(unittest.TestCase): + """The stream-into-btrfs-receive pump, against a real child on a real pipe. + + btrfs receive is stood in for by a shell that drains stdin, so the pipe + blocks the way the real one does and the child is only reaped when its + stdin is actually closed. + """ + + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + root = Path(self.tmp.name) + self.state_dir = root / "state" + self.state_dir.mkdir() + (self.state_dir / "state.json").write_text("{}") + self.top = root / "image-top" + self.received = root / "received" + self.ctx = types.SimpleNamespace(target=root / "mnt", state_dir=self.state_dir) + self.procs = [] + + def stand_in(self, script): + """Patch Popen so btrfs receive is this shell script instead, keeping + the caller's stdin pipe and stderr file.""" + + spawn = subprocess.Popen # before the patch below replaces it + + def popen(cmd, **kwargs): + self.assertEqual(cmd[:2], ["btrfs", "receive"]) + proc = spawn(["sh", "-c", script], **kwargs) + self.procs.append(proc) + return proc + + patch = mock.patch.object(phases_impl.subprocess, "Popen", side_effect=popen) + patch.start() + self.addCleanup(patch.stop) + + def drains_stdin(self): + self.stand_in(f"cat > {shlex.quote(str(self.received))}") + + def stream_file(self, data): + path = Path(self.tmp.name) / "omarchy-root.btrfs" + path.write_bytes(data) + return path + + def test_streams_the_file_through_and_reports_done(self): + self.drains_stdin() + phases_impl._receive_root_image(self.ctx, self.top, self.stream_file(b"x" * (9 << 20))) + + self.assertEqual(self.received.read_bytes(), b"x" * (9 << 20)) + self.assertEqual(json.loads((self.state_dir / "state.json").read_text())["phase_progress"], 1.0) + + def test_a_failed_receive_raises_with_its_error_output(self): + self.stand_in("cat > /dev/null; echo 'ERROR: chunk failed' >&2; exit 1") + with self.assertRaisesRegex(RuntimeError, "chunk failed"): + phases_impl._receive_root_image(self.ctx, self.top, self.stream_file(b"stream")) + + def test_a_read_error_off_the_medium_still_closes_and_reaps_the_receive(self): + # EIO out of a dying stick is the failure the verify machinery exists + # for. If stdin were left open, btrfs receive would sit blocked on it + # and hold the staging mount busy for the caller's silent umount. + self.drains_stdin() + with self.assertRaises(OSError): + phases_impl._receive_root_image(self.ctx, self.top, DyingStream(b"y" * (9 << 20))) + + proc = self.procs[0] + self.assertEqual(proc.wait(timeout=10), 0) + self.assertEqual(self.received.read_bytes(), b"y" * (8 << 20)) + + +class DyingStream: + """A stream path whose reads give one chunk and then fail like a medium + that has gone away mid-install.""" + + def __init__(self, data): + self.data = data + + def stat(self): + return types.SimpleNamespace(st_size=len(self.data)) + + def open(self, mode): + return self + + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + def read(self, size): + if self.data: + chunk, self.data = self.data[:size], b"" + return chunk + raise OSError(errno.EIO, "Input/output error") + + if __name__ == "__main__": unittest.main() From a8bdcafd220d69e6e333e941eb0406e2bcbc5e40 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Thu, 27 Aug 2026 17:37:11 +0200 Subject: [PATCH 58/67] Cover the release sweep, and let device-mapper judge its own closes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit omarchybot measured the gap: with `exit $close_failed` mutated to `exit 0`, ./test/all stayed green — the script whose exit status decides whether the user is told to pull the stick had no test at all. It has one now, driving every branch against stubbed swapoff/findmnt/umount/dmsetup/cryptsetup: the swap hunt and its ordering ahead of the unmount, a tree that stays mounted, a mapper that stays open, the CRYPT- gate against an LVM mapping and against a mapper that is already gone, mappers named as arguments, and the stage mount outside the target tree. The mutation above now fails it. Making the close's outcome observable turned up a real edge in the check. `[[ -e /dev/mapper/$mapper ]]` asks udev, which lags in both directions: the node can outlive a good close for a moment, which would withhold the removal offer on a machine that is fine (the false negative omarchybot went looking for and could not produce), and it is never created at all when device-mapper runs with udev disabled, which would hide a mapper that really did stay open. dmsetup info asks the kernel and is right either way — and it is what the failure report on the very next line already calls. Reviewing that change turned up the next edge: dmsetup answers rc 1 both for a mapper that is provably gone and for not being able to answer at all, so a transient dm failure would have slipped an open crypt mapper past the close with the release still exiting 0 — fail open, in the one direction this script must not. The two answers differ only in the message ("Device does not exist." is the driver's own word for ENXIO; the unreachable-control-node shape was reproduced live), so mapper_uuid now tells them apart and an unanswerable query fails the release, at the gate and at the post-close probe alike. Both new branches are mutation-checked, as are three test hardenings from the same review: ordering assertions that FAIL instead of aborting the suite under set -e when a call goes missing, a dmsetup stub that speaks the real messages and matches the -c flag positionally rather than by a glob a mapper name could satisfy, and an EXIT trap over the sandboxes. Co-Authored-By: Claude Opus 5 (1M context) Co-Authored-By: Claude Fable 5 --- .../local/bin/omarchy-release-install-target | 54 +++- test/unit/release-install-target-test.sh | 255 ++++++++++++++++++ 2 files changed, 300 insertions(+), 9 deletions(-) create mode 100755 test/unit/release-install-target-test.sh diff --git a/configs/airootfs/usr/local/bin/omarchy-release-install-target b/configs/airootfs/usr/local/bin/omarchy-release-install-target index dd5ef045b..d9a251650 100755 --- a/configs/airootfs/usr/local/bin/omarchy-release-install-target +++ b/configs/airootfs/usr/local/bin/omarchy-release-install-target @@ -27,6 +27,28 @@ set -uo pipefail +# device-mapper's word on a mapper: prints its UUID and returns 0 when it +# exists, 1 when it provably does not ("Device does not exist." is the +# driver's own answer for ENXIO), and 2 when dmsetup could not answer at all +# (control node unreachable, driver not responding) — the same rc 1 as a +# missing mapper, told apart only by the message. The distinction matters +# because 1 is the idempotent no-op and 2 must never pass for it: a mapper +# that cannot be proven gone has to fail the release, and this exit status +# is what stands between the user and pulling the medium. +dm_err=$(mktemp) +trap 'rm -f "$dm_err"' EXIT +mapper_uuid() { + local uuid + if uuid=$(dmsetup info -c --noheadings -o uuid "$1" 2>"$dm_err"); then + printf '%s\n' "$uuid" + return 0 + fi + grep -q "Device does not exist" "$dm_err" && return 1 + echo "could not query mapper $1:" >&2 + cat "$dm_err" >&2 + return 2 +} + target="${1:?usage: omarchy-release-install-target [mapper ...]}" shift mappers=("$@") @@ -71,16 +93,30 @@ for mapper in "${mappers[@]}"; do # Crypt mappings only — a foreign mapper under the target (or a stale name # passed as an argument) is not ours to tear down, and cryptsetup would # remove it without complaint. A missing mapper falls out here too, which - # keeps the no-op run silent the way idempotency wants. - uuid=$(dmsetup info -c --noheadings -o uuid "$mapper" 2>/dev/null) + # keeps the no-op run silent the way idempotency wants — but only on the + # driver's word for missing: an unanswerable query fails the release. + uuid=$(mapper_uuid "$mapper") + case $? in + 1) continue ;; + 2) close_failed=1; continue ;; + esac [[ $uuid == CRYPT-* ]] || continue - if ! cryptsetup close "$mapper" 2>/dev/null && [[ -e /dev/mapper/$mapper ]]; then - { - echo "could not close mapper $mapper; holders:" - fuser -vm "/dev/mapper/$mapper" 2>&1 - dmsetup info "$mapper" 2>&1 - } >&2 - close_failed=1 + # dmsetup, not the /dev/mapper node, decides whether the close took: the + # node is udev's and lags in both directions — still there for a moment + # after a good close (which would withhold the offer on a machine that is + # fine), and never there at all when device-mapper runs with udev disabled + # (which would hide a mapper that really did stay open). Only its provable + # absence counts as closed; an unanswerable probe fails like a held one. + if ! cryptsetup close "$mapper" 2>/dev/null; then + mapper_uuid "$mapper" >/dev/null + if (( $? != 1 )); then + { + echo "could not close mapper $mapper; holders:" + fuser -vm "/dev/mapper/$mapper" 2>&1 + dmsetup info "$mapper" 2>&1 + } >&2 + close_failed=1 + fi fi done diff --git a/test/unit/release-install-target-test.sh b/test/unit/release-install-target-test.sh new file mode 100755 index 000000000..75c6223bb --- /dev/null +++ b/test/unit/release-install-target-test.sh @@ -0,0 +1,255 @@ +#!/bin/bash +# +# omarchy-release-install-target is the one source of truth for letting go of +# the install target, and its exit status is load-bearing: the dashboard's +# removal offer and its immediate `reboot -ff` both key on it, so a sweep that +# quietly gives up must not report success. This drives the script with +# stubbed swapoff/findmnt/umount/dmsetup/cryptsetup on PATH, so every branch — +# the swap hunt, the crypt gate, the failure exits — is checked without a disk. + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +SCRIPT="$ROOT/configs/airootfs/usr/local/bin/omarchy-release-install-target" + +fails=0 +check() { # desc, expected_rc, actual_rc, [needle in output], [output] + local desc=$1 want=$2 got=$3 needle=${4:-} out=${5:-} + if [[ $got != "$want" ]]; then + echo "FAIL: $desc (rc want=$want got=$got)"; fails=1; return + fi + if [[ -n $needle && $out != *"$needle"* ]]; then + echo "FAIL: $desc (missing '$needle' in: $out)"; fails=1; return + fi + echo "ok: $desc" +} + +check_absent() { # desc, needle, haystack + local desc=$1 needle=$2 hay=$3 + if [[ $hay == *"$needle"* ]]; then + echo "FAIL: $desc (unexpected '$needle' in: $hay)"; fails=1; return + fi + echo "ok: $desc" +} + +call_order() { # desc, earlier-call-pattern, later-call-pattern + # First match of each pattern in $CALLS, by line. A missing call is its own + # failure, not a script abort: the bare grep pipelines here would otherwise + # kill the whole test under set -euo pipefail exactly when they had + # something to say. + local desc=$1 first second + first=$(grep -n "$2" <<<"$CALLS" | head -1 | cut -d: -f1 || true) + second=$(grep -n "$3" <<<"$CALLS" | head -1 | cut -d: -f1 || true) + if [[ -z $first || -z $second ]]; then + echo "FAIL: $desc (missing call: '$2' -> ${first:-none}, '$3' -> ${second:-none})" + fails=1; return + fi + if (( first >= second )); then + echo "FAIL: $desc ('$2' at line $first, '$3' at line $second)"; fails=1; return + fi + echo "ok: $desc" +} + +# A sandbox whose stub tools record every call to $BOX/calls and answer from +# fixture files, so a case is set up by writing those files. The stubs model +# the kernel's view rather than a script's: a successful `cryptsetup close` +# drops the mapper from the table dmsetup then reports on, which is what makes +# the close's success or failure observable the way the script observes it. +BOXES=() +trap '(( ${#BOXES[@]} == 0 )) || rm -rf "${BOXES[@]}"' EXIT + +new_box() { # -> sets BOX to a fresh sandbox with default (nothing to do) state + BOX=$(mktemp -d) + BOXES+=("$BOX") + export BOX + mkdir -p "$BOX/bin" + : >"$BOX/calls" + : >"$BOX/swaps" # /proc/swaps body, header already stripped + : >"$BOX/sources" # findmnt -o SOURCE -R + : >"$BOX/mapper_mounts" # findmnt -o TARGET -S /dev/mapper/ + : >"$BOX/uuids" # " " per line; absent = no such mapper + echo 1 >"$BOX/mounted_rc" + echo 0 >"$BOX/umount_rc" + echo 0 >"$BOX/close_rc" + + cat >"$BOX/bin/sync" <<'EOF' +#!/bin/bash +echo "sync" >>"$BOX/calls" +EOF + cat >"$BOX/bin/tail" <<'EOF' +#!/bin/bash +# Only /proc/swaps is faked; anything else goes to the real tail. +if [[ ${*: -1} == /proc/swaps ]]; then cat "$BOX/swaps"; exit 0; fi +exec /usr/bin/tail "$@" +EOF + cat >"$BOX/bin/swapoff" <<'EOF' +#!/bin/bash +echo "swapoff $*" >>"$BOX/calls" +EOF + cat >"$BOX/bin/findmnt" <<'EOF' +#!/bin/bash +echo "findmnt $*" >>"$BOX/calls" +case "$*" in + *"-o SOURCE"*) cat "$BOX/sources" ;; + *"-o TARGET"*) cat "$BOX/mapper_mounts" ;; + *) echo "stub findmnt tree" ;; # the failure diagnosis +esac +EOF + cat >"$BOX/bin/mountpoint" <<'EOF' +#!/bin/bash +exit "$(cat "$BOX/mounted_rc")" +EOF + cat >"$BOX/bin/umount" <<'EOF' +#!/bin/bash +echo "umount $*" >>"$BOX/calls" +exit "$(cat "$BOX/umount_rc")" +EOF + cat >"$BOX/bin/dmsetup" <<'EOF' +#!/bin/bash +echo "dmsetup $*" >>"$BOX/calls" +# Broken mode: dmsetup cannot answer at all — the message shape a real one +# prints when the control node is unreachable (reproduced non-root), which +# the script must keep apart from a mapper that is provably gone. +if [[ -e $BOX/dmsetup_broken ]] || + { [[ -e $BOX/dmsetup_broken_after ]] && + (( $(grep -c '^dmsetup' "$BOX/calls") > $(cat "$BOX/dmsetup_broken_after") )); }; then + echo "/dev/mapper/control: open failed: Permission denied" >&2 + echo "Command failed." >&2 + exit 1 +fi +mapper="${*: -1}" +uuid=$(awk -v m="$mapper" '$1 == m { print $2 }' "$BOX/uuids") +[[ -n $uuid ]] || { echo "Device does not exist." >&2; exit 1; } +# `info -c -o uuid` is the gate's query; a bare `info` is the failure +# report. The flag is matched positionally: a glob over $* would also hit a +# mapper name containing '-c'. +if [[ ${2:-} == -c ]]; then echo "$uuid"; else echo "Name: $mapper"; fi +EOF + cat >"$BOX/bin/cryptsetup" <<'EOF' +#!/bin/bash +echo "cryptsetup $*" >>"$BOX/calls" +rc=$(cat "$BOX/close_rc") +if (( rc == 0 )); then + awk -v m="${*: -1}" '$1 != m' "$BOX/uuids" >"$BOX/uuids.new" + mv "$BOX/uuids.new" "$BOX/uuids" +fi +exit "$rc" +EOF + cat >"$BOX/bin/fuser" <<'EOF' +#!/bin/bash +echo " USER PID ACCESS COMMAND" +EOF + chmod +x "$BOX"/bin/* + TARGET="$BOX/mnt" + mkdir -p "$TARGET" +} + +run_release() { # extra mapper args -> sets RC, OUT (stderr), CALLS + set +e + OUT=$(PATH="$BOX/bin:$PATH" bash "$SCRIPT" "$TARGET" "$@" 2>&1 >/dev/null) + RC=$? + set -e + CALLS=$(cat "$BOX/calls") +} + +# ── An idempotent no-op: nothing mounted, no swap, no mapper ───────────────── +new_box +run_release +check "an untouched target releases silently" 0 "$RC" +check_absent "no unmount is attempted with nothing mounted" "umount" "$CALLS" +check_absent "no mapper is closed with nothing open" "cryptsetup" "$CALLS" + +# ── The ordinary release: swap off, tree unmounted, crypt mapper closed ────── +new_box +echo 0 >"$BOX/mounted_rc" +printf '%s file 4194300 0 -2\n/dev/zram0 partition 8388604 0 100\n' "$TARGET/swap/swapfile" >"$BOX/swaps" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +run_release +check "a mounted encrypted target is fully released" 0 "$RC" +check "the target's swapfile is deactivated" 0 0 "swapoff $TARGET/swap/swapfile" "$CALLS" +check_absent "swap outside the target is left alone" "/dev/zram0" "$CALLS" +check "the tree is unmounted" 0 0 "umount -R $TARGET" "$CALLS" +check "the mapper off the mount table is closed" 0 0 "cryptsetup close omarchy_root" "$CALLS" +# Swap is an invisible holder: with it still on, the unmount reports busy. +call_order "swapoff comes before the unmount" '^swapoff' '^umount' + +# ── A tree that stays mounted fails the release ────────────────────────────── +new_box +echo 0 >"$BOX/mounted_rc" +echo 1 >"$BOX/umount_rc" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +run_release +check "a failed unmount fails the release" 1 "$RC" "could not unmount $TARGET" "$OUT" +check "the holders are named on the way out" 0 0 "ACCESS COMMAND" "$OUT" +check_absent "a target still mounted is not swept for mappers" "cryptsetup" "$CALLS" + +# ── A mapper that stays open fails the release too ─────────────────────────── +new_box +echo 0 >"$BOX/mounted_rc" +echo 1 >"$BOX/close_rc" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +run_release +check "a mapper that stays open fails the release" 1 "$RC" "could not close mapper omarchy_root" "$OUT" +check "the unmount still happened" 0 0 "umount -R $TARGET" "$CALLS" + +# ── The crypt gate: cryptsetup would happily remove what is not ours ───────── +new_box +echo 0 >"$BOX/mounted_rc" +echo "/dev/mapper/vg0-data[/]" >"$BOX/sources" +echo "vg0-data LVM-9f2cIkjHqf0e" >"$BOX/uuids" +run_release +check "a non-crypt mapper is left alone" 0 "$RC" +check_absent "cryptsetup is never called on an LVM mapping" "cryptsetup" "$CALLS" + +new_box +echo 0 >"$BOX/mounted_rc" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" # no uuids entry: gone already +run_release +check "a mapper that is not there falls out silently" 0 "$RC" +check_absent "no close is attempted on a missing mapper" "cryptsetup" "$CALLS" + +# ── A dmsetup that cannot answer must not pass for "already gone" ──────────── +new_box +echo 0 >"$BOX/mounted_rc" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +touch "$BOX/dmsetup_broken" +run_release +check "an unanswerable mapper query fails the release" 1 "$RC" \ + "could not query mapper omarchy_root" "$OUT" +check_absent "no close is attempted on an unproven mapper" "cryptsetup" "$CALLS" + +# ── dmsetup dying between the close and its probe is still a failed close ──── +new_box +echo 0 >"$BOX/mounted_rc" +echo 1 >"$BOX/close_rc" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +echo 1 >"$BOX/dmsetup_broken_after" # the gate's query answers; every later call breaks +run_release +check "an unanswerable post-close probe fails the release" 1 "$RC" \ + "could not close mapper omarchy_root" "$OUT" + +# ── Mappers named as arguments: the open-but-never-mounted window ──────────── +new_box +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +run_release omarchy_root +check "a named mapper is closed with nothing mounted" 0 "$RC" "cryptsetup close omarchy_root" "$CALLS" + +# ── A mapper mounted outside the target tree ───────────────────────────────── +new_box +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +echo "/run/omarchy-install/image-top" >"$BOX/mapper_mounts" +run_release omarchy_root +check "a stage mount outside the target is unmounted" 0 "$RC" \ + "umount -R /run/omarchy-install/image-top" "$CALLS" +call_order "the stage mount goes before the close" '^umount -R /run' '^cryptsetup close' + +if (( fails )); then + echo "omarchy-release-install-target: FAILED" + exit 1 +fi +echo "ok: omarchy-release-install-target sweeps and reports" From f41d074f688d8452e48ddaced18fc742c17aecbf Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Thu, 27 Aug 2026 17:37:11 +0200 Subject: [PATCH 59/67] Cover the two things the release's verdict decides MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The other half of omarchybot's coverage note: TARGET_RELEASED=yes could be mutated to =no with ./test/all still green, though it decides both what the finish screen promises about the medium and whether the button does the medium-free `reboot -ff` or the graceful reboot that still pages off it. Telling the user the stick may come out while the release actually failed is how a machine gets its medium pulled mid-read, so the verdict is now tested instead of trusted. The reboot choice lived inline in the driver, so it moves into reboot_now() beside release_target and warm_reboot_path — the same verdict, now callable. The test sources the dashboard up to its driver (everything above is definitions) and exercises release_target, render_finish and reboot_now against a stubbed release helper: a clean sweep leaves the target released and invites the medium out, a failed one withdraws the offer, logs the helper's holder diagnosis, and keeps the graceful reboot. Both mutations fail it. The tty is a fifo rather than a file: the dashboard reopens it with > for every write, which a console shrugs off and a regular file answers by truncating away everything drawn before. jq is stubbed with the rest of the tools — just enough of it for the target query — so the verdict the test reports is the dashboard's, not the host package set's. Co-Authored-By: Claude Opus 5 (1M context) Co-Authored-By: Claude Fable 5 --- .../usr/local/bin/omarchy-install-dashboard | 23 ++- test/unit/dashboard-release-gate-test.sh | 157 ++++++++++++++++++ 2 files changed, 172 insertions(+), 8 deletions(-) create mode 100755 test/unit/dashboard-release-gate-test.sh diff --git a/configs/airootfs/usr/local/bin/omarchy-install-dashboard b/configs/airootfs/usr/local/bin/omarchy-install-dashboard index fa449d89f..47544346b 100755 --- a/configs/airootfs/usr/local/bin/omarchy-install-dashboard +++ b/configs/airootfs/usr/local/bin/omarchy-install-dashboard @@ -642,6 +642,20 @@ warm_reboot_path() { gum --version >/dev/null 2>&1 || true } +reboot_now() { + # Which reboot the button gets is the other half of the release's verdict. + # Released: nothing anywhere is dirty and the medium may already be gone, + # so skip the stop jobs that would page it back in. Not released: the + # finish screen told the user to leave the medium in, which keeps the + # graceful path's cold reads safe, and systemd gets its own chance at the + # unmount the release could not do. + if [[ $TARGET_RELEASED == yes ]]; then + systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true + else + systemctl reboot 2>/dev/null || reboot 2>/dev/null || true + fi +} + reboot_prompt() { local prompt_pad @@ -997,14 +1011,7 @@ if reboot_prompt; then exit 0 fi printf '%s' "$CLEAR" >"$TTY_PATH" 2>/dev/null || true - if [[ $TARGET_RELEASED == yes ]]; then - systemctl reboot -ff 2>/dev/null || reboot -f 2>/dev/null || true - else - # The target is still (partly) held, so let systemd unmount what the - # release could not: the finish screen told the user to leave the medium - # in, which keeps the graceful path's cold reads safe. - systemctl reboot 2>/dev/null || reboot 2>/dev/null || true - fi + reboot_now fi exit 0 diff --git a/test/unit/dashboard-release-gate-test.sh b/test/unit/dashboard-release-gate-test.sh new file mode 100755 index 000000000..700bc63c0 --- /dev/null +++ b/test/unit/dashboard-release-gate-test.sh @@ -0,0 +1,157 @@ +#!/bin/bash +# +# The dashboard asks omarchy-release-install-target to let go of the target +# before it offers the install medium's removal, and that release's verdict +# decides two things the user acts on: whether the finish screen says the +# stick may come out, and whether the button does the medium-free +# `reboot -ff` or the graceful reboot that still pages off it. A release that +# failed while the screen said otherwise is how a machine gets its stick +# pulled mid-read, so the verdict is tested here rather than trusted. +# +# The dashboard's own driver is a screenful of top-level code, so the file is +# sourced up to it: everything above is definitions, which is all this needs. + +set -euo pipefail + +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd) +DASHBOARD="$ROOT/configs/airootfs/usr/local/bin/omarchy-install-dashboard" + +fails=0 +check() { # desc, expected, actual + local desc=$1 want=$2 got=$3 + if [[ $got != "$want" ]]; then + echo "FAIL: $desc (want '$want', got '$got')"; fails=1; return + fi + echo "ok: $desc" +} + +check_has() { # desc, needle, haystack + local desc=$1 needle=$2 hay=$3 + if [[ $hay != *"$needle"* ]]; then + echo "FAIL: $desc (missing '$needle' in: $hay)"; fails=1; return + fi + echo "ok: $desc" +} + +check_absent() { # desc, needle, haystack + local desc=$1 needle=$2 hay=$3 + if [[ $hay == *"$needle"* ]]; then + echo "FAIL: $desc (unexpected '$needle' in: $hay)"; fails=1; return + fi + echo "ok: $desc" +} + +BOX=$(mktemp -d) +trap 'rm -rf "$BOX"' EXIT +mkdir -p "$BOX/bin" "$BOX/omarchy" +printf 'OMARCHY\n' >"$BOX/omarchy/logo.txt" +printf '{"target": "%s/mnt", "started_at": 0, "finished_at": 62}\n' "$BOX" >"$BOX/state.json" + +# The release helper the dashboard shells out to, standing in for the real +# sweep: it echoes the diagnosis a failing sweep writes to stderr and takes +# its exit status from a fixture, which is the whole contract between them. +cat >"$BOX/bin/omarchy-release-install-target" <<'EOF' +#!/bin/bash +echo "release-helper $*" >>"$BOX/calls" +echo "could not unmount $1; holders: init(1)" >&2 +exit "$(cat "$BOX/release_rc")" +EOF +cat >"$BOX/bin/systemctl" <<'EOF' +#!/bin/bash +echo "systemctl $*" >>"$BOX/calls" +EOF +cat >"$BOX/bin/reboot" <<'EOF' +#!/bin/bash +echo "reboot $*" >>"$BOX/calls" +EOF +cat >"$BOX/bin/ttfx" <<'EOF' +#!/bin/bash +exit 0 +EOF +# Just enough jq that the test does not hinge on the host having the real +# one: the target query answers from the state file, anything else answers +# empty (install_duration then falls through to its log parsing). +cat >"$BOX/bin/jq" <<'EOF' +#!/bin/bash +prog= +for arg in "$@"; do + case $arg in -*) ;; *) prog=$arg; break ;; esac +done +if [[ $prog == *.target* ]]; then + target=$(sed -n 's/.*"target": *"\([^"]*\)".*/\1/p' "${@: -1}") + echo "${target:-/mnt}" +fi +EOF +chmod +x "$BOX"/bin/* + +# The dashboard's definitions, without the driver that would run an install. +awk '/^\[\[ -e \$TTY_PATH \]\] \|\| exit 2$/ { exit } { print }' "$DASHBOARD" >"$BOX/dashboard-defs.sh" + +# Run one scenario: source the definitions, set the release verdict the way +# the given helper exit status would, call $1, and leave the tty output, the +# log and the recorded calls behind for the parent to assert on. +scenario() { # function-to-call, release_rc -> sets TTY_OUT, LOG_OUT, CALLS + local fn=$1 release_rc=$2 reader + echo "$release_rc" >"$BOX/release_rc" + : >"$BOX/calls" + : >"$BOX/install.log" + # The tty is a fifo, not a file: the dashboard reopens it with > for every + # write, which a real console shrugs off and a regular file answers by + # truncating away everything drawn before. An fd held open on the write end + # keeps the reader alive across those reopens. + : >"$BOX/tty.out" + rm -f "$BOX/tty.fifo" + mkfifo "$BOX/tty.fifo" + cat <"$BOX/tty.fifo" >>"$BOX/tty.out" & + reader=$! + ( + export BOX + PATH="$BOX/bin:$PATH" + exec 9>"$BOX/tty.fifo" + OMARCHY_DASHBOARD_TTY="$BOX/tty.fifo" OMARCHY_PATH="$BOX/omarchy" \ + source "$BOX/dashboard-defs.sh" "$BOX/install.log" "$BOX/state.json" -- true + release_target + echo "$TARGET_RELEASED" >"$BOX/released" + [[ $fn == release_target ]] || "$fn" + ) + wait "$reader" + TTY_OUT=$(cat "$BOX/tty.out") + LOG_OUT=$(cat "$BOX/install.log") + CALLS=$(cat "$BOX/calls") + RELEASED=$(cat "$BOX/released") +} + +# ── The verdict itself ─────────────────────────────────────────────────────── +scenario release_target 0 +check "a clean sweep leaves the target released" "yes" "$RELEASED" +check_has "the helper is given the target from the state file" "release-helper $BOX/mnt" "$CALLS" +check_absent "a clean sweep logs no failure" "could not release" "$LOG_OUT" + +scenario release_target 1 +check "a failed sweep withdraws the release" "no" "$RELEASED" +check_has "the failure is logged" "release: could not release $BOX/mnt" "$LOG_OUT" +check_has "the helper's holder diagnosis is forwarded to the log" "holders: init(1)" "$LOG_OUT" + +# ── What the finish screen promises ────────────────────────────────────────── +scenario render_finish 0 +check_has "a released target invites the medium out" "You can now remove the install medium" "$TTY_OUT" + +scenario render_finish 1 +check_has "a held target asks for the medium to stay" \ + "Leave the install medium in until the reboot completes" "$TTY_OUT" +check_absent "and never promises the opposite in the same breath" \ + "You can now remove" "$TTY_OUT" + +# ── What the button does ───────────────────────────────────────────────────── +scenario reboot_now 0 +check_has "a released target reboots without stop jobs" "systemctl reboot -ff" "$CALLS" + +scenario reboot_now 1 +check_has "a held target reboots gracefully" "systemctl reboot" "$CALLS" +check_absent "a held target keeps its stop jobs" "-ff" "$CALLS" + +if (( fails )); then + echo "dashboard release gate: FAILED" + exit 1 +fi +echo "ok: the dashboard's removal offer and reboot follow the release" From c4d0614a54c3efe7875721f01baeb819d991b118 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 28 Aug 2026 15:33:06 +0200 Subject: [PATCH 60/67] Model the forgiven close: cryptsetup fails, the mapper is gone anyway MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 7's one finding (Codex's, confirmed by omarchybot by enumerating the fixtures): the release test never exercised the forgiveness branch, where cryptsetup close returns nonzero while device-mapper's table no longer holds the mapping — the transient shape round 5 went looking for on hardware and could not produce. The stub only removed the mapper on a successful close, so replacing the post-close probe with an unconditional failure would have survived the suite; the cost of that regression is a healthy machine told to leave the medium in and take the graceful reboot. The cryptsetup stub gains a close_removes fixture — exit 1, mapping gone — and the new scenario asserts such a release still exits 0 with nothing reported. Mutation-checked with the exact regression named in the review: the probe replaced by an unconditional failure now fails both new checks. Co-Authored-By: Claude Fable 5 --- test/unit/release-install-target-test.sh | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/test/unit/release-install-target-test.sh b/test/unit/release-install-target-test.sh index 75c6223bb..d22fd7ac2 100755 --- a/test/unit/release-install-target-test.sh +++ b/test/unit/release-install-target-test.sh @@ -129,7 +129,9 @@ EOF #!/bin/bash echo "cryptsetup $*" >>"$BOX/calls" rc=$(cat "$BOX/close_rc") -if (( rc == 0 )); then +# close_removes: the transient shape — cryptsetup reports a failure while +# the mapping is nevertheless gone from the kernel's table. +if (( rc == 0 )) || [[ -e $BOX/close_removes ]]; then awk -v m="${*: -1}" '$1 != m' "$BOX/uuids" >"$BOX/uuids.new" mv "$BOX/uuids.new" "$BOX/uuids" fi @@ -222,6 +224,21 @@ check "an unanswerable mapper query fails the release" 1 "$RC" \ "could not query mapper omarchy_root" "$OUT" check_absent "no close is attempted on an unproven mapper" "cryptsetup" "$CALLS" +# ── A close that errors while the mapper is provably gone is forgiven ──────── +# The transient shape round 5 could not reproduce on hardware: cryptsetup +# reports a failure, yet device-mapper's table no longer holds the mapping. +# The post-close probe must read that as closed — failing it would tell a +# healthy machine to leave the medium in and take the graceful reboot. +new_box +echo 0 >"$BOX/mounted_rc" +echo 1 >"$BOX/close_rc" +touch "$BOX/close_removes" +echo "/dev/mapper/omarchy_root[/@]" >"$BOX/sources" +echo "omarchy_root CRYPT-LUKS2-2b0e-omarchy_root" >"$BOX/uuids" +run_release +check "a failed close with the mapper gone still releases" 0 "$RC" +check_absent "and reports nothing to hold the medium for" "could not close" "$OUT" + # ── dmsetup dying between the close and its probe is still a failed close ──── new_box echo 0 >"$BOX/mounted_rc" From 8329635324993fc229e3b5deafc8b4c67925c071 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Fri, 28 Aug 2026 22:39:21 +0200 Subject: [PATCH 61/67] Wrap the root image stream in an outer zstd layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The per-extent zstd:15 inside the send stream cannot see past a 128 KiB extent, so the stream still carries its protocol framing raw plus redundancy that only exists across extents. One whole-stream pass reclaims 11% (3.75 GB -> 3.34 GB measured on the 2026-08-27 image): level 15 and the 128 MiB long-range window each carry their own fixed slice of that, and past either lies ~0.3% for minutes of build time or a stream a stock zstd -d refuses. Levels, fast levels, bigger windows, and a trained dictionary were all swept; dedupe found 44.9 KB to share across the whole image, so this layer is the whole win. The receive pipeline grows a zstd -dc stage but keeps its shape: the orchestrator still owns the read off the medium (a dying stick must surface as its OSError, not as a decompressor exit code), the extents inside still land on disk as they arrived, and the verify service now hashes the smaller file with its size-based timeout following along. Closing the parent's copy of the decoded pipe's read end is load-bearing — with it open, a receive that dies early never EPIPEs the decompressor and the pipeline blocks instead of failing; the new test hangs without that close. corrupt-image-test used to find the stream by the send magic the layer now hides, so its fixture flips one hex digit of the recorded sha256 instead — same verdict, same halt, and the digest is unique ASCII the plainest grep can find. (Binary needle-grepping was prototyped and works, but only via -F with a raw-byte pattern file: ugrep, which Omarchy installs as grep, decodes -P \xNN escapes as UTF-8 codepoints, so high bytes silently never match. Not worth the ceremony when the checksum file is sitting right there.) That the verify truly reads the whole image stays the slow-medium scenario's business. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_017uQPDXd2ean2CgUUtWcVjQ --- README.md | 2 +- builder/build-iso.sh | 2 +- builder/build-root-image.sh | 17 +++- .../system/omarchy-root-image-verify.service | 6 +- configs/airootfs/root/.automated_script.sh | 6 +- .../local/bin/omarchy-wait-root-image-verify | 2 +- .../omarchy-iso/orchestrator/phases_impl.py | 84 +++++++++++++------ test/integration.d/corrupt-image-test.sh | 42 +++++----- test/integration.d/pxe-test.sh | 2 +- test/unit/test_root_image.py | 83 +++++++++++++----- test/unit/wait-root-image-verify-test.sh | 10 +-- 11 files changed, 176 insertions(+), 80 deletions(-) diff --git a/README.md b/README.md index 2a753c823..ba783776c 100644 --- a/README.md +++ b/README.md @@ -130,7 +130,7 @@ Scenarios under `test/integration.d/` boot a real ISO install in QEMU and assert The first scenario is `factory-reset`: it proves `omarchy-system-factory-reset` hands a machine on without destroying a shared ESP. The installed ESP gets a Windows entry with payload plus a second Linux cloned under a foreign machine-id with its own boot directory and UKIs; a real factory reset is then driven through a guest pty, and the harness asserts the foreign entries survive both the staged reset and first-boot provisioning, that the old Omarchy identity is fully retired, and that the machine reaches first-boot setup unattended. -`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, overwrites a few bytes deep inside the root image (the recorded sha256 stays right; the bytes under it don't, like a badly flashed stick), and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. +`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, flips one digit of the root image's recorded sha256 (so the image no longer hashes to what the medium claims — the same verdict a badly flashed stick produces), and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. Artifacts — screenshots, the fixtured/staged/final `limine.conf`, the reset typescript, and the factory-reset log — land under `test-runs/-integration/runs/-/`, and `--no-preview` skips the `imv` review just like the acceptance harness. diff --git a/builder/build-iso.sh b/builder/build-iso.sh index 3c37e5cb6..d7bdaa714 100755 --- a/builder/build-iso.sh +++ b/builder/build-iso.sh @@ -341,7 +341,7 @@ sed "/^\[options\]/a CacheDir = /var/cache/omarchy/mirror/offline/" \ iso_subdir="$(sed -n 's/^install_dir="\(.*\)"$/\1/p' "$build_cache_dir/profiledef.sh")/$(sed -n 's/^arch="\(.*\)"$/\1/p' "$build_cache_dir/profiledef.sh")" [[ $iso_subdir == */x86_64 ]] || { echo "ERROR: could not read install_dir/arch from profiledef.sh: '$iso_subdir'" >&2; exit 1; } root_image_dir="$build_cache_dir/work/iso/$iso_subdir" -root_image_stream="$root_image_dir/omarchy-root.btrfs" +root_image_stream="$root_image_dir/omarchy-root.btrfs.zst" mkdir -p "$root_image_dir" # Builds before the move left the stream (and, briefly, its checksum) in the # persistent build cache, where they would ship inside the squashfs alongside diff --git a/builder/build-root-image.sh b/builder/build-root-image.sh index 94f1da631..9457bb3cb 100755 --- a/builder/build-root-image.sh +++ b/builder/build-root-image.sh @@ -13,6 +13,15 @@ # measured at roughly half the time of extracting the same packages with pacman, # independent of CPU count; the per-machine delta is a small pacstrap after. # +# The stream itself then gets an outer zstd layer (STREAM_COMPRESS below). The +# per-extent compression above cannot see past a 128 KiB extent, so the send +# stream still carries its protocol framing uncompressed plus redundancy that +# only exists across extents; one whole-stream pass reclaims ~11% (measured on +# a 3.75 GB stream — and the level barely matters, the long-range window does). +# The installer decompresses it in the receive pipe, where zstd -d is orders of +# magnitude faster than any install medium; the extents inside still land on +# disk as they are. +# # Usage: build-root-image.sh ... # # pacman.conf Config whose repositories resolve every package; its @@ -51,6 +60,12 @@ fi # stores the extents as they arrive, so neither choice affects install speed; # the installed system writes new data at its own mount option either way. IMAGE_COMPRESS="compress-force=zstd:15" +# Outer layer over the whole send stream. Level 15 and the 128 MiB long-range +# window are each worth a few hundred MB/-tens of MB respectively over the +# defaults; beyond either lies ~0.3% for minutes of build time (level 19+) or +# a stream a stock `zstd -d` refuses (--long>27 exceeds the decoder's default +# window limit). -T0 is a pure win: ~25 s on a many-core builder, same bytes. +STREAM_COMPRESS=(zstd -q -15 --long=27 -T0) # Name of the subvolume inside the stream. The orchestrator looks for this name # after `btrfs receive` (phases_impl.ROOT_IMAGE_SUBVOLUME). IMAGE_SUBVOLUME="omarchy-root" @@ -190,5 +205,5 @@ sync btrfs property set -ts "$root" ro true rm -f "$output" mkdir -p "$(dirname "$output")" -btrfs send -q --compressed-data -f "$output" "$root" +btrfs send -q --compressed-data "$root" | "${STREAM_COMPRESS[@]}" -o "$output" echo "Root image stream: $(du -h "$output" | cut -f1) at $output" diff --git a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service index f1a146ff5..02e9da73e 100644 --- a/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service +++ b/configs/airootfs/etc/systemd/system/omarchy-root-image-verify.service @@ -7,8 +7,8 @@ Description=Verify the Omarchy root image on the install medium # This is the only verifier: the orchestrator's prepare_install_target phase # collects the verdict, waits for the unit if it is still running, and fails # the install if it could not run. -ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs -ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.sha256 +ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.zst +ConditionPathExists=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.zst.sha256 [Service] Type=oneshot @@ -20,7 +20,7 @@ RemainAfterExit=yes # forever. On timeout Result=timeout tells the wait helper to report a slow # medium rather than a corrupt one. WorkingDirectory=/run/archiso/bootmnt/arch/x86_64 -ExecStart=/usr/bin/sha256sum --check --strict omarchy-root.btrfs.sha256 +ExecStart=/usr/bin/sha256sum --check --strict omarchy-root.btrfs.zst.sha256 # Background work: never compete with the console or anything interactive. Nice=19 IOSchedulingClass=idle diff --git a/configs/airootfs/root/.automated_script.sh b/configs/airootfs/root/.automated_script.sh index be0a947ee..22b68fe84 100644 --- a/configs/airootfs/root/.automated_script.sh +++ b/configs/airootfs/root/.automated_script.sh @@ -58,8 +58,8 @@ if [[ ${OMARCHY_INSTALL_DEBUG:-} == "1" ]]; then fi # Warm the page cache for the root image and bundled packages while the user -# works through the wizard. The install streams ~5GB of root image into btrfs -# receive and then pacstraps a few packages out of the offline mirror; on media +# works through the wizard. The install streams a multi-GB root image into +# btrfs receive and then pacstraps a few packages out of the offline mirror; on media # slower than the unpack the install is read-bound and every byte cached here # is a byte it never waits for. On faster media this costs nothing but # otherwise-idle bandwidth: the medium is untouched while the user types, and @@ -72,7 +72,7 @@ warm_offline_mirror() { local mirror=/var/cache/omarchy/mirror/offline local budget_kb spent_kb=0 size_kb path # The orchestrator's ROOT_IMAGE_STREAM. - local image=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs + local image=/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.zst [[ ${OMARCHY_NO_PREFETCH:-} == 1 ]] && return 0 diff --git a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify index b33deabd0..7fc409618 100755 --- a/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify +++ b/configs/airootfs/usr/local/bin/omarchy-wait-root-image-verify @@ -20,7 +20,7 @@ set -uo pipefail UNIT=omarchy-root-image-verify.service BOOT_MEDIUM_MOUNT=/run/archiso/bootmnt -STREAM="$BOOT_MEDIUM_MOUNT/arch/x86_64/omarchy-root.btrfs" +STREAM="$BOOT_MEDIUM_MOUNT/arch/x86_64/omarchy-root.btrfs.zst" CHECKSUM="$STREAM.sha256" fail() { echo "$1" >&2; exit 1; } diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 8893897c6..a38db9bfa 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -128,10 +128,18 @@ def _omarchy_nvim_package() -> str: # The root image: a `btrfs send --compressed-data` stream of the invariant # target system (build-root-image.sh), unpacked onto the target in place of -# pacstrapping it. build-iso.sh ships it as a plain file on the ISO, read -# straight off the boot medium, with sha256sum output for it next to it. The -# subvolume name is what build-root-image.sh sends. -ROOT_IMAGE_STREAM = Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs") +# pacstrapping it, shipped behind an outer whole-stream zstd layer (the +# per-extent compression inside the stream cannot reach the send framing or +# redundancy that spans extents; the outer pass is ~11% of the stream). +# build-iso.sh ships it as a plain file on the ISO, read straight off the +# boot medium, with sha256sum output for it (the compressed file) next to it. +# The subvolume name is what build-root-image.sh sends. +ROOT_IMAGE_STREAM = Path("/run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.zst") +# Decompresses the outer layer in the receive pipe. --long=27 mirrors the +# compressing side's window: it is within the decoder's default 128 MiB +# acceptance limit, but saying it here keeps the pair visibly in step with +# STREAM_COMPRESS in build-root-image.sh. +ROOT_IMAGE_DECOMPRESS = ("zstd", "-dc", "--long=27") ROOT_IMAGE_SUBVOLUME = "omarchy-root" # The live ISO starts omarchy-root-image-verify.service at boot: `sha256sum -c` # of the stream, running while the user is in the configurator. It is the only @@ -653,23 +661,43 @@ def _umount_tree(root: Path, attempts: int = 20) -> None: def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> None: - """Pipe the stream into btrfs receive, publishing progress for the - dashboard as a fraction of stream bytes consumed.""" + """Pipe the stream through the outer-layer decompressor into btrfs + receive, publishing progress for the dashboard as a fraction of stream + bytes consumed (compressed bytes — the fraction of the medium read, which + is what the wait is made of). + + This loop keeps the read off the medium for itself rather than handing + zstd the file: a read error here is the dying-medium case the verify + machinery exists for, and it must surface as this process's OSError, not + as a decompressor exit code to reverse-engineer. + """ total = stream_path.stat().st_size errors = ctx.state_dir / "btrfs-receive.err" with errors.open("w") as err: + unzstd = subprocess.Popen( + [*ROOT_IMAGE_DECOMPRESS], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=err, + ) proc = subprocess.Popen( ["btrfs", "receive", "-q", str(top)], - stdin=subprocess.PIPE, + stdin=unzstd.stdout, stderr=err, ) - assert proc.stdin is not None + assert unzstd.stdin is not None + assert unzstd.stdout is not None + # The receive holds the decoded pipe now. Dropping this copy of its read + # end is load-bearing: with it open, a receive that dies early would + # never turn zstd's writes into EPIPE, and the whole pipeline — this + # loop included — would block on full pipes instead of failing. + unzstd.stdout.close() sent = 0 last_report = 0.0 try: with stream_path.open("rb") as stream: while chunk := stream.read(8 << 20): - proc.stdin.write(chunk) + unzstd.stdin.write(chunk) sent += len(chunk) now = time.monotonic() if now - last_report >= 0.5: @@ -679,31 +707,37 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No pass finally: # Whatever happened above — including the EIO off a dying medium that - # the whole verify machinery exists to catch — btrfs receive must not - # be left blocked on an open stdin. It would hold the caller's staging - # mount busy, and that umount is check=False: the mount would leak onto - # the target filesystem with nothing said about it. - code = _close_receive(proc) - if code != 0: - raise RuntimeError(f"btrfs receive failed: {errors.read_text(errors='replace').strip()}") + # the whole verify machinery exists to catch — neither child must be + # left blocked on an open stdin. A blocked receive would hold the + # caller's staging mount busy, and that umount is check=False: the + # mount would leak onto the target filesystem with nothing said. + unzstd_code, code = _close_receive(unzstd, proc) + if unzstd_code != 0 or code != 0: + # Both children share the error file, so the full story is in the + # detail either way; the headline names the stage that broke the pipe. + stage = "btrfs receive" if code != 0 else "root image decompression" + raise RuntimeError(f"{stage} failed: {errors.read_text(errors='replace').strip()}") _write_phase_progress(ctx, 1.0) -def _close_receive(proc: subprocess.Popen) -> int: - """Close btrfs receive's stdin and reap it, returning its exit status. +def _close_receive(unzstd: subprocess.Popen, proc: subprocess.Popen) -> tuple[int, int]: + """Close the pipeline's intake and reap both children, returning + (decompressor, receive) exit statuses. Never raises: it runs on the way out of a failing read, where the original - exception is the one worth keeping. Closing flushes, so a receive that has - already died answers with a BrokenPipeError of its own; the wait is - unbounded because a receive that has read EOF is committing the subvolume, - which on slow media is legitimately slow and must not be killed. + exception is the one worth keeping. Closing flushes, so a decompressor + that has already died answers with a BrokenPipeError of its own; on EOF it + drains what it holds and exits, which ends the receive's stdin in turn. + The receive's wait is unbounded because a receive that has read EOF is + committing the subvolume, which on slow media is legitimately slow and + must not be killed. """ - if proc.stdin is not None: + if unzstd.stdin is not None: try: - proc.stdin.close() + unzstd.stdin.close() except OSError: pass - return proc.wait() + return unzstd.wait(), proc.wait() def _write_phase_progress(ctx: InstallContext, fraction: float) -> None: diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh index d7199aa24..48041f8e8 100755 --- a/test/integration.d/corrupt-image-test.sh +++ b/test/integration.d/corrupt-image-test.sh @@ -3,45 +3,47 @@ # A corrupt install medium is refused before the disk is touched. The root # image ships on the ISO with its sha256 beside it; omarchy-root-image-verify # checks it at boot and the installer's pre-flight phase takes that verdict. -# Damage the image data on a copy of the ISO (the checksum stays right, the -# bytes under it don't: a badly flashed stick), autoinstall from it, and +# Flip one digit of the recorded checksum on a copy of the ISO (so the hash +# of the image no longer matches what the medium claims — the same verdict a +# badly flashed stick's damaged image produces), autoinstall from it, and # assert: the unit fails, the install halts in "Preparing install target" # telling the user to re-flash, nothing after that phase ran, and the target -# disk still has no partition table. +# disk still has no partition table. (That the verify truly reads the whole +# multi-GB image is the slow-medium scenario's business, where the read is +# what is being timed.) # # Boots the ISO itself, not the installed base image, so it needs no base. source "$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/base-test.sh" CORRUPT_ISO="$BASE_DIR/corrupt.iso" -STREAM=arch/x86_64/omarchy-root.btrfs +STREAM=arch/x86_64/omarchy-root.btrfs.zst VERIFY_UNIT=omarchy-root-image-verify.service STATE=/run/omarchy-install/state.json # ------------------------------------------------------------------ fixture -# A copy of the ISO with 16 bytes of the root image overwritten (0xFF) a -# third of the way in, deep in extent data. ISO9660 carries no per-file integrity -# data, so patching in place leaves everything else on the medium intact -# (cp --reflink makes the copy free on btrfs). The stream is found by the -# NUL-terminated magic every btrfs send stream starts with; its length comes -# from the ISO's directory. +# A copy of the ISO with one hex digit of the recorded sha256 flipped. +# ISO9660 carries no per-file integrity data, so patching in place leaves +# everything else on the medium intact (cp --reflink makes the copy free on +# btrfs). The 64-char digest is plain ASCII and unique on the ISO, so plain +# grep -F finds the checksum file's bytes; the flip stays within the hex +# alphabet so sha256sum reports a clean FAILED, not a format complaint. corrupt_iso() { - local start size offset - local damage='\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff' + local digest flip start - size=$(bsdtar -tvf "$ISO" "$STREAM" | awk '{print $5}') - [[ $size =~ ^[0-9]+$ ]] || { echo "no root image on the ISO at $STREAM" >&2; return 1; } + digest=$(bsdtar -xOf "$ISO" "$STREAM.sha256" | awk '{print $1}') + [[ $digest =~ ^[0-9a-f]{64}$ ]] || { echo "no recorded sha256 for $STREAM on the ISO" >&2; return 1; } + [[ ${digest:0:1} == 0 ]] && flip=1 || flip=0 - log "Copying the ISO and corrupting its root image" + log "Copying the ISO and mis-recording the root image's checksum" rm -f "$CORRUPT_ISO" cp --reflink=auto "$ISO" "$CORRUPT_ISO" - start=$(grep -Pboa -m1 'btrfs-stream\x00' "$CORRUPT_ISO" | cut -d: -f1) - [[ -n $start ]] || { echo "btrfs send stream magic not found in the ISO image" >&2; return 1; } + start=$(grep -Fboa -m1 -- "$digest" "$CORRUPT_ISO" | cut -d: -f1 || true) + [[ -n $start ]] || { echo "recorded checksum not found in the ISO image" >&2; return 1; } - offset=$((start + size / 3)) - echo -ne "$damage" | dd of="$CORRUPT_ISO" bs=1 seek="$offset" conv=notrunc status=none - log "Image damaged at stream offset $((size / 3))" + printf '%s' "$flip" | dd of="$CORRUPT_ISO" bs=1 seek="$start" conv=notrunc status=none + log "Recorded sha256 flipped: ${digest:0:8}... now reads ${flip}${digest:1:7}..." } # -------------------------------------------------------------------- phases diff --git a/test/integration.d/pxe-test.sh b/test/integration.d/pxe-test.sh index af56baf26..7aed05377 100755 --- a/test/integration.d/pxe-test.sh +++ b/test/integration.d/pxe-test.sh @@ -102,7 +102,7 @@ assert_nbd_medium() { check "kernel cmdline pinned copytoram=n" \ ssh_live_root "grep -q 'copytoram=n' /proc/cmdline" check "root image stream is reachable on the netboot medium" \ - ssh_live_root "test -f /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs" + ssh_live_root "test -f /run/archiso/bootmnt/arch/x86_64/omarchy-root.btrfs.zst" } assert_installed_system() { diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 51dae5f4b..6d80acf14 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -116,7 +116,7 @@ def test_passes_when_helper_exits_zero(self): def test_corrupt_medium_raises_with_the_helpers_message(self): stderr = ("install medium is corrupt: re-flash it\n" "sha256 mismatch on the root image\n" - "omarchy-root.btrfs: FAILED") + "omarchy-root.btrfs.zst: FAILED") with self.helper_returns(1, stdout="boot medium: /dev/sda1 ...\n", stderr=stderr): with self.assertRaises(RuntimeError) as raised: phases_impl.verify_root_image_stream(self.ctx) @@ -145,7 +145,7 @@ class PublishVerifyProgressTest(unittest.TestCase): def test_mirrors_hasher_read_position(self): with tempfile.TemporaryDirectory() as tmp: - stream = (Path(tmp) / "omarchy-root.btrfs").resolve() + stream = (Path(tmp) / "omarchy-root.btrfs.zst").resolve() stream.write_bytes(b"\0" * 4096) states = iter(["activating", "active"]) @@ -169,7 +169,7 @@ def unit_property(prop): def test_missing_stream_is_a_no_op(self): with mock.patch.object(phases_impl, "ROOT_IMAGE_STREAM", - Path("/nonexistent/omarchy-root.btrfs")), \ + Path("/nonexistent/omarchy-root.btrfs.zst")), \ mock.patch.object(phases_impl, "_verify_unit_property") as show: phases_impl._publish_verify_progress(ctx=None) show.assert_not_called() @@ -220,7 +220,7 @@ def setUp(self): self.state_dir = Path(self.tmp.name) / "state" self.state_dir.mkdir() self.top = self.state_dir / "image-top" - self.stream = Path(self.tmp.name) / "omarchy-root.btrfs" + self.stream = Path(self.tmp.name) / "omarchy-root.btrfs.zst" self.stream.write_bytes(b"stream") self.ctx = types.SimpleNamespace(target=self.target, state_dir=self.state_dir) @@ -467,11 +467,13 @@ def test_factory_snapshot_joins_the_unit_before_anything_else(self): class ReceiveRootImageTest(unittest.TestCase): - """The stream-into-btrfs-receive pump, against a real child on a real pipe. + """The stream-through-zstd-into-btrfs-receive pump, against real children + on real pipes. - btrfs receive is stood in for by a shell that drains stdin, so the pipe - blocks the way the real one does and the child is only reaped when its - stdin is actually closed. + The decompressor is the real zstd on real fixture streams; btrfs receive + is stood in for by a shell that drains stdin, so its pipe blocks the way + the real one does and the child is only reaped when its stdin actually + reaches EOF (which now takes the decompressor exiting first). """ def setUp(self): @@ -488,11 +490,17 @@ def setUp(self): def stand_in(self, script): """Patch Popen so btrfs receive is this shell script instead, keeping - the caller's stdin pipe and stderr file.""" + the caller's pipes and stderr file. The zstd child spawns for real. + + Fixtures must be built before this arms: subprocess.run resolves + Popen through the patched module attribute too. + """ spawn = subprocess.Popen # before the patch below replaces it def popen(cmd, **kwargs): + if cmd[0] == "zstd": + return spawn(cmd, **kwargs) self.assertEqual(cmd[:2], ["btrfs", "receive"]) proc = spawn(["sh", "-c", script], **kwargs) self.procs.append(proc) @@ -505,34 +513,71 @@ def popen(cmd, **kwargs): def drains_stdin(self): self.stand_in(f"cat > {shlex.quote(str(self.received))}") - def stream_file(self, data): - path = Path(self.tmp.name) / "omarchy-root.btrfs" - path.write_bytes(data) + def compressed(self, data): + return subprocess.run( + ["zstd", "-q", "-c", "--long=27"], input=data, + stdout=subprocess.PIPE, check=True, + ).stdout + + def stream_file(self, data, raw=False): + """The shipped artifact: DATA behind the outer zstd layer (or, with + raw=True, a file that never was a zstd stream).""" + path = Path(self.tmp.name) / "omarchy-root.btrfs.zst" + path.write_bytes(data if raw else self.compressed(data)) return path def test_streams_the_file_through_and_reports_done(self): + stream = self.stream_file(b"x" * (9 << 20)) self.drains_stdin() - phases_impl._receive_root_image(self.ctx, self.top, self.stream_file(b"x" * (9 << 20))) + phases_impl._receive_root_image(self.ctx, self.top, stream) self.assertEqual(self.received.read_bytes(), b"x" * (9 << 20)) self.assertEqual(json.loads((self.state_dir / "state.json").read_text())["phase_progress"], 1.0) def test_a_failed_receive_raises_with_its_error_output(self): + stream = self.stream_file(b"stream") self.stand_in("cat > /dev/null; echo 'ERROR: chunk failed' >&2; exit 1") - with self.assertRaisesRegex(RuntimeError, "chunk failed"): - phases_impl._receive_root_image(self.ctx, self.top, self.stream_file(b"stream")) + with self.assertRaisesRegex(RuntimeError, "btrfs receive failed.*chunk failed"): + phases_impl._receive_root_image(self.ctx, self.top, stream) + + def test_a_receive_that_dies_at_once_fails_instead_of_blocking(self): + # With the parent still holding a read end of the decoded pipe, a + # dead receive would never EPIPE the decompressor and the pump would + # block on full pipes forever. The payload outsizes every buffer in + # the pipeline so that hang is reachable. + stream = self.stream_file(b"x" * (9 << 20)) + self.stand_in("exit 1") + with self.assertRaisesRegex(RuntimeError, "btrfs receive failed"): + phases_impl._receive_root_image(self.ctx, self.top, stream) + + def test_a_corrupt_outer_layer_fails_as_decompression(self): + # The boot-time hash normally rejects this first; the pipe is the + # backstop when the medium rots between hash and receive. + stream = self.stream_file(b"not a zstd frame at all", raw=True) + self.drains_stdin() + with self.assertRaisesRegex(RuntimeError, "decompression failed"): + phases_impl._receive_root_image(self.ctx, self.top, stream) + self.assertEqual(self.procs[0].wait(timeout=10), 0) def test_a_read_error_off_the_medium_still_closes_and_reaps_the_receive(self): # EIO out of a dying stick is the failure the verify machinery exists - # for. If stdin were left open, btrfs receive would sit blocked on it - # and hold the staging mount busy for the caller's silent umount. + # for. If stdin were left open, the zstd child would sit blocked on it + # — and btrfs receive behind it — holding the staging mount busy for + # the caller's silent umount. Random payload so the compressed stream + # outgrows the 8 MiB first chunk and the EIO lands mid-stream. + payload = os.urandom(12 << 20) + stream = DyingStream(self.compressed(payload)) self.drains_stdin() with self.assertRaises(OSError): - phases_impl._receive_root_image(self.ctx, self.top, DyingStream(b"y" * (9 << 20))) + phases_impl._receive_root_image(self.ctx, self.top, stream) proc = self.procs[0] self.assertEqual(proc.wait(timeout=10), 0) - self.assertEqual(self.received.read_bytes(), b"y" * (8 << 20)) + received = self.received.read_bytes() + # What made it through is a flushed prefix of the payload: the + # truncated tail died in the decompressor, not in the receive. + self.assertTrue(payload.startswith(received)) + self.assertLess(len(received), len(payload)) class DyingStream: diff --git a/test/unit/wait-root-image-verify-test.sh b/test/unit/wait-root-image-verify-test.sh index e4c19ccf0..9eddf41e9 100755 --- a/test/unit/wait-root-image-verify-test.sh +++ b/test/unit/wait-root-image-verify-test.sh @@ -32,8 +32,8 @@ run_helper() { # loadstate, active_seq, start_rc, [result] -> sets RC and OUT local loadstate=$1 active_seq=$2 start_rc=$3 result=${4:-exit-code} local box; box=$(mktemp -d) mkdir -p "$box/bin" "$box/medium/arch/x86_64" "$box/sys/block/sdz/queue" - : >"$box/medium/arch/x86_64/omarchy-root.btrfs" - : >"$box/medium/arch/x86_64/omarchy-root.btrfs.sha256" + : >"$box/medium/arch/x86_64/omarchy-root.btrfs.zst" + : >"$box/medium/arch/x86_64/omarchy-root.btrfs.zst.sha256" # WITH_HASHER_PROC=1: a fake hasher (pid 4242) holds the stream at pos 512 # of 1024 bytes, and the helper draws progress into $box/progress. The shim @@ -41,9 +41,9 @@ run_helper() { # loadstate, active_seq, start_rc, [result] -> sets RC and OUT local mainpid=0 if [[ -n ${WITH_HASHER_PROC:-} ]]; then mainpid=4242 - head -c 1024 /dev/zero >"$box/medium/arch/x86_64/omarchy-root.btrfs" + head -c 1024 /dev/zero >"$box/medium/arch/x86_64/omarchy-root.btrfs.zst" mkdir -p "$box/proc/4242/fd" "$box/proc/4242/fdinfo" - ln -s "$box/medium/arch/x86_64/omarchy-root.btrfs" "$box/proc/4242/fd/7" + ln -s "$box/medium/arch/x86_64/omarchy-root.btrfs.zst" "$box/proc/4242/fd/7" printf 'pos:\t512\nflags:\t0100000\n' >"$box/proc/4242/fdinfo/7" fi echo "none mq-deadline kyber [bfq]" >"$box/sys/block/sdz/queue/scheduler" @@ -59,7 +59,7 @@ echo sdz EOF cat >"$box/bin/journalctl" <<'EOF' #!/bin/bash -echo "omarchy-root.btrfs: FAILED" +echo "omarchy-root.btrfs.zst: FAILED" EOF # systemctl show -p LoadState|ActiveState --value ; systemctl start ... cat >"$box/bin/systemctl" < Date: Sun, 30 Aug 2026 16:37:37 +0200 Subject: [PATCH 62/67] Reap a decompressor its receive never joined; name both broken stages Round 8's two code findings on the receive pipeline. If the second Popen raises, the decompressor was left alive and blocked on a stdin nobody would ever feed -- btrfs-progs is guaranteed on the live ISO, so the trigger is close to unreachable, but a leaked child is the kind of close-to that turns a loud failure into a wedged teardown. The half-built pipeline now closes both of the decompressor's pipes and reaps it before re-raising; the new test drives the real zstd against a receive that cannot spawn and asserts the child was reaped (mutation-checked: without the cleanup, the child is found alive). And when both children fail, the headline no longer guesses: exit codes cannot distinguish a corrupt outer layer starving the receive from a dead receive EPIPEing the decompressor, so the message names both stages and the shared error file -- which carries zstd's own verdict -- disambiguates. Two tests pin the shape from both directions: junk input with a failing receive, and a receive that dies at once taking zstd with it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YVPbW91RD1JkdAK5dXxJab --- .../omarchy-iso/orchestrator/phases_impl.py | 41 +++++++++++++++---- test/unit/test_root_image.py | 41 ++++++++++++++++++- 2 files changed, 73 insertions(+), 9 deletions(-) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index a38db9bfa..4795500cf 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -680,11 +680,28 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No stdout=subprocess.PIPE, stderr=err, ) - proc = subprocess.Popen( - ["btrfs", "receive", "-q", str(top)], - stdin=unzstd.stdout, - stderr=err, - ) + try: + proc = subprocess.Popen( + ["btrfs", "receive", "-q", str(top)], + stdin=unzstd.stdout, + stderr=err, + ) + except Exception: + # A half-built pipeline has no receive to drain the decompressor, + # and _close_receive below never runs: close both of its pipes so + # it exits (EOF on stdin; EPIPE once its output has nowhere to + # go) and reap it, then let the original exception tell the + # story. btrfs is on every live ISO, so this is close to + # unreachable -- but a leaked child blocked on stdin is the kind + # of close-to that turns a loud failure into a wedged teardown. + for pipe in (unzstd.stdin, unzstd.stdout): + if pipe is not None: + try: + pipe.close() + except OSError: + pass + unzstd.wait() + raise assert unzstd.stdin is not None assert unzstd.stdout is not None # The receive holds the decoded pipe now. Dropping this copy of its read @@ -714,8 +731,18 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No unzstd_code, code = _close_receive(unzstd, proc) if unzstd_code != 0 or code != 0: # Both children share the error file, so the full story is in the - # detail either way; the headline names the stage that broke the pipe. - stage = "btrfs receive" if code != 0 else "root image decompression" + # detail either way. When both fail, the order of death is not + # knowable from exit codes alone -- a corrupt outer layer kills zstd + # and starves the receive, while a dead receive EPIPEs zstd -- so + # the headline names both instead of guessing a culprit, and the + # detail (zstd's "premature end", the receive's own complaint) + # disambiguates. + if unzstd_code != 0 and code != 0: + stage = "root image decompression and btrfs receive both" + elif code != 0: + stage = "btrfs receive" + else: + stage = "root image decompression" raise RuntimeError(f"{stage} failed: {errors.read_text(errors='replace').strip()}") _write_phase_progress(ctx, 1.0) diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index 6d80acf14..bf5d10cd9 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -544,12 +544,49 @@ def test_a_receive_that_dies_at_once_fails_instead_of_blocking(self): # With the parent still holding a read end of the decoded pipe, a # dead receive would never EPIPE the decompressor and the pump would # block on full pipes forever. The payload outsizes every buffer in - # the pipeline so that hang is reachable. + # the pipeline so that hang is reachable. The EPIPE takes the + # decompressor down too, so the headline names both. stream = self.stream_file(b"x" * (9 << 20)) self.stand_in("exit 1") - with self.assertRaisesRegex(RuntimeError, "btrfs receive failed"): + with self.assertRaisesRegex(RuntimeError, "decompression and btrfs receive both failed"): + phases_impl._receive_root_image(self.ctx, self.top, stream) + + def test_a_corrupt_layer_with_a_failing_receive_names_both(self): + # The corrupt-medium shape the headline used to get backwards: zstd + # dies on the junk, the starved receive exits nonzero after it, and + # the message blamed the receive alone. Neither death order is + # provable from exit codes, so both are named and the shared detail + # file carries zstd's own verdict. + stream = self.stream_file(b"junk that is no zstd frame", raw=True) + self.stand_in("cat > /dev/null; exit 1") + with self.assertRaisesRegex(RuntimeError, "decompression and btrfs receive both failed"): + phases_impl._receive_root_image(self.ctx, self.top, stream) + + def test_a_receive_that_cannot_spawn_reaps_the_decompressor(self): + # The half-built pipeline: on the live ISO btrfs-progs is guaranteed, + # so a receive that cannot spawn is close to unreachable -- but a + # decompressor left alive and blocked on a stdin nobody will ever + # feed turns that loud failure into a wedged teardown. + stream = self.stream_file(b"x" * (9 << 20)) + spawn = subprocess.Popen + zstd_procs = [] + + def popen(cmd, **kwargs): + if cmd[0] == "zstd": + proc = spawn(cmd, **kwargs) + zstd_procs.append(proc) + return proc + raise FileNotFoundError("btrfs") + + patch = mock.patch.object(phases_impl.subprocess, "Popen", side_effect=popen) + patch.start() + self.addCleanup(patch.stop) + with self.assertRaises(FileNotFoundError): phases_impl._receive_root_image(self.ctx, self.top, stream) + (zstd,) = zstd_procs + self.assertIsNotNone(zstd.poll()) + def test_a_corrupt_outer_layer_fails_as_decompression(self): # The boot-time hash normally rejects this first; the pipe is the # backstop when the medium rots between hash and receive. From 826a9bcd421d29ad703143ba0b400342ed64c088 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Sun, 30 Aug 2026 16:37:37 +0200 Subject: [PATCH 63/67] Corrupt the image bytes again, located by the ISO's own metadata Round 8's sharpest observation: when the outer zstd layer hid the btrfs-stream magic, the corrupt-image fixture retreated to flipping a digit of the recorded checksum -- which proves the gate refuses a mismatch, but no longer proves the digest covers the image bytes at all. A build that hashed the wrong or a stale file would still mismatch, still pass, and still ship. That guard mattered precisely when the layer added a second candidate to hash. The fixture damages the artifact itself again. Its coordinates come from the ISO9660 directory records -- xorriso reports the stream's start LBA and byte size, so the flip is placed by filesystem metadata rather than content search (the zstd frame magic opens every mirror package and countless squashfs blocks on this ISO), and a bounds check proves the byte lies inside the file's single contiguous extent. One byte +1 mod 256, a MiB into the payload: bit-rot as a flashed stick actually serves it. libisoburn joins the runner's dependency line. Verified end to end on a built ISO: the boot-time hash caught the flip, the install halted in the pre-flight phase with the re-flash advice, eight of eight assertions green. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YVPbW91RD1JkdAK5dXxJab --- test/integration | 2 +- test/integration.d/corrupt-image-test.sh | 63 +++++++++++++++--------- 2 files changed, 40 insertions(+), 25 deletions(-) diff --git a/test/integration b/test/integration index 7dd77e6b8..4c8408a80 100755 --- a/test/integration +++ b/test/integration @@ -73,7 +73,7 @@ else done fi -omarchy-pkg-add qemu-full edk2-ovmf socat imagemagick tesseract tesseract-data-eng mtools +omarchy-pkg-add qemu-full edk2-ovmf socat imagemagick tesseract tesseract-data-eng mtools libisoburn export OMARCHY_INTEGRATION_ISO OMARCHY_INTEGRATION_ISO=$(realpath "$ISO") diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh index 48041f8e8..576f1f599 100755 --- a/test/integration.d/corrupt-image-test.sh +++ b/test/integration.d/corrupt-image-test.sh @@ -3,14 +3,15 @@ # A corrupt install medium is refused before the disk is touched. The root # image ships on the ISO with its sha256 beside it; omarchy-root-image-verify # checks it at boot and the installer's pre-flight phase takes that verdict. -# Flip one digit of the recorded checksum on a copy of the ISO (so the hash -# of the image no longer matches what the medium claims — the same verdict a -# badly flashed stick's damaged image produces), autoinstall from it, and -# assert: the unit fails, the install halts in "Preparing install target" -# telling the user to re-flash, nothing after that phase ran, and the target -# disk still has no partition table. (That the verify truly reads the whole -# multi-GB image is the slow-medium scenario's business, where the read is -# what is being timed.) +# Flip one byte inside the image stream on a copy of the ISO (a badly +# flashed stick's damage, for real), autoinstall from it, and assert: the +# unit fails, the install halts in "Preparing install target" telling the +# user to re-flash, nothing after that phase ran, and the target disk still +# has no partition table. Damaging the artifact rather than the recorded +# digest also proves the digest covers the bytes that ship: a build hashing +# the wrong or a stale file could not pass here. (That the verify truly +# reads the whole multi-GB image is the slow-medium scenario's business, +# where the read is what is being timed.) # # Boots the ISO itself, not the installed base image, so it needs no base. @@ -23,27 +24,41 @@ STATE=/run/omarchy-install/state.json # ------------------------------------------------------------------ fixture -# A copy of the ISO with one hex digit of the recorded sha256 flipped. -# ISO9660 carries no per-file integrity data, so patching in place leaves -# everything else on the medium intact (cp --reflink makes the copy free on -# btrfs). The 64-char digest is plain ASCII and unique on the ISO, so plain -# grep -F finds the checksum file's bytes; the flip stays within the hex -# alphabet so sha256sum reports a clean FAILED, not a format complaint. +# A copy of the ISO with one byte flipped inside the root image stream +# itself. ISO9660 files are contiguous extents with no per-file integrity +# data, so patching in place leaves everything else on the medium intact +# (cp --reflink makes the copy free on btrfs). +# +# The stream's extent comes from the ISO9660 directory records themselves: +# xorriso reports the file's start LBA and byte size, so the flip is placed +# by filesystem metadata, not by searching for content (the zstd frame +# magic opens every mirror package and countless squashfs blocks on this +# ISO, so a content search would need fingerprinting games). ISO9660 files +# are single contiguous extents, which is what makes lba*2048 + offset a +# byte address inside this file and no other. corrupt_iso() { - local digest flip start - - digest=$(bsdtar -xOf "$ISO" "$STREAM.sha256" | awk '{print $1}') - [[ $digest =~ ^[0-9a-f]{64}$ ]] || { echo "no recorded sha256 for $STREAM on the ISO" >&2; return 1; } - [[ ${digest:0:1} == 0 ]] && flip=1 || flip=0 + local lba size start off orig flipped - log "Copying the ISO and mis-recording the root image's checksum" + log "Copying the ISO and corrupting one byte inside the root image stream" rm -f "$CORRUPT_ISO" cp --reflink=auto "$ISO" "$CORRUPT_ISO" - start=$(grep -Fboa -m1 -- "$digest" "$CORRUPT_ISO" | cut -d: -f1 || true) - [[ -n $start ]] || { echo "recorded checksum not found in the ISO image" >&2; return 1; } - printf '%s' "$flip" | dd of="$CORRUPT_ISO" bs=1 seek="$start" conv=notrunc status=none - log "Recorded sha256 flipped: ${digest:0:8}... now reads ${flip}${digest:1:7}..." + # "File data lba: xt , startlba , blocks , filesize , path" + read -r lba size < <(xorriso -indev "$ISO" -find "/$STREAM" -exec report_lba 2>/dev/null | + awk -F, '/File data lba/ { gsub(/ /, ""); print $2, $4 }') + [[ $lba =~ ^[0-9]+$ && $size =~ ^[0-9]+$ ]] || + { echo "xorriso could not report the extent of $STREAM" >&2; return 1; } + start=$((lba * 2048)) + + # Deep enough to model bit-rot in the payload, and provably inside the + # file's extent; +1 mod 256 so the byte always changes. + off=$((start + 1048576)) + ((1048576 < size)) || { echo "stream unexpectedly small: $size bytes" >&2; return 1; } + orig=$(dd if="$CORRUPT_ISO" bs=1 skip="$off" count=1 status=none | od -An -tu1 | tr -d ' ') + flipped=$(( (orig + 1) % 256 )) + printf "\\$(printf '%03o' "$flipped")" | + dd of="$CORRUPT_ISO" bs=1 seek="$off" conv=notrunc status=none + log "Stream extent at LBA $lba ($size bytes); byte at ISO offset $off flipped ($orig -> $flipped)" } # -------------------------------------------------------------------- phases From 7577f67a276f1e5f70b459aa19936a2809c04bd8 Mon Sep 17 00:00:00 2001 From: Omabot Date: Mon, 31 Aug 2026 05:37:08 -0700 Subject: [PATCH 64/67] Describe the corruption the scenario actually injects 826a9bc moved the flip from the recorded digest to the image bytes; the README still told the reader a hex digit of the sha256 gets changed, which is the one thing that scenario no longer does. Co-Authored-By: Claude Opus 5 (1M context) Co-Authored-By: Codex XHigh --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ba783776c..b7edd96de 100644 --- a/README.md +++ b/README.md @@ -130,7 +130,7 @@ Scenarios under `test/integration.d/` boot a real ISO install in QEMU and assert The first scenario is `factory-reset`: it proves `omarchy-system-factory-reset` hands a machine on without destroying a shared ESP. The installed ESP gets a Windows entry with payload plus a second Linux cloned under a foreign machine-id with its own boot directory and UKIs; a real factory reset is then driven through a guest pty, and the harness asserts the foreign entries survive both the staged reset and first-boot provisioning, that the old Omarchy identity is fully retired, and that the machine reaches first-boot setup unattended. -`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO, flips one digit of the root image's recorded sha256 (so the image no longer hashes to what the medium claims — the same verdict a badly flashed stick produces), and autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. +`corrupt-image` proves a bad install medium is refused before the disk is touched. It copies the ISO and flips one byte a megabyte into the root image stream itself — addressed from the extent xorriso reports out of the ISO9660 directory records, so the damage lands in the shipped bytes and nowhere else — then autoinstalls from that copy: `omarchy-root-image-verify.service` (which hashes the image at boot, while a user would be in the configurator) must fail, the install must halt in its pre-flight phase with the "re-flash" advice on screen, and the target disk must still have no partition table. It boots the ISO itself rather than the base image, so it also works standalone with `OMARCHY_INTEGRATION_ISO` set. Artifacts — screenshots, the fixtured/staged/final `limine.conf`, the reset typescript, and the factory-reset log — land under `test-runs/-integration/runs/-/`, and `--no-preview` skips the `imv` review just like the acceptance harness. From 1f94df15ba252a2ae89856954cdceb411b4560df Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 31 Aug 2026 22:30:39 +0200 Subject: [PATCH 65/67] Prove the pristine stream verifies before corrupting it Round 9 caught the fixture's comment claiming more than the scenario proved: a build that hashed the wrong or a stale file would mismatch on the pristine ISO and on the corrupted copy alike, and every assertion would still pass -- and in --reuse-base and standalone runs no install ever boots the untouched ISO to catch it implicitly. The fixture now proves the claim instead of retracting it: before flipping anything it extracts the recorded digest and recomputes sha256 over the stream's extent host-side, refusing with both digests named on mismatch. Also makes the extent-failure diagnostic reachable: under set -e the read off an empty pipe killed the scenario before the guard could say what went wrong (round 9's second find, verified in a scratch shell on both sides of the fix). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YVPbW91RD1JkdAK5dXxJab --- test/integration.d/corrupt-image-test.sh | 33 +++++++++++++++++++----- 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/test/integration.d/corrupt-image-test.sh b/test/integration.d/corrupt-image-test.sh index 576f1f599..fc56a1969 100755 --- a/test/integration.d/corrupt-image-test.sh +++ b/test/integration.d/corrupt-image-test.sh @@ -8,10 +8,13 @@ # unit fails, the install halts in "Preparing install target" telling the # user to re-flash, nothing after that phase ran, and the target disk still # has no partition table. Damaging the artifact rather than the recorded -# digest also proves the digest covers the bytes that ship: a build hashing -# the wrong or a stale file could not pass here. (That the verify truly -# reads the whole multi-GB image is the slow-medium scenario's business, -# where the read is what is being timed.) +# digest proves the digest is sensitive to the shipped bytes at that +# offset; the fixture separately proves the pristine stream matches its +# recorded digest host-side, so a build hashing the wrong or a stale file +# fails even in --reuse-base and standalone runs, where no install ever +# boots the untouched ISO. (That the verify truly reads the whole multi-GB +# image is the slow-medium scenario's business, where the read is what is +# being timed.) # # Boots the ISO itself, not the installed base image, so it needs no base. @@ -44,12 +47,30 @@ corrupt_iso() { cp --reflink=auto "$ISO" "$CORRUPT_ISO" # "File data lba: xt , startlba , blocks , filesize , path" + # || true: under set -e a read off an empty pipe would kill the scenario + # before the guard below could say what went wrong. read -r lba size < <(xorriso -indev "$ISO" -find "/$STREAM" -exec report_lba 2>/dev/null | - awk -F, '/File data lba/ { gsub(/ /, ""); print $2, $4 }') - [[ $lba =~ ^[0-9]+$ && $size =~ ^[0-9]+$ ]] || + awk -F, '/File data lba/ { gsub(/ /, ""); print $2, $4 }') || true + [[ ${lba:-} =~ ^[0-9]+$ && ${size:-} =~ ^[0-9]+$ ]] || { echo "xorriso could not report the extent of $STREAM" >&2; return 1; } start=$((lba * 2048)) + # Before damaging anything, prove the pristine artifact verifies: the + # digest recorded at build time must match the bytes that ship, computed + # here over the extent just located. Every booted install checks this + # implicitly, but --reuse-base skips the install and this scenario runs + # standalone, and in those modes nothing else would catch a build that + # hashed the wrong or a stale file. + local recorded computed + recorded=$(xorriso -osirrox on -indev "$ISO" -extract "/$STREAM.sha256" "$BASE_DIR/stream.sha256" 2>/dev/null && + awk '{print $1; exit}' "$BASE_DIR/stream.sha256") || true + [[ ${recorded:-} =~ ^[0-9a-f]{64}$ ]] || + { echo "could not read the recorded digest for $STREAM" >&2; return 1; } + computed=$(dd if="$ISO" bs=1M iflag=skip_bytes,count_bytes skip="$start" count="$size" status=none | sha256sum | cut -d' ' -f1) + [[ $computed == "$recorded" ]] || + { echo "pristine image does not match its recorded digest (recorded ${recorded:0:8}..., shipped ${computed:0:8}...)" >&2; return 1; } + log "Pristine stream matches its recorded digest (${recorded:0:8}...)" + # Deep enough to model bit-rot in the payload, and provably inside the # file's extent; +1 mod 256 so the byte always changes. off=$((start + 1048576)) From 759702de3f79f3d5b7eb054128d4784d5bf7e126 Mon Sep 17 00:00:00 2001 From: Jonny Heggheim Date: Mon, 31 Aug 2026 22:30:39 +0200 Subject: [PATCH 66/67] Reap the decompressor on BaseException, not just Exception Round 9's completeness note: a KeyboardInterrupt aimed at the orchestrator's PID alone (the process-group kill and a terminal Ctrl-C both reach zstd directly) could land inside the second Popen, and `except Exception` would skip the half-built-pipeline cleanup on its way out. The block re-raises, so BaseException swallows nothing. The new test raises KeyboardInterrupt from the receive's spawn and asserts the decompressor was reaped; reverting the one word fails exactly that test. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YVPbW91RD1JkdAK5dXxJab --- .../omarchy-iso/orchestrator/phases_impl.py | 5 +++- test/unit/test_root_image.py | 26 +++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py index 4795500cf..bc8e905e4 100644 --- a/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py +++ b/configs/airootfs/usr/share/omarchy-iso/orchestrator/phases_impl.py @@ -686,7 +686,7 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No stdin=unzstd.stdout, stderr=err, ) - except Exception: + except BaseException: # A half-built pipeline has no receive to drain the decompressor, # and _close_receive below never runs: close both of its pipes so # it exits (EOF on stdin; EPIPE once its output has nowhere to @@ -694,6 +694,9 @@ def _receive_root_image(ctx: InstallContext, top: Path, stream_path: Path) -> No # story. btrfs is on every live ISO, so this is close to # unreachable -- but a leaked child blocked on stdin is the kind # of close-to that turns a loud failure into a wedged teardown. + # BaseException, not Exception: a KeyboardInterrupt aimed at this + # process alone must run the same cleanup on its way out (the + # block re-raises, so nothing is swallowed). for pipe in (unzstd.stdin, unzstd.stdout): if pipe is not None: try: diff --git a/test/unit/test_root_image.py b/test/unit/test_root_image.py index bf5d10cd9..72262d686 100644 --- a/test/unit/test_root_image.py +++ b/test/unit/test_root_image.py @@ -587,6 +587,32 @@ def popen(cmd, **kwargs): (zstd,) = zstd_procs self.assertIsNotNone(zstd.poll()) + def test_an_interrupt_during_spawn_reaps_the_decompressor_too(self): + # A KeyboardInterrupt aimed at this process alone (not the process + # group, which reaches zstd directly) lands wherever the interpreter + # happens to be -- including inside the second Popen. It is a + # BaseException, so an `except Exception` would skip the cleanup and + # leak the blocked decompressor while the orchestrator unwinds. + stream = self.stream_file(b"x" * (9 << 20)) + spawn = subprocess.Popen + zstd_procs = [] + + def popen(cmd, **kwargs): + if cmd[0] == "zstd": + proc = spawn(cmd, **kwargs) + zstd_procs.append(proc) + return proc + raise KeyboardInterrupt + + patch = mock.patch.object(phases_impl.subprocess, "Popen", side_effect=popen) + patch.start() + self.addCleanup(patch.stop) + with self.assertRaises(KeyboardInterrupt): + phases_impl._receive_root_image(self.ctx, self.top, stream) + + (zstd,) = zstd_procs + self.assertIsNotNone(zstd.poll()) + def test_a_corrupt_outer_layer_fails_as_decompression(self): # The boot-time hash normally rejects this first; the pipe is the # backstop when the medium rots between hash and receive. From f684071aad0878a8a46db71f297c913edada16d9 Mon Sep 17 00:00:00 2001 From: Omabot Date: Sat, 5 Sep 2026 05:30:00 -0700 Subject: [PATCH 67/67] Stop OCR-waiting on the free-space unencrypted confirm The quattro merge brought in 9a7325e, which removed the two `wait_for_screen "without encryption"` waits because tesseract cannot read gum's highlighted "Yes, install without encryption" button under any preprocessing. The free-space block this branch adds carries a third copy of that wait, in a place git had no reason to conflict on, so the merge left it behind: `omarchy-iso-test --free-space` would spend 30s waiting for text that never resolves, then die under `set -e` before the install ever starts. Same treatment as the two sibling sites -- capture the screen and continue; a Ctrl+C toggle that failed to apply still surfaces as an encrypted install stalling at the LUKS prompt with SSH never coming up. Co-Authored-By: Claude Opus 5 (1M context) --- bin/omarchy-iso-test | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bin/omarchy-iso-test b/bin/omarchy-iso-test index 768e66679..75f389f37 100755 --- a/bin/omarchy-iso-test +++ b/bin/omarchy-iso-test @@ -713,7 +713,8 @@ drive_configurator() { capture_console "success-installer-12-free-space-confirm-encrypted" if ! $ENCRYPT; then press ctrl-c # toggles the confirm to the unencrypted flow - wait_for_screen "without encryption" 30 + # Same highlighted button OCR cannot read as the full-disk confirm below. + sleep 2 capture_console "success-installer-13-free-space-confirm-unencrypted" fi press ret