Repository navigation
Speed up installs and add reproducible native benchmarks - #1
Draft
ryankirkman wants to merge 40 commits into
Draft
ryankirkman wants to merge 40 commits into
ryankirkman wants to merge 40 commits into
Conversation
Reuse the official unattended fixture, collect boot and package evidence, and control QEMU through a recorded filesystem mailbox. Keep mutable VM storage outside synced workspaces.
…ion evidence Add pinned guest image construction with version and package-reason checks, identity scrubbing, native compression, and a validated portable live-only QEMU bundle. Extend the VM supervisor for matched direct-kernel boots and real readiness bounds; reject interrupted or inequivalent comparisons.
Append a verified mkinitcpio late hook without rebuilding the release ISO, preserving original hooks and live directory modes. Reboot onto the target without installer media, retain serial evidence, and require conservative whole-boot measurements for the overall 2x target.
Copy payload files using an embedded manifest while retaining live directory modes. Validate the supplementary ISO's embedded qcow2 against the validated image digest without allocating another multi-gigabyte copy.
Reject shared machine IDs, SSH host keys, pacman master keys, or filesystem UUIDs. Record actual supplementary media digests, topology and cache preconditioning, and enforce matching fixtures. Add a separate opt-in package-prefetch experiment without changing verification or defaults. Reject stale image-build success metadata.
…uild Preserve original phase timings, 941-package inventory, clean package-file validation, installed-root boot evidence, and separately collected public identities/service diagnostics. Label this as calibration, not a matched speedup sample. Add a durable guest-build driver and retain the upstream ISO license.
Use a standard public Ubuntu runner with real KVM and sufficient disk gates, pinned source/media, no persistent checkout credentials and small evidence-only artifacts. Run three alternating fresh pairs for the original image candidate, then continue with three separate no-prefetch pairs even if the first valid comparison misses 2x. Seal complete evidence before disk reclamation and use the conservative full host-clock gate.
GitHub rejected runner.temp in job-level env before creating a job. Set the same temporary paths in the first shell step through GITHUB_ENV.
…irmware installs Preserve the successful 181-second native stock calibration and image-build failure. Keep package-owned paths during image cleanup and retain bounded diagnostics on failure. Add verified firmware ISO fixtures with an independent media-free reboot gate, complete package file counts, and cold source-page evidence. Run both image variants through three fresh alternating pairs; no complete-install speedup is claimed before validation.
Stage PR145's pinned dashboard as a separate candidate after ordinary image-overlay activation. Require both sync calls to succeed before immediate guest reboot, retain graceful fallback and standalone validation, and collect installed boot critical-path diagnostics after timing. Existing workflow selection and both initial variants remain unchanged; this commit does not claim a speedup.
… variants The first booted image candidate matched all package versions but failed file validation because fresh child subvolumes hid packaged log directories. Seed child mounts with checked archive copies before replaying mounts, guard exact upstream source and unsafe or nested-subvolume layouts, and retain invalid trial diagnostics separately. Preserve the validated native cold control and image build plus the failed local candidate. Measure three fresh alternating pairs for each of image, no-prefetch, and guarded fast-reboot variants; no speedup is claimed from failed installs.
Record the first fully validated 2.19x native pair and the following failed boot repetition without claiming a complete three-pair result. Retain original run evidence, expand read-only rescue context, and require evidence seal verification before comparisons while preserving explicit raw-calibration compatibility.
Select four stock firmware calibration attempts, preserve full failure journals and the first real console smoke failure, and bind QEMU's generic file backend to the owned live process and exact serial path. Retain Plymouth source review and focused diagnostic contracts. This checkpoint awaits the fresh console smoke result before the diagnostic workflow is launched.
Record smoke-02 root login, nonce and UID proof, all ten read-only diagnostic command markers, unchanged original serial bytes, and the retained invalid installation verdict. Preserve exact tested source and screenshots without changing the running native diagnostic workflow.
Select only the early-verification direct-restore candidate after four stock calibration installs passed. Keep firmware boot, verified cold source media, 600-second install and 180-second standalone-boot deadlines, and three fresh control/candidate pairs with all validation gates.
Retain all 142 original artifacts from native run 34000377432 with their manifest and independent validation record. Record the rejected finish-overlap proposal as documentation only. The direct-restore benchmark remains frozen at its running commit.
Record all 266 original artifacts from run 34001279394 and the independently verified six successful installations and standalone reboots. Report median observed speedup 1.968526x and conservative lower bound 1.898330853x: the complete direct-restore series missed the 2x threshold. Preserve the existing benchmark workflow revision.
Overlap the unchanged locate-index command with boot preparation and run the complete foreground animation during finalization. Preserve both worker joins, package and boot validation, snapshot ordering, and installer-success/target-release gates. Retain focused contracts, actual guest engine/UI evidence and exact payload provenance; select three fresh firmware/cold matched pairs. This new candidate has no native performance result yet.
Execute contract copies against an owned temporary serial file without inspecting or accessing a host serial device. Preserve all 76 original artifacts from the pre-pair failure and the 14 passing isolated contract results. Runtime candidate, native runner and workflow remain unchanged; the next trial rebuilds its fixtures and starts three fresh matched pairs.
Run overlap preparation contracts before expensive image manufacture, and collect read-only advisory log events after the SSH readiness clock is recorded. Preserve focused passing contracts and actual preparation equivalence evidence, plus the source-only UFW batching assessment. Candidate runtime and the currently running native trial remain unchanged; no whole-suite pass is claimed.
Replace the PID-namespace-sensitive /proc assertion with actual child readiness and inherited-pipe EOF, including a live-child negative control. The targeted test passed; production cleanup and the active native trial remain unchanged.
Exact source snapshot of local commit 2749593bac0fff3d336de6c6d7519266eb672d56, tree 446a506. Retains all original run 34005907730 evidence, the updated performance report and source-only Limine hash audit. Candidate median 2.0773053657713194x; conservative lower bound 1.9995695373707232x, below the strict 2x threshold. No pending firewall/logging runtime or workflow changes included.
Preserve the rejected all-call logging cost and the narrower component evidence. Keep package-owned files unchanged; the new candidate has no native performance result yet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeated package database queries and per-step logging processes add avoidable installer overhead. The default changes batch package presence and successful-install verification into one read-only pacman query, and use Bash's built-in timestamp formatter. Package selection, failure handling, log format and child-shell isolation are preserved. Component results and regression checks are documented in
docs/install-speed.md; they are not full-install speedup claims.The larger installation experiments are opt-in benchmark overlays. They build on the prebuilt Btrfs image installer from upstream PR #145 and PR #113, whose authors deserve credit for that design. This branch adds reproducible same-package image preparation, preserves image files and metadata when creating child subvolumes, disables redundant package prefetch, tests guarded reboot and earlier verification, and optionally uses direct target writes. Full image verification and final sync/release checks remain required.
The latest complete trial, run 34005907730, completed three fresh alternating pairs with all six installations and media-free reboots validated. Observed control readiness was 155.341–166.325 seconds and candidate readiness was 74.847–75.465 seconds. Median observed speedup was 2.077305×; the conservative lower bound was 1.999569537×, including measured SSH polling uncertainty. The strict 2× threshold remains unmet. All 180 original sample seal hashes were independently rechecked; all 270 original artifact files and the original false verdict are retained. Complete evidence.
That measured candidate moves the unchanged locate-index command into the existing joined user-finalization branch and runs the complete foreground logo animation while installation finishes. Indexing and both finalization branches must complete before validation and the factory snapshot; the completion screen and media-removal prompt still wait for installer success and target release. Moving the ordinary filesystem scan later can change the newly created paths it observes.
Earlier results remain separately preserved: run 33996768275 produced one valid 2.19046× conservative pair before a later candidate readiness failure, and the complete direct-write trial 34001279394 reached 1.898330853× conservatively across three valid pairs. The first overlap attempt stopped at a host contract portability check before paired samples. Its isolated test fix preceded the fresh complete trial above; no samples were reused. The earlier intermittent readiness failure remains unexplained.
The fresh firewall/logging trial 34009848220 stopped at its early firewall host contract before ISO download, calibration, image construction or paired installations. Its failure evidence is retained and the test portability issue is under review; it produced no candidate installation or speed result. It moves the unchanged firewall leaf to the start of the joined user branch and exposes the built-in-clock logger through one temporary private read-only mount only during serial system setup. Firewall completion must still precede user/SSH/Tailscale/indexing work; those three shorter setup invocations keep their original logger. Package-owned helper bytes and metadata remain intact. The manifest and preflight require
serial-system-finalizer-onlyscope. This candidate has no native performance result yet.The earlier all-call logging design was rejected after its full four-call TCG component median increased from 14.641953 to 15.879115 seconds. The narrower design then passed fresh root/user/failure, read-only mount, concurrent parent isolation and cleanup checks, and improved every fresh component pair: median 15.316387 to 12.076732 seconds (1.268256×). These are bounded component results, not full-install claims. The component stopwatch excludes the controller's one-time module import; the native clock includes it. Both failed fixture attempts, the all-call regression, exact sources and fresh system-only evidence are preserved. The disposable guest's diagnostic Plymouth setting is excluded from the native fixtures. A separate source-only Limine hash-launch audit also makes no runtime or speedup claim.
To reproduce the latest completed candidate with the workflow's Linux/KVM prerequisites and 50 GiB free, use fresh output directories:
python3 test/benchmarks/install-speed/native-ci/run-native-experiment.py \ --repo "$PWD" --work /tmp/omarchy-speed-work --evidence /tmp/omarchy-speed-evidence \ --boot-method firmware --source-cache cold \ --install-timeout 600 --standalone-reboot-timeout 180 \ --variants image-no-package-prefetch-fast-reboot-early-verify-direct-restore-overlapTo reproduce the current unmeasured candidate instead, select
--variants image-no-package-prefetch-fast-reboot-early-verify-direct-restore-overlap-firewall-loggingwith the same fresh-pair protocol.Acceptance requires matching all 941 package versions and explicit/dependency reasons, successful
pacman -Qkwith complete file counts, fresh machine/SSH/pacman/Btrfs identities, and a second ordinary boot after removing installation media. Trials alternate three fresh pairs on matched hardware; failures remain excluded and retained. Timing covers VM start through installed-root SSH. Image manufacture is excluded; per-install verification is included. Results apply to this unencrypted Omarchy 4.0.2 KVM fixture, not physical USB installations or a production ISO release.