feat(os): normalize the Linux setup header so new images measure the kernel file

[kvinwang]

Makes RTMR[1] independent of the host's QEMU version, for images built from now
on. Existing images keep their current measurement byte for byte.

Fixes the same root cause as #1183, but by removing the QEMU-version dependency
rather than modelling it — see "Relationship to #1183".

Problem

TDX CVMs on QEMU 10.2 fail verification with an RTMR[1] mismatch:

RTMR1 mismatch: expected=4ffc8bcd…, actual=74f00f63…

QEMU is the boot loader for -kernel: it fills in the setup-header fields the
Linux boot protocol expects a boot loader to supply — type_of_loader,
ramdisk_image/ramdisk_size, heap_end_ptr, cmd_line_ptr — and serves the
result over fw_cfg. OVMF measures those bytes into RTMR[1].

Upstream commit
a7542a38f399
("x86/loader: Don't update kernel header for CoCo VMs", first in 10.2.0) widened
the pre-existing SEV-only skip to every confidential guest:

-    if (!sev_enabled() && protocol > 0) {
+    if (!MACHINE(x86ms)->cgs && protocol > 0) {
         memcpy(setup, header, MIN(sizeof(header), setup_size));
     }

Measured on a real image, the rewrite touches 4 regions, 11 bytes:

0x0210..0x0212  orig=[00,01]   patched=[b0,81]                type_of_loader, loadflags
0x0219..0x021f  orig=[00 x6]   patched=[10,54,7f,c0,62,a9]    ramdisk_image, ramdisk_size
0x0224..0x0226  orig=[a0,50]   patched=[00,fe]                heap_end_ptr
0x022a..0x022b  orig=[00]      patched=[02]                   cmd_line_ptr

The real problem is not the arithmetic — it is that what to compute depends on
a version the host declares. A knob the verifier must consult is a knob the
host can lie about.

Fix: normalize on both sides, for new images

• Image build — os/image/normalize-kernel-header.py zeroes the
  boot-loader-written fields in the shipped bzImage, wired into
  os/image/assemble.sh so both the Yocto and mkosi backends get it, and
  records "kernel_header_normalized": true in metadata.json.
• Firmware — 0007-OvmfPkg-QemuKernelLoaderFsDxe-normalize-setup-header.patch
  zeroes them again in the kernel blob, before it is measured and loaded.
• dstack-mr — for such an image RTMR[1] is a plain Authenticode hash of the
  kernel file, on every QEMU version and at every guest memory size. That
  lifts the memory_size == 2 GiB || >= 0xB0000000 restriction on the
  no-image-download path.

Normalization is idempotent, so it lands on the same bytes whether or not QEMU
rewrote anything.

The field set comes from the boot protocol, not from QEMU

Zero every field Documentation/arch/x86/boot.rst types as write (the
boot loader fills it in, the kernel supplies no value), and clear
CAN_USE_HEAP in loadflags. Leave every modify field alone.

That decouples the rule from QEMU's behavior, so it does not go stale when QEMU
changes, and it rules out the obvious trap: modify fields carry real
kernel-supplied values, and code32_start (0x214 = 0x100000) is the
protected-mode entry point.

It changes two bytes of the kernel

Every write field is already zero in a freshly built bzImage except
heap_end_ptr (0x224 = 0x50a0). Zeroing it is safe on every boot path:

• boot.rst types it write (obligatory): the boot loader writes it, the
  kernel supplies no value.
• init_heap() reads it only when the boot loader has set CAN_USE_HEAP,
  which the kernel builds clear and this also clears.
• On the EFI-stub path the real-mode setup code never runs.
• The PE headers occupy 0x40..0x170; no setup-header field overlaps them.

Secure Boot is not planned, so there is no signature to invalidate.

Existing images are untouched

dstack-mr is embedded in KMS (CvmVerifier), so it must keep verifying every
CVM that is already deployed. Those images ship an OVMF that does not normalize,
and on QEMU ≤ 10.1 — which is all of production today — their RTMR[1] covers
QEMU's rewritten copy. That path is unchanged, including patch_kernel() and
the memory-size restriction that goes with it.

Regenerating the measurement for a currently shipped image reproduces the
document it already carries byte for byte, so os_image_hash and every
on-chain whitelist entry stay valid:

$ dstack-mr tdx-measurement-cbor ~/.dstack/images/dstack-0.6.0 | sha256sum
da94efbb4437fec1986b91ae2815180c6a7d779e778bb64135bfe4bda8a4bc2e
$ sha256sum ~/.dstack/images/dstack-0.6.0/measurement.tdx.cbor
da94efbb4437fec1986b91ae2815180c6a7d779e778bb64135bfe4bda8a4bc2e

Which behavior applies is declared by the image, never by the host:

path	signal	read by
image download	kernel_header_normalized in metadata.json	ImageInfo → Machine::normalized_setup_header
no image download (lite)	the same flag, mirrored into the measurement document	TdxOsImageMeasurement

The flag is omitted from the CBOR when false, so pre-normalization documents
encode exactly as they always did — the document version does not move.

An old image on QEMU ≥ 10.2 stays broken, exactly as it is today; the remedy is
to re-emit it, which is the direction this PR exists to enable.

Verification

Real TDX hardware, both QEMU generations. The same image was booted as a
live CVM on a QEMU 8.2.2 host and on a QEMU 10.2.1 host:

	QEMU 8.2.2	QEMU 10.2.1
MRTD	78cb3ad7…	8ed3f63f…	differ — page-add ordering
RTMR0	68102e7b…	131221a6…	differ — generated ACPI tables
RTMR1	60db95c7…8f79d200	60db95c7…8f79d200	identical
RTMR2	f873ce9b…	f873ce9b…	identical

8.2.2 is the load-bearing case: it does rewrite the setup header, so passing
there proves the firmware undid a real rewrite rather than merely agreeing with
itself. 10.2.1 then shows the digest did not move. On both hosts the first
RTMR[1] event is 880962f4…eccaf30 — the Authenticode SHA-384 of the shipped
bzImage, cross-checked with an independent implementation — and each CVM's own
CCEL event log replays to exactly the RTMRs in its quote.

dstack-verifier accepts both, over the no-image-download path with an
unreachable download URL:

$ dstack-verifier --config verifier-no-download.toml --verify qemu-10-2-capture.json
{"is_valid":true,"details":{"quote_verified":true,"event_log_verified":true,
 "os_image_hash_verified":true,"acpi_tables_verified":true,...}}

Full mkosi image build (./os/mkosi/build.sh image) passes end to end:
the OVMF patch applies and the firmware rebuilds, assemble.sh invokes the
normalization, and the resulting image's kernel, measurement document and
sha256sum.txt are self-consistent.

Offline, normalize(QEMU's rewrite at 1 GiB / 2 GiB / 2.5 GiB / 0xB0000000 / 768 GiB) == normalize(original) for all five, and normalization is idempotent.
That is what lets the memory-size restriction go for normalized images.

New tests:

• os/tests/test-kernel-header-normalization.sh parses the field table out of
  both the image-build script and the OVMF patch and compares them. If they
  ever drift, every CVM fails on RTMR[1] with nothing pointing at why. It also
  checks the modify fields survive, that normalization is idempotent, and
  that a non-bzImage or a truncated one is rejected at build time while OVMF
  leaves a non-bzImage alone. It runs from the mkosi acceptance suite, i.e.
  from build.sh lint and the existing mkosi-build workflow.
• dstack-mr: the flag actually reaches the digest — the two branches select
  different kernel bytes, and only the pre-normalization one moves with guest
  RAM.
• dstack-types: ImageInfo parses the flag out of metadata.json, and
  treats its absence as the pre-normalization behavior. That is the field the
  image-download path reads; losing it there is a silent attestation failure.
• dstack-types: the flag round-trips, and a pre-normalization document does
  not drift — the flag stays absent and the bytes are unchanged.
• tdx-lite-normalized-attestation.json and
  tdx-lite-normalized-qemu-10-2-attestation.json pin both captures above, so
  the version-independence property is checked in CI rather than asserted.
  The two existing lite fixtures are unchanged and still pass, which is the
  regression check for deployed images.
• os/mkosi/tests/acceptance.sh asserts the patch stays applied and the
  assemble hook stays wired.

Relationship to #1183

#1183 fixes the same bug by recording both digests in the measurement document
and picking one from the host-declared vm_config.qemu_version. It is the
right shape if old images must keep working on QEMU ≥ 10.2.

This PR instead makes the question disappear for new images, and leaves old
images exactly as they are. The two are compatible: if #1183 lands first, this
builds on it and old images gain 10.2 support too.

Drive-by

tools/vm-runner could not launch current images at all — its rootfs check only
accepted rootfs.img.verity, so rootfs.img.parted.verity failed with
"Unsupported rootfs image format" before QEMU started. It also omitted
qemu_version from vm_config, which makes the verifier fall back to 9.1.0 and
pick the wrong MEM.PAGE.ADD ordering on a QEMU 8 host. Both fixed; the hardware
capture above went through it.

Full workspace suite, clippy -D warnings and prek are clean. Follow-up
cleanup tracked in #1185.

[Copilot]

🔵 Needs a closer look

It changes firmware measurement behavior and core TDX attestation semantics (RTMR[1]) across multiple components, which warrants careful human security review despite the added tests/fixtures.

Pull request overview

This PR makes TDX RTMR[1] (kernel measurement) independent of host QEMU version by normalizing the Linux setup header both in the shipped kernel image and in OVMF right before measurement/loading. It then simplifies dstack-mr to treat RTMR[1] as the plain Authenticode SHA-384 of the shipped bzImage, updates measurement/document versions, and refreshes verifier fixtures accordingly.

Changes:

• Normalize bootloader-owned Linux setup-header fields in the shipped bzImage (build-time) and again in OVMF (measurement-time) to eliminate QEMU-dependent bytes.
• Simplify dstack-mr kernel measurement logic to hash the kernel “as shipped” (dropping QEMU-header-patching logic and related memory-size constraints).
• Update documentation, tests, fixtures, and tools/vm-runner behavior to match the new measurement semantics.

File summaries

File	Description
tools/vm-runner/vm-runner.py	Emits qemu_version into vm_config when detectable; accepts newer rootfs .verity naming.
os/yocto/layers/meta-dstack/recipes-core/dstack-ovmf/dstack-ovmf/0007-OvmfPkg-QemuKernelLoaderFsDxe-normalize-setup-header.patch	OVMF-side setup-header normalization before measuring/loading the kernel blob.
os/yocto/layers/meta-dstack/recipes-core/dstack-ovmf/dstack-ovmf_git.bb	Wires the new OVMF patch into the Yocto build.
os/tests/test-kernel-header-normalization.sh	Cross-checks the normalization field table between the Python script and the OVMF patch; exercises idempotence and invariants.
os/mkosi/tests/acceptance.sh	Ensures mkosi build wiring keeps applying the OVMF patch and the assemble hook.
os/mkosi/components/ovmf/ovmf.sh	Updates component cache key inputs to include the new OVMF patch.
os/mkosi/components/ovmf/ovmf-build.sh	Applies the new OVMF patch in the mkosi OVMF build flow.
os/image/README.md	Documents why the shipped kernel differs from raw build output (normalization rationale and invariants).
os/image/normalize-kernel-header.py	Build-time normalization implementation for bzImage setup-header fields.
os/image/assemble.sh	Hooks normalization into image assembly before measurements/checksums are computed.
dstack/verifier/src/verification.rs	Bumps measurement cache version to invalidate stale cached RTMR computations.
dstack/verifier/fixtures/tdx-lite.README.md	Updates fixture provenance/assumptions for normalized-kernel measurements.
dstack/verifier/fixtures/tdx-lite-getquote.json	Refreshes lite fixture capture content for the new measurement behavior.
dstack/verifier/fixtures/tdx-lite-attestation.json	Refreshes lite fixture capture content for the new measurement behavior.
dstack/dstack-types/src/lib.rs	Updates TDX measurement document version/semantics documentation (v3 → v4 meaning shift).
dstack/dstack-mr/src/tdx.rs	Switches RTMR[1] computation to hash shipped kernel; adjusts memory-split constant naming/usage.
dstack/dstack-mr/src/main.rs	Updates CLI help text related to measurement split feature wording.
dstack/dstack-mr/src/machine.rs	Updates call sites for the simplified rtmr1_log API.
dstack/dstack-mr/src/kernel.rs	Removes QEMU kernel-header patching logic; RTMR[1] now measures the shipped kernel hash.
docs/security/security-model.md	Documents the removal of the host-declared QEMU dependency for kernel measurement.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.