Krun microVM backends

maelys-warden-krun-driver is an optional Apple Silicon helper for running a Linux workload in a libkrun microVM. It is packaged independently so the portable C library and the core maelys-warden installation never acquire a libkrun, firmware or Hypervisor dependency.

The release contract is deliberately narrow:

• macOS 15 or newer on Apple Silicon;
• libkrun 1.19.4, linked directly through a stable Homebrew opt path;
• a fail-closed 60-GiB candidate ceiling while that release uses HVF's default 36-bit IPA. The global ABI 10 ceiling remains 64 GiB for direct Containerization; Krun rejects the unsupported value during preparation;
• la bottle Warden maelys-warden-libkrun, construite avec les seules features blk,external-kernel-only. Le patch versionné supprime à la compilation le chargement de libkrunfw et refuse un démarrage sans noyau externe ;
• maelys-warden-guest-kernel, le noyau Linux/arm64 brut commun également chargé par le backend Containerization direct ; Krun le sélectionne avec krun_set_kernel(..., KRUN_KERNEL_FORMAT_RAW, ...) et n'exécute plus le noyau qui est l'unique noyau admissible ;
• the shared, reproducible Linux/arm64 Warden guest bundle, including the content-pinned maelys-init inserted into the deterministic ext4 root by Warden's materializer;
• Transparent Socket Impersonation disabled before the explicit vsock is created;
• an ad-hoc Apple Silicon signature carrying the Hypervisor and dynamic library validation entitlements; no paid Developer ID is required.

Warden does not copy libkrun into its driver archive. Homebrew owns that dependency, its upgrades and its license metadata. The released driver uses maelys-warden-libkrun, a headless build containing the upstream blk feature plus Warden's compile-time external-kernel-only feature. The official libkrun/krun/libkrun formula remains supported as the explicit upstream-gpu runtime variant; it is not silently substituted for the headless release closure.

Install

Install the Warden-owned headless runtime and optional driver:

brew install maelys-dev/tap/maelys-warden-libkrun
brew install maelys-dev/tap/maelys-warden-krun-driver

maelys-warden-libkrun has arm64 Sequoia and Tahoe bottles. Its source formula applies the reviewed, digest-pinned external-kernel-only patch, then builds upstream libkrun with --no-default-features --features blk,external-kernel-only; it does not link libkrunfw, libepoxy, virglrenderer or MoltenVK. The public libkrun ABI still contains GPU entry points, but they do not imply that the graphics implementation or its dylibs are present. otool -L, not the symbol table, is the normative dynamic-closure check.

The Warden formula remains usable without this package. Optional discovery and the ABI 10 candidate adapter canonicalize and hash the driver, package manifest, common guest manifest, static guest init, static Bubblewrap, common raw guest kernel and the complete selected pinned dylib closure. The default headless closure contains only libkrun. An explicitly packaged upstream-GPU closure retains libkrunfw, libepoxy, virglrenderer and MoltenVK and the same generated Seatbelt profile machinery. Discovery validates the exact MWKR-PROBE/8 proof and accepts only digest-pinned OCI linux/arm64 targets. K5 connects the sealed K4 ext4 artifact to the K2 MWDP supervisor and admits only its exact capability subset: view mappings, tmpfs, captured stdio and network NONE. K6 adds a separate krun-bubblewrap candidate for recursive filesystem access rules. K7 lets both candidates expose the standard proxy frontend through a pinned guest relay and the unique host Egress enforcement engine. K8 wraps the host VMM driver in Warden's behaviorally proven deny-default Seatbelt profile. There is deliberately no low-level workload command line to build application integrations around; callers use TrustedRunSpec through the Warden CLI, C facade or SDKs.

Default discovery checks the driver beside Warden and the documented Homebrew/system prefixes. Absence or an unsupported package is harmless. An explicit MAELYS_KRUN_DRIVER=/absolute/path is different: an invalid package is reported as an error and never downgraded to another backend.

Deux fermetures libkrun, un seul compilateur Seatbelt

Le profil hôte n'est ni un fichier SBPL écrit à la main pour la variante headless, ni du code dupliqué. Le backend C transforme d'abord le manifeste du paquet en une fermeture normalisée et épinglée :

Le générateur C parcourt la fermeture effective. Il accorde uniquement file-read* et file-map-executable aux répertoires des dylibs réellement admises et lie chaque chemin canonique à un paramètre sandbox-exec -D. La variante v4 produit donc une entrée runtime ; le manifeste v1 historique en produit cinq et conserve intégralement le profil nécessaire à la formule officielle GPU. Modifier le nombre de bibliothèques ne change ni la politique de réseau, ni les droits sur le driver, le firmware, le rootfs ou les sockets privées.

La sélection est fail-closed : un manifeste v3 ne peut pas nommer une dylib graphique, un manifeste v1 doit fournir les trois versions graphiques exactes, et les deux variantes doivent lier le manifeste du noyau commun. Toute identité ou tout digest modifié invalide le candidat avant le démarrage de la VM.

Host/guest lifecycle

The driver creates an AF_UNIX listener before entering the VM and records the node's device/inode. krun_add_vsock_port maps guest port 4519 to that listener and maps three different byte streams on 4521–4523. maelys-init connects to host CID 2, becomes a child subreaper, creates a new process group and arms PR_SET_PDEATHSIG. For the simple krun candidate, it sends MI/6 READY only after an explicit host MI/6 RUN and a close-on-exec error pipe prove that the workload execve() succeeded; a missing executable therefore produces an explicit bounded MI/6 ERROR workload-exec-<errno> instead of a false READY. For krun-bubblewrap, the same pipe proves that the trusted Bubblewrap process was admitted. Bubblewrap then performs the workload exec, so a missing executable is reported after READY as the exact nonzero terminal outcome of the pinned Bubblewrap supervisor (MI/6 EXIT 1 0 today), without inventing a synthetic workload status.

guest -> host  MI/6 HELLO
host  -> guest MI/6 SPEC ... root-sha256
host  -> guest MI/6 RUN
guest -> host  MI/6 READY root-sha256
guest -> host  MI/6 ERROR bounded-reason
host  -> guest MI/6 SIGNAL 15
host  -> guest MI/6 STOP
guest -> host  MI/6 EXIT 0 0 user-usec system-usec maxrss-kib scratch-bytes root-mode cleanup

HELLO prouve uniquement que le PID 1 scellé a démarré. SPEC transporte le RunSpec borné et le SHA-256 attendu du rootfs. L'invité relit le bloc /dev/vda et refuse tout écart avant RUN. RUN est l'autorisation explicite de créer le workload. READY est plus tardif et plus fort : il renvoie le digest réellement mesuré, que le driver compare au digest attendu ; the workload (or pinned Bubblewrap supervisor) passed preflight and exec admission. These states are never collapsed into one message.

Before workload admission, relay failures use MI/6 ERROR egress-relay-<state>. The one-byte state identifies only the bounded startup stage (P upstream preflight, L loopback listener, A allocation, E event loop, W watch registration, or a deadline/transport failure); it never carries a path, credential, destination or policy value.

The three stdio sockets are kernel-bounded streams with native backpressure; there is no line protocol, transcoding or merged stdout/stderr channel. Kernel printk and PID-1 diagnostics remain on the separate virtio console and cannot corrupt workload output.

The guest bootstrap arguments use the private --maelys-workload delimiter; the generic -- token is reserved by libkrun's built-in init path and would truncate the workload argv before it reaches maelys-init.

EOF on the Warden control channel kills the workload group. EOF on the stdin stream is only workload stdin EOF. SIGTERM/SIGINT received by the host is forwarded; after five seconds the host requests STOP and kills the VMM worker. The Unix socket is unlinked only if device/inode still identify the node it created. The VMM worker also installs a detached EVFILT_PROC/NOTE_EXIT watcher for its driver parent. If that parent disappears—even through uncatchable SIGKILL—the worker exits with 137 instead of leaving an orphan microVM consuming host resources. The expected parent PID is captured before fork() and checked again after the kqueue watch is armed, closing the reparent-to-launchd window.

MI/6 is newline-delimited. The guest parser is incremental: it retains a partial command across reads, processes several coalesced commands in order, bounds buffered input and kills the workload on malformed or oversized input. MI/6 remains private to the VM boundary. K2 wraps the host Warden-to-driver boundary in the already shared, bounded MWDP/1 framing; it does not replace the guest protocol with a second interpretation of MWDP.

Maintainer build and package

The host binary is intentionally outside make all:

brew install pkg-config
make krun-driver-check
make krun-driver-analyze

Build the complete common guest bundle after explicitly preparing its pinned builder image:

make warden-guest-builder
make warden-guest-bundle
make warden-guest-bundle-check

The physical test root must be a K4 materialized ext4 containing the exact verified Warden /maelys-init; warden-guest-bundle-check verifies its source bundle before the gate runs. Exercise the real Hypervisor.framework/vsock/lifecycle path—not only the fake-library gate—with:

make test-krun-e2e KRUN_TEST_ROOTFS=/absolute/path/to/rootfs

For a release, sign the driver with driver/krun/maelys-warden-krun-driver.entitlements, then run:

scripts/package-krun-driver.sh \
  build/release/bin/maelys-warden-krun-driver

The packager refuses a wrong architecture, wrong dependency version, unstable link path, missing entitlement, invalid or non-ad-hoc signature, or an invalid common guest bundle. It emits the archive, adjacent checksum and a requirements.json binding the exact driver and guest-manifest digests.

Package proof and private driver protocol

The proof line remains available to release CI:

maelys-warden-krun-driver --probe --proof-fd 3 3>&1

Expected proof:

MWKR-PROBE/8 api=1 payload=11 kernel=external-raw-warden init=maelys-pid1 mi=6 root-measure=sha256 rusage=usec,maxrss-kib root-overlay=tmpfs-bounded tsi=disabled-vsock root-disk=ext4-ro virtiofs4=linux-simplified inner-bwrap=sealed-root egress-vsock=4520 stdio-vsock=4521,4522,4523 network-total=validated

linux-simplified is intentional for host workspace views. macOS user IDs are not Linux guest identities: preserving a host UID such as 501 makes a private 0700 workspace inaccessible after Bubblewrap enters its user namespace. libkrun's simplified mode presents each object as owned by the requesting guest identity while retaining host permission bits. Warden remains the authority for which canonical host directory is exposed and whether its VirtioFS device is read-only or read-write. The sealed ext4 root continues to preserve the OCI UID/GID metadata exactly; this choice affects only explicit host workspace views.

Every local relink of the production driver is followed by an ad-hoc signature using the repository entitlement file. The build then verifies com.apple.security.hypervisor and com.apple.security.cs.disable-library-validation before declaring the target complete. Release signing may replace that ad-hoc signature, but an unsigned or entitlement-free E2E driver is never produced silently.

Normal execution accepts only --control-fd 4. That descriptor is a private AF_UNIX stream between Warden and the driver; sealed root identity, workload argv, environment, mount table, guest-kernel identity and lifecycle commands travel in bounded MWDP/1 frames. The driver repeats root and view identity checks before configuring libkrun. Il revalide aussi le chemin canonique, device/inode, mode, taille et SHA-256 du noyau, puis appelle krun_set_kernel(..., KRUN_KERNEL_FORMAT_RAW, ...). The trusted init is already present in the ext4 and its digest is bound by both artifact.seal and the root digest; an image-supplied file at that path was removed during materialization. These values never appear in the driver's argv. The driver emits no private receipt: Warden is the sole owner of the public execution receipts v4/v5 and validates MWDP/1 EXIT against the actual driver process status.

MI/6 remains the separate, internal driver↔maelys-init protocol over the AF_UNIX/vsock mapping. K1 exposes the ABI 8 backend and seals package evidence; K2 supplies the operational outer supervisor. The backend's prepare operation admits only a sealed K4 root artifact; K5 lowers that artifact and its guest views, then reports the workload outcome through correlated READY/EXIT frames.

Exact security boundary of the K5 candidate

libkrun documents that its VMM shares the caller's host security context. The K5 candidate is nevertheless selectable because Warden seals and revalidates the OCI/ext4 artifact, disables ambient guest networking, supervises the VM domain and records the exact Krun mechanism in receipt v3. The K5 release did not claim that the host VMM process was deny-default confined; the current K8 contract adds that independent Seatbelt defense-in-depth layer. Likewise, Egress mediation is an explicit capability delivered by K7, while inner Bubblewrap belongs only to the distinct K6 candidate. Both remain absent from the K5 capability mask. There is no silent promotion of the restricted krun candidate to those stronger contracts.

Audit against the operational Apple backend

This section records the implementation audit performed against the original feat/production-krun-driver lineage and the completed Executor ABI 10 Apple backend. The middle column reports the shipped 0.39.0 state; the last column keeps the original K5-relative reading of what each boundary still required, and notes where that work has since landed.

The current code is therefore useful rather than disposable. Its libkrun configuration, TSI disabling, device/inode-safe socket cleanup, guest subreaper, process-group termination, parent-death watcher, packaging and physical E2E fixture form the lower half of the final backend. K1–K4 now provide the Warden-facing adapter, common guest trust bundle, artifact contract and K5 execution bridge between that sealed artifact and the VM.

Target architecture

There are still three protocols and descriptor spaces:

1. MWDP/1 is host Warden to host driver on the driver's private fd 4;
2. MI/6 is host driver to guest maelys-init over vsock 4519;
3. application fd 3 and network fd 4 are workload conventions and are not exposed by the first Krun release.

The Krun product surface supports captured stdio, network NONE and the Egress proxy frontend. A Warden-owned guest relay crosses AF_VSOCK port 4520 to the execution-private Egress AF_UNIX listener. The Maelys-aware fd4 frontend remains rejected because Unix descriptor passing cannot cross the VM kernel boundary. ISOLATED_DUPLEX also remains rejected until a guest byte-stream protocol is specified.

Decisions fixed before implementation

Executor ABI 10 is the resource contract

Krun uses the same TargetSpec, target-aware probe, prepared artifact and receipt machinery as the other OCI candidates. ABI 9 added the canonical root disposition and scratch cleanup; ABI 10 adds an opaque, digest-bound execution budget. Krun applies its vCPU/memory fields through libkrun and its workload fields through the common MI/6 cgroup-v2 runtime. No backend-specific public vtable is needed.

Warden owns the private Krun VMM worker, but receipt v7 deliberately records its host-controller evidence as applied: false, scope krun-vmm-worker and source not-applicable. Physical macOS 26 gates proved that a finite RLIMIT_CPU is incompatible with libkrun 1.19.4: applying it before hv_vm_create() prevents guest readiness, while applying it after the guest control connection stalls the MI/6 exchange. The implementation and its controller handshake were therefore removed before release rather than attesting a mechanism that does not work. A future host watchdog must be implemented and physically proven before this receipt field may become true.

The first target contract is intentionally narrow:

kind:       OCI
platform:   linux/arm64
execution:  native
image:      immutable sha256 digest

linux/amd64 is rejected during probe_target, before prepare. libkrun on Apple Silicon does not provide Rosetta for Linux, and a target must never be relabelled arm64 merely because the host driver is arm64.

OCI materialization belongs above the driver

The existing --rootfs DIR interface is not an acceptable public target: directory ownership and mode checks do not bind all descendants, OCI whiteouts, hardlinks, xattrs or later mutations. The driver also must not shell out to Docker, Colima or Apple Container. Doing so would make candidate semantics depend on an unrelated daemon and would move mutable resolution past planning.

Warden therefore needs a content-addressed OCI store with this pipeline:

REFERENCE@sha256 + linux/arm64
        |
        v
verify index/manifest/config/layer descriptor digests
        |
        v
stream layers into a content-addressed logical inode graph
  - use a bounded path hash index; whiteouts leave tombstones until a bounded
    deterministic compaction instead of shifting the full graph per deletion
  - reject absolute paths and `..`
  - implement OCI whiteouts exactly
  - never resolve an archive-created symlink through the host filesystem
  - validate hardlink targets inside the logical Linux namespace
  - bound files, inodes, bytes, path length and expansion ratio
  - preserve Linux case, Unicode bytes, uid/gid and admitted mode semantics
  - reject xattrs, sparse/global-PAX records and every special file in v1
  - clear setuid/setgid unless a later policy explicitly opts in
        |
        v
write directory entries and inodes with a pinned libext2fs materializer
  - no host-filesystem staging tree and no `mke2fs -d APFS_DIRECTORY`
  - no journal; fixed UUID, timestamps, hash seed and feature set
  - canonical inode/allocation order derived from the logical graph
  - bounded image size; every unused block is deterministically zeroed
        |
        v
read-only raw ext4 artifact + immutable metadata
        |
        v
atomic rename into store/objects/<manifest-hex>/<os>-<architecture>

The macOS implementation deliberately does not ask libarchive to interpret tar paths. Libarchive supplies only the bounded decompression filters; Warden's raw tar reader constructs the logical Linux graph from the original path bytes. This separation is load-bearing because the macOS libarchive pathname layer normalizes Unicode. No named APFS root tree is ever created, and Warden never mounts an attacker-authored filesystem image: it mounts only ext4 structures that its pinned materializer generated from the validated graph.

An OCI index produced by Docker Buildx may also contain provenance attestation manifests, using the descriptor contract documented by BuildKit. Warden ignores an entry only when it has the exact non-runnable Buildx shape: OCI/Docker image-manifest media type, platform unknown/unknown, annotation vnd.docker.reference.type=attestation-manifest, and a valid vnd.docker.reference.digest. An arbitrary artifact, an incomplete attestation marker or an unknown descriptor remains a hard error. Attestation payloads are not execution targets and are not interpreted as proof by the offline importer.

The public CLI auto-detects an OCI image-layout directory or an OCI archive. K4 delivers the deliberately offline surface:

maelys-warden image inspect ./tool.oci.tar
maelys-warden image import ./tool.oci.tar --platform linux/arm64

The user does not calculate or type a digest during the normal single-image import. Warden verifies every OCI descriptor, computes the manifest and root artifact digests, and prints the immutable reference to use with run. If the source contains several manifests or platforms, import fails and asks for an explicit selector discovered with image inspect:

maelys-warden image import ./bundle.oci --platform linux/arm64 \
  --digest sha256:0123...

--store /absolute/path is an advanced helper option used by tests and integrators; the public CLI normally uses $MAELYS_WARDEN_OCI_STORE, then $XDG_DATA_HOME/maelys-warden/oci, then $HOME/.local/share/maelys-warden/oci. The store and its private descendants must be owned by root or the invoking user and use mode 0700.

--oci-layout is not a public format switch: content detection is structural. --image is not used to invent provenance for local bytes. A registry reference may be retained as non-authoritative metadata, but only verified digests identify the imported artifact. Online image pull later writes into the same store and requires --platform whenever the registry index is multi-platform.

The product surface distinguishes offline import from later online registry work. Store inspection and removal were completed by K9:

# delivered since K4
maelys-warden image inspect SOURCE
maelys-warden image import SOURCE [--platform PLATFORM] [--digest DIGEST]
maelys-warden image list
maelys-warden image remove IMMUTABLE-REFERENCE

# not accepted by the current CLI
maelys-warden image pull REFERENCE [--platform PLATFORM]

inspect never mutates the store. import never contacts a registry. pull may contact a registry and stores the resolved digest, never a mutable tag as authority. A platform may be inferred only when the verified source contains exactly one platform; host architecture is not used to guess.

Each imported directory contains:

objects/<manifest-sha256-without-prefix>/<os>-<architecture>/
├── root.ext4       # deterministic raw filesystem, mode 0400
├── artifact.json   # audit metadata, mode 0600
└── artifact.seal   # minimal machine-verifiable launch contract, mode 0600

This is also the retention contract. Compressed descriptors and temporary file-content blobs are verified and consumed inside the private staging directory, then deleted and the parent directory synchronized before the atomic rename. They are never part of the published store object. The ordered manifest, config and layer digests retained in artifact.json are sufficient to identify and reproduce the source closure; re-materialization requires the original OCI source.

artifact.json records the manifest/config/layer/root digests, the trusted Warden init digest, platform, logical entry count, resource ceilings, materializer identity and deliberate transformations. The compact seal is what the backend parses:

MWOCI/3
manifest=sha256:<64 lowercase hex>
config=sha256:<64 lowercase hex>
init=sha256:<64 lowercase hex>
bwrap=sha256:<64 lowercase hex>
relay=sha256:<64 lowercase hex>
platform=linux/arm64
root=sha256:<64 lowercase hex>
root-size=<decimal bytes>
materializer=maelys-warden-oci-materializer/5

Ces deux nombres ne versionnent pas le même contrat : MWOCI/3 est la version de l'enveloppe du seal ; /4 est l'identité de l'algorithme matérialiseur qui a produit et validé le filesystem ext4. Le consommateur vérifie les deux.

The materializer removes all three reserved paths from the image graph and injects the Warden-owned static arm64 /maelys-init, /maelys-bwrap and /maelys-guest-supervisor. The guest manifest, seal, ext4 root digest and candidate evidence bind the same helper bytes. Image-supplied helpers are therefore never executable by K6/K7.

The seal carries a stable materializer-format identity, not the Warden product version. Concrete Warden and libext2fs versions remain in artifact.json for audit, while compatible Warden upgrades can consume the same sealed root. An unknown materializer identity is rejected with an explicit re-import diagnostic. Publication fsyncs the root and every affected directory before reporting the immutable artifact.

Warden 0.30 advances that identity from /3 to /4: the ext4 writer now accounts for every reserved inode and requires an independently clean e2fsck -fn result. Pre-0.30 roots are rejected and must be re-imported rather than silently inheriting the weaker structural proof.

maelys-warden run --image ... consumes only an already imported sealed artifact and fails with an actionable error when it is absent. K4 prepare opens root and seal with O_NOFOLLOW, verifies owner/mode, hashes the opened files and pins device, inode, mode, size and digest for both. K5 spawn reopens and revalidates those identities, then passes the already-pinned raw artifact identity in the launch payload. The driver repeats the checks before krun_add_disk(..., read_only=true) and krun_set_root_disk_remount(..., "ext4", ...). libkrun's built-in init runs from a Warden-owned dummy virtiofs root, switches onto the sealed block root, then executes the Warden /maelys-init already contained in that root. Each trusted workspace mapping becomes a separate tagged krun_add_virtiofs4 device. maelys-init mounts only the workspace tags and guest destinations contained in the sealed payload, using component-wise no-symlink resolution. The guest receives an explicit working directory, argv and empty or explicit environment. krun_set_root() is not part of that production path.

This is a deliberate macOS security decision. An ordinary APFS volume is usually case-insensitive and applies Unicode normalization rules that differ from Linux. Extracting README and readme, or two byte-distinct normalized names, can otherwise authenticate a deterministic but semantically wrong tree. A raw Linux filesystem also preserves OCI uid/gid faithfully; a non-root macOS extractor cannot. If the deterministic ext4 materializer is not available or cannot prove these properties, the candidate is unavailable—no fallback extracts a production rootfs onto ordinary APFS.

This prohibition includes intermediate staging: layer entries are never materialized as named files on APFS. Pathnames exist only as validated Linux metadata in the content-addressed logical graph and, finally, as ext4 directory entries. A shortcut such as mke2fs -d over an APFS directory is forbidden because it would reintroduce case and normalization collisions before ext4 is created. The journal is disabled for this read-only root; UUID, timestamps, directory hash seed, ext4 feature flags and inode/allocation order are fixed so equivalent imports can be byte-identical.

Warden never accepts or mounts an attacker-authored filesystem image. It constructs ext4 structures itself from the validated logical graph, so untrusted influence is limited to admitted names, contents, sizes and metadata values—not raw superblock, bitmap, inode or directory-record bytes. This also reduces the attack surface presented to the guest kernel's ext4 parser: an inner-kernel compromise could bypass Bubblewrap and invalidate the composite candidate's FS_ACCESS_RULES, even though the outer VM boundary would remain.

The implementation uses the mature libext2fs userspace API from e2fsprogs; it does not implement an ext4 writer from scratch. The dependency version, build identity and selected ext4 feature set are pinned and recorded in the materializer identity. Absence or mismatch makes the candidate unavailable. Since Warden 0.30, the parser, logical graph, helper substitution and writer are compiled into the private deterministic libmaelys-oci.a; the installed materializer command is a thin adapter and the archive is not a public ABI. Every generated test root must pass e2fsck -fn, independent read-only debugfs assertions and the physical K9 Linux boot gate. See OCI materialization architecture.

The v1 materializer rejects block/character devices, FIFOs and sockets, and clears setuid/setgid bits while recording that transformation in the artifact metadata and digest input. Supporting those Linux image semantics later requires an explicit policy and a new materializer contract, never an implicit change.

Guest helpers are trusted Warden inputs, never image authority

For the former directory-root gate, libkrun's memory-backed overlay API can replace /maelys-init. That is not sufficient for the production block-root path: libkrun switches away from the dummy virtiofs root before it executes the requested init, so files in that overlay are no longer visible.

K4 therefore removes any image-provided maelys-init node from the logical Linux graph immediately before ext4 materialization and inserts the exact static arm64 init from the verified common Warden bundle. Its digest is bound both by the raw-root digest and explicitly by artifact.seal; the backend requires it to equal the pinned bundle digest. K5 lets libkrun's built-in init switch to that ext4 root and then executes /maelys-init. K6 applies the same image-independent trust rule to Bubblewrap. K7 extends it to the relay: MWOCI/3 removes an image-owned /maelys-guest-supervisor, inserts the pinned static helper, and binds its digest in the seal, guest manifest and evidence.

The existing package location already has the right generic name:

libexec/maelys-warden/guest/linux-arm64/
├── maelys-init
├── bwrap
├── maelys-guest-supervisor
├── probe-amd64
└── manifest.json

The Apple backend does not use maelys-init, but both drivers share the same reproducible build, manifest parser, digest validation and package provenance. There must not be a second Krun-only build of bwrap or relay.

Egress remains the only network enforcement engine

The driver continues to call krun_disable_implicit_vsock() followed by one explicit vsock device with zero TSI feature bits. It never enables TSI, virtio-net, passt, gvproxy or ambient guest IP networking.

For a mediated plan, Warden starts one authenticated Egress endpoint and maps a second guest port to its private AF_UNIX listener:

workload -> 127.0.0.1:private-port -> blind guest relay
         -> AF_VSOCK CID 2 port 4520 -> libkrun AF_UNIX mapping
         -> Egress HTTP/SOCKS listener -> allowed destination

The relay has no policy and no credential database. The workload receives secret-free HTTP_PROXY, HTTPS_PROXY, ALL_PROXY URLs and emptied NO_PROXY. The execution-private AF_UNIX endpoint selects an immutable Egress principal; Egress alone resolves DNS, pins addresses, applies the allowlist, enforces quotas and emits network receipts. The lifecycle channel on 4519 and the Egress stream on 4520 are never multiplexed.

The host VMM process needs its own confinement

The Hypervisor boundary protects the host kernel from the guest, but libkrun and its driver still run with the caller's macOS credentials. Warden must launch the driver under a dedicated Seatbelt profile that grants only:

• execution and mapping of the pinned driver and selected libkrun closure;
• read-only access to the sealed root artifact and helper bundle;
• access to the private execution directory and its exact AF_UNIX sockets;
• inherited stdio/control descriptors and required Hypervisor/I/O Kit/Mach services established by a physical behavioral probe;
• no ambient outbound network and no unrelated home/workspace access.

The profile is compiled by Warden from trusted paths. It is not supplied by the image or driver. If the deny-default profile cannot boot and destroy a real microVM, the Krun candidate is unavailable rather than launched without host confinement. This was delivered as K8 in 0.25.0 under the profile identity maelys.warden.krun-host-seatbelt/v1, which candidate evidence and receipt v3 initially bound; receipt v4 preserves it while adding the selected runtime-closure evidence.

Candidate guarantees

K6 implements the filesystem rows below, K7 implements proxy mediation and K8 confines the host VMM driver. Host Seatbelt is defense in depth around the VMM; it contributes no MIR capability bit of its own.

The simple candidate is eligible for coarse OCI policies. A MIR requiring portable filesystem access rules selects krun-bubblewrap or rejects; it never silently weakens to the simple candidate.

Racine éphémère en 0.35

Les deux candidats Krun abaissent ROOT_EPHEMERAL_WRITE dans le même maelys-init que Containerization direct. Le driver ne rend jamais le disque ext4 writable : il transporte uniquement le mode et la borne scellés.

Le gate physique écrit, renomme, hardlink et whiteout des entrées, remplit le tmpfs jusqu'à ENOSPC, force un workload, réadmet ensuite le même root et compare le SHA-256 du lower. Le receipt v5 porte la couche ephemeral-root-overlay; v4 reste disponible uniquement pour les exécutions readonly historiques.

Receipt v3 records mechanisms outermost to innermost:

krun:
  seatbelt-host -> libkrun-vm

krun-bubblewrap:
  seatbelt-host -> libkrun-vm -> bubblewrap

The guest init and relay digests belong to evidence and the network section. They are not advertised as independent isolation layers. Candidate-level effective capabilities remain normative because lifecycle and mediation are cross-layer guarantees.

Unlike Apple Container, the libkrun VM is structurally owned by the VMM worker process. Killing and reaping that worker returns Hypervisor resources to the host kernel; there is no independent service that may keep the domain alive. This makes PROCESS_TREE_LIFECYCLE attainable, and Warden now advertises it on both candidates because the parent-death, escalation and physical no-orphan gates below pass.

Implementation milestones

K-1 — Packaged prototype hardening — delivered in 0.19.0

This milestone is complete in commit ac8977a and release 0.19.0. It hardened the diagnostic driver before the ABI backend is built on it:

• never signal a worker PID after waitpid() has reaped it;
• capture the expected driver PID before fork(), then verify parent identity after arming the worker's kqueue watcher;
• create a fresh AF_VSOCK stream for every failed connect attempt;
• make receipt temporary-path truncation and descriptor ownership explicit;
• release a libkrun context if restoring the firmware working directory fails;
• send MI/6 HELLO, wait for explicit MI/6 RUN, and send MI/6 READY only after close-on-exec proves workload exec success on the then-simple candidate;
• parse partial and coalesced MI/6 commands incrementally with a hard bound.

Delivered gate: fake-driver lifecycle tests and the Linux guest-helper build pass; a missing workload on the simple candidate produces no READY, fragmented/coalesced control messages are handled deterministically, and killing the outer driver leaves no worker.

K0 — Freeze shared driver framing — implemented in 0.20.0

The former Apple-only frame codec is now the internal warden_driver_protocol module. The operational Apple backend and Swift driver speak MWDP/1; backend-specific launch payloads remain independently versioned. The production Krun executable is still diagnostic-only until K2, so K0 proves the second implementation with a fake Krun driver using the same role-aware state machine rather than pretending that a sealed Krun launch path already exists.

Delivered gate: exact big-endian golden frames, invalid flags, oversized and truncated payloads, fragmented/coalesced stream reads, deadlines and duplicate/out-of-order transitions are covered. Apple physical native, translated, graceful and forced lifecycle tests remain green; the fake Krun driver completes HELLO -> LAUNCH -> READY -> SIGNAL -> ACK -> STOP -> ACK -> EXIT.

K1 — Turn the executable into an ABI 8 backend — implemented in 0.21.0

Add src/backend_krun.c, maelys_executor_register_krun() and optional driver discovery beside the main executable, Homebrew opt prefix and documented system prefixes. The core never links libkrun. Registration canonicalizes and hashes the driver, guest manifest, init and selected runtime closure; it records exact versions and refuses unsafe ownership or writable components.

probe_target accepts only digest-pinned OCI linux/arm64 and returns native target evidence. The backend is configured as an OCI VM candidate with a measured cold-start cost. Missing package, unsupported host or failed proof is a non-fatal absence during default discovery; an explicit override fails loudly.

Merge gate: planner permutation, unobserved candidate, wrong platform, changed driver/library/helper and ABI mismatch tests all fail before prepare.

K2 — Replace the diagnostic CLI with MWDP supervision — implemented in 0.21.0

Retain --probe for packaging, but make normal execution accept only the private control fd. Move every current rootfs path, argv, explicit environment, admission timeout and VM resource setting into bounded binary launch payload v2. No secret is placed in argv, inherited ambient environment, logs or a driver-local receipt. K7 versions the backend-specific contract as payload 6 to carry the optional pinned Egress endpoint and bounded relay parameters.

The outer driver owns the worker running krun_start_enter; it handles control EOF as forced stop. Warden owns the deadlines:

SIGNAL -> MI/6 SIGNAL -> bounded grace
STOP   -> MI/6 STOP   -> kill VMM worker -> bounded reap
EOF    -> emergency STOP and cleanup

The worker's parent-death watcher remains the last-resort protection against an uncatchable outer-driver death. MI/6 EXIT is parsed and correlated with the worker status; disagreement makes outcome unknown, never fabricated.

On the simple candidate, the guest sends READY only after a close-on-exec error pipe proves the target exec succeeded. On the K6 composite, READY instead proves admission of the trusted Bubblewrap layer; the terminal EXIT still reports an inner workload exec failure exactly. Both MWDP/1 and MI/6 consume bounded frames from persistent buffers; neither assumes one read() equals one command. The K0/K9 gates below must preserve partial-frame, coalesced-frame, oversized-length and trailing-byte coverage as the private MI/6 path moves behind the shared driver machinery.

Delivered gate: normal exit, ready failure, a trailing-byte payload, graceful signal followed by forced stop, and controller EOF are driven through the production driver against a fake libkrun. The historical K2 payload v2 carried CPU, RAM and explicit environment values reach libkrun exactly. Every admitted run returns a launch-correlated EXIT; premature failure returns ERROR; the VMM worker is bounded and reaped. The former --run/driver-local-receipt surface has been removed. Physical Hypervisor/vsock coverage uses the same controller protocol. Payload 7 is the current K9 launch contract: it adds explicit MI/6 RUN admission and three private stdio streams to payload 6 without changing MWDP/1.

K3 — Build one common guest TCB bundle (delivered in 0.21.0)

The reproducible Warden guest bundle now contains static arm64 maelys-init, Bubblewrap, maelys-guest-supervisor and the amd64 probe under its current v4 manifest. The shared maelys-warden-oci-tools package owns this layout once; Apple Containerization direct and Krun bind its identities without copying it. Package checks reject dynamic ELF, wrong architecture, symlinks, unsafe modes and digest mutation. Krun historical launch payload v3 carried the verified init bytes and proved the memory-backed overlay path for the former directory-root gate. K4/K6 reuse the same bundle identity and insert the init and Bubblewrap into the sealed ext4 production root; payload 8 added the relay, optional Egress bridge, explicit RUN gate and private stdio transport. Payload 9 ajoute l'identité complète du noyau brut commun.

Merge gate: an image without init/bwrap/relay succeeds; image-provided files at the reserved paths are never executed; tampered bundle bytes fail before VM start; double builds reproduce the same manifest and artifacts.

K4 — Implement the offline OCI store (delivered in 0.21.0)

The shared optional OCI tools package now supplies the private store helper plus maelys-warden image inspect SOURCE and maelys-warden image import SOURCE. It auto-detects OCI layout directories and archives from their content, validates descriptor closure and safely applies layers through pinned libext2fs 1.47.4. The artifact records the manifest, platform, config, ordered layers, trusted init, raw-root identity, resource ceilings and deliberate transformations. Any image-provided /maelys-init is removed before the Warden-owned init is inserted. No mutable tag is stored as authority.

The Krun backend's prepare performs read-only lookup and copies the root and seal identities into its artifact. It performs no network request and starts no process. K5 spawn revalidates them immediately before LAUNCH. Package and guest-bundle dependency manifests are parsed as bounded JSON objects with duplicate keys and alternate escaped field spellings rejected; values in a comment-like string or unrelated subtree cannot satisfy a dependency binding.

Delivered gate: traversal, absolute path, symlink pivot, hardlink escape, malformed whiteout, digest mismatch, decompression bomb, partial import, root/seal post-prepare mutation, unknown media types, forbidden special files and ambiguous multi-image/platform input all fail closed. README/readme and byte-distinct Unicode normalization forms survive as separate ext4 entries. OCI uid/gid is written directly into ext4. No named APFS staging root exists. Equivalent imports produce byte-identical roots; unused blocks are zero and covered by the digest. Entry, inode, uncompressed-byte, expansion-ratio and total-image ceilings bound host allocation.

K5 — Lower rootfs and workspace views (implemented for 0.21.0)

The backend replaces krun_set_root() with a read-only raw root disk plus krun_set_root_disk_remount. It retains a Warden-owned dummy virtiofs root only for libkrun's built-in init and its required mount points; that init switches to ext4 and executes the sealed Warden /maelys-init. Each trusted workspace mapping is a separate tagged krun_add_virtiofs4 device. maelys-init mounts only the workspace tags and guest destinations contained in the sealed payload, using component-wise no-symlink resolution. It applies an explicit working directory, argv and empty or explicit environment; the driver environment is never inherited into the guest. Since 0.32, libkrun boots a fixed maelys-init --remote-spec command and payload 11 sends the canonical RunSpec over MI/6 after boot. Workload argv, environment and credentials no longer enter the observable kernel command line. Canonical UTF-8 strings, including quotes and control characters other than NUL, are bounded by the 1 MiB protocol frame rather than the historical 1024-byte cmdline budget.

Gate: rootfs write fails, read-only workspace write fails, read-write workspace succeeds, omitted paths are invisible, mount destinations cannot escape or mask the injected TCB, OCI uid/gid is visible as recorded, forbidden special metadata stays absent, source identity mutation is rejected, malformed or oversized MI/6 RunSpecs fail before VM admission, and the public CLI emits a receipt v7 whose ordered layers attest host Seatbelt, Krun and an optional ephemeral-root overlay, plus exact resource application and observations.

K6 — Add krun-bubblewrap (implemented in 0.23.0)

The distinct candidate uses the exact static arm64 Bubblewrap from the common guest bundle. Payload 5 introduced the bounded rule table it transports; the current payload 11 still carries it unchanged. maelys-init mounts trusted views first, runs the same Bubblewrap namespace and mount lowering against a trusted preflight child, then starts Bubblewrap and reports READY. For this composite, READY attests confinement admission, not successful exec of the image workload; a missing workload subsequently produces the exact nonzero Bubblewrap outcome (EXIT 1 for the currently pinned helper).

The inner layer deliberately does not use --unshare-net. NETWORK_NONE is provided by the outer VM configuration: no virtio-net device, TSI disabled and no route. AF_VSOCK is not scoped by Linux network namespaces. Without mediation, the only mapped port is lifecycle port 4519; its host listener accepts the single trusted maelys-init connection and is then closed, while the control socket itself is close-on-exec and never reaches Bubblewrap or the workload. With mediation, K7 adds and attests the distinct relay port 4520 outside the inner Bubblewrap network view; no ambient or inferred vsock mapping is allowed.

The composite mounts a fresh procfs with --proc /proc after entering its new PID namespace. It never exposes the ext4 root's inherited /proc view to the workload. Apple Container + Bubblewrap now provides the same semantic outcome after explicitly clearing the outer masked/read-only proc paths; Krun obtains it from its complete guest root without that Apple configuration step. If the mount fails, Bubblewrap admission fails before READY; there is no fallback to the inherited tree.

Only recursive READ/WRITE rules with ERROR or SKIP missing-path semantics are accepted. DENY, literal rules and PROTECT_CREATE fail before VM spawn. Automated gates cover image-helper replacement, helper digests, payload parsing, capability separation and unsupported rules. The physical Apple Silicon gate is:

make KRUN_TEST_ROOTFS=/absolute/root.ext4 test-krun-bubblewrap-e2e

It proves recursive workspace enforcement, a fresh readable procfs inside the new PID namespace, direct-egress denial and the documented READY followed by the exact nonzero Bubblewrap outcome for a missing composite workload.

K7 — Connect the existing Egress adapter through vsock (implemented in 0.24.0, hardened in 0.25.0)

maelys-init/relay bootstrap now has a bounded AF_VSOCK upstream mode and explicit ready/failure handshake. The Executor-Egress adapter creates the same per-execution endpoint-bound principal and listener contract as Apple, while the Krun driver maps the listener to fixed guest port 4520. The proxy URLs have no userinfo, so no secret enters the launch payload or guest kernel cmdline. Start order is Egress ready, VM ready, relay ready, then workload. Cleanup reverses that order and validates socket identity before unlink.

Delivered gates: standard HTTP and SOCKS clients work in integration coverage; the release-blocking Krun matrix additionally boots the intentionally minimal sealed OCI fixture and runs its static AF_INET HTTP probe. The probe validates the credential-free proxy environment, an authorized response and receipt; direct external connect and another loopback port fail; endpoint identity cannot be reused across executions; backpressure, half-close, Egress disappearance, per-principal connection quotas and relay saturation remain bounded. A workload seccomp filter also denies socket(AF_VSOCK) and all io_uring entry points before exec, while the sibling relay retains its required AF_VSOCK access.

K8 — Add host Seatbelt confinement and enrich receipts (implemented since 0.25.0)

Warden compiles and behaviorally probes the dedicated driver profile before advertising either Krun candidate. Candidate evidence binds the profile identity, driver, the selected complete pinned dylib closure, firmware, rootfs, guest bundle, Egress endpoint contract and exact libkrun proof. Frozen schema v3 keeps its historical GPU-closure contract for existing archives; new OCI executions no longer accept it. Receipt v4 records the normative outer-to-inner order seatbelt-host, krun, then optional bubblewrap, and additionally records the runtime variant, libkrun feature set, admitted component versions, closureEvidence digest and the common raw-kernel contract. The Krun layer records 2 vCPUs, 1024 MiB RAM and the aggregate helper evidence without private paths or credentials.

The receipt does not discover libraries by inspecting a mutable process after launch. Backend construction canonicalizes and hashes the files first; spawn revalidates their identities; the receipt snapshots the exact admitted closure and its evidence. With the released headless package it contains:

{
  "guestKernel": {
    "contract": "maelys.warden.guest-kernel/v1",
    "format": "raw-arm64",
    "loader": "libkrun-external",
    "evidence": "sha256:..."
  },
  "runtime": {
    "variant": "warden-headless",
    "features": ["blk", "external-kernel-only"],
    "components": [
      {"name": "libkrun", "version": "1.19.4"}
    ],
    "closureEvidence": "sha256:..."
  }
}

With an explicitly packaged official closure, variant becomes upstream-gpu and components also names libepoxy, virglrenderer and MoltenVK. Thus two executions using the same MIR but different runtime closures cannot have the same backend evidence.

Merge gate: a denied host file and denied direct host network are tested from a driver fixture; capability-name/mask mismatch, reordered layers and evidence mutation invalidate the receipt.

K9 — Product surface and release gates (delivered in 0.28.0)

The C facade, CLI and Python/Node SDKs already use an imported OCI artifact without adding Krun-only target semantics. --backend krun and --backend krun-bubblewrap remain exact diagnostic constraints; normal users let the planner choose. Document installation, image import, run, receipt and removal. K5 enables exact-proof auto-discovery; K9 completes the remaining image-management commands and full product matrix.

Required matrix on Apple Silicon:

captured-stdio is a product contract, not merely three fd arguments to krun_add_virtio_console_default: workload stdout is byte-exact and binary-clean, stderr remains separate, and neither kernel printk nor maelys-init diagnostics contaminate stdout. K9 uses private AF_VSOCK streams 4521–4523 rather than the console and physically sends five MiB including every possible byte value through stdin/stdout, plus an independent stderr marker. The gate also proves graceful/forced lifecycle, store list/remove, and pre-VM rejection of missing or altered target evidence.

Run make check, GCC/Clang, ASan/UBSan, TSan for host components, fake-libkrun protocol tests and real Hypervisor/vsock tests. Docker cannot validate HVF, Seatbelt, libkrun firmware loading or actual VM teardown.

Expected code touchpoints

Operational K8 contract and later completion

The Krun package is advertised for the K8 subset because all of the following are simultaneously true:

• a public TrustedRunSpec can select it without a private rootfs argument;
• the image digest, platform, raw-root artifact, driver, libkrun closure and guest helpers are bound into one prepared artifact and revalidated at spawn;
• the exact capability set is enforced or rejected without fallback;
• NETWORK_NONE and proxy-mediated plans are exact; mediated traffic reaches only the per-execution Egress listener through AF_VSOCK 4520;
• Warden can gracefully stop, forcibly destroy and reap the complete VM domain;
• receipt v7 identifies host Seatbelt, the selected Krun mechanism, optional inner Bubblewrap/root overlay, exact VM/cgroup/Egress budgets, measured usage, controller failsafe and sealed target/helper evidence;
• installation absence remains harmless, while explicit selection explains exactly which proof failed.

K6 extends that base with inner Bubblewrap, K7 adds Egress/vsock to both candidates and K8 adds host Seatbelt. Every unavailable guarantee stays absent from the capability mask and cannot be selected accidentally.

References: libkrun, its stable C API, the upstream 36-bit IPA limit, and the official Homebrew tap.