The Linux SVSM project

If legacy networks are like individual homes with a few doors where a handful of people have the key, then cloud-based environments are like apartment complexes that offer both higher density and greater flexibility, but which include more key holders and potential entry points. The importance of protecting virtual machines (VMs) running in these environments — from both the host and other tenants — has become increasingly clear. The Linux Secure VM Service Module (SVSM) is a new, Rust-based, open-source project that aims to help preserve the confidentiality and integrity of VMs on AMD hardware.

The resource sharing that makes multi-tenant cloud environments so efficient can also expose the memory, caches, and registers of VMs to unauthorized access. As a response, confidential computing seeks to preserve the confidentiality and integrity of VMs from other VMs as well as from the host-machine owners. This is of particular concern for cloud providers that must meet their clients' stringent security requirements in order to sell their services. Availability is not usually part of the security goals because untrusted providers (potential attackers in these threat models) usually have direct physical access to the hosts themselves.

When performing sensitive operations on an untrusted cloud infrastructure, many resources, including the host BIOS, hypervisor, device drivers, virtual machine manager (VMM), and other VMs, cannot be fully trusted. With such a reduced trusted computing base (TCB), the root of trust usually falls to dedicated hardware components that are separate from the CPU and the rest of the hardware. The SVSM acts as an intermediary between the guest hypervisor and the firmware of these components on AMD processors. Within the context of operating systems, a "service module" can be defined as a separate entity whose main goal is to perform operations on behalf of the kernel. Since the kernel itself does not need to be able to perform such operations anymore, its ability to do so can be limited by the hardware, stopping a potential abuse from attackers.

In particular, Linux SVSM offers services to interact with the AMD Secure Processor (ASP), which is a key component of AMD's Secure Encrypted Virtualization (SEV) technology. The "Zen 3" architecture introduced with third-generation AMD EPYC processors uses the ASP to protect both the memory and register states of secured guests; the services Linux SVSM provides take advantage of these hardware capabilities. Linux SVSM provides secure services in accordance with the SVSM specification to help minimize the attack surface on guest machines. Its release was announced on the linux-coco confidential-computing mailing list, where the community is actively discussing development-related topics. Linux SVSM is an effort in the direction of virtualized confidential computing. Understanding this requires an introduction to the most recent SEV features.

SNP features used by Linux SVSM

The AMD Secure Nested Paging (SNP) feature is one of the confidential-computing extensions introduced with the "Zen 3" microarchitecture. Linux SVSM makes extensive use of two SNP features: the Reverse Map Table (RMP) and the Virtual Machine Privilege Levels (VMPLs); it also makes use of a special area known as the Virtual Machine Saving Area (VMSA). The VM state, which is a complete snapshot of the running guest's CPU registers, is saved in the VMSA whenever the VM exits back to the hypervisor.

SNP provides memory-integrity protection using a DRAM-loaded, per-host RMP. The RMP contains an entry for every physical page on the system and keeps track of the ownership and permissions of each so as to (for example) trigger a page fault when a third-party attempts to write where it should not. The RMP thus acts as an extra step in the page-table walking sequence. Some of the RMP use cases include preventing data corruption, data aliasing, and page-remapping attacks. The RMP holds the mapping for each physical page and its corresponding guest page; therefore, only one guest page can be mapped per physical page. Further, an attacker may attempt to change the physical page mapped to a guest page behind the guest's back; the RMP will thwart such attacks.

Before using a page, the guest must first validate its RMP mapping (the RMP entries include a valid bit, that is checked by hardware in the last step of the nested page walk). This is usually done during initial boot as part of the kernel's page-table preparation with the PVALIDATE instruction. The hypervisor is responsible for managing the RMP in cooperation with the SVSM and hardware checks have been implemented to ensure that the hypervisor does not misuse the RMP.

SNP also introduces the concept of Virtual Machine Privilege Levels (VMPLs), which range from zero to three, for enhanced hardware security control within VMs; VMPL0 is the highest level of privilege and VMPL3 the lowest, resembling x86 protection rings. VMPLs increase access-control granularity and can trigger exits from the VM when some virtual CPU (vCPU) attempts to access a resource that it should not. A new page that is assigned to, and validated by, a guest receives all permissions at VMPL0. The guest can later use the RMPADJUST instruction to change this for higher privilege levels. For example, a guest running at VMPL1 can remove the execute permission for that page from vCPUs running at VMPL2 or higher. Again, this type of operation normally occurs during boot. The VMSA of each guest contains its VMPL level, which cannot be modified after launch unless the SVSM directly modifies the VMSA.

Linux SVSM makes use of these (and other) new SNP features. It runs at VMPL0 while the guest OS runs at VMPL1, meaning that the SVSM will perform all guest operations that require VMPL0 on behalf of the OS. The SVSM could also provide other services (e.g. potentially live migration) in a secure manner. All requests from the guest use an API defined in the SVSM specification and must follow protocol specifications for each service type. Relying on Linux SVSM to handle certain operations drastically hardens the TCB because the sensitive work is offloaded from large programs (such as the Linux kernel) that have many attack vectors to the smaller SVSM. Further, multiple subsystems (such as kernel randomization) that are now targets due to the expansion of cloud virtualization will not require the same levels of auditing because they become unprivileged.

The Linux SVSM execution flow

Linux SVSM is not an operating system; rather, it is a standalone binary loaded by the hypervisor. The SVSM benefits from the strong static guarantees of the Rust language, from both a security and memory perspective and for safe synchronization. The Linux SVSM logic comprises both its internal setup and VM guest request handling. Analyzing the Linux SVSM execution flow is an effective way to get a better understanding. This flow consists of the following four phases:

Jump to Rust. The SVSM is the first guest code executed by the hypervisor after a VM is launched. The boot process starts at VMPL0 within the bootstrap processor (BSP). A small amount of assembly code performs basic initialization before quickly jumping to higher-level, standalone Rust code. Even in Rust, though, some operations need to be executed from within unsafe blocks (e.g. writing to MSRs or dereferencing pointers). Linux SVSM relies on the x86_64 Rust crate for most of its page handling.

Kernel components initialization. SVSM, running on the BSP, performs some checks to verify that the provided memory addresses are correct and that it is indeed running at VMPL0 with proper SEV features. The SVSM also comes with serial output support and its own dynamic memory allocator (a slab allocator for allocations up to 2KB and a buddy scheme for allocations greater than that). All of these components are initialized and other OS housekeeping occurs as well.

Launch of APs and OVMF. When running the guest under SMP, the BSP initializes the rest of auxiliary processors (APs), preparing a VMSA for each of them. Upon start, the APs jump to the SVSM request loop. The BSP locates the Open Virtual Machine Firmware (OVMF) BIOS, prepares its VMSA to run at VMPL1, and then requests the hypervisor to use the new VMSA to run the OVMF code. OVMF eventually starts the execution of the guest Linux kernel, which also runs at VMPL1. The SVSM is contained in the guest's address space, but it is not accessible by it. Whenever the guest OS needs to perform a privileged VMPL operation (such as validating its pages) it will communicate with the SVSM following one of the predefined protocols. At this point the SVSM is out of the picture while the guest kernel runs, at least until that kernel makes a service request. The initialization process is complete.

Request loop. Once everything is up and running, the process of handling requests within the SVSM begins. When the guest needs to execute something at VMPL0 (such as updating the RMP with a page validation) or to request other services from the SVSM (like virtual TPM operations), it follows the SVSM API and requests the hypervisor to run the VMPL0 VMSA that is associated with the SVSM, triggering a context switch. At that point, the hypervisor resumes the SVSM by issuing a VMRUN instruction via the VMPL0 VMSA of the SVSM. The request is processed; upon completion, the SVSM instructs the hypervisor to resume the guest VMPL1 VMSA.

Throughout this process, the SVSM executes with the SEV "Restricted Injection" feature active. This feature disables virtual interrupt queuing and limits the event-injection interface to just the #HV ("hypervisor injection") exception. The SVSM runs with interrupts disabled and does not expect any event injection, which would result in the SVSM double-faulting and terminating. This mode of operation is enforced to further reduce the security exposure within the SVSM and simplifies the handling of interruptions.

What's next?

Linux SVSM requires updated versions of the host and guest KVM, QEMU, and OVMF subsystems. These modifications are currently either under development or making their way upstream. As of this writing, the SVSM repository includes initialization scripts that clone repositories with needed changes to ease the process for developers. The current focus is on Linux support; however, the SVSM specification itself is OS-independent.

Linux SVSM is an open-source project under active development. As such, it is accepting public contributions. Support for the ability to run under different x86 privilege levels is currently being developed. Once the SVSM is able to offload all the security operations, we will be able to provide additional services, such as live VM migration. The SVSM privilege-separation model also permits the existence of a virtual Trusted Platform Module (virtual TPM). You can find recent discussions regarding design possibilities for a potential vTPM on the linux-coco mailing list. The Linux SVSM may also benefit from finer security granularity, documentation, community participation, etc. There are many open development fronts and opportunities to be part of the process and learn Rust from a systems perspective along the way. We welcome all contributions to the project.