Using Python to investigate EFI and ACPI

In a talk that could easily be seen as a follow-on to his PyCon 2015 talk, Josh Triplett presented at LinuxCon North America on using Python to explore the low-level firmware of today's systems. The BIOS Implementation Test Suite (BITS) provides an environment that hearkens back to the days of BASIC, PEEK, and POKE, as he demonstrated at PyCon in Montréal in April, but it is much more than that. In Seattle at LinuxCon, he showed that it can also be used to look at and use the Extensible Firmware Interface (EFI) and Advanced Configuration and Power Interface (ACPI) code in a system—all from Python.

Triplett started his talk with a bit of nostalgia: pictures of various home computers from the 1980s (e.g. Commodore 64, TRS-80, Apple II) in his slides [PDF]. He polled the room to see which were the first computers used by those in the room before showing a picture of the first IBM PC, which was his first computer. There was a common element to all of those early home computers, he said: they provided access to the low-level hardware of the system. These days, we have lost a lot of that access because the operating system mediates access to the hardware.

The IBM PC ran DOS, which accessed the hardware through the Basic Input/Output System (BIOS). There were fixed data tables and addresses in the BIOS for accessing the hardware. Various system services (e.g. disk, display, serial ports) were available via interrupts. If the BIOS did not know about the hardware, the system couldn't talk to it.

EFI and ACPI came along "to solve every problem BIOS ever had and quite a few it didn't", Triplett said. The key concept behind both is "extensibility", but they also have a reputation for being "subtle, complicated, and quick to anger", he said. The operating system and bootloader both use the facilities provided by EFI and ACPI, but it is mostly done from C code.

BITS came about because of a need to access BIOS, EFI, and ACPI without writing any C code. Triplett (and his father, Burt Triplett, both of whom work for Intel) ported Python to run in the GRUB bootloader, which allowed using the language to poke at the low-level firmware. As with his PyCon presentation, his slides were displayed and his demos were run from within the BITS environment in a virtual machine (VM) on his laptop.

He uses KVM with Open Virtual Machine Firmware (OVMF), which provides Unified EFI (UEFI, the successor to EFI) from Tiano for use in a VM. It is much safer to play with EFI and ACPI in a VM, he said, so that if things "blow up", they won't also take your system with it.

BITS has a full Python interpreter that runs in ring 0 on x86 systems. That gives it the same privilege level that an operating system running on x86 has. In addition, many of the Python standard library modules are available in BITS, along with a few modules that provide useful types and functions for BITS, EFI, ACPI, and so on.

Triplett then demonstrated using the BITS Python to access memory from a specific address in the firmware, which showed the path where he had built OVMF:

    >>> import bits
    >>> from ctypes import *

    >>> mem = (c_char * 128).from_address(0xf1390)
    >>> print bits.dumpmem(mem)
    00000000: 2f 68 6f 6d 65 2f 6a 6f 73 68 ... /home/josh...
    ...

The ctypes module provides access to C data types (like c_char) from Python. The code creates an array of 128 bytes from the address specified (found using strings on the binary), which can then be manipulated by the Python code—and dumped to the screen using a utility function from the bits module.

He then moved on to look at ACPI:

    >>> import acpi
    >>> acpi.get_table_list()
    ['APIC', 'DSDT', 'FACP', 'FACS', 'HPET', 'RSDP', 'RSDT', ...
    >>> print bits.dumpmem(acpi.get_table('RSDP'))
    00000000: 52 53 ...			    RSD.PTR..BOCHS..
    ...

RSDP is the ACPI Root System Description Pointer and the "RSD PTR" string—represented as "RSD.PTR" in the memory dump—is how BIOS finds that data structure in the firmware. RSDP contains information about the ACPI version and the original equipment manufacturer (OEM) that provided it (for QEMU, its ACPI descends from the Bochs emulator, thus "BOCHS" as the OEM ID). The RSDP also has a pointer to the Root System Description Table (RSDT), which points to the rest of the system description tables, including those that describe hardware and other features that are specific to that particular system. The acpi.parse_table() function can be used to examine these tables, as Triplett demonstrated.

In the classic-BIOS world, serial ports can be found by consulting a table at a fixed address but, in a modern system, hardware is found differently. Resources are discovered using identifiers such as "COM1" (for the first serial port). Those identifiers lead to various descriptors in the ACPI tables that specify everything needed to talk to the device (address, interrupts, etc.). In today's systems, this is how all of the built-in hardware is discovered. By using acpi.display_resources() and acpi.parse_descriptor(), Triplett was able to show some of the "guts" of the COM1 "current resource settings" (_CRS), including its 0x3f8 address—which he then used to output a string from a loop in Python using bits.outb().

Both ACPI and UEFI are huge specifications, each with thousands of pages of documentation. ACPI is largely concerned with how to find the hardware in the system, while UEFI is the way to get modern "BIOS" services. Triplett proceeded to demonstrate accessing UEFI using the "efi" module that comes with BITS.

He started by printing out the EFI system table data structure, which is accessed in BITS with efi.system_table. That table provides a bunch of information about the system and its EFI firmware, including things like pointers to the standard input and output as well as to the boot and runtime services provided by EFI. The system table is what gets passed to an EFI program (e.g. a bootloader) as one of its arguments.

In EFI, objects are identified using globally unique identifiers (GUIDs). Triplett showed how to retrieve and use those GUIDs to access various pieces of information. For example, the ACPI table GUID can be used to retrieve a pointer to the ACPI RSDP, which is how that data structure is found on modern systems (rather than searching for the magic "RSD PTR" string as BIOS does).

Triplett then demonstrated how to clear the screen using the ClearScreen() function in the "text output protocol" that is attached to the standard output channel (i.e. ConOut) in the system table. He also showed using the "file system protocol" to access files in the EFI filesystem.

That was all a prelude to his final act, however. He noted that he had done graphics before in talks about BITS (e.g. the Mandelbrot set in his PyCon presentation), so that was now "boring". This time, he used the "input text protocol" along with graphics to build an simple, interactive Tron-inspired game. It drew a line that would change direction based on arrow-key input and would halt when the line intersected itself. Triplett clearly knows how to finish up a conference talk as, once again, there was nice round of applause to acknowledge his amusing hackery—and the talk as a whole.

[I would like to thank the Linux Foundation for travel assistance to Seattle for LinuxCon North America.]

Index entries for this article
Conference	LinuxCon North America/2015

---

to post comments