Embedded Linux: Small Root Filesystems
The Root Filesystem
The Linux kernel works hand in hand with what is called the root filesystem. This is the filesystem upon which the root directory can be mounted and which contains the files necessary to bring the system to a state where other filesystems can be mounted and user space daemons and applications started. Most desktop and server distributions make use of two kinds of root filesystems: the initial root filesystem and the real root filesystem. The former is used to mount and run the latter.
The directory structure for a root filesystem can be extremely minimal, as we'll see in a moment, or it can contain the usual set of directories including /dev, /bin, /etc, and /sbin, among others that you see in any desktop Linux distribution.
The kernel boot process concludes with the init code (see init/main.c) whose primary purpose is to create and populate an initial root filesystem with a set of directories and files. It then tries to launch the first user mode process to run an executable file found on this initial filesystem. This first process ("init") is always given process ID 1.
There are three ways for the kernel to find the file that will be run by the init process. The first method is to use a file specified at boot time with the init= kernel parameter. If this parameter is not set, the kernel tries a series of locations to find a file named "init". These include /sbin/init, /etc/init, and /bin/init. If all these fail, the kernel tries to run any shell it finds at /bin/sh. If this last fallback is not found, the kernel will print an error saying that no init could be found.
Once the init process is started it typically begins to launch other user space programs. On a desktop or server system this is known as the sysvinit process and includes the set of scripts found (typically) under /etc/rc.d. The name sysvinit comes from the mechanism used in System V Unix, which defined the naming scheme used for directories and files. On embedded and small footprint systems the init process may be a set of custom designed scripts or even a single application. Some desktop distributions are also beginning to replace sysvinit with alternatives designed with faster booting in mind.
The early root filesystem: Initial Ramdisks
The initial root filesystem is known as the initial ramdisk because the filesystem lives in a disk image created by the kernel in RAM. In a desktop or server system, the initial ramdisk is used to load drivers and initialize an environment so that an external storage system (disk or network attached storage) can be mounted. The switch from the initial root filesystem to the real root filesystem is called a pivot. The pivot causes the real root filesystem to be mounted over the initial root filesystem. When that happens, a new init process from the real root filesystem is launched and takes over the process id of 1. At that point the initial ramdisk is no longer needed and the memory can be freed.
In an embedded system the initial ramdisk might be the only filesystem ever mounted, since it contains all the user mode applications required. Alternatively, the initial ramdisk might mount a flash drive or other local storage yet still not pivot. Instead the mounted storage might be used as a source for user space applications. In these cases the initial ramdisk is never cleared because the pivot never happens.
In the 2.4 kernel, the initial ramdisk was referred to as the initrd image. It was created as a filesystem inside a file. The file was mounted and a filesystem created inside of it. A directory structure for the initial ramdisk was copied into this filesystem. The file was then unmounted, compressed and provided to the kernel via the initrd= kernel parameter at boot time.
The initrd file could only be loaded at boot time from an external source (except for the MIPS kernel, which allowed you to embed the image into the kernel). The original ram disk mechanism for the 2.4 kernel created a synthetic fixed sized block device that needed the filesystem driver used when the initrd was created, such as ext2, in order to work with file data. At the end of the boot process the initrd image had to be unmounted in order to clean up memory usage before switching to a more complete root filesystem.
In the 2.6 kernel the process of creating and using the initial ramdisk has been somewhat simplified. First, the files are simply collected together in an compressed CPIO file, now referred to as the initramfs instead of initrd. The initramfs file is always embedded in the kernel (for all hardware platforms) even if you don't create one yourself. If you don't, a default CPIO archive is created automatically by the kernel build process.
Second, there is no external filesystem required at boot time for the initramfs. Instead, the initramfs is unpacked in a special ramfs-based filesystem called the rootfs. The ramfs filesystem support is built into the kernel and cannot be disabled, so it's always available. Because it doesn't use backing store, it's a simpler system than the mechanism used in the 2.4 kernel. And when the boot process is done with the initramfs, a more complete root filesystem (such as one found on disk) can be directly mounted over it without worrying about wasting a lot of memory.
Why use the initramfs?
It would be ideal if the kernel could boot into a minimal state that knew just enough to bring the system to a useful state for the user or environment it will run in. This minimal state would allow the kernel to be as small as possible with as few options compiled in as possible. This is exactly why you use an initramfs.
On any system, and most especially on resource limited systems, you want to keep the kernel itself small and dynamically load only those driver modules that are required to make the system finish booting. Most desktop systems use the initramfs to determine what kind of hard drive or other storage is available with a complete root filesystem. In this case the initramfs contains boot scripts and driver modules relevant to bringing up the system. These files are only kept around temporarily while the real root filesystem is mounted and the real init process is started. Because the variety of desktop hardware is large, the initramfs can end up being large and fairly sophisticated as it tries to guess what kind of hardware is about to be mounted.
On small systems the situation can be much different. There may not be any additional storage available to hold another, more complete root filesystem. In that case the initramfs becomes the real root filesystem. Because the initramfs is running out of RAM, it will contain only those files and directories absolutely necessary to run the system.
Alternatively, a small system might use a dedicated flash drive with read only access to prevent accidental destruction of the bootable system. In that case the initramfs will contain boot scripts that mount the flash device and perform a little trickery to simulate writeable partitions so the system can operate normally.
Creating an initramfs
It's possible to recreate an initial ramdisk that mirrors your running desktop using the mkinitrd script. The problem with using this script is that you're recreating your desktop environment. That's not likely what you're looking for in your embedded system or even a live CD. So we need to look at creating the initramfs manually.
The kernel source includes the text file ramfs-rootfs-initramfs.txt under Documentation/filesystems. In this file, under the section titled "Populating initramfs" are instructions for creating a very minimal initramfs. This includes a minimal set of device files, the /proc, /sys and /mnt directories, an init script and a BusyBox binary. We'll get to BusyBox in a moment.
Start by creating a directory called "myinitfs":
mkdir myinitfs
Add some basic directories:
mkdir -p myinitfs/{boot,proc,sys,mnt,sbin,dev,lib,usr/bin}
Not all of these are required but you'll want them around to populate with useful tools in your initramfs anyway. Next, add the required device files. If your kernel and user space processes need to be able to output messages then the minimal root filesystem will need a console device. This is created with the mknod command.
mknod -m 644 myinitfs/dev/console c 5 1
If your system is booting from a CD and the root filesystem is in a compressed filesystem image on the CD then you'll also need a loop device.
mknod -m 644 myinitfs/dev/loop0 b 7 0
Of course, your embedded system doesn't have to output messages to a console and it certainly doesn't have to mount any filesystems, so neither of these are required. But if you're creating a live CD you'll want them.
After creating the directory structure and adding these two devices, we copy in a shell script for our init program and a compiled copy of the BusyBox binary. The content of the shell script and the makeup of the BusyBox binary are the keys to getting your small system running.
Starting Small: BusyBox
BusyBox is the workhorse of embedded systems. It is a collection of commonly used Unix utilities rolled together into a single binary. The command line utilities usually have fewer options than their standalone counterparts but tend to be functionality similar. The primary goal of BusyBox is to provide a full featured set of utilities for resource limited systems.
BusyBox is a well designed package that is extremely easy to use. A graphical configuration utility similar in style to the curses-based kernel configuration utility allows you to choose the utilities you need. The Unix utilities are referred to as applets and the configuration utility lets you pick which applets to include in the binary. The choice of which applets to include depends entirely on the system you're trying to create. For a live CD that mounts a compressed file system from the CD as the real root filesystem (over the initramfs) you would include utilities like losetup, mount and umount, gzip, and tar, along with the basic ls, ash, grep, mkdir, mknod and so forth.
The build process for BusyBox is simple. Unpack the BusyBox archive in the current directory (where myinitfs is located). This creates a BusyBox directory. In that directory, create your configuration:
make menuconfig
You'll be prompted to save the configuration, which you should do. In the configuration you should be certain to specify the directory where the build should be installed. While not absolutely required, it saves a copying step later. In the latest version of BusyBox, 1.2.1, look under the BusyBox Settings->Installation Options menu and set the install directory to "../myinitfs".
After configuration, you simply build and install the binary:
make
make install
Getting Bigger: LFS
Before looking at the init script I want to mention that, although BusyBox can provide just about everything you need to get the system booted and even provide a runtime environment on its own, you might need far more user space support. If you're looking to extend your system to a full distribution, be sure to look at the LinuxFromScratch.org web site, known more commonly as LFS. Here you'll find step by step instructions on how to build a complete distribution.
The LFS is often used in live CD distributions as the runtime system that is loaded from a compressed filesystem off a CD by a BusyBox-based initramfs. Building a live CD from scratch in this manner is a great way to learn what a Linux distribution is all about, from the kernel on up through KDE and GNOME.
A live CD init script
At this point you've created a minimal set of utilities and a directory structure suitable for booting (sans the kernel, of course). But you still need the all important init script that kicks things off for the user space environment.
I've worked with this init script for some time, which is based on the init script found in an older version of the LFS live CD. It assumes the use of UnionFS and SquashFS for mounting and using compressed filesystem image files from the CD.
In my next article in this series I'll look at how and why you would use
compressed filesystems like SquashFS along with UnionFS to boot your
system.
| Index entries for this article | |
|---|---|
| GuestArticles | Hammel, Michael J. |