Learning with Linux From Scratch [LWN.net]

May 19, 2004

This article was contributed by Ladislav Bodnar

In the midst of all the excitement surrounding the release of Fedora Core 2 this week, some of the smaller projects that announced new versions at about the same time might have escaped attention. One of them was Linux From Scratch 5.1.

As the popularity of Linux increases, many people ask: "Is there a fast and fun way to learn the ins and outs of the Linux operating system? Are there any entertaining alternatives to conventional training courses and books?" For many, the answer might very well be Linux From Scratch (LFS), a book that provides step-by-step instructions to build a complete Linux operating system from source code available for download on the Internet.

Linux From Scratch is a mature project. Its beginnings date back to December 1999 when version 1.0 was released. In it, the book's author Gerard Beekmans explains the purpose of the new "distribution":

I started this document about 6 months ago. I tried a few Linux distributions and came to the conclusion that there wasn't a distribution I totally liked. Every distribution has its own advantages and disadvantages, but I was never satisfied with what I had (although Debian comes very close to what I want), so I decided to explore the possibility of building my own Linux distribution using nothing but source code of programs. As I found out there's quite a bit of work involved, but it's also a lot of fun and you really learn a lot by doing it, since you need to configure every single aspect of the system. This forces you to read a lot of manuals on how to configure various software. It also gives you total control over your system (well, that's the idea). You know exactly what software is installed, how it is configured and where all the configuration files reside.

Yes, Linux From Scratch is primarily about learning. Although the final product can indeed be used as a distribution in its own right, the road that one has to walk in order to get to the destination is too tedious to turn it a regular routine. Installing Linux From Scratch is even harder than installing Gentoo: there is no Portage to do the hard work and all compiling has to be done with the classic UNIX tools of configure, make and make install. But this is where the educational value of Linux From Scratch manifests itself. The process is possibly the most practical way to learn about every detail regarding file structures, processor optimizations, configuration files, security matters and thousands of other issues. How much would you pay for a commercial Linux training course? Linux From Scratch is a great resource which will not only teach you the very basics of Linux, it will do so in a most entertaining way, all for free.

If you decide to embark on this experience, consider these prerequisites:

Make sure that you have an existing Linux installation on your hard disk. Any recent distribution will do, as long as it is complete with a GCC compiler and relevant development tools.
Download the LFS packages. All the required LFS packages can be downloaded from LFS mirrors, either individually, or as one complete tarball. Alternative methods using P2P networks or a wget script are also available.
Download the LFS book. The LFS book contains around 200 pages and can be downloaded in HTML or PDF formats.

Although trying to absorb 200 highly technical pages split into 9 chapters might sound like a lot of hard work, the truth is that a great deal of the book consists of reference information, such as package descriptions and listings of program files and their dependencies. The preface and the first chapter can be skimmed over - they contain little beside a foreword, acknowledgment, conventions, changelog and other general information. Chapter 2 explains how to create a new partition, format it with an ext2 file system and mount it. Chapter 3 lists packages needed to build LFS, while chapter 4 details the final preparations before the actual build, inclusive of setting up the build environment.

The real meat of the book starts in Chapter 5, which contains instructions about compiling and installing a minimal Linux system. At this stage, all compilation is done with tools "borrowed" from the host environment, but with static linking to system libraries to gain "independence" from the host system. This will ensure that the newly compiled tools still work in a subsequent, "chroot-ed" stage. The compile process starts with GNU Binutils and continues with GCC, Linux header files, Glibc, Tcl, several essential GNU utilities, and Perl. After installing Glibc, both Binutils and GCC have to be recompiled for the second time to link them against the new Glibc. The compiling of most packages will only take a minute or two, with the exception of Glibc and GCC, which will take a lot longer. However, the time it takes to compile the packages can be utilized for reading the relevant sections in the book, which provide detailed information about such interesting matters as the purpose of the many available GCC compiler flags and other related topics.

Chapter 6 starts with mounting the proc and devpts file systems, followed by a chroot into the newly compiled base LFS partition. The next step is to create a standard UNIX directory structure. If you are fairly new to the world of Linux, this is a great chapter to learn about file permissions, passwords, users and groups, log files, and also about creating devices in the /dev directory. Next comes a detailed explanation on compiling Glibc, including notes on locales, how to configure the dynamic loader, and a list of commands provided by Glibc. A very useful chapter indeed! In order to enable dynamic linking to system libraries, all applications compiled in the previous chapter need to be recompiled here for the second time (or, in the case of Binutils and GCC, for the third time). The remaining system packages will also be compiled in this chapter. Interestingly, beside GCC 3.3.3, the book also recommends installing GCC 2.95.3, which will be used exclusively for compiling the Linux kernel; the well-tested older compiler is said to be more suitable for building a rock-solid kernel than any of the new GCC 3.x series.

The full Linux kernel is finally compiled in chapter 8. However, to get there, one still needs to go through the short, but important chapter 7 - another invaluable section of the book providing all the necessary bootscripts and filled with information about setting up the system clock, the syslog daemon, and networking. The kernel compilation chapter does not deal with kernel configuration issues; it merely provides instructions to compile a default kernel, with a suggested alternative of copying an existing kernel configuration file from a known working system. The very final step of the book is to configure the Grub boot loader (previous versions of LFS used lilo, but version 5.1 switched to Grub) to make the newly compiled Linux system bootable.

Completing all the steps in the book will probably kill a whole weekend, but besides the freshly acquired knowledge and experience, the brand new Linux system on your hard disk is very bare-bones and not particularly useful. So how can you make it useful? By moving on to the next book - the 413-page Beyond Linux From Scratch (BLFS). This is a priceless resource with detailed instruction on how to compile many common applications, including essential utilities, server packages (Apache, MySQL, Samba...), desktop environments (XFree86 + KDE, GNOME, XFce...), OpenOffice.org, multimedia and printing packages, and many other open source applications. At this stage, you'll probably start craving a binary Linux distribution, one that can be installed and is ready to use in 20 minutes. But even if you don't plan on further package compiling, the BLFS book is a great reference for those moments when you do need to compile applications, with many tricks, workarounds and guidelines.

Linux From Scratch is a wonderful project. It should become a compulsory reading material for all Linux training courses, and something that every Linux enthusiast should complete at least once. This would also create another interesting side effect: people who tend to be quick in expressing dissatisfaction on the distributions' mailing lists and forums would probably show a lot more respect for the developers. Installing a ready-made distribution is a trivial task. Building up a set of 4 CDs containing a stable, secure and reliable operating system, plus thousands of applications, is most definitely not.

(Log in to post comments)

Related projects

Posted May 20, 2004 8:08 UTC (Thu) by dank (guest, #1865) [Link]

LFS is great stuff. Its focus on good documentation is wonderful.

However, once you learn what LFS teaches, and you want to use it in a production environment, you really want to have an automated build process stored in a source code control system, not a book. (At least I do. Having an automated build means being able to sleep at night without worrying you forgot a step...)

It's worth mentioning two projects with similar "build it all from scratch" goals as LFS, but which focus on automated rather than manual build.

ptxdist is a tiny, 100% cross-compiled distribution of Linux by Robert Schwebel. Without any chroot'ing or rebooting, it builds an entire small Linux system from pristine source tarballs with the bare minimum set of patches.

crosstool is my smallish shell script that builds a gcc/glibc toolchain entirely from pristine source taballs with the bare minimum set of patches. It supports most recent or interesting binutils, gcc, and glibc versions, and just about every CPU type glibc supports. If you're so inclined, you can easily use it to build Linux-targeted distributed compile farms regardless of whether your workstations run Linux, Windows, MacOSX, or Solaris. Lots of folks who normally would prefer downloading a precompiled toolchain are finding that it's easy to build their own with crosstool.

The two independent projects share a common patch repository containing just the bare minimum patches for gcc, glibc, and binutils to get the toolchain to compile and work (many are simply small patches pulled from the upstream cvs).

For a full distribution, try Gentoo

Posted May 20, 2004 12:33 UTC (Thu) by Duncan (guest, #6647) [Link]

Then of course, when one wants a full distribution with an automated build
process but still self compileable, one can turn to the source based
meta-distribution, Gentoo. Or, go the other way, and move to Gentoo from
a traditional binary distribution, as many do when they want more control.
Gentoo supplies the first part, the staticly compiled host system, for
you, as a bootable LiveCD, or you can do the chroot thing and build from
an existing host system (as I'm doing). In either case, if one chooses a
stage-one install, the initial stage-one tarball includes the first
statically compiled base system utils (as necessary for configuring and
compiling the rest of the system), device nodes, boot scripts, file system
layout, and GCC. One compiles the second round of the base tools from
there, after which you are at stage-two (also available in tarball form,
for those that don't want to bother with the basics). From there, one
compiles the base system applications for which there isn't a lot of
alternative (including things like perl, python, the manpage system, etc),
ending up at stage three (again also available as a tarball, for those
wishing to take the simple shortcut route).

A stage-three Gentoo system has all the basics for which there is little
choice, but remains barely functional, not even including a syslogger, as
there are several choices available. However, this is where life gets
interesting, as one can begin to make choices on packages, not just the
features to include on packages that MUST be installed. This is where
Linux from Scratch and other basic systems leave off, but it's where
Gentoo really begins, as one now chooses the packages for the rest of
their system, and compiles and installs them, using Gentoo's source based
but automated and dependency checked Portage build system, with its two
base commands, emerge, which does the high-level stuff such as dependency
checking, and ebuild, generally called by emerge, but also usable on its
own, to fetch, unpack, configure, compile, and install, individual
packages. The latter command, ebuild, is especially useful for those
wishing to do additional customization either to the applied patches, or
to the configure and compile, before installation.

I'm personally in the process of moving from Mandrake to Gentoo, because I
wanted the faster availability and more continuous upgrade cycle of
Gentoo, and was ready for the greater level of control and higher level of
configurability it offered. (Tho Mandrake isn't to bad itself, if one
chooses to compile from SRPM rather than install the binaries directly,
and then takes advantage of the additional configuration options available
when doing so, and if one runs x86. However, availability of upgrades
still lags, on anything but x86, the big reason I'm switching, as I run
x86_64 aka AMD64, rather less well supported by Mandrake, to date, with
KDE 3.2 missing on the platform well after KDE 3.2.1 was released
upstream, tho it's available now.)

Duncan
(Hard at the process of discovering the advantages and rising to the
challenges of a new-to-him distribution! =:^)

Related projects

Posted May 21, 2004 8:48 UTC (Fri) by ca9mbu (guest, #11098) [Link]

If you're after an automated way of installing LFS then I'd like to recommend nALFS. It is intended that this will become the primary testing tool for LFS releases, in order to maintain the quality of our releases across a range of architectures. Although this automates the install, it doesn't provide any kind of package management. For that, BLFS outlines some of the issues involved, and points the avid reader to the Hints project where several package-management tools are discussed in more detail.

Regards,

Matt Burgess
LFS Coordinator.