The grumpy editor's e-book reader

Your editor recently "celebrated" yet another birthday; one asks "which birthday?" at the risk of making him grumpy indeed. During that celebration, a surprising present turned up, in the form of an Amazon Kindle book reader. That presents an opportunity to play with a new toy, something your editor is not known for turning down, even when the toy is as problematic as the Kindle. In the process, your editor turned up some free software which helps to make the device rather more useful.

The Kindle is a device which inspires mixed feelings. It is a nice piece of hardware, showing what can be done with electronic-ink displays attached to a Linux-based system. It is small, light, and able to store a long list of books. The display is nicely readable, even in strong outdoor light, though it is also somewhat slow to respond and changes pages with a distracting black flash. The built-in cellular modem makes the acquisition of books from almost anywhere easy; no computer is required. The keyboard is awful, but one does not normally expect to do a lot of typing on such a device.

What one does expect to do with it is to read. There is a great deal of written material on the net, much of it available under free (or, at least, "freely distributable") licenses. But your editor cannot be the only one who, despite being an avid reader, lacks enthusiasm for reading a novel from the computer screen after spending the working day staring at that same screen. Kindle-like devices offer a solution to this problem; they are portable and much easier on the eyes. They are still not as nice as a real book, but they are nice enough to make much of that online content more accessible. Electronic book readers are an interesting class of gadget, even for those of us who have no real interest in ditching words printed on dead trees.

Your editor's Kindle showed up just a couple of days after the widely-reported "memory hole" incident, in which Amazon deleted copies of two George Orwell books from the devices of customers who had "purchased" them. LWN has said little about this event for the simple reason that there is very little to add; the unbooking of 1984 must be shocking even to the most irony-challenged among us. But one could well add that Amazon has made it clear that running Linux does not necessarily make a device serve its owner. The Kindle is very much a closed, captive platform under the control of its vendor. As far as your editor can tell, nobody has yet claimed to have achieved root access on a Kindle 2 device.

Managing the device

One of your editor's first questions was, naturally: can the Kindle be loaded with books which do not come from Amazon? Many of us have materials in PDF format sitting around already. A lot of free software documentation is, for some strange reason, not available via the Amazon store. That is also true of the bulk of the public-domain and freely-licensed material which has been made available on the net. There are good reasons to put all of this stuff onto a reader where it will be available, someday, when the urge to read it strikes. There are a few other motivations as well: avoidance of payments, avoidance of DRM, privacy, protection against having material deleted by Big Bezos, etc.

The Kindle has a USB port, and it shows up as a normal USB mass storage device. So gaining access to the library from a Linux machine is a straightforward thing to do. Among other things, this access can be used to back up one's books, perhaps giving a degree of permanence to a book collection which, otherwise, appears to be subject to a discouraging degree of control from outside. It would also be nice to be able to place new files there; that, too, is possible, with one little problem: the Kindle 2 lacks a PDF reader.

Now, one could come up with no end of choice words for whoever thought that an electronic book without PDF capability made any sense whatsoever. But it's better to do something about it. Your editor spent some time searching for answers before stumbling across an interesting program named calibre. This GPLv3-licensed, Python-implemented, multi-platform program aims to solve the the PDF problem, and quite a bit more as well. It is, in fact, a general electronic book library manager with support for a number of reader devices.

Installing calibre was only mildly painful. It requires Python 2.6 and a fairly wide range of libraries; your editor's Fedora 11 machine (the time and courage to return to Rawhide have been lacking) does not have a calibre package, but all of the dependencies are available. The installation instructions are based on the idea that feeding text from a web site directly into a root shell is a good idea, but one can get around that. There is also the installation of a udev rule which sets the permissions for USB-connected Sony readers (for the Kindle, the program doesn't open the USB device directly).

Let it be said: calibre (version 0.5.14) needs a lot of work. The interface is strange and requires a certain amount of figuring out. If it can't communicate with the reader, there is no real information as to why (one hint: it expects that the device filesystem will be automatically mounted at plugin time, something your editor has disabled on his system). It will happily tell you that a given book is not available in the right format for the reader, but is short on information on how to get it into an appropriate format. One needs to be prepared to spend some time just messing with it. These gripes notwithstanding, calibre has the makings of a nice tool.

The main screen is dominated by a list of books in the library; clicking on the "reader" icon at the top yields a list of books on the device instead. One could imagine a useful combined listing mode which showed all books, with a concise indication of where they are to be found, but calibre does not do that. There's a pretty - but relatively useless - "browse by cover" mode. As with the Kindle itself, the book list is a single, flat listing, with no provision for organizing the books into hierarchies. This can only get painful as the list of books grows. Yes, this is 2009, we do everything with tags now, and calibre supports tags. Your editor would still like directories. Call them "bookshelves" if that fits the theme better. It would also be really nice if calibre could treat a group of files as all being part of a single book. No such luck.

The "view" operation can be used to read a book on the Linux system. It opens an internal reader for a number of formats; this reader seems to fail, silently, fairly often. For PDF files, calibre just starts evince, which works just fine.

The "send to device" button is, naturally enough, the way to move a specific file (or set of files) to the reader. There doesn't appear to be a "just keep the two in sync" option; books all must be loaded onto the reader explicitly. It would sure be nice if "send to device" would just convert the file into an appropriate format if need be, but that doesn't happen; it throws up a dialog saying that the transfer isn't possible. In other words, the user must explicitly perform a format conversion on (say) a PDF file before it can be sent to the Kindle. "Convert E-books" does that, providing a nice set of options on how the conversion is to be done. It works, but it should work automatically.

The conversion of PDF files into the "MOBI" format understood by the Kindle works reasonably well, but the books suffer somewhat in the translation. Paragraph breaks tend to vanish, page headers and footers get mixed into the text, and so on. The formatting of code samples loses little details like indentation. All told, the result isn't quite what it should be; it seems like it should be possible to do better.

One nice feature built into calibre is the "fetch news" operation. The program can go to the web sites of a large number of publications, download the current edition of whatever news is published there, and convert it into a format suitable for loading into the reader. If you leave calibre running, it can perform regular downloads, keeping the library populated with current newspaper and library editions. Needless to say, this feature is appealing when compared with the payment-required newspaper offerings from Amazon. On the other hand, as a web publisher, your editor does have a certain affection for the "payment required" mode of operation.

Also worthy of note is Savory, a repackaging of the calibre format-conversion code which runs on the Kindle itself. This tool lurks on the device as a daemon; whenever it sees a new PDF file, it goes off and converts it to the MOBI format automatically. Getting Savory to work can be a bit tricky, but, once the right incantations have been made, it works as advertised. The process is quite slow (the Kindle is not known for the data-crunching power of its CPU) and the end results are, not surprisingly, about the same as those obtained by using calibre directly - with one difference. Savory's conversion process actually makes two copies, one of which is a series of PNG images reflecting the real appearance of the source PDF file. The images have their own problems (they are not amenable to searching, for example), but they do look nicer. In summary, Savory is a nice enhancement, but it's in no way the same as having the device be able to just display PDF files natively.

Source

The Kindle is based on GPL-licensed software. Since Amazon is distributing this software with the device, it is required by the GPL to either include the source with the device or include a written offer to provide the source. Your editor read through all of the fine print, including absolute restrictions on modifying the device and discouraging stuff like:

Big brother does, indeed, know what you are reading. But your editor could not find the written source offer. So, technically, it would appear that Amazon is in violation of the GPL. That said, Amazon has made the source available for each version of the operating software shipped with Kindle devices; one simply needs to know where to look for it.

That source distribution takes the form of a 140MB compressed tarball, which, in turn, contains 44 other compressed tarballs. Among other things, Amazon has tossed in the source for the 2.6.22.19 kernel, GCC, bootchart, powertop, and iptables. There is no separate patch to the kernel, but a quick diff shows significant changes, mostly in the form of the addition of support for the Freescale i.MX27 and i.MX31 processor architectures. The i.MX27 code is not currently upstream, even in the 2.6.31-rc kernels. Amazon has patched in drivers for a Freescale PATA controller, i2c controllers, the Kindle "five-way controller," the "Fiona" keyboard, a "magnetic sensor" device, a trackball device, a video output device, a "run time integrity checker" device, a number of electronic ink devices, and far more. There's also the yaffs2 filesystem and, bizarrely, a version of Andi Kleen's superseded unlocked_fasync patch. None of this code is upstream, and there appears to be little interest in getting it there.

Sadly, the Kindle is a closed device, so there is little point in trying to build and boot this code. That integrity checker device seems likely to get in the way. The unhackable nature of the device does not come as a surprise; that is how things tend to be done these days. But one can still wish that things were different. A user-modifiable Kindle would not just be more resistant to Orwellian monitoring and control; it could also be extended in ways that Amazon never dreamed of. Maybe it could even get a PDF reader. What a fun device that could be.

---

(Log in to post comments)