|
|
Subscribe / Log in / New account

Calibrating Calibre 1.0

By Jonathan Corbet
August 27, 2013
File management does not seem to be a one-size-fits-all proposition. Thus, while general-purpose file managers exist, it often appears that much of the development effort goes into domain-specific management tools. We have a whole set of photo management applications, for example, and even more music managers. When it comes to electronic books, though, there seems to only be one viable project out there: Calibre. LWN last looked at Calibre almost exactly two years ago. The recent version 1.0 release provides an obvious opportunity to see what has been happening with this fast-moving utility.

While installing the new version, your editor quickly learned that some things have not changed. The download page still recommends against using versions of the program packaged by distributors, "as those are often buggy/outdated". Instead, one is supposed to install the project's binary release, done by instructing a Python interpreter, running as root, to fetch a program from a web site and execute it, sight unseen. After all, what could possibly go wrong? For those who might balk at handing the keys to their systems to a remote site in that way, it is also possible to download a simple tarball and run the program from there. Source is also available, naturally, but the build process has a lot of dependencies and is not for those in a hurry.

One unfortunate antifeature that has not changed is the program's habit of phoning home on startup, providing the user's IP address, system type, Calibre version, and an identifying UUID to a Calibre server. One assumes that most distributors have disabled this reporting, but binaries retrieved directly from the site still do it. On the other hand, it's worth noting that Calibre is good about not breaking things with its (frequent) updates. Plugins installed years ago still work without problems (or even rebuilding), for example, and your editor's book library itself has never had trouble with Calibre updates. Calibre is developing rapidly, but some care has clearly gone into avoiding pain for users when they upgrade.

New in 1.0

In one sense, there is not a whole lot in the 1.0 release itself that calls for the big version number bump. Evidently, it was simply time to declare that Calibre had reached that milestone. That said, there are a couple of new features in this release, the most visible of which is the grid view for books in the library. This view, as one might expect, simply shows [Calibre's grid browser] book covers in a matrix arrangement; it joins the longstanding list view and the relatively useless animated "cover browser" as the third way of selecting a book from the library. The grid view is nice; it can be thought of as analog to arranging one's books with the cover forward, while the list view is like looking at the spines.

The 1.0 release also advertises a much faster database backend for managing book information. With a library containing several dozen books, it is hard to notice the difference; those with much larger libraries may have a different experience. But large libraries present challenges of their own in Calibre; the interface feels increasingly unwieldy as the number of books grows. A set of bookshelves can be organized as the owner wishes; a Calibre library is somewhat less flexible. There is, for example, no mechanism for imposing any sort of hierarchy on books. Yes, this is the 21st century, and we all use tags for everything now, but, as Herbert Simon pointed out years ago in The Sciences of the Artificial, human brains think in terms of hierarchies.

That said, one helpful feature that has been added (in the 0.9.28 release in April 2013) is "virtual libraries," which are a sort of saved-search mechanism. One can create a virtual library containing only books from a specific author, say, or based on tags. Virtual libraries don't really provide much that Calibre's search mechanism didn't already do, but they make frequent searches easier and faster to get to.

Calibre has long been the definitive tool for the conversion of books between formats. Over time it has been gaining features for the editing of books as well. New in 1.0 is an editor allowing the modification of the [TOC
editor] table of contents; it can perform simple tasks like adding an entry for an interesting location, or it can be used to create a new table of contents from the beginning. The automatic metadata downloading system can look further afield for book covers, descriptions, and more. The embedding of fonts within books is now fully supported. On the input side, it is now possible to import books in the Microsoft Word (.docx) format; for output, the PDF generation engine has been rewritten with, it is claimed, much better results.

Beyond that, of course, the program continues to gain support for more devices, more ebook formats, and more web sites as sources of content. Calibre's ability to download the contents of an online magazine issue and package them into a book on a reader device remains one of its more useful feature; it makes offline reading easy. In general, the addition of features continues at such a pace that many of them are hidden at the outset and must be manually added to a toolbar (by way of the "preferences" screen) to be accessible. Suffice to say that Calibre developer Kovid Goyal does not seem to agree with the minimalist line of thought — with regard to features or configuration options — that seems to dominate some other projects.

All those features and options are certainly welcome, but there is no doubt that Calibre could benefit from some focused design and usability work. [Download dialog] The interface is cluttered and useful functionality can be hard to find. Who can say, off the top of their head, why there are two search bars on the main screen? (One of them is actually a place to enter a name to attach to a "saved search"). Or consider a dialog like the one shown to the right. What line of reasoning would lead to the "cancel" button being placed right in the middle of all the other options? In general, Calibre gives the impression of being a large collection of (mostly useful) tools haphazardly thrown into a big box.

Finally, one significant "new" feature (new since the last review, but it was available in 0.9) is proper support for Android devices. Linux systems still struggle to support the MTP protocol in any sort of general way, but Calibre's internal MTP implementation seems to work just fine. Calibre recognizes Android devices when they are plugged in, and can tell multiple devices apart; it can be configured to manage or ignore each device independently. There is a simple rule system that allows different types of files to be directed to different locations on the device. It all works quite well, though interoperability with the Android Kindle app is minimal. The third-party plugins that deal so nicely with books on an actual Kindle device are unable to read Kindle books on an Android device.

DRM

In general, of course, DRM continues to be one of the biggest disincentives for anybody considering the purchase of an electronic book. It should be possible to buy a book and read it using the device — and the software — of one's choice. One should not have to worry about things like having one's books deleted as a consequence of crossing a national border. Books should not be subject to lock-in to a particular device or the continuing good will of a remote corporation. Someday, hopefully, the book industry will free itself of DRM; until then, people who are unwilling to break the DRM on books they buy will not be able to make full use of a library manager like Calibre.

One should note, of course, that there are quite a few publishers selling DRM-free electronic books now. Patronizing those publishers seems like a good way to ensure that they continue to make DRM-free books available. Calibre, of course, has long had a built-in "get books" feature that can search for electronic books from a large number of sellers.

In summary: Calibre remains the definitive tool for the management of electronic books for Linux — or for any other operating system. The needed functionality is there, and it continues to develop at a fast pace. Indeed, the pace is surprisingly fast given that the lead developer is responsible for the vast majority of the work — nearly 75% of the commits in the Calibre git repository. Perhaps, someday, somebody will set out to create something that is prettier, perhaps based on Calibre's book management and format-conversion engine. But, for now, Calibre seems to be the only show in town; happily, it is a highly functional and useful tool.


to post comments

Calibrating Calibre 1.0

Posted Aug 27, 2013 22:35 UTC (Tue) by nix (subscriber, #2304) [Link] (13 responses)

As an aside, calibre's internal mtp implementation is (a Python binding around) the existing libmtp project. Calibre is quite good about using existing libraries where possible. I just wish it had the same attitude to existing user interface elements, because its UI is quite thoroughly confounding, though thankfully one can just use its command line for conversion jobs.

(And where oh where did you find Alastair Reynolds's _Troika_ in ebook form? Nearly all his recent novellas are available only in murderously expensive hardcover form as far as I can see: yes, he's good, but not worth nearly a pound a page.)

Troika

Posted Aug 27, 2013 23:02 UTC (Tue) by corbet (editor, #1) [Link] (11 responses)

Amazon has Kindle versions of Troika and a few of his other novellas.

Troika

Posted Aug 27, 2013 23:58 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (10 responses)

Amazon says it's 'unavailable for purchase' :(

Troika

Posted Aug 28, 2013 2:08 UTC (Wed) by happynut (guest, #4117) [Link] (4 responses)

Its available for kindle at amazon.com, but not at amazon.ca or amazon.co.uk. Follow Corbet's link; it worked for me. Since you said "a pound a page" I'm assuming you checked amazon.co.uk...

Troika

Posted Aug 28, 2013 7:54 UTC (Wed) by johill (subscriber, #25196) [Link] (1 responses)

That's interesting, the link also says unavailable for me (from Germany) - do they do something with geoIP? That's odd though, I've bought many ebooks on amazon.com (and in fact my kindle is still registered in the US)

Troika

Posted Aug 28, 2013 9:40 UTC (Wed) by hummassa (subscriber, #307) [Link]

The link works from Brazil, too.

Troika

Posted Aug 29, 2013 6:30 UTC (Thu) by ayeomans (guest, #1848) [Link] (1 responses)

From the UK, when I tried that link I get the message "Kindle titles are available for UK customers on Amazon.co.uk." (where the "Buy" button is normally), Only the hardback is (expensively) available.

Troika

Posted Aug 29, 2013 13:24 UTC (Thu) by dlang (guest, #313) [Link]

The problem is that the Authors have historically sold publishers the rights to publish books only in specific parts of the world.

this makes sense when you are talking about printed books, where the publisher needs to have a distribution chain setup in that part of the world to be able to sell the books.

But unfortunately, the e-book version suffers from the same distribution limits, even when they don't really make sense.

Troika

Posted Aug 28, 2013 2:09 UTC (Wed) by ghane (guest, #1805) [Link] (4 responses)

Maybe they ran out, and are running "cp" for more copies?

Oh wait, Amazon like Jon better than it likes you? :-)

Troika

Posted Aug 30, 2013 17:32 UTC (Fri) by nix (subscriber, #2304) [Link] (3 responses)

Jon lives in the US, while I live in the UK like Alastair so, it seems, am not allowed to read his stuff unless I want to spend on the order of £100 per novella. Wonderful. :/

Troika

Posted Aug 30, 2013 19:55 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (1 responses)

Surely you mean £10? I'd expect a lot more than a novella at £100…

Troika

Posted Sep 2, 2013 9:44 UTC (Mon) by nix (subscriber, #2304) [Link]

Unfortunately not. These things were expensive chapbooks when they were printed -- now, years out of print, you're talking £70+. With a lot of +.

Troika

Posted Aug 30, 2013 23:21 UTC (Fri) by klossner (subscriber, #30046) [Link]

I have observed successful purchases of geographically-restricted digital products from Amazon by routing the web traffic through a proxy in a non-restricted country. Evidently the customer's home address and credit card currency are not considered.

Calibrating Calibre 1.0

Posted Sep 3, 2013 2:37 UTC (Tue) by kmself (guest, #11565) [Link]

Thanks for the review, Jon. And in particular your comments on the update mechanism -- I'm seeing similar mechanisms increasingly, and they raise my hackles...

On a related note: does anyone know how to change the default PDF viewer Calibre uses? It's beyond useless, and I'd much prefer either xpdf or evince.

0.9.31+dfsg-1 under Debian.

Calibrating Calibre 1.0

Posted Aug 28, 2013 4:07 UTC (Wed) by drag (guest, #31333) [Link] (5 responses)

I haven't looked at Calibre in a couple years. It totally scared me away from it due to the author introducing trivial local root exploits into any Linux system that installed it.

http://lwn.net/Articles/465311/
http://1337day.com/exploit/16889

I am still too scared to install it, unfortunately. Which is a shame because it does actually tend to do a decent job at managing ebooks if I remembered correctly.

Calibrating Calibre 1.0

Posted Aug 28, 2013 7:47 UTC (Wed) by fb (guest, #53265) [Link]

Thanks for pointing this out. I mean it, truly appreciated.

I used Calibre for a while some years ago, and probably would install it again without much thought should I need to perform conversions between e-book formats.

Calibrating Calibre 1.0

Posted Aug 28, 2013 8:20 UTC (Wed) by Trou.fr (subscriber, #26289) [Link] (1 responses)

Note that Debian replaces the setuid helper with a custom component relying on other (standard) suid programs : http://bazaar.launchpad.net/~calibre-packagers/calibre/de...

Calibrating Calibre 1.0

Posted Aug 28, 2013 8:34 UTC (Wed) by TomH (subscriber, #56149) [Link]

Fedora also replaces it with a non-setuid dummy script: http://pkgs.fedoraproject.org/cgit/calibre.git/tree/calib...

Calibrating Calibre 1.0

Posted Aug 28, 2013 9:02 UTC (Wed) by rsidd (subscriber, #2582) [Link]

Thanks for that.

Calibrating Calibre 1.0

Posted Aug 30, 2013 17:34 UTC (Fri) by nix (subscriber, #2304) [Link]

The setuid helper has not existed since Nov 5 2011. It's udisks all the way now (which is also pretty horrible software, IMNSHO).

Calibrating Calibre 1.0

Posted Aug 28, 2013 8:02 UTC (Wed) by fb (guest, #53265) [Link] (1 responses)

Good thing about LWN is that I come here to read about Calibre, and learn not only of its history of security vulnerabilities but also of why I should not use Google Play.

The history with Google Play linked through "One should not have to worry about things like having one's books deleted as a consequence of crossing a national border." only makes me appreciate my Kindle even more.

The Kindle gets bashed (in some portions of the internet) because of Amazon's use of the kill switch on Orwell's 1984 some years ago, but honestly I find it the perfect /trustworthy/ reading device (ok, short of having a "Debian/Ubuntu for e-readers") as it can spend its entire life with the "Airplace Mode" ON with books being copied through USB.

Calibrating Calibre 1.0

Posted Sep 6, 2013 6:35 UTC (Fri) by eternaleye (guest, #67051) [Link]

A device running Android (or Replicant) with FBReader would be a good bet. If you go with Replicant and FBReader, it's all FLOSS software, and FBReader can (AFAIU) pull from Calibre's OPDS server mode over the network.

Bam, a fully trustworthy e-reader device.

Google Play bug

Posted Aug 30, 2013 11:57 UTC (Fri) by erwbgy (subscriber, #4104) [Link] (2 responses)

Apparently the Google Play issue was due to a bug - from Gizmodo:

Update: Google has reached out to use to explain that this is not normal or intended behavior for the Google Books app, but instead the result of a bug. Google Play stores will not allow additional purchases while in territories where they do not operate, but this active removal of content was apparently an isolated issue unrelated to normal Google Books DRM behavior.

Google Play bug

Posted Aug 31, 2013 12:26 UTC (Sat) by njwhite (guest, #51848) [Link] (1 responses)

Still a pretty severe bug though, that I can't imagine happening with software which hadn't been written with DRM in mind. I've never seen a music playing program decide it would delete all of its music library after an update (though saying that, I suppose it wouldn't suprise me if I heard of something DRM loving like iTunes doing that).

Google Play bug

Posted Aug 31, 2013 19:10 UTC (Sat) by raven667 (subscriber, #5198) [Link]

I have seen programs try to sync something with nothing and end up with nothing which is a frustrating bug

DRM

Posted Sep 2, 2013 17:34 UTC (Mon) by dwmw2 (subscriber, #2063) [Link]

FWIW there are plugins for Calibre which let you easily import DRM-afflicted books, stripping the DRM from them automatically as you go.

I usually run Kindle4PC or Adobe Digital Editions under wine, and then import the download books into Calibre.

A Digital Editions purchase gives you a .acsm file which is essentially an XML blob describing the download. Would be really nice if someone reverse-engineered that and we had a tool which would do it all, rather than needing to run DE under wine and then de-DRM the resulting download. A few beers and a night with mitmproxy should do it.

Would be nice if we had a Kindle implementation which worked the same, of course, but that seems much less likely to happen (and to keep working if it did).


Copyright © 2013, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds