LWN.net Logo

Coming soon: Sphinx 1.0

By Jonathan Corbet
June 23, 2010
Your editor, naturally enough, has a certain interest in systems designed for the documentation of code and other programming-related topics. The perfect system is hard to find. Word processors tend to just get in the way; they put the focus on appearance instead of content, and the result is hard to process with automated tools. Text-oriented systems tend to work better, but one need only write a book chapter or two in <?DocBook?> before even Emacs-strengthened little fingers collapse under the strain. TeX is a great tool, but, as your editor's dog-eared copy of the TeXbook will attest, attaining (and maintaining) the proper level of TeXpertise requires a non-trivial commitment of time. So the search for better alternatives goes on. In this context, your editor recently stumbled across Sphinx, which is headed toward a 1.0 release one of these days.

Sphinx was originally created for use in the generation of the Python documentation, but it has grown to the point that quite a few other projects (including Bazaar, Blender, Calibre, Django, Mayavi, etc) are using it as well. It remains heavily oriented toward code documentation, and it has a number of features (such as extracting documentation strings from functions) to support that goal, but it's not limited to that role.

At the core of Sphinx is the reStructuredText markup language. It's a simple language, aimed at minimizing excessive decoration and making the source look as much like the final product as possible. So paragraphs are delineated by blank lines, *italics* becomes italics, bulleted list items begin with asterisks, etc. It's a straightforward, low-ceremony way of getting the text into the system.

What Sphinx has done, though, is to add a long list of features and directives on top of reStructuredText. Much of it is intended to make code documentation easy; for example, a C function would be introduced with:

    .. cfunction:: void kthread_bind(struct task_struct *k, unsigned int cpu);

When the documentation is generated, this code will be marked up as desired (Pygments is used for that task) and incorporated into the document. Similar directives exist for the documentation of variables, exceptions, classes, methods, command-line options, and so on.

Beyond that, there are a number of features intended to help the creation of extensive, multi-volume documents. Sphinx can generate tables of contents which span multiple documents. Index entries will automatically be made from function definitions, but there's a mechanism for adding manual index entries as well. Code examples can be included directly from source files - an invaluable feature for anybody trying to keep documentation and code in sync. There is also an extension mechanism allowing projects to add any special directives they may need.

Sphinx can generate output in HTML format or in PDF by way of either LaTeX or rst2pdf. There is a theme mechanism allowing the customization of the output, which generally looks pretty nice, if perhaps just a little on the gaudy side.

The current version is 0.6.7, which was released in early June. The development version is 1.0b2, as of this writing. The headline feature for 1.0 appears to be "domains," which provide collections of directives for specific programming languages. Domains, in other words, are another step in turning Sphinx from a Python-specific tool into something more generally applicable. Another important new feature is a builder which can output documents in the Epub format for mobile reader devices. Some new themes have been added. There's quite a bit more; see the release notes for details. While 1.0 is still in development, a look through the discussions on the project's mailing list suggests that quite a few people are using it.

This article, clearly, is a superficial overview. To truly describe the ups and downs of a tool like Sphinx would require writing a book with it - something your editor was unable to find the time to do by the weekly LWN deadline. There are larger projects currently under consideration, though, and Sphinx is very much on the radar for those. Any tool which makes the creation of high-quality documentation easier can only be a good thing.


(Log in to post comments)

Doxygen?

Posted Jun 24, 2010 14:22 UTC (Thu) by kraftcheck (guest, #35072) [Link]

How does this compare to existing tools such as doxygen?

Doxygen?

Posted Jun 24, 2010 15:24 UTC (Thu) by corbet (editor, #1) [Link]

I'm not hugely familiar with doxygen, but my understanding is that it's heavily oriented toward reference manuals with the text actually included within the source itself. Sphinx manuals are separate entities. My sense is that Sphinx would be better if there is a lot of text aside from the source documentation itself, but I'm not speaking from a position of great experience.

Doxygen?

Posted Jun 25, 2010 8:13 UTC (Fri) by spaetz (subscriber, #32870) [Link]

Doxygen pulls everything out of the code and works fully automatic AFAIK.

sphynx works on a bit more semiautomatic way (which leads to way prettier and better understandable documentation). You can have a class or a function be autodocumented, but you can also easily drop unneeded functions that draw attention away from how the code really works, or from API that is supposed to be non-public. You can easily reorder the way functions are listed, you can add introductory (or inbetween) text and comments.

Sphynx can provide more of a reference book with a structure that you can decide on, yet it pulls out as much (or little) autodocumentation from out of the source code, so you don't have to duplicate documentation.

halibut

Posted Jun 24, 2010 22:11 UTC (Thu) by lonely_bear (subscriber, #2726) [Link]

I did not use it, but encounter it when I was looking around for typeseting documents.

http://www.chiark.greenend.org.uk/~sgtatham/halibut/

Coming soon: Sphinx 1.0

Posted Jun 24, 2010 23:13 UTC (Thu) by tseaver (subscriber, #1544) [Link]

FWIW, my business partner has written a book with Sphinx:

http://bfg.repoze.org/book

The book is made from the same ReST source as the online docs:

http://docs.repoze.org/bfg/1.2/

which you can see in source form here:

http://hg.repoze.org/repoze.bfg/file/e4241664d137/docs

http://bzr.repoze.org/repoze.bfg/trunk/files/head%3A/docs/

http://git.repoze.org/?p=repoze.bfg/.git;a=tree;f=docs;h=...

We were able to self publish the book at CreateSpace / Amazon:

https://www.createspace.com/3422488

http://www.amazon.com/repoze-bfg-Web-Application-Framewor...

We have switched to using Sphinx for all project-related documentation,
and would recommend it heartily to anyone.

Copyright © 2010, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds