More information on the X11R7.0/X11R6.9 release [LWN.net]

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 10:06 UTC (Mon) by oak (guest, #2786)
In reply to: More information on the X11R7.0/X11R6.9 release by khim
Parent article: More information on the X11R7.0/X11R6.9 release

> Now, the question: why do you think autotools are evil, but gas, ld, gcc
> and make are not. They are solving the same problem!

Toolchain takes care of building, autotools take care of configuring
the software for building. These are completely different things.

Autotools is intended for configuring software for the environment
where it is built, and for most purposes it works quite fine for this.
If you instead try to do cross-configuring and cross-compiling, this base
assumption is false. As a result autotools shoots itself to both feet
and falls on its face.

For more information, see e.g. this:
http://www.scratchbox.org/documentation/general/tutorials...
(I also had a FOSDEM presentation on this subject a few years ago)

Base system tools like Glibc and coretools support cross-compilation,
most of the software using autotools doesn't and even if it would,
the Autotools cross-compilation support mainly consists of replacing
a test with a guess. In general this seems somewhat fragile, so
personally I would prefer being able to set these options directly
/ explicitly. (Note: OpenEmbedded has hacks for making it easier
to coax Autotools build systems to work with cross-compilation.)

Additionally, Autotools expects the underlying system to be POSIX and
maybe even some GNU variant of this. If it's not the case, Autotools
makes building the software much harder as if Autotools stuff doesn't
work, most of the Linux software doesn't have a functioning build system.
For a fun exercise, try building Gnome GARNOME using uClibc + Busybox
as base instead of Glibc + GNU core/file/text/etc/tools.

Much finer would be to use just Autoconf for default/optional configuring
of the software and for building depend completely on GNU Make. Even
Autoconf could be nowadays to a large extent removed if code is standard
ANSI-C, all dependencies support pkg-config (which was designed to
deal with the dependency stuff) and software doesn't have configurable
parts.

(Log in to post comments)

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 14:29 UTC (Mon) by pizza (subscriber, #46) [Link]

Cross-compiling is an interesting beast, but by and large, autotools works here too. Granted, I don't do any desktop-type cross compiling.

All of the problems I've seen with autoconf screwing up on cross-compiliation environments are due to its users (ie the software devlopers, not the end-user trying to build) not knowing how to use it properly. A tweak to the autoconf.ac file later, and bang, it works. The assumptions about paths are the biggest offenders here; but more often than not these problems are due to things being hardcoded rather than mis-mis-configured.

Cross-compilation environments are unusual in that instead of having to figure things out, everything about the target environment is known. Many of the tests run aren't relevant, as you mentioned, and thus should be explicitly specified, but if not, it's not the fault of autoconf, but the original developer not allowing stuff to be overridden to begin with. File and Library paths are the worst offenders here. It's particularly frustrating when your target architechure is the same as the cross-tool architechure, albeit with different libraries (as you mentioned in your example)

That said, it's worth repeating that the autotools are under continual development, and are always improving. Bugs are fixable, but only if reported.

Still, the bottom line is that autoconf/automake/etc solves far more problems than it creates. It's complicated, but so is the problem it's trying to solve.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 18:55 UTC (Mon) by oak (guest, #2786) [Link]

> Still, the bottom line is that autoconf/automake/etc solves far more
> problems than it creates. It's complicated, but so is the problem
> it's trying to solve.

I'm not so convinced of this, or that that the problem is relevant anymore. If it's complicated, it's anyway going to require developer to do something also to his software, not just it's build system.

Build-deps could be handled with pkg-config much easily and writing something that implements the standard configure API (configure options and writing the defines to a config.h file) shouldn't be that hard to script.

I see there two approaches to the configuration problem:

Autotools approach: Complex tool solving 99% of the problem and at the same time making the last 1% about impossible. Additionally, instead of aborting when finding a problem, it tries to go around it (the Microsoft way?) which will fail in non-obvious ways when developers do similar things with additional assumptions
New system friendly approach: few simple tools and APIs which solve 90% of the problem and let the last 10% to be moderate manual effort. Additionally, this would stop if it finds incompatibility (the Unix way?)

For example if you want to port software to a system that doesn't have all the GNU/Posix tools or has versions with incompatible / missing options, your options in case 1) are:

Re-write the software build system completely from scratch, OR
Port a huge number (several dozens) of the GNU tools for the system first

Whereas in case 2), all that could be required might be to port GNU Make and pkg-config first and then compile the source.

The basic problem of Autoconf is that it's skitsofrenic, at the same time both too trusting and too paranoid. It both tests how to deal with trivialities which different working would surprise SW developers, and includes a huge number of assumptions about the underlying system. :-)

Better would be if Autotools would require a small set of very well documented/specified system functionality (commands and their options) and it would have a test suite to test this autotools / configure compatibility. This could be Autotools system API. This way SW developers could trust that underlying system has a certain functionality instead of Autotools "dealing" with the incompatibility and SW specific checks breaking in interesting ways. It would be good if as a result there would be less runnable code in configure scripts and it would be more readable (for reasons, see list below).

For non-autotools provided checks there has to be some better way than how autotools deals with SW specific checks. When the extensions (m4 macros) are installed, it's AFAIK not e.g. differentiated whether they are for build tools or for target tools.

Here's a list of lesser problems with autoconf:

Configure depends on a lot of tools and expects them to have certain options. However, the scripts don't log everything they do, and check whether everything succeeds[1]
Running configure etc. takes more time than building the actual software, especially if software re-building is using ccache to speed it up (and code is C, not C++). It would be interesting to know how much of a typical Gentoo build goes to running Autotools and how much to compiling the actual software
Configure scripts / automake makefiles are larger than the actual code for smaller projects. This is a problem if one would want to security audit what the whole SW will do when installed (from source). I know that people could autogenerate the configure and Makefiles, but that doesn't have any standard API and although somebody could check whether the C-code does something dubious, nobody's going to fully review configure scripts...
Autoconf versions are incompatible with each other and developers don't always know how they are incompatible, they just use the version their distro happens to have

[1] I've e.g. seen a problem where less compatible "sed" implementation resulted in successful build without any errors/warnings, the produced code just was not working. We knew the reason only after writing a wrapper for all shell commands which logged what command line tools were run, with which options and return codes.