More information on the X11R7.0/X11R6.9 release [LWN.net]

More information on the X11R7.0/X11R6.9 release

Posted Dec 25, 2005 1:37 UTC (Sun) by khim (subscriber, #9252)
In reply to: More information on the X11R7.0/X11R6.9 release by Zarathustra
Parent article: More information on the X11R7.0/X11R6.9 release

If you don't know how to use make, maybe you should be programming in VisualBasic or Perl.

And you can not write you programs without C compiler and assembler in hex code then you should not program at all, right ?

Fundamentally the goal of C, Make and Autotools is the same: reduce amount of hand-written code and make the same code usable on different systems. Sometimes things are not going well, C compiler miscompiles code, make forget to recompile something (especially parallel make) and creates mess, etc.

Now you happily swallow problems with C compilers and/or Make (it's not easy to write portable makefile BTW) but claim autotools do not solve real problems and are not needed. Sure: you can do anything doable with autotools with just Make and C compiler. Or with C compiler without Make. Or with just assembler. Or even with hex editor. But. With autotools you are doing less manual work. For the same end result (if you are carefull - and if you are not carefull nothing will help). What's not to like ?

Now, the question: why do you think autotools are evil, but gas, ld, gcc and make are not. They are solving the same problem!

NOf course autotools have bugs. Of course autotools can be misused. But the same is true for as, ld, make or gcc. Why are so hostile to autotools but not to your C compiler and linker ?

And if you need real problem autotools are solving - it's amount of hand-written code. Few simple lines per shared library, few lines per optional feature, etc. Portability can be achieved as well - but that's not the only goal. I'm yet to see the same thing done without autotools or some other makefile-generator with comparable amount of hand-written code.

(Log in to post comments)

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 10:06 UTC (Mon) by oak (guest, #2786) [Link]

> Now, the question: why do you think autotools are evil, but gas, ld, gcc
> and make are not. They are solving the same problem!

Toolchain takes care of building, autotools take care of configuring
the software for building. These are completely different things.

Autotools is intended for configuring software for the environment
where it is built, and for most purposes it works quite fine for this.
If you instead try to do cross-configuring and cross-compiling, this base
assumption is false. As a result autotools shoots itself to both feet
and falls on its face.

For more information, see e.g. this:
http://www.scratchbox.org/documentation/general/tutorials...
(I also had a FOSDEM presentation on this subject a few years ago)

Base system tools like Glibc and coretools support cross-compilation,
most of the software using autotools doesn't and even if it would,
the Autotools cross-compilation support mainly consists of replacing
a test with a guess. In general this seems somewhat fragile, so
personally I would prefer being able to set these options directly
/ explicitly. (Note: OpenEmbedded has hacks for making it easier
to coax Autotools build systems to work with cross-compilation.)

Additionally, Autotools expects the underlying system to be POSIX and
maybe even some GNU variant of this. If it's not the case, Autotools
makes building the software much harder as if Autotools stuff doesn't
work, most of the Linux software doesn't have a functioning build system.
For a fun exercise, try building Gnome GARNOME using uClibc + Busybox
as base instead of Glibc + GNU core/file/text/etc/tools.

Much finer would be to use just Autoconf for default/optional configuring
of the software and for building depend completely on GNU Make. Even
Autoconf could be nowadays to a large extent removed if code is standard
ANSI-C, all dependencies support pkg-config (which was designed to
deal with the dependency stuff) and software doesn't have configurable
parts.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 14:29 UTC (Mon) by pizza (subscriber, #46) [Link]

Cross-compiling is an interesting beast, but by and large, autotools works here too. Granted, I don't do any desktop-type cross compiling.

All of the problems I've seen with autoconf screwing up on cross-compiliation environments are due to its users (ie the software devlopers, not the end-user trying to build) not knowing how to use it properly. A tweak to the autoconf.ac file later, and bang, it works. The assumptions about paths are the biggest offenders here; but more often than not these problems are due to things being hardcoded rather than mis-mis-configured.

Cross-compilation environments are unusual in that instead of having to figure things out, everything about the target environment is known. Many of the tests run aren't relevant, as you mentioned, and thus should be explicitly specified, but if not, it's not the fault of autoconf, but the original developer not allowing stuff to be overridden to begin with. File and Library paths are the worst offenders here. It's particularly frustrating when your target architechure is the same as the cross-tool architechure, albeit with different libraries (as you mentioned in your example)

That said, it's worth repeating that the autotools are under continual development, and are always improving. Bugs are fixable, but only if reported.

Still, the bottom line is that autoconf/automake/etc solves far more problems than it creates. It's complicated, but so is the problem it's trying to solve.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 18:55 UTC (Mon) by oak (guest, #2786) [Link]

> Still, the bottom line is that autoconf/automake/etc solves far more
> problems than it creates. It's complicated, but so is the problem
> it's trying to solve.

I'm not so convinced of this, or that that the problem is relevant anymore. If it's complicated, it's anyway going to require developer to do something also to his software, not just it's build system.

Build-deps could be handled with pkg-config much easily and writing something that implements the standard configure API (configure options and writing the defines to a config.h file) shouldn't be that hard to script.

I see there two approaches to the configuration problem:

Autotools approach: Complex tool solving 99% of the problem and at the same time making the last 1% about impossible. Additionally, instead of aborting when finding a problem, it tries to go around it (the Microsoft way?) which will fail in non-obvious ways when developers do similar things with additional assumptions
New system friendly approach: few simple tools and APIs which solve 90% of the problem and let the last 10% to be moderate manual effort. Additionally, this would stop if it finds incompatibility (the Unix way?)

For example if you want to port software to a system that doesn't have all the GNU/Posix tools or has versions with incompatible / missing options, your options in case 1) are:

Re-write the software build system completely from scratch, OR
Port a huge number (several dozens) of the GNU tools for the system first

Whereas in case 2), all that could be required might be to port GNU Make and pkg-config first and then compile the source.

The basic problem of Autoconf is that it's skitsofrenic, at the same time both too trusting and too paranoid. It both tests how to deal with trivialities which different working would surprise SW developers, and includes a huge number of assumptions about the underlying system. :-)

Better would be if Autotools would require a small set of very well documented/specified system functionality (commands and their options) and it would have a test suite to test this autotools / configure compatibility. This could be Autotools system API. This way SW developers could trust that underlying system has a certain functionality instead of Autotools "dealing" with the incompatibility and SW specific checks breaking in interesting ways. It would be good if as a result there would be less runnable code in configure scripts and it would be more readable (for reasons, see list below).

For non-autotools provided checks there has to be some better way than how autotools deals with SW specific checks. When the extensions (m4 macros) are installed, it's AFAIK not e.g. differentiated whether they are for build tools or for target tools.

Here's a list of lesser problems with autoconf:

Configure depends on a lot of tools and expects them to have certain options. However, the scripts don't log everything they do, and check whether everything succeeds[1]
Running configure etc. takes more time than building the actual software, especially if software re-building is using ccache to speed it up (and code is C, not C++). It would be interesting to know how much of a typical Gentoo build goes to running Autotools and how much to compiling the actual software
Configure scripts / automake makefiles are larger than the actual code for smaller projects. This is a problem if one would want to security audit what the whole SW will do when installed (from source). I know that people could autogenerate the configure and Makefiles, but that doesn't have any standard API and although somebody could check whether the C-code does something dubious, nobody's going to fully review configure scripts...
Autoconf versions are incompatible with each other and developers don't always know how they are incompatible, they just use the version their distro happens to have

[1] I've e.g. seen a problem where less compatible "sed" implementation resulted in successful build without any errors/warnings, the produced code just was not working. We knew the reason only after writing a wrapper for all shell commands which logged what command line tools were run, with which options and return codes.