More information on the X11R7.0/X11R6.9 release [LWN.net]

More information on the X11R7.0/X11R6.9 release

Posted Dec 23, 2005 22:39 UTC (Fri) by beagnach (guest, #32987) [Link]

seconded. this is not slashdot.

More information on the X11R7.0/X11R6.9 release

Posted Dec 23, 2005 22:51 UTC (Fri) by drag (subscriber, #31333) [Link]

No he's right. I misread what he was saying.

That's what I get for firing off a quick reply before running of to work and I would like to apologize to him for jumping the gun.

Imake sucks and on that he and I agrees.

The thing is that with autotools it's known and usefull for people that don't know the code.

I can take code directly off of the cvs server and run the autoconf stuff to generate a make file, compile, and install the software without having to know anything about the source code that I am dealing with..

Autotool/Autoconf/Autowhatever takes into account not only GNU/Linux-style systems, but other systems like windows, and dozens of other platforms and other toolchains.

This should help people with porting efforts, I expect. If your going to write a makefile for portable software you'd have to have intiment knowledge of not only your system your working on, but every other system that you expect people will want to use your software on.

I understand that it's probably not easy to deal with. But it's not going to be easy to make a portable code base with manually building make files either.

Maybe something like scons or whatnot would of been better, but it's not like those don't have their own problems either.

autoconf is OK, dunno about the rest

Posted Dec 24, 2005 11:28 UTC (Sat) by kleptog (subscriber, #1183) [Link]

I think there's some confusion here. Just because you use autoconf, doesn't mean the Makefiles are automatically generated.

For example, PostgreSQL uses autoconf. Why? Because it has a standard interface for specifying extensions and where they are. Because it has standard methods for determine if header file X exists, what is the signature of signal, does the compiler understand inline, does the function pstat exist, etc? You should have seen the mess it was before.

I think people seriously overestimate source code compatability once you step outside of gcc/glibc or linux/*BSD. PostgreSQL has some of the best code I've seen, no #ifdef scattered through the code. The code is written for sane platforms and the quirks are dealt with in the ports directory.

The results of all this creates one header file to #include and one Makefile.global with various parameters substituted. All the other Makefiles are hand written. And it definitly increases portability because now no-one needs to write code to differentiate different releases of AIX or to remember whether the C compiler can do 64-bit arithmetic, because configure simply tells you what you need to know. If a new release of BSD changed the default signal semantics to POSIX, we wouldn't even notice, because configure would set the flag and the code will be compiled accordingly.

I agree with the comments about libtool, I havn't seen a compelling reason to use it, it seems to cause more problems that it solves. PostgreSQL compiles and links shared libraries on a dozen different platforms without that much difficulty. Completely autogenerating makefiles seems like a mess too, but you don't need to do that. autoconf, by itself, *is* useful.

autoconf is OK, dunno about the rest

Posted Dec 25, 2005 7:01 UTC (Sun) by evgeny (guest, #774) [Link]

> Just because you use autoconf, doesn't mean the Makefiles are automatically generated.

Agreed here absolutely. In all my projects, I generate only two files with autoconf - a config.h and a Make.conf, which are #included into all other source files and makefiles, respectively. This also greatly helps in portabilty to other platforms lacking autoconf support: as a rule, these are pretty API/ABI-stable, so a replacement for "configure" is trivially written in the native script language which basically fills in a predefined template modulus a few options the user might want to alter (equivalents of --with-options etc).

Having said that, there is, undeniably, quite a way ahead of autoconf for many possible improvements.

More information on the X11R7.0/X11R6.9 release

Posted Dec 25, 2005 1:37 UTC (Sun) by khim (subscriber, #9252) [Link]

If you don't know how to use make, maybe you should be programming in VisualBasic or Perl.

And you can not write you programs without C compiler and assembler in hex code then you should not program at all, right ?

Fundamentally the goal of C, Make and Autotools is the same: reduce amount of hand-written code and make the same code usable on different systems. Sometimes things are not going well, C compiler miscompiles code, make forget to recompile something (especially parallel make) and creates mess, etc.

Now you happily swallow problems with C compilers and/or Make (it's not easy to write portable makefile BTW) but claim autotools do not solve real problems and are not needed. Sure: you can do anything doable with autotools with just Make and C compiler. Or with C compiler without Make. Or with just assembler. Or even with hex editor. But. With autotools you are doing less manual work. For the same end result (if you are carefull - and if you are not carefull nothing will help). What's not to like ?

Now, the question: why do you think autotools are evil, but gas, ld, gcc and make are not. They are solving the same problem!

NOf course autotools have bugs. Of course autotools can be misused. But the same is true for as, ld, make or gcc. Why are so hostile to autotools but not to your C compiler and linker ?

And if you need real problem autotools are solving - it's amount of hand-written code. Few simple lines per shared library, few lines per optional feature, etc. Portability can be achieved as well - but that's not the only goal. I'm yet to see the same thing done without autotools or some other makefile-generator with comparable amount of hand-written code.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 10:06 UTC (Mon) by oak (guest, #2786) [Link]

> Now, the question: why do you think autotools are evil, but gas, ld, gcc
> and make are not. They are solving the same problem!

Toolchain takes care of building, autotools take care of configuring
the software for building. These are completely different things.

Autotools is intended for configuring software for the environment
where it is built, and for most purposes it works quite fine for this.
If you instead try to do cross-configuring and cross-compiling, this base
assumption is false. As a result autotools shoots itself to both feet
and falls on its face.

For more information, see e.g. this:
http://www.scratchbox.org/documentation/general/tutorials...
(I also had a FOSDEM presentation on this subject a few years ago)

Base system tools like Glibc and coretools support cross-compilation,
most of the software using autotools doesn't and even if it would,
the Autotools cross-compilation support mainly consists of replacing
a test with a guess. In general this seems somewhat fragile, so
personally I would prefer being able to set these options directly
/ explicitly. (Note: OpenEmbedded has hacks for making it easier
to coax Autotools build systems to work with cross-compilation.)

Additionally, Autotools expects the underlying system to be POSIX and
maybe even some GNU variant of this. If it's not the case, Autotools
makes building the software much harder as if Autotools stuff doesn't
work, most of the Linux software doesn't have a functioning build system.
For a fun exercise, try building Gnome GARNOME using uClibc + Busybox
as base instead of Glibc + GNU core/file/text/etc/tools.

Much finer would be to use just Autoconf for default/optional configuring
of the software and for building depend completely on GNU Make. Even
Autoconf could be nowadays to a large extent removed if code is standard
ANSI-C, all dependencies support pkg-config (which was designed to
deal with the dependency stuff) and software doesn't have configurable
parts.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 14:29 UTC (Mon) by pizza (subscriber, #46) [Link]

Cross-compiling is an interesting beast, but by and large, autotools works here too. Granted, I don't do any desktop-type cross compiling.

All of the problems I've seen with autoconf screwing up on cross-compiliation environments are due to its users (ie the software devlopers, not the end-user trying to build) not knowing how to use it properly. A tweak to the autoconf.ac file later, and bang, it works. The assumptions about paths are the biggest offenders here; but more often than not these problems are due to things being hardcoded rather than mis-mis-configured.

Cross-compilation environments are unusual in that instead of having to figure things out, everything about the target environment is known. Many of the tests run aren't relevant, as you mentioned, and thus should be explicitly specified, but if not, it's not the fault of autoconf, but the original developer not allowing stuff to be overridden to begin with. File and Library paths are the worst offenders here. It's particularly frustrating when your target architechure is the same as the cross-tool architechure, albeit with different libraries (as you mentioned in your example)

That said, it's worth repeating that the autotools are under continual development, and are always improving. Bugs are fixable, but only if reported.

Still, the bottom line is that autoconf/automake/etc solves far more problems than it creates. It's complicated, but so is the problem it's trying to solve.

More information on the X11R7.0/X11R6.9 release

Posted Dec 26, 2005 18:55 UTC (Mon) by oak (guest, #2786) [Link]

> Still, the bottom line is that autoconf/automake/etc solves far more
> problems than it creates. It's complicated, but so is the problem
> it's trying to solve.

I'm not so convinced of this, or that that the problem is relevant anymore. If it's complicated, it's anyway going to require developer to do something also to his software, not just it's build system.

Build-deps could be handled with pkg-config much easily and writing something that implements the standard configure API (configure options and writing the defines to a config.h file) shouldn't be that hard to script.

I see there two approaches to the configuration problem:

Autotools approach: Complex tool solving 99% of the problem and at the same time making the last 1% about impossible. Additionally, instead of aborting when finding a problem, it tries to go around it (the Microsoft way?) which will fail in non-obvious ways when developers do similar things with additional assumptions
New system friendly approach: few simple tools and APIs which solve 90% of the problem and let the last 10% to be moderate manual effort. Additionally, this would stop if it finds incompatibility (the Unix way?)

For example if you want to port software to a system that doesn't have all the GNU/Posix tools or has versions with incompatible / missing options, your options in case 1) are:

Re-write the software build system completely from scratch, OR
Port a huge number (several dozens) of the GNU tools for the system first

Whereas in case 2), all that could be required might be to port GNU Make and pkg-config first and then compile the source.

The basic problem of Autoconf is that it's skitsofrenic, at the same time both too trusting and too paranoid. It both tests how to deal with trivialities which different working would surprise SW developers, and includes a huge number of assumptions about the underlying system. :-)

Better would be if Autotools would require a small set of very well documented/specified system functionality (commands and their options) and it would have a test suite to test this autotools / configure compatibility. This could be Autotools system API. This way SW developers could trust that underlying system has a certain functionality instead of Autotools "dealing" with the incompatibility and SW specific checks breaking in interesting ways. It would be good if as a result there would be less runnable code in configure scripts and it would be more readable (for reasons, see list below).

For non-autotools provided checks there has to be some better way than how autotools deals with SW specific checks. When the extensions (m4 macros) are installed, it's AFAIK not e.g. differentiated whether they are for build tools or for target tools.

Here's a list of lesser problems with autoconf:

Configure depends on a lot of tools and expects them to have certain options. However, the scripts don't log everything they do, and check whether everything succeeds[1]
Running configure etc. takes more time than building the actual software, especially if software re-building is using ccache to speed it up (and code is C, not C++). It would be interesting to know how much of a typical Gentoo build goes to running Autotools and how much to compiling the actual software
Configure scripts / automake makefiles are larger than the actual code for smaller projects. This is a problem if one would want to security audit what the whole SW will do when installed (from source). I know that people could autogenerate the configure and Makefiles, but that doesn't have any standard API and although somebody could check whether the C-code does something dubious, nobody's going to fully review configure scripts...
Autoconf versions are incompatible with each other and developers don't always know how they are incompatible, they just use the version their distro happens to have

[1] I've e.g. seen a problem where less compatible "sed" implementation resulted in successful build without any errors/warnings, the produced code just was not working. We knew the reason only after writing a wrapper for all shell commands which logged what command line tools were run, with which options and return codes.