| |||||||||||||
Kamp: A Generation Lost in the BazaarKamp: A Generation Lost in the BazaarPosted Aug 20, 2012 15:21 UTC (Mon) by corbet (editor, #1)In reply to: Kamp: A Generation Lost in the Bazaar by patrick_g Parent article: Kamp: A Generation Lost in the Bazaar Lots of stuff shipped with a collection of makefiles...Makefile.sunos, Makefile.hpux, Makefile.irix, ... That and lots of #ifdef lines.
(Log in to post comments)
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 18:10 UTC (Mon) by samroberts (subscriber, #46749) [Link]
Also, imake, but even the X project it was written for has abandoned it. Disliking autotools shouldn't lead you into the belief that nothing can be worse!
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 19:43 UTC (Mon) by wahern (subscriber, #37304) [Link]
Shipping a bunch of different Makefiles isn't necessarily worse. Consider that in the intervening 20 years, all of the extant platforms have converged significantly, due in no small part to POSIX and SUS. Today, apart from major feature differences for which there's no simple solution, there are fewer incompatibilities. Papering them over requires some ifdef's, yes, but it's not that bad. Using separate Makefiles (1) allows automatic builds to work in the common case, and (2) simplifies fixing issues because the learning curve wrt the build for that project is shallow.
In the modern context, the value of autotools and similar is less, and the relative cost higher.
The bigger issue today is remaining "portable" between Linux and Windows. That causes many times more headaches than in even the most backwards of Makefile hacks.
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 21:23 UTC (Mon) by cmccabe (guest, #60281) [Link]
I hate how the discussion about build systems gets framed as a binary choice between using hand-rolled Makefiles and using autotools. There are other build systems in the world, like CMake! And they even have portability to Windows, which autotools does not.
The so-called portability of autotools is a mirage, anyway. autotools requires you to write portable shell scripts in order to be portable. Since most developers couldn't do this to save their lives, it is effectively Linux-only.
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 22:29 UTC (Mon) by nix (subscriber, #2304) [Link]
People writing "configure.ac"s with significant shell fragment content (which is surely going to be checking for something more complex than the mere presence or absence of functions) but yet who can't even be bothered to read the Shellology chapter in the Autoconf manual or even use the freely-available POSIX spec on opengroup.org when writing their shell fragments... well, words fail me. People like that *deserve* to lose.
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 22:35 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
You mean 99% of developers who simply treat autotools as a magic box that prints cryptic incantations on their consoles?
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 1:11 UTC (Tue) by cmccabe (guest, #60281) [Link]
Let's just consider a random configure.ac that happens to be laying around on my computer.
http://ceph.com/git/?p=ceph.git;a=blob;f=src/gtest/config...
There's something non-portable about this one. Can you spot what it is?
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 2:04 UTC (Tue) by hummassa (subscriber, #307) [Link]
That it needs pthreads?
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 5:38 UTC (Tue) by pbonzini (subscriber, #60935) [Link]
"test x == y" is unportable.
But it's otherwise a very well-written configure.ac
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 20:40 UTC (Tue) by cmccabe (guest, #60281) [Link]
Good find. I hope this puts to rest the idea that you need to write "significant shell content" to run into portability problems with autotools. You only need to write one line, or half a line.
By the way, gtest has since switched to CMake.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 22:36 UTC (Tue) by nix (subscriber, #2304) [Link]
For that project, who cares? Ceph is intrinsically nonportable, IIRC.
(Arguing that Autoconf is awful because it includes shell fragments is also vitiated in the presence of a project that uses any manually-written shell scripts at all. As most do.)
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 6:15 UTC (Wed) by cmccabe (guest, #60281) [Link]
As a sometime contributor to Ceph, I can definitely say that no, it is NOT "intrinsically nonportable." The servers which implement the filesystem itself are all userspace processes, which should, in theory, be portable to any POSIX OS. In fact, you can see a few places in the source where someone added #ifdef DARWIN and similar things. There was never a serious effort to get things working well on non-Linux platforms, just due to limited resources and interest. But that may change in the future.
And yes, the kernel component is Linux-only, but there are other ways to access the filesystem.
> (Arguing that Autoconf is awful because it includes shell
All the standalone shell scripts that are currently in the Ceph server repo are used for testing purposes. That's why they're in directories with names like 'qa', 'test', and so forth. Later on, the project started moving more towards Python for testing... but I digress.
Would you be ok with a build system that required you to write Makefiles in C? Oh, but your project includes .c files, so your arguments are vitiated. Come on, this is absurd, and you know it.
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 7:40 UTC (Wed) by cmccabe (guest, #60281) [Link]
In case it's not clear, the opinions about autotools here are my own, not necessarily those of other Ceph developers. I'm not claiming to speak for them (although if you are reading this, guys, consider ditching autotools).
C.
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 13:38 UTC (Wed) by nix (subscriber, #2304) [Link]
I'd be unhappy with a build system that required me to write makefiles in C because C is a bad language for writing makefiles in. However, I note that makefiles are themselves little more than dependencies tied to shell script fragments, so clearly the shell and make are relatively tightly associated (indeed, make knows internally what characters are common shell metacharacters, and stuff like that). configure scripts can in any case accept C fragments too (and Pascal fragments, C++ fragments and the like). They are used where they are suitable, i.e. when trying to figure out properties of the compilation environment.
What I was trying to point out is that the nonportability of shell script is a bad argument against autoconf if you use any other shell script in your source tree, because you are just as likely to introduce nonportabilities there as in the configure.ac (and they would be just as hard to detect). The portability or otherwise of C is irrelevant to this argument: your objection is a non sequitur.
configure scripts depend on the shell environment in any case, so allowing you to introduce your own shell fragments introduces no new dependencies, which was very important to the Autoconf designers. (Again, the manual *says* all this. This is not secret hidden knowledge.)
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 18:27 UTC (Wed) by cmccabe (guest, #60281) [Link]
> However, I note that makefiles are themselves little more than
> dependencies tied to shell script fragments, so clearly the shell and make > are relatively tightly associated (indeed, make knows internally what > characters are common shell metacharacters, and stuff like that).
Again, this is not a binary choice between plain old Makefiles and autotools. My choice is CMake.
CMake has its own scripting language which is portable across platforms (including Windows), versioned, and specifically designed for this use-case. Using the policy mechanism you can tell CMake "act as if you were CMake 2.6." Anyone with a version of CMake higher than or equal to 2.6 can then compile your project exactly as you intended. Shell scripts don't have any of these things.
Yes, shell scripts are appropriate for some use cases. But a build system is just not one of them. Let's use the right tool for the job rather than trying to explain how with a little duct tape, a plunger can be used as a hairbrush.
At the end of the day, nothing can make your code portable but you. But the build system can help you or hurt you. If it closes off platforms to you, like autotools closes off Windows to you, that hurts. If it forces you to write shell scripts to do everything, that hurts portability too. I've spent a long time fixing build breakages that resulted from autotools' poor choices. The question of what kind of Makefile.am or configure.ac a superintelligent Zen master who has studied AIX, IRIX, csh, tcsh, bash, zsh, dash, and every other shell out there might write is not interesting to me. In the real world, autotools = not portable.
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 19:21 UTC (Wed) by hummassa (subscriber, #307) [Link]
That is really not my experience. I prefer CMake to autotools for new code, but CMake was not really crosscompiler-friendly the last time I checked. Autotools is really much more complicated, but it will even work in MinGW/CygWin without a lot of problems... and it will suport crosscompiling without a lot of EXTRA effort. Yes, it is possible that CMake works well with MSVC, but I wouldn't know about it.
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 20:17 UTC (Wed) by boudewijn (subscriber, #14185) [Link]
I know that it does work well with msvc. I also know that it works well when cross-compiling from linux to windows. Maybe the setup is a bit more involved than with autotools, maybe it's not -- maybe the people feel it is more involved just don't remember the effort they expended on the autotools setup when they had to work with it for the first time. And yes, cmake works very well with mingw on windows itself.
Really, people who have an opinion on cmake and autotools in relation to windows, whether it's with mingw, msvc, or icc, but haven't checked out the way the kde-windows project uses cmake should check it out. Doing that will make their life _much_ easier.
Kamp: A Generation Lost in the Bazaar Posted Aug 24, 2012 1:16 UTC (Fri) by cmccabe (guest, #60281) [Link]
CMake has supported cross-compiling for a long time. http://www.vtk.org/Wiki/CMake_Cross_Compiling
Various projects may have trouble with cross-compiling, because their authors did something problematic (like assuming that they could build a binary with the target compiler and then run it, or using hard-coded paths in a dumb way). However, this is just as true with autotools. Fix the bugs and move on.
Kamp: A Generation Lost in the Bazaar Posted Aug 23, 2012 5:46 UTC (Thu) by quotemstr (subscriber, #45331) [Link]
> can't even be bothered to read the Shellology chapter
I read the chapter and decided to ignore it. I'm not interested in working around ancient HP-UX bugs or supporting systems too old to have decent support for here-documents.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 0:33 UTC (Tue) by HelloWorld (guest, #56129) [Link]
CMake is a steaming pile of crap and the sooner it dies, the better.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 1:56 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Yet it works and works great. It can build usable project files for XCode and MSVS, and it's the only real build project that can do it.
Scons and waf are also nice.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 11:42 UTC (Tue) by cortana (subscriber, #24596) [Link]
Cross compiling with it is a significant pain. The way dependencies are searched for is extremely hard to debug--it either works or it fails with no indication of why. I usually end up having to hack debug statements throughout /usr/share/cmake-2.8/Modules in order to work out what's going on. IIRC the breakage is usually caused by the macros for finding dependencies not taking into account that CMAKE_FIND_ROOT_PATH would ever be used. There is also the magical handling of the toolchain file (why can't the relevant variables be set with -D on the command line?), but that is a minor issue that can be easily worked around once the necessary amount of hair has been pulled out.
Some day I hope to use a build system that does not make me pine for the autotools, but that day has not yet come to pass.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 11:46 UTC (Tue) by cortana (subscriber, #24596) [Link]
I'll add that I evaluate a build system by considering how it caters to the needs of the following categories of user:
1. Project developers
CMake and other systems that are sadly popular such as SCons or waf, fall over when considering the needs of those who aren't project developers themselves. For all its warts, autotools is the only system that caters adequately to the needs of all of the above groups.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 14:06 UTC (Tue) by hummassa (subscriber, #307) [Link]
CMake is the second best build system, far far far behind autotools. Yes, it does not cater well to Group 4 (especially if cross-compiling is needed), but it's OK for the other three.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 15:27 UTC (Tue) by pboddie (subscriber, #50784) [Link]
I've spent the last few days cross-compiling, and although the overhead of the build system in question may involve the use of autotools by some projects, my own rant would have something to do with people writing and promoting new tools whilst ignoring use-cases that result in those new tools being completely useless for various groups of people.
I've seen quite a bit of build system evolution and apathy towards cross-compiling in my time - just search the Python bug-tracker for the debris of attempts to get Python's build system to stop doing "I need to run myself on the host now!" and actually work for cross-compilation - and hoping that target devices will be powerful enough for everyone to forget about cross-compilation isn't a useful strategy.
I suppose people can argue that the "anarchy" has not only resulted in the construction of autotools - a dubious claim, since autotools does consider things that later tools ignore or miss, and thus does seem to have had some thought given to its functionality - but also the lack of maintenance of autotools and the proliferation of other tools that don't always measure up. Those things have more to do with the perceived cost of fixing things or developing new things than any particular development strategy, though.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 14:22 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
I felt a little sick reading this:
>For all its warts, autotools is the only system that caters adequately to the needs of all of the above groups.
Autotools at most is an equal-opportunity serial murder, which chooses its victims from any of the four listed groups indiscriminately.
Kamp: A Generation Lost in the Bazaar Posted Aug 22, 2012 17:21 UTC (Wed) by dashesy (subscriber, #74652) [Link]
I enjoyed your analogy, I may borrow it for many discussions at work I guess.
I wish LWN adds the "Unread comments by $USER"
Kamp: A Generation Lost in the Bazaar Posted Aug 23, 2012 22:09 UTC (Thu) by nix (subscriber, #2304) [Link]
Recent versions of the autotools add parallel murder support if you set the AUTOMURDER_JOBS environment variable.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 18:22 UTC (Tue) by boudewijn (subscriber, #14185) [Link]
Well, actually I gave it a first try not that long ago, and I actually found it really easy to cross-compile and package my first application from Linux for Window using cmake. If you have to hack around like you describe, I suspect you're actually taking the wrong approach to begin with.
On the other hand, the cmake systems for all the dozens of dependencies I used in that case had been tested in KDE's wonderful emerge project beforehand, and that helped a lot.
Kamp: A Generation Lost in the Bazaar Posted Aug 21, 2012 8:05 UTC (Tue) by BlueLightning (subscriber, #38978) [Link]
Funny, those are usually my sentiments about autotools when I am forced to deal with them....
Kamp: A Generation Lost in the Bazaar Posted Aug 20, 2012 22:55 UTC (Mon) by marcH (subscriber, #57642) [Link]
> Disliking autotools shouldn't lead you into the belief that nothing can be worse!
Wow, a quadruple negation. Totally appropriate for a discussion about autotools; well done!
|
|||||||||||||