This is stupid argument... [LWN.net]

That's why I've said "not just distribution guys".

Posted Nov 9, 2011 13:42 UTC (Wed) by khim (subscriber, #9252) [Link]

Bundling unrelated libraries with your project is a big no-no on Linux.

I don't know why will you ever want to add "unrelated" library - usually you bundle related library :-)

Indeed, even GNU Hello does that.

Whether or not you agree with it, most distributions have a policy of "no bundled libraries."

That's different. That's for packagers in distributions. And yes, autotools support them just fine, too. Aforementioned GNU Hello will only use bundled library if gettext is not available on the host system (and on host system with GLibC it's always available).

Have you ever actually developed a piece of cross-platform software using CMake or Scons?

Sure. In fact that's why I'm so against them. I do know them - and I hate them. We use Scons here for Native Client development - and I hate it. Simple things which are trivial with autotools (for example: "compile the same sources for four different platforms: one native and three cross" or "compile the some sources twice using different compilers to compare output") become serious problem and require a lot of kludges.

You will be able to develop software faster and with fewer hassles.

Somehow I'm not seeing this. Not only build is unbearably long (this is Scons problem, CMake does not share this particular problem), but I have nothing like "make distcheck". A lot of similar simple amenities are lost when you start using "modern advanced build systems".

Imagine if Lennart had been discussing systemd with someone making the same arguments as you.

I'd laugh very loud.

installation/uninstallation

...are hard to do with SysV init scripts...

standard adherence

...every distribution is unique, there are no standard amond sysv-init based systems...

portability between distros

...does not exist: if you'll try to move startup script from RedHat to Debian or back it'll not work... even if we'll forget about things like Gentoo...

Would Lennart have accepted (his own) argument, and abandoned the systemd project?

Why would he abandon it? The right answer (and this is exactly what Lennart did) is to discuss the common tasks people need to perform and translate "sysv-init solutions" to "systemd solutions". In fact Lennart spent sizable effort doing it.

He knew that making better software sometimes requires breaking compatibility with the old, obsolete software.

Bwa-ha-ha. Don't make me explode. The very description of systemd is systemd is a system and service manager for Linux, compatible with SysV and LSB init scripts (emphasis mine). One of the PulseAudio FAQs is Can I get OSS and ALSA applications to work with PulseAudio? and answer is, of course, yes.

Lennart is low-level guy, not GNOME guy. He knows compatibility is important. That is why his creations are accepted as a replacement for the "old way". Apparently "new, improved" build systems are created by guys who don't know that. Oh, well, their loss, then.

Compatibility can never be perfect, I understand that, but when I'm presented with "new super-puper build system" and ask how to do simple and important (to me) tasks I usually hear either "nobody needs that" or "you can probably invent a kludge" or even "patches are welcome". Sorry, guys, but it's your resposibility to provide backward-compatibility with "the old way" in your creation, not mine. I can live with rare and insignificant problems but when I ask trivial, simple, obvious, question "how to add autoconfiscated library to my build" and hear some tales about how I should not do that... well, sorry, no way I'd accept this build system.

But when it comes to build systems, people are still repeating the old myth that there are no viable alternatives to autotools-- that everything else is somehow suspect or tainted, that nobody will ever be able to learn the new thing. This is complete BS.

Sorry, but no, this is not a "complete BS". You can say anything you want about Lennart, but the important fact is that both systemd and pulseaudio do include support for important predecessors while CMake, Scons, Waf and others in their arrogance ignore autotools. Well, it's their problem, not mine. They may be interesting for people who need to support Windows more then they need to support GNU, but in the GNU world autotools will remain standard till someone invents good and compatible replacement.

That's why I've said "not just distribution guys".

Posted Nov 10, 2011 1:22 UTC (Thu) by cmccabe (guest, #60281) [Link]

I've never actually used SCons. I used Cons, which was the predecessor system. Cons was fairly slow, but not more so than autotools. CMake is quite fast.

> Simple things which are trivial with autotools (for example: "compile the
> same sources for four different platforms: one native and three cross" or
> "compile the some sources twice using different compilers to compare
> output") become serious problem and require a lot of kludges.

I don't see why cross-compilation would be any harder (or easier) on CMake versus autotools.

I think you are missing the point of my systemd vs. SysV init comparison. SysV init sucks. Everyone who is technically knowledgeable in this area admits this. But system administrators are very familiar with it. It is standardized in the Filesystem Hierarchy Standard.

You might argue that that standardization is a mirage, because every distribution uses it differently. In the same way, autotools standardization is a mirage because everyone uses a slightly different set of macros and generator programs. Some projects don't even use automake at all, but simply autoconf. Some projects have wrapper scripts, and some do not. Some projects check in certain generated files and others don't. And so, on as we discussed.

Like autotools, SysV wins in every way except the way that's actually important: actually being good! If you are going to recommend that people use an inferior build system because of compatibility and the difficulty of retraining, you should also recommend that they use SysV init, for the same reasons.

In fact, whenever systemd comes up in a discussion, you see a lot of posts by system administrators who are scared of the change-- and rightly so. They know that there will be a period when they will not be as familiar with the new system as the with old. They also know that the new system will have rough edges and possibly a regression or two. But guess what-- positive change can never happen when people refuse to learn something new.

That's why I've said "not just distribution guys".

Posted Nov 10, 2011 6:32 UTC (Thu) by khim (subscriber, #9252) [Link]

Cons was fairly slow, but not more so than autotools.

Sadly this is not true. Autotools scale, scons does not - it's that simple.

CMake is quite fast.

Yup. That's why I'm not concentrating on speed when I discuss "modern build system". Some of them can be rejected outright for that reason alone, but some of them are fast.

I don't see why cross-compilation would be any harder (or easier) on CMake versus autotools.

It's harder with SCons. CMake actually somewhat acceptable here. The fact that cross-compilation was added so late is still felt sometimes, but yes, the difference is not that large today. It's still large enough to be a pain, though.

In the same way, autotools standardization is a mirage because everyone uses a slightly different set of macros and generator programs.

Sure, but there are big difference: I can not drop init script from Debian in RedHat and hope that it'll work. I can add any autoconfiscated project in my own project - and that works (sometimes with minor issues, but it works).

And so, on as we discussed.

And as discussed it's minor issue. I'm deeply involved (enough for that difference to matter) with just a couple of projects. Ok, may be half-dozen, but still enough to count them on my hands. But I'm using many dozens of different other projects and they all looks the same from the outside: "./configure ; make ; make install".

If you are going to recommend that people use an inferior build system because of compatibility and the difficulty of retraining, you should also recommend that they use SysV init, for the same reasons.

Nope. As I've said: all "new" init systems (upstart, systemd) have support for "old" SysV init scripts - and they had SysV init compatibility support from the day one. They understand that they must support "old way" of doing things till "new way" takes over. Transition takes years - and all that time old way should be supported seamlessly. Autotools compatibility in build systems either does not exist at all or is an afterthought. The question arises: if developers of these systems can not make even such fundamental thing "right" then what can they do right? And indeed - there are numerous other shortcomings in these build systems.

In fact, whenever systemd comes up in a discussion, you see a lot of posts by system administrators who are scared of the change-- and rightly so.

Sure. And then you see a lot of posts where they are relived to find out that they can still use most their SysV init tricks with systemd.

But guess what - positive change can never happen when people refuse to learn something new.

Sure. But positive change can and should happen piecemeal. People must have the right to learn "something new" at their own pace. Sometimes (when change is big like with pulseaudio or systemd) disruption is inevitable - but the authors should still try to reduce it as much as possible. Build system creators stance is "forget everything you knew, you should learn our new system right away" - and this just will not do.

For example cross-compilation: I can not use my usual way (./configure --target=blah-blah-blah"). Instead I must learn to write .cmake files right away. Why? I can understand the desire to get rid of M4. Ok. But why get rid of all the amenities offered by autoconf at the same time? Ah, you want to "build new world free from problems of the old one". Ohkay. That's you choice. If you just want to create bunch of new problems - you can do that. Just... do that somewhere where I'm not involved, Ok?

That's why I've said "not just distribution guys".

Posted Nov 10, 2011 21:43 UTC (Thu) by cmccabe (guest, #60281) [Link]

> Sadly this is not true. Autotools scale, scons does not - it's that simple.

As I said before, I don't have any direct experience with scons. Thanks for posting the link to Electric Cloud's analysis. It does seem that scons has significant overhead.

Just because scons doesn't scale, though, we can't conclude that autotools does scale. The comparison in that article is not against automake, but against gmake, which is basically an implementation of plain old make. In my experience, CMake is much faster than autotools.

You keep claiming that not having a "configure" command is some kind of fundamental stumbling block for you when understanding other build systems. Really? The 'cmake' command is pretty much the CMake equivalent of configure. Like configure, you can use it to set up CFLAGS, specify build options, and so forth. Also like autotools, once you've invoked this command once, you don't have to do it again.

I have no doubt that upstart and systemd have some kind of support for dropping in sysV init scripts. But to a system administrator trying to understand why some daemon is not starting the way he wants, that is cold comfort. In practice, using the new system require retraining and maybe even (gasp!) learning some new commands. Positive change won't happen until we make it happen.

It's not important

Posted Nov 10, 2011 23:28 UTC (Thu) by khim (subscriber, #9252) [Link]

Just because scons doesn't scale, though, we can't conclude that autotools does scale. The comparison in that article is not against automake, but against gmake, which is basically an implementation of plain old make.

Automake uses simple make "behind the scenes" so it's speed is similar to make.

In my experience, CMake is much faster than autotools.

Yes, but difference is constant, it does not depend on size of project. I can live with one munite null build: not perfect, but acceptable. But Scons can easily take minutes just to say that you don't need to build anything!

You keep claiming that not having a "configure" command is some kind of fundamental stumbling block for you when understanding other build systems.

This is not a stumbling block. This is litmus test. If people don't bother to even provide such a simple thing then what else have they decided to redo without a good reason? In case of CMake the asnwer is: almost everything. Perhaps first-class Windows support requires that, I don't know, but since I'm not interested in first-class Windows support...

The 'cmake' command is pretty much the CMake equivalent of configure.

It's not backward compatible, it assumes the whole world adopted CMake already, it provides no way to use existing autoconfiscated projects as subprojects. Basically: it's my way or the highway. Not a good way to offer replacement. When KDE4 and GNOME3 did that lots of people decided to keep old version alive for as long as possible. With autotools this decision makes even more sense because autotools developers have no intention to drop everything in favor of CMake so you can continue to use autotools and ignore CMake.

I have no doubt that upstart and systemd have some kind of support for dropping in sysV init scripts.

They both work as drop-in replacements for Sysv init. You can just replace Sysv init with upstart and systemd - and everything just continue to work. However. Systemd decided not to support upstart compatibility - and as a result Ubuntu ignored it.

In practice, using the new system require retraining and maybe even (gasp!) learning some new commands.

Sure. This is where systemd largely dropped the ball. But at least it provided a cheat sheet and tried to support the same capabilities. CMake (like most "modern" systems) decided to redo everything. For example: why "./configure CC=/my/super/duper/cc" becomes "cmake -DCMAKE_C_COMPILER=/my/super/duper/cc" ?

CMake was clearly built by people who wanted to "throw away all the legacy" - eventually they added most of it back, but in mangled and crippled form. And they arrogantly assume that everyone will want to adopt CMake: all tutorials assume I want to convert everything to CMake right away and that I'll never want to go back. This level of arrogance does not inspire confidence, sorry.

That's why I've said "not just distribution guys".

Posted Nov 11, 2011 16:48 UTC (Fri) by nix (subscriber, #2304) [Link]

FWIW, autotools scales reasonably well. It only really falls over speed-wise when you start using things like gnulib, which because they test for every bug you can possibly imagine tend to take minutes to run a multimegabyte configure script.

The actual build, well, since cmake generates makefiles, the problem must be something other than cmake-versus-GNU Make. And it is. Firstly, a lot of projects, especially older projects, are rife with recursive make, which the advent of parallel compilation has made particularly clear was a bad idea from the start. But nonrecursive makefiles aren't hard to write with Automake, even for larger projects (see e.g. ImageMagick for an example). The payoff is just relatively low for the work that needs doing. The second problem is Libtool, which thought it is much faster than it was, really works at the wrong level: it should be part of Automake, so that the generated makefiles directly contain the appropriate magic incantations to generate shared libraries. Unfortunately if you fix that you break the people who are running Libtool directly, and those were its original user base: Automake integration came later.

That's why I've said "not just distribution guys".

Posted Nov 16, 2011 0:17 UTC (Wed) by cmccabe (guest, #60281) [Link]

Yeah, but having to cram everything into one giant Makefile sucks. It's like putting all your code into one .c file because your compiler is to dumb to handle multiple files. It's 2011. We shouldn't have work around these kind of tool limitations.

That's why I've said "not just distribution guys".

Posted Nov 16, 2011 2:53 UTC (Wed) by nix (subscriber, #2304) [Link]

You don't have to cram everything into one giant makefile. You just say

include $(shell find . -mindepth 2 -name Makefile)

then have subsidiary makefiles say, in part, something like

ifeq ($(words $(MAKEFILE_LIST)),1)
all:
make -C .. # path to toplevel makefile from here
else
# everything else
endif

And now you have lots of independent fragments of makefiles drawing on a common library declared in the toplevel makefile, new ones are automatically picked up, the makefiles can themselves be automatically built using make rules and the changes picked up, and anyone typing make in subdirectories gets an automatic make from the toplevel instead. (With further trivial adjustments you can arrange to automatically build only the subdir's targets, and automatically compute the toplevel makefile, and have all of that boilerplate inserted with a single $(eval ...) so the people writing those subsidiary makefiles never need to know all this stuff.)

Not only is this not hard, it is stuff which it is quite easy to autogenerate if you so desire. It is also stuff which is designed into GNU Make (hence the auto-reloading feature of "include" if the included makefiles are rebuilt), so it is not in any way a "workaround".

(Non-GNU makes are often a lost cause in this area. Anyone who's not using GNU Make is just trying to be difficult by this point: GNU Make runs everywhere and is more capable than just about all the competition.)

That's why I've said "not just distribution guys".

Posted Nov 16, 2011 21:55 UTC (Wed) by cmccabe (guest, #60281) [Link]

I believe you just described how the Android build system works, except that they use a Python script to locate Android.mk files, rather than the find command. If you could find a way to apply this trick to autoconf projects, you probably could speed up quite a few builds-- at the cost of making things even more complex. The other alternative is just to use CMake, which will do this all for you, without any boilerplate. But I guess you're probably sick of hearing about that by now :)