Weekly Edition
Return to the Development page
| |||||||||||||
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News)
KDE.News continues its KDE4 series with this look at the CMake-based build system. "Our working relationship aside, CMake has greatly improved the process of building KDE. Projects using CMake take less time to get started, since there is less time spent fighting with the build system. One KDE developer says, 'CMake doesn't make you want to shoot yourself with a nailgun when building your project anymore.'"
(Log in to post comments)
add sh/bash to pain list Posted Feb 21, 2007 23:15 UTC (Wed) by b7j0c (subscriber, #27559) [Link] make is indeed painful. cons seemed an interesting alternative, although i can't find anyone depending on it.
likewise sh. i would love to see a distro take a leap and roll out a interactive shell based on a modern scripting language. zoidberg/perl is neat, not sure if there is a python equivalent.
hey while we're at it, replace c with d!
more inertia in unix tools than meets the eye....
python shell Posted Feb 21, 2007 23:28 UTC (Wed) by cortana (subscriber, #24596) [Link] I discovered this today while playing with ipython:
ipython -p sh
Pretty neat!
add sh/bash to pain list Posted Feb 21, 2007 23:56 UTC (Wed) by atai (subscriber, #10977) [Link] try http://squirrelsh.sourceforge.net/
add sh/bash to pain list Posted Feb 22, 2007 12:47 UTC (Thu) by nix (subscriber, #2304) [Link] The syntax for invoking subprocesses in there is *painful*. (Also, what about e.g. pipes and redirection? Matrix maths is much rarer in shell scripts than interaction with subprocesses...)
add sh/bash to pain list Posted Feb 22, 2007 0:00 UTC (Thu) by vmole (guest, #111) [Link] Make is not painful, aside from the idiotic tab-only indent. (Notice that cmake still relies on make). Make works fine, especially the GNU version.
What is painful is the standard GNU build system with recursive Makefiles, autoconf (wrong-headed, but livable), automake (bad), and the-tool-that-shall-not-be-named (okay, I'll write it: libtool. Watch out for the Shoggoth).
There's a lot of inertia in Unix tools because most of the tools work just fine. The trouble is that leads some to believe they're all good...
add sh/bash to pain list Posted Feb 22, 2007 1:25 UTC (Thu) by ajross (subscriber, #4563) [Link] I agree wholeheartedly. Make is simple and easy, with really just one true design flaw (whitespace-sensitive syntax). But it doesn't solve the whole problem: in particular, it doesn't provide any built-in support for portable build schemes or compile-time configuration management.
So a whole bunch of other crap gets layered on top, with varying degrees of success (autotools, despite the mind-twisting weirdness, at least has the advantage of being quite featureful and *very* widely supported). And the resulting complexity leads the community to declare "make sucks!".
And thus is spawned an unending stream of make-like build systems, most of which don't do much more than fix the whitespace-sensitive syntax. Not that this necessarily applies to CMake, about which I know nothing. If I had to pick a target for this flame, I'd choose Ant or SCons...
add sh/bash to pain list Posted Feb 22, 2007 3:39 UTC (Thu) by b7j0c (subscriber, #27559) [Link] > Make is simple and easy, with really just one true design flaw> (whitespace-sensitive syntax)
you don't count its reliance on build file timestamps as a design flaw?
add sh/bash to pain list Posted Feb 22, 2007 3:57 UTC (Thu) by ajross (subscriber, #4563) [Link] you don't count its reliance on build file timestamps as a design flaw? Um, no? Sorry, but I guess I don't understand your thinking here. Automatically tracking dependencies and rebuilding targets that depend on "changed" files is, I think, an unarguably useful feature. And it inherently requires that there be some mechanism for telling when a file has changed. Now, certainly there are many mechanisms for tracking that information. But given that make is using the one provided by the operating system, I fail to see why any of the alternatives would be preferable.
add sh/bash to pain list Posted Feb 22, 2007 5:45 UTC (Thu) by b7j0c (subscriber, #27559) [Link] > but I guess I don't understand your thinking here
a changed timestamp does not imply a changed file
add sh/bash to pain list Posted Feb 22, 2007 6:05 UTC (Thu) by ajross (subscriber, #4563) [Link] Indeed, you've discovered quite a conundrum there. For extra credit, please detail a solution that doesn't involve, say, computing an MD5 hash on every dependency of every target for every invocation. Or integrating your build tool into a source control system. Or the filesystem.
Just stop, basically. If you have an idea for (yet) a(nother) better make tool, then write it and stop with the uber-k00l one-liners, they don't make you look as smart as I suspect you think they do.
add sh/bash to pain list Posted Feb 22, 2007 7:07 UTC (Thu) by cate (subscriber, #1359) [Link] Some makefiles build some files in an other directory, compare the files and eventualy they copy in the right location. It is an hack. It should be included in the makefile syntax.
make is also not so good (with a simple syntax) to handle double output files and semi circular dependencies (i.e. one makefile line to compile and generate the dependencies (.d files) for next make run)
The use of "touch" in a makefile show also design flaw to makefile syntax
I'll happy only after extentions of makefile (or a new make like tools) that could compile efficently (with short and simple syntax) also TeX and LaTeX files.
add sh/bash to pain list Posted Feb 22, 2007 12:50 UTC (Thu) by nix (subscriber, #2304) [Link] Looking at the most monstrous makefiles I could find (glibc and gcc) the occasional `move-if-change' is not a killer.
What does make things nastier is the need for -stamp files everywhere to model X-to-many dependencies and cope with parallel make...
add sh/bash to pain list Posted Feb 22, 2007 12:52 UTC (Thu) by nix (subscriber, #2304) [Link] I hate to say it, but
$(DVIOBJS): %.dvi : %.tex
simply is *not* killingly annoying nor difficult and certainly doesn't require any extensions to make(1).
add sh/bash to pain list Posted Feb 22, 2007 13:24 UTC (Thu) by cate (subscriber, #1359) [Link] It is an hack (echo, while, grep, tee, rm). Lucky there is only one implementation of TeX engine, with a fixed string.And very annoying for bigger project, which includes more dvi files (plus ev. the pdflatex) and a lot of includes. And to complicate: usually user will use bibtex, or some filter for indexes.
make is very good for tree dependencies (and timestamp), but it is not suited (and designated) for general build use.
Maybe because of Makefile programmer, but in a lot of programs you see: "run make clean before to compile the program" (when you change configuration, you patches programs, or now often, before/after a new ./configure). Is this not contrary to the spirit of make? Also kernel had such problems, showing that it is not easy to tell make the right dependencies.
I know that a lot of such problems are due to bad makefile programers, but this show that makefile is not more well suited for today build process.
add sh/bash to pain list Posted Feb 22, 2007 16:52 UTC (Thu) by khim (subscriber, #9252) [Link] Is this not contrary to the spirit of make? Also kernel had such problems, showing that it is not easy to tell make the right dependencies. It's not easy to tell the right dependencies period. make has nothing to do with this: you can create perfectly manageable make-based system (see kernel 2.6) or you can create mess with any other system. Things like mod_ssl hack (one file has the C program, perl script to regenerale this program and data for said perl script - you should run perl+gcc or just gcc dependyng on type of changes) can not be sanely handled by ANY build system.
add sh/bash to pain list Posted Feb 23, 2007 22:16 UTC (Fri) by nix (subscriber, #2304) [Link] Quite so.
*Any* large system will soon acquire strange code generators and autobuild
It can get insanely elaborate, e.g. gcc's horde of build tools using files
glibc is another good example, with automatic tree walkers finding the
The question of whether you put that stuff in CMake libraries or in files
add sh/bash to pain list Posted Feb 24, 2007 10:25 UTC (Sat) by k8to (subscriber, #15413) [Link] Agreed hairy tricky tools are often enough needed. Agreed where they live for the sake of where they live doesn't matter. It's a question worth exploring if you can make viewing and editing the expression of that more sane one way or the other, I think.
add sh/bash to pain list Posted Feb 22, 2007 18:52 UTC (Thu) by b7j0c (subscriber, #27559) [Link] what is with the chain jerking? your response has nothing intelligent to offer other than a thinly-veiled insult predicated on nothing at all.
you asked what the weakness was, i told you. this does in fact impact users of very large build projects.
add sh/bash to pain list Posted Feb 23, 2007 23:33 UTC (Fri) by JohnNilsson (guest, #41242) [Link] How about this:Introduce a 'computable' file.
A computable file is a function from a specified set of files to a bytestream.
When a computable file is read the stream is computed and cached.
If the specified 'input'-files changes, inotify triggers a deletion of the cache.
A 'buildsystem' is thus only a set of computable files.
make shmake Posted Feb 22, 2007 6:31 UTC (Thu) by ldo (subscriber, #40946) [Link] Make is not painful,... Yes it is. The only kind of dependency it can express is a build-dependency. No provision for expressing other things, like include-dependencies. Hence the need for layering all kinds of other cruft on top to get around this limitation.
make shmake Posted Feb 22, 2007 12:54 UTC (Thu) by nix (subscriber, #2304) [Link] Per-language dependency analysis plugins to make would be nice, I agree, probably conditionalized on pattern rules, e.g
%.c %.h: $(dependency-plugin c)
which loads, say, /usr/libexec/make/dependencies/c.so with a defined interface and uses it to append dependencies to .c and .h spotted in target lists...
make shmake Posted Feb 22, 2007 19:32 UTC (Thu) by oak (subscriber, #2786) [Link] > The only kind of dependency it can express is a build-dependency.> No provision for expressing other things, like include-dependencies. As you found out, Make is not supposed to be a/b/c/d/e/f-language pre-processor. You ask that kind of information from the compiler of the corresponding language. With C-language and GCC compiler you could use a rule like this:
# automatic include file dependencies
deps: $(SRCFILES)
gcc $(CPPFLAGS) -MM $^ > deps
-include deps
make shmake Posted Mar 1, 2007 19:54 UTC (Thu) by nix (subscriber, #2304) [Link] It would be nice if that sort of boilerplate could be generalized outsomehow. Mega-duplication (or the reliance on system-specific stuff like *BSD makefile fragments) both are nasty...
make shmake Posted Mar 2, 2007 23:33 UTC (Fri) by ldo (subscriber, #40946) [Link] Jam already has this sort of thing built-in. So much simpler than trying to add yet another layer of complexity on top of make.
make shmake Posted Mar 5, 2007 13:42 UTC (Mon) by nix (subscriber, #2304) [Link] jam strikes me more as a language to build build systems in than a build system.
I remember trying to set the equivalent of CFLAGS and prefix with Boost... it took a couple of dozen lines of patches to the build system just to do that, and a couple of patches to the jam source bundled with Boost.
Sorry, but I prefer Automake's nasty verbose makefiles to *that*.
add sh/bash to pain list Posted Feb 22, 2007 8:22 UTC (Thu) by lamikr (guest, #2289) [Link] I agree, make itself is not bad at all.More proplematic things happens usually when you need to modify for example "configure.in" files to be able to get changes to "configure" file.
(For example to get rid from the "build and run test" for a certain library. This is quite common if you are using cross-compiler, because then the test executable can not usually be executed in the host computer where the build is performed.)
add sh/bash to pain list Posted Feb 22, 2007 0:16 UTC (Thu) by tetromino (subscriber, #33846) [Link] > shell based on a modern scripting languageOn my machine, a newly-started bash process uses <500K of unshared memory. A newly started python process uses over 1.6M of unshared memory. So if you are like me, having a couple dozen terminal windows open at once, using a python-based shell would mean quite a bit of memory wasted. Now consider that every rc script is a shell script, and that the launchers for a significant percentage of programs are in fact shell scripts, and you can see why python shells aren't exactly popular. In fact, some people are considering switching to minimalistic shells like ash (~100K) and busybox (even less) for non-interactive use.
As for interactive use, I generally don't find myself using a lot of advanced shell features. For-loops are about the most advanced shell syntax I use in interactive mode. For anything more complex, I find that it's generally a better idea to write a brief perl script and save it for later use - for one thing, you don't need to retype a twenty-line function when you need it a month later.
add sh/bash to pain list Posted Feb 22, 2007 3:36 UTC (Thu) by b7j0c (subscriber, #27559) [Link] > A newly started python process uses over 1.6M of unshared memory. So if you> are like me, having a couple dozen terminal windows open at once, using a > python-based shell would mean quite a bit of memory wasted
but is it wasted? when i hear "wasted" or "bloat", i ascribe it to resources consumed that do not provide a benefit. a python or perl enabled shell does indeed provide real benefits to users. at least in the case of perl (using a shell like zoidberg), you now have the entirety of CPAN available for interactive use, well integrated with shell variables you set. this isn't a waste.
and even if you have twenty shells open, thats under 40MB. i can't speak for you but i have 2gb on my desktop, why not use it?
add sh/bash to pain list Posted Feb 22, 2007 5:26 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link] it's only useful if you are going to use those features. if you aren't going to need them it doesn't matter how neat they are, they are still wasting ram.
while I have 2G on my laptop, and 4G on my server at home, I would prefer to use it for more useful things (including caching directory entries) rather then holding stuff that I just won't use.
add sh/bash to pain list Posted Feb 22, 2007 3:06 UTC (Thu) by k8to (subscriber, #15413) [Link] I've used a lot of "more full featured language as a shell" tools over the years. I've given all of them fair shakes, tried to use them in real environments for weeks or months.
In the end they all suck.
Reasonable languages tend towards explicit. Shell really needs to do a lot of implicit to make life typing the stuff in reasonable. This is why tcl and perl fare much better as shells than python does, for example, since python syntax actually shows you what's a number and what's a string, and so on, wheras perl for example likes to hide that. But even perl and (even better suited to being a shell) tcl tend to fall down with requiring too much obnoxious syntax to do things that are just plain easy with bourne.
Shell is an interactive program first, a language only very secondarily (don't let the kool-aid drinkers say otherwise). A programming language can only really be a shell very secondarily. This is one of the reasons Monad sucks, for example.
add sh/bash to pain list Posted Feb 22, 2007 5:59 UTC (Thu) by drag (subscriber, #31333) [Link] To see the ultimate expression of this check out Microsoft's Monad.
It's a .NET scripting language trying to pass itself off as a user interface.
The healthier example is ipython. And that is nice in the way that it doesn't pretend to be anything it's not, which is a improved python shell (which has always existed) to make a easier to use and freindlier python development environment.
Bash isn't that bad. They've been steadily improving it as time goes on.
I don't know how stupid or obvious this is, but I just learned about it not long ago, but the backticks are now deprecieted.
The improved method is to use (). Like such:
echo $(echo $(echo $(echo 'hello world')))
So that would be impossible with backticks, or at least so hard I don't know how to do it. It's certainly a lot more convient.
Maybe it's time to re-evaulate it? I donno.
If you want a thing that is specificly designed to be easy to use is:
It's pretty neat. One of things about it is that it supports some of the newer Freedesktop.org standards like the *.desktop files.
For example if you go:
Then it will look up the mime type and then launch your default application for that file. It's kinda new and a lot of the <application>.desktop files are broken but it's nice non-the-less.
Kind a like the shell for normal people.
add sh/bash to pain list Posted Feb 22, 2007 6:16 UTC (Thu) by drag (subscriber, #31333) [Link] Here are a few things I found real quick.Bash debugger little article about using bash completion a longer article from IBM about bash improvements
add sh/bash to pain list Posted Feb 22, 2007 12:58 UTC (Thu) by nix (subscriber, #2304) [Link] $(...) has been in POSIX for, oh, years: I know I was using it on Solaris in 1996. I think it dates all the way back to ksh88.
And you're right, there's no call to use `` anymore at all. They're not even easier to read than $(...).
add sh/bash to pain list Posted Feb 22, 2007 3:07 UTC (Thu) by mmarq (guest, #2332) [Link] "" hey while we're at it, replace c with d! ""
you mean this ? : http://www.digitalmars.com/d/
Great idea. Since the "focus" sooner or later is going to change around 86x_64, it could be the cherry on top of the cake...
add sh/bash to pain list Posted Feb 22, 2007 7:58 UTC (Thu) by tzafrir (subscriber, #11501) [Link] bash with programmable completion for every program is very nice indeed (I know that zsh had it long before bash, but you had to program it manually, and bash now comes with programmable completion for nearly any command) .
As for a nice interactive shell: http://fishshell.org/screenshots.html
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 9:14 UTC (Thu) by sylware (guest, #35259) [Link] cmake...Need a C++ compiler, fine for KDE/QT since it's C++, it's a nono for C apps. Complexity? Having a look at the code, they seems to be in the same "brain damaged" league as the autotools. BSD license... erk! Won't ever contribute: "I give code, you give code", where is my dear GPL/LGPL license. Cross compilation for apps or plumbering for creating cross compilation toolchains... nil... or well hidden. Command line use ways worse than autotools. Well, cmake will want me to kill myself like the autotools.
Still waiting for a new GPL/LGPL C build systems in UNIX philosophy (cf wikipedia about it)... Hu? Should write it? Sorry have other projects... back to the autotools in the mean time.
But cmake has understood very well something and we are a lot to agree: we need a more maintainable build system.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 16:53 UTC (Thu) by aleXXX (subscriber, #2742) [Link] > Need a C++ compiler, fine for KDE/QT since it's C++, it's a nono> for C apps.
You need a C++ compiler to compile CMake. Then you can perfectly build C,
> Complexity? Having a look at the code, they seems to be in the
I don't know what you mean with this.
> BSD license... erk! Won't ever contribute: "I give code, you give
Not that much different from XFree, png, jpeg and others I think.
> Cross compilation for apps or plumbering for creating cross compilation
Cross compiling is currently not built-in, but is an often requested
> Command line use ways worse than autotools.
You don't have to use the -DSOME_VARIABLE=some_value syntax, you can use
Alex
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Mar 7, 2007 11:33 UTC (Wed) by sylware (guest, #35259) [Link] >> Need a C++ compiler, fine for KDE/QT since it's C++, it's a nono>> for C apps.
> You need a C++ compiler to compile CMake. Then you can perfectly build C,
"nono for C apps" means it's wrong, not impossible. What developers want is to reduce in size and complexity the software stack in order to be more maintainable. That's common sense for developers, especially in the "libre" software world. To be dependent on a highly complex C++ compiler for the build system for C apps is ways overboard.
>> Complexity? Having a look at the code, they seems to be in the
> I don't know what you mean with this.
I said that the difficulty/complexity of CMake in regards of the autotools is not that far away (too close maybe)...
>> BSD license... erk! Won't ever contribute: "I give code, you give
> Not that much different from XFree, png, jpeg and others I think.
We do not live in a perfect world, and because other projects are badly licensed does not grant you to bad licensed your software too. But bringing a new /recent build system with a BSD license... God! It's way above my tolerance limit.
>> Cross compilation for apps or plumbering for creating cross compilation
> Cross compiling is currently not built-in, but is an often requested
Are you jocking? A brand new build system that does not deal with cross-compilation at its core??? Do you know that "normal"(build=host=target) compilation is a special case for cross(canadian) compilation? Maybe you are ignoring, you know, the little fact that all the "libre" software community if pushing hard its software stack on embedded systems where cross compilation is mandatory? That's a colossal core design mistake, so colossal that it could be mistaken with sabotage.
>> Command line use ways worse than autotools.
> You don't have to use the -DSOME_VARIABLE=some_value syntax, you can use
I talk command line and you speak ncurses/windows GUI? Are you making a fool of me?
Well, cmake will want me to kill myself HARDER than the autotools.
>> Still waiting for a new GPL/LGPL C build systems in UNIX philosophy (cf >> wikipedia about it)... Hu? Should write it? Sorry have other projects... >> back to the autotools in the mean time.
>> But cmake has understood very well something and we are a lot to agree: >> we need a more maintainable build system.
Please, trash your current cmake and redo it using C, with a GPL/LGPL license and this time without forgetting core functionalities right at the start.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 10:37 UTC (Thu) by shablool (guest, #28774) [Link] From my experience, SCons (www.scons.org) is much more simple and intuitive then CMake.It's a pity that the KDE team did not adopt it as their build system.
S.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 14:00 UTC (Thu) by k8to (subscriber, #15413) [Link] This has been my experience as well, but it doesn't seem universal. I'll gladly suffer from multiple build systems if that means I can dump autotools and ant.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 15:43 UTC (Thu) by gnb (subscriber, #5132) [Link] They tried it and decided it wasn't what they were looking for. Seemsfair enough to me. That fact that it's exactly what some people want doesn't necessary make it the right tool for every project.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 22, 2007 16:59 UTC (Thu) by aleXXX (subscriber, #2742) [Link] We tried scons and didn't get it working within something like 6 months.Dependencies, running tools built during the built (without being installed), RPATH handling, system introspection, portability to OSX, MSVC and more wasn't implemented or good enough.
We didn't get any support from the scons developers.
We had to modify so much we had bascially a fork of scons.
You have to learn python to use Scons.
You can use all of python in build files, not necessarily a good thing.
That's how the most simple cmake file looks:
ADD_EXECUTABLE(helloworld main.c)
And CMake isn't tied to Makefiles. You can also build KDE using XCode on
Alex
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 24, 2007 10:32 UTC (Sat) by k8to (subscriber, #15413) [Link] Not to try to quibble with your decision making (it sounds completely reasonable to me), but two objections:
- You have to learn python to use Scons.
Feature, not bug. All make systems are specialized mini-languages which tend to slowly sprawl. Python is a relatively mature well-stewarded language which many people already know, and hews more closely to most people's programming experience than most build-tools do. In essence, learning python is easier for many people than learning Make.
- And CMake isn't tied to Makefiles.
If this is a response to the parent (which I can no longer see), apologies. But of course, neither is Scons.
The Road to KDE 4: CMake, a New Build System for KDE (KDE.News) Posted Feb 27, 2007 10:16 UTC (Tue) by aleXXX (subscriber, #2742) [Link] > - You have to learn python to use Scons.> > Feature, not bug. All make systems are specialized mini-languages which > tend to slowly sprawl. Python is a relatively mature well-stewarded > language which many people already know, and hews more closely to most > people's programming experience than most build-tools do. In essence, > learning python is easier for many people than learning Make.
That's something one can probably argue about with no end.
We had the same discussion in KDE: "a real programming language is much better than some strange macro language". But now that we have CMake since one year, these arguments are gone. The CMake language just works (for what it's supposed to do).
> - And CMake isn't tied to Makefiles.
You are right, that's not a reply to the direct parent. I just noted that much of the discussion was about "make", and you can use cmake without make.
Alex
|
|||||||||||||