User: Password:
|
|
Subscribe / Log in / New account

KS2011: Afternoon topics

KS2011: Afternoon topics

Posted Oct 26, 2011 21:44 UTC (Wed) by mathstuf (subscriber, #69389)
In reply to: KS2011: Afternoon topics by pj
Parent article: KS2011: Afternoon topics

Build systems can be replaced and things are fine. Make less so. There are many things that replace make that do not support things like -j, -k, -n, or other nicities for debugging broken builds.

Personally, I'm more partial to CMake than other build systems for C/C++. Some languages have it better: Python has setuptools or distribute, and Haskell has cabal. These are built with the language in mind and work fine until other things are needed (e.g., asciidoc/xmlto for man pages, doxygen for documentation, and non-standard testing suites (especially when language bindings are concerned and you can't build/use the bound language without hacks)). Others languages still have it hard such as Java with maven or whatever the XML horror du jour is.

What I *really* want is a build system that generates Makefiles (or Ninja files, or Visual Studio projects, or XCode projects, or what have you) (CMake), defaults to out of source builds (CMake and whatever autotools magic glibc has that no one bothers to copy), has a declarative syntax (cabal), and has no need to ship generated files (CMake, raw Makefiles).

I have hacks for CMake to handle LaTeX builds (including double and triple+bibtex passes) with out-of-source builds, symlinking my dotfiles, generating cross-referenced doxygen, and more, but I think a build system that supports something more akin to make's generator rules (something like Prolog/Mercury maybe?) would be nicer to work with (CMake's escaping and argument parsing is less than ideal to manage with complicated things). Implicit know-how of supporting system copies and bundled libraries with automatic switches (which can be disabled if there are patches which make in-the-wild copies not work) would be wonderful as well. CMake's external_project_add gets close, but still has some rough edges (such as needing manual switches for system copies support).


(Log in to post comments)

KS2011: Afternoon topics

Posted Oct 27, 2011 19:52 UTC (Thu) by pbonzini (subscriber, #60935) [Link]

Autotools supports out-of-tree builds with zero effort from the autotools user. All programs using autotools should support mkdir build; cd build; ../configure && make.

KS2011: Afternoon topics

Posted Oct 27, 2011 20:11 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Yes, you can configure out-of-tree, but you can't autoreconf -i -f out-of-tree. I want *zero* files in the source tree that are not meant to be committed. So yes, autotools does support out-of-tree builds, but it still leaves a dirty source tree behind.

Out-of-source *build* is probably bad wording. More precisely, .gitignore should be empty and git status (and the equivalent in the other VCS's) should also be empty starting with no generated files.

KS2011: Afternoon topics

Posted Oct 29, 2011 19:05 UTC (Sat) by fuhchee (guest, #40059) [Link]

"I want *zero* files in the source tree that are not meant to be committed."

The cure to that is to mean to commit autoconf/automake-generated files.

KS2011: Afternoon topics

Posted Oct 31, 2011 20:46 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

> The cure to that is to mean to commit autoconf/automake-generated files.

Which, IMO, is worse than having them in the source tree after their generation. The autoconf/automake *generated* files belong in the build directory.

KS2011: Afternoon topics

Posted Oct 31, 2011 20:55 UTC (Mon) by fuhchee (guest, #40059) [Link]

"The autoconf/automake *generated* files belong in the build directory."

If you say so. :-)

KS2011: Afternoon topics

Posted Nov 1, 2011 14:30 UTC (Tue) by nix (subscriber, #2304) [Link]

Well, that's true if your source directory is a version-control checkout, because anything else is a recipe for conflict hell as different developers use slightly different versions of the generating tools (producing slightly different output).

It's certainly not true if your source directory is a release tarball (or other release medium). Autoconf et al should have been run for you by that point, and the result tested. That way end users don't need anything but a shell and ordinary build tools to run the build. (This is one area where cmake falls down: all the builders need a copy of it.)

KS2011: Afternoon topics

Posted Nov 1, 2011 14:41 UTC (Tue) by fuhchee (guest, #40059) [Link]

"anything else is a recipe for conflict hell"

In practice, if people are pragmatic, it's fine.
Developers can regenerate the files at will with any version that works.
In the case of version control branch merge problems, regenerate them again.

KS2011: Afternoon topics

Posted Oct 28, 2011 18:51 UTC (Fri) by anatolik (subscriber, #73797) [Link]

Most builds systems are work fine until you have a small project (less than 10K files), but all of them suck when the project grows. Basically all these build systems are not scalable.

I really like the idea of Tup build system http://github.com/gittup that stores graph into local sqlite database and reparses/rebuilds graph only when files are changed - this makes iterative development much more pleasant. Another cool feature is dependencies autodiscovering - under the hood it uses fuse (and fuse-like) library for that (this works on linux, macosx and windows). And the third feature that I like is "monitor" - inotify based daemon that updates graph of dependencies in background while you change files in your editor.

I made some experiments with my project (100K) and found that null build takes ~1.6 sec without monitor and 0.09 sec with monitor. Null build for my gmake based build system on the same project takes 42 secs (it parses makefiles files, builds graph of dependencies, scans files for modification, but does not run any commands).

KS2011: Afternoon topics

Posted Oct 28, 2011 20:13 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

That actually looks to be close to what I'd like to see for the build system itself. The rules for "how to build" are declared in the top-level and then piped through later. It also appears to have very little in the way of hard-coded knowledge about C/C++ and extending it to also support things like latex pipelines, documentation builds, and more would be well-supported.

It doesn't support -B or -n make flags though. The percentage output is nice. However, it doesn't seem to support out-of-source builds (I sort of hacked bootstrap.sh to do the bootstrap successfully, but there's no further support to get a full build working). The code base looks clean, so maybe I can get a patch and convince the maintainer to accept out-of-source as an option.

KS2011: Afternoon topics

Posted Oct 28, 2011 20:38 UTC (Fri) by anatolik (subscriber, #73797) [Link]

> It doesn't support -B
One of the mottos of the project - "you'll never need to do clean build". Things like "clean" and -B are used in case if build is broken by incorrect dependency information (esp it happens often for recursive make). Tup provides "build consistency" - the output is always correct because dependencies are always correct.

> make -n
"tup todo"

> it doesn't seem to support out-of-source builds
AFAIK the tup author in favor of adding it. It is better to contact the maillist as I am not sure about his plans.

Oh, yeah, I remembered 4th thing that I like in tup - "buffered output". Output from commands is always printed atomically. No more interlaced output! The interlaced output is especially annoying if you have an error in one of widely used header files - this makes error messages really difficult to read.

KS2011: Afternoon topics

Posted Oct 28, 2011 21:00 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

> One of the mottos of the project - "you'll never need to do clean build". Things like "clean" and -B are used in case if build is broken by incorrect dependency information (esp it happens often for recursive make). Tup provides "build consistency" - the output is always correct because dependencies are always correct.

So if I update a system library or header, it will relink the relevant parts? That's usually the corner case I've run into.

> AFAIK the tup author in favor of adding it. It is better to contact the maillist as I am not sure about his plans.

That's good.

> Oh, yeah, I remembered 4th thing that I like in tup - "buffered output". Output from commands is always printed atomically. No more interlaced output! The interlaced output is especially annoying if you have an error in one of widely used header files - this makes error messages really difficult to read.

I usually do `make -kj8` followed by `make` to do this, so this should help that. I rarely do parallel builds from vim however (<F8> is bound to "build", autodetecting CMake, make, pdflatex, cabal, rpmbuild, and a few others based on the current file) so interlaced output never really bothered me there.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds