|
|
Log in / Subscribe / Register

GNU Make 4.0 released

From:  Paul Smith <psmith-AT-gnu.org>
To:  info-gnu-AT-gnu.org
Subject:  GNU Make 4.0 released
Date:  Wed, 09 Oct 2013 02:47:09 -0400
Message-ID:  <1381301229.8253.231.camel__44157.1754091003$1381301351$gmane$org@homebase>

GNU Make 4.0 has been released and is available now for download:

http://ftp.gnu.org/gnu/make/make-4.0.tar.gz   b5e558f981326d9ca1bfdb841640721a
http://ftp.gnu.org/gnu/make/make-4.0.tar.bz2  571d470a7647b455e3af3f92d79f1c18

Or one of the mirrors listed in http://www.gnu.org/order/ftp.html
Get redirected to a nearby mirror at http://ftpmirror.gnu.org/make/

        The make utility automatically determines which pieces of a
        large program need to be recompiled, and issues commands to
        recompile them.

More information about make at: http://www.gnu.org/software/make/

Bug reports/announcements at: https://savannah.gnu.org/projects/make/


Some new features available in this release of GNU Make:

* New feature: GNU Guile integration
  This version of GNU make can be compiled with GNU Guile integration.
  GNU Guile serves as an embedded extension language for make.
  See the "Guile Function" section in the GNU Make manual for details.
  Currently GNU Guile 1.8 and 2.0+ are supported.

* New command line option: --output-sync (-O) enables grouping of output by
  target or by recursive make.  This is useful during parallel builds to avoid
  mixing output from different jobs together giving hard-to-understand
  results.

* New command line option: --trace enables tracing of targets.  When enabled
  the recipe to be invoked is printed even if it would otherwise be suppressed
  by .SILENT or a "@" prefix character.  Also before each recipe is run the
  makefile name and linenumber where it was defined are shown as well as the
  prerequisites that caused the target to be considered out of date.

* New command line option argument: --debug now accepts a "n" (none) flag
  which disables all debugging settings that are currently enabled.

* New feature: The "job server" capability is now supported on Windows.
  Implementation contributed by Troy Runkel <Troy.Runkel@mathworks.com>

* New feature: The .ONESHELL capability is now supported on Windows.  Support
  added by Eli Zaretskii <eliz@gnu.org>.

* New feature: "!=" shell assignment operator as an alternative to the
  $(shell ...) function.  Implemented for compatibility with BSD makefiles.
  Note there are subtle differences between "!=" and $(shell ...).  See the
  description in the GNU make manual.
  WARNING: Backward-incompatibility!
  Variables ending in "!" previously defined as "variable!= value" will now be
  interpreted as shell assignment.  Change your assignment to add whitespace
  between the "!" and "=": "variable! = value"

* New feature: "::=" simple assignment operator as defined by POSIX in 2012.
  This operator has identical functionality to ":=" in GNU make, but will be
  portable to any implementation of make conforming to a sufficiently new
  version of POSIX (see http://austingroupbugs.net/view.php?id=330).  It is
  not necessary to define the .POSIX target to access this operator.

* New function: $(file ...) writes to a file.

* Setting the -r and -R options in MAKEFLAGS inside a makefile now works as
  expected, removing all built-in rules and variables, respectively.

Plus other enhancements, and over 80 bugs fixed since the last release.
_______________________________________________
Make-alpha mailing list
Make-alpha@gnu.org
https://lists.gnu.org/mailman/listinfo/make-alpha



to post comments

GNU Make 4.0 released

Posted Oct 9, 2013 14:16 UTC (Wed) by josh (subscriber, #17465) [Link] (74 responses)

Meanwhile, many users are still stuck on 3.81, because 3.82 introduced an intentionally backward-incompatible change to the handling of pattern rules, which breaks real makefiles such as those in the Linux kernel: https://savannah.gnu.org/bugs/index.php?33034

This bug is the reason why make 3.82, released in 2011, still sits in Debian experimental and hasn't yet graduated to unstable where it might form part of a release. Make 4.0 seems likely to suffer the same fate.

GNU Make 4.0 released

Posted Oct 9, 2013 15:35 UTC (Wed) by cesarb (subscriber, #6266) [Link] (6 responses)

Looking at http://pkgs.fedoraproject.org/cgit/make.git/, Fedora seems to already be using make 3.82. Ubuntu seems to be at 3.81 (http://packages.ubuntu.com/make). I believe openSUSE is also at 3.82.

GNU Make 4.0 released

Posted Oct 9, 2013 16:56 UTC (Wed) by BlueLightning (subscriber, #38978) [Link] (5 responses)

Haven't those all been patched by each distro to fix the regressions?

GNU Make 4.0 released

Posted Oct 9, 2013 17:12 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

Yes. For Fedora, you can find the patches at http://pkgs.fedoraproject.org/cgit/make.git/tree/

The spec file has the list of upstream bugs

http://pkgs.fedoraproject.org/cgit/make.git/tree/make.spec

GNU Make 4.0 released

Posted Oct 9, 2013 18:51 UTC (Wed) by josh (subscriber, #17465) [Link] (3 responses)

I've checked Fedora, and their patches fix various issues in 3.82, but not this issue with mixing pattern rules and normal rules. If you know of any distro that has fixed that particular regression, I'd be very interested in the patch.

GNU Make 4.0 released

Posted Oct 14, 2013 3:38 UTC (Mon) by salimma (subscriber, #34460) [Link] (2 responses)

How does Fedora compile its kernel then - presumably they either carry patches to the kernel makefiles, or the newer kernels in Fedora already support the new makefile format?

GNU Make 4.0 released

Posted Oct 15, 2013 5:11 UTC (Tue) by madscientist (subscriber, #16861) [Link] (1 responses)

The Linux kernel makefiles were fixed back in 2010, Linux 2.6.34, so they no longer use the now-illegal syntax and build just fine with GNU make 3.82, so the kernel in current Fedora releases (for example) compiles just fine. What Josh was asking was if Fedora had patched GNU make to undo the change and make the syntax legal again.

GNU Make 4.0 released

Posted Oct 15, 2013 6:05 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

No. Fedora would never do that. It is a upstream decision.

GNU Make 4.0 released

Posted Oct 9, 2013 16:50 UTC (Wed) by ppedroni (subscriber, #6592) [Link]

Gentoo has 3.82 in stable as well.

GNU Make 4.0 released

Posted Oct 9, 2013 18:10 UTC (Wed) by Kayden (guest, #89093) [Link] (12 responses)

Wow. I knew automake broke with incompatible changes all the time (which already seemed bizarre to me), but they broke make too? That's just awful.

It's okay for projects to be "done"...

GNU Make 4.0 released

Posted Oct 9, 2013 19:08 UTC (Wed) by josh (subscriber, #17465) [Link] (11 responses)

I'm definitely happy that GNU Make is still actively developed. I really look forward to the untangled parallel make output (-O) and the --trace debugging facility.

I'd just like the regressions fixed so I can actually *use* the new make.

GNU Make 4.0 released

Posted Oct 9, 2013 20:17 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (10 responses)

Yeah, after using ninja for a while, the untangled output is a great fix. It's unfortunate it's behind a flag though :( .

GNU Make 4.0 released

Posted Oct 9, 2013 22:46 UTC (Wed) by madscientist (subscriber, #16861) [Link] (9 responses)

You can add "export GNUMAKEFLAGS=-Otarget" (or similar) to your ~/.bashrc file and forget about it, if you decide you always want it enabled.

There was discussion on the mailing lists about making this the default. In the end it was decided we needed more experience with it first. It may become the default in the future though.

GNU Make 4.0 released

Posted Oct 9, 2013 23:05 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (8 responses)

Ah, that's good to know. If it's worth anything, +1 on making it the default :) . This should actually fix the problem with colored output from CMake where the color "leaks" out because it is done like:

cmake --print-percentage
cmake --start-color green "friendly output"
command

and command output can get in between the "friendly output" parts before color is turned back off.

GNU Make 4.0 released

Posted Oct 10, 2013 3:03 UTC (Thu) by madscientist (subscriber, #16861) [Link] (7 responses)

Unfortunately it will "fix" this problem by disabling colorized support altogether. That's because cmake, like most tools, only generates colorized output when it believes it's generating output to a terminal. This is a good thing since it means you don't get a lot of bogus control characters in your log files.

However, since in order to implement output-sync we must redirect the output of recipes to a file so it can be sync'd, tools that are invoked by make are not attached to a TTY.

It would be nice if such tools had an option that said "I know stdout doesn't look like a terminal, but please generate colorized output anyway". But as far as I know it doesn't.

GNU Make 4.0 released

Posted Oct 10, 2013 6:24 UTC (Thu) by josh (subscriber, #17465) [Link] (6 responses)

Most programs with --color options default to "auto" (colorize to terminals only), but have an "always" option to unconditionally colorize. That option is useful when you want to pipe to less -R, for instance. GCC's -fdiagnostics-color option has an "always" as well.

GNU Make 4.0 released

Posted Oct 10, 2013 13:26 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (1 responses)

Yeah, but I hope tools which write make files never use "always". It would screw up parsing of error output by vim and likely other tools as well. CMake doesn't have an always since the color support is really an implementation detail (all "cmake -E cmake_*" commands are internal, which includes the progress reporting and color commands).

GNU Make 4.0 released

Posted Oct 10, 2013 15:03 UTC (Thu) by josh (subscriber, #17465) [Link]

Makefiles shouldn't directly use color "always", but the caller of a makefile could.

Also, perhaps make could provide some convenient make variable or environment variable to look at that indicates if the final output destination is a TTY?

GNU Make 4.0 released

Posted Oct 10, 2013 13:59 UTC (Thu) by madscientist (subscriber, #16861) [Link] (3 responses)

Yes but as far as I know, cmake doesn't have such an option. And as mathstuf points out setting it to "always" inside the makefile will cause more problems than it solves. Basically you need to outsource the decision to make itself, since make is the program that knows if stdout is going to be a TTY or not. However, today there's no way for a makefile to determine whether make's output is a TTY (well I guess you can probably figure it out using $(guile ...)... you'd have to do it once at the top-level makefile and communicate it through the environment to any recursive invocations).

GNU Make 4.0 released

Posted Oct 10, 2013 14:04 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (2 responses)

Well, there is "tty_out ?= $(test -t 1 && echo yes || echo no)". Then setting a variable based on it. It even allows forcing it to "yes" for those cases.

GNU Make 4.0 released

Posted Oct 10, 2013 14:10 UTC (Thu) by madscientist (subscriber, #16861) [Link] (1 responses)

$(test ...)? What's that? If you meant $(shell test ...) that won't work because the shell function always redirects stdout (it's basically like backticks in the shell).

You could test stderr instead and assume that if it was a TTY, then stdout is intended to be a TTY as well, I suppose. There are lots of fairly common situations where this would fail though.

GNU Make 4.0 released

Posted Oct 10, 2013 14:13 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Oops, yeah. I had originally written it with "!=", but figured I'd go for compatibility, then messed it up :) .

GNU Make 4.0 released

Posted Oct 9, 2013 18:40 UTC (Wed) by ploxiln (subscriber, #58395) [Link] (36 responses)

I ran into this "pattern + static target" rule limitation in the linux kernel makefiles a couple of years ago, but only because I was using a rather old kernel on an embedded platform. I found it was already fixed somewhere in the 2.6.3x releases and backported to a stable patch-release of 2.6.32.y.

Others have mentioned some distros that use make 3.82; Arch Linux is another one of them, and applies relatively few patches, and manages to build everything in its repos: https://projects.archlinux.org/svntogit/packages.git/tree...

You'll also notice, in the link to the set of patches fedora applies, the "pattern + static target" thing is not patched out. Make just ends up running into a bunch of not-quite-right usages in various corners of makefiles, and also has a few "legitimate" regressions. I think it's a strength of open source that we're not always hobbled by backwards-compatibility forever, and we can eventually fix usage everywhere and make everything more sane.

GNU Make 4.0 released

Posted Oct 10, 2013 1:03 UTC (Thu) by torquay (guest, #92428) [Link] (32 responses)

    I think it's a strength of open source that we're not always hobbled by backwards-compatibility forever, and we can eventually fix usage everywhere and make everything more sane.

No. That type of thinking just leads to chaos. Maybe if one or two projects did that in the entire software stack, we can put up with the pain. The problem is that too many projects do that, accumulating in overall massive API and ABI breaks from one distro version to another. This leads to massive pain on the users.

It's fine for a project to change/improve things internally, but DO NOT break the user interface in minor versions.

This is what Linus Torvalds meant when he said "I'd really like to see some of the kernel culture spread into userspace", when complaining that many userspace projects break compatibility with other software, which is something he's steadfastly avoided in the kernel itself.

Or if you prefer Linus Torvald's words unfiltered, here's a message on LKML, exemplifying his stance.

Another quote from Torvalds: "Way too many projects seem to think that the code is more important than the user, and they break things left and right, and they don't apologize for it, because they feel that they are ‘fixing’ the code and doing the right thing."

Corollary: No project is more important that the users. Do not put your users through pain, even if that means putting in a bit more effort on your part.

GNU Make 4.0 released

Posted Oct 10, 2013 9:52 UTC (Thu) by khim (subscriber, #9252) [Link] (31 responses)

Corollary: No project is more important that the users. Do not put your users through pain, even if that means putting in a bit more effort on your part.

Sure, but you are still mixing issues. Users ≠ developers. It's not their job to fix software programs—they are supposed to create other great things: may be art, may be some physical goods, or perhaps just move stuff around. But they are not supposed to fix the code and thus code should never break under them.

Now, software developers are different: their work (or their life passion, does not matter) is code. Writing code, fixing code, that kind of stuff. It's perfectly fine to break stuff which is only ever touched by developers. I gave recent example from kernel already, here is one from glibc.

Microsoft (which spends enormous efforts to make sure “Joe Average” will not see compatibility problems) is in the same position (it's not uncommon to see that you need MSVC6 or MSVC2005 to develop something because older versions are not 100% backward compatible) and this is Apple's position, too.

Developer is not user, if s/he does not know how to fix bugs s/he should not touch code in the first place.

P.S. It's true that one should still try to keep breakage to the minimum when one develops software for developers, sure. But we are discussing one small (and easy to adjust to!) change which was introduced three years ago!

GNU Make 4.0 released

Posted Oct 10, 2013 11:07 UTC (Thu) by torquay (guest, #92428) [Link] (16 responses)

    P.S. It's true that one should still try to keep breakage to the minimum when one develops software for developers, sure. But we are discussing one small (and easy to adjust to!) change which was introduced three years ago!

This might be one relatively minor instance, but it is a manifestation of an unhealthy culture. The so far tolerated way of developing user space components needs fixing, no matter what the motivation for developing the software is in the first place. Breaking APIs and ABIs willy nilly is a bad idea.

GNU Make 4.0 released

Posted Oct 10, 2013 13:01 UTC (Thu) by khim (subscriber, #9252) [Link] (15 responses)

This might be one relatively minor instance, but it is a manifestation of an unhealthy culture.

Really? Then, of course, you can show us illuminating examples of these “healthy cultures” which keep APIs stable and where one insignificant breakage once per three years is considered “big deal”. Logic is simple: if it's really is as poisonous as you say then surely all these “unhealthy cultures” (which include Linux, Windows, MacOS, ChromeOS, Android, etc) are dying and are soon to be replaced with these “healthy cultures”! Except reality is different.

Of course some “healthy cultures” (by your definition) exist. Take TeX, for example: it's extremely stable and uber-compatible, but to call it “healthy cultures” will be an overstatement: it stagnated for years til it was replaced with XeTeX which finally brought few much-requested changes and yes, it also broke some packages here and there.

We know that ABI stability is important because empirical evidence is there. We have no idea if keeping API stable or not is good thing: what meager evidence we have does not give us conclusive answers.

Breaking APIs and ABIs willy nilly is a bad idea.

Or course, but when you lump together ABIs, APIs and mix “willy nilly” with “once per three years” your argument quickly loses it's power.

GNU Make 4.0 released

Posted Oct 10, 2013 13:59 UTC (Thu) by Otus (subscriber, #67685) [Link] (12 responses)

> “unhealthy cultures” (which include Linux

Don't know about the others, but seems to me a regression like this would have been reverted in Linux when reported. This is not an internal interface, it's ABI (just textual like e.g. much of sysfs for Linux).

It's not a big deal if a regression sometimes slips through, but they ought to be fixed if at all possible.

GNU Make 4.0 released

Posted Oct 10, 2013 16:11 UTC (Thu) by khim (subscriber, #9252) [Link] (11 responses)

It's not an ABI since it's not used when program is run. It's only used when program is build. And kernel is known for such breakage. You can not just grab random new version of kernel headers and build userspace program. You can read lovely discussion on the topic here. Note that opinions of kernel developers differ and not all of them are in favor of fixing the stuff. Yet, such changes sometimes are introduced on purpose (as happened recently).

It's not a big deal if a regression sometimes slips through, but they ought to be fixed if at all possible.

Nope. They should be fixed when feasible. Gentoo was able to mark make 3.82 stable two years ago which strongly suggests that breakage was not as severe as people like to portray it.

GNU Make 4.0 released

Posted Oct 10, 2013 19:49 UTC (Thu) by lambda (subscriber, #40735) [Link] (9 responses)

If you actually read further in the discussion you refer to about not being able to grab a random new version of kernel headers and build userspace programs, you'll notice that there was substantial pushback and the headers were eventually fixed. It's too bad that there was a release that broke compatibility, but David Miller pushed back hard and restored the missing functionality. That's an example of a healthy culture in play.

The perf API issue is a bit different. That's a place where the former API simply did not work; it was fundamentally wrong. There's really nothing to be done that could completely fix the situation, so in trading off between preserving binary compatibility and preserving source compatibility, they choose to retain binary compatibility. It's not an ideal situation, the real world sometimes has messy cases that you need to deal with, but it's still an example of a healthy culture that does its best to preserve compatibility.

On the other hand, the GNU Make change simply broke working Makefiles, without any consideration of maintaining compatibility. There was no reason for it; they could have just forbidden the actually broken rules, rather than the ones that actually worked.

It's not a matter of never breaking compatibility with anything. Sometimes, there's nothing you can do. Sometimes the "feature" that someone was depending on is actually a security vulnerability, and you need to block access to it to prevent exploitation. The "healthy culture" part is trying to do everything you can to preserve backwards compatibility, and considering it much more strongly than some code cleanliness issue of "it'll be too complicated to support this use case that we never documented in the past."

GNU Make 4.0 released

Posted Oct 10, 2013 20:20 UTC (Thu) by khim (subscriber, #9252) [Link] (8 responses)

On the other hand, the GNU Make change simply broke working Makefiles, without any consideration of maintaining compatibility.

s/working/accidentally working/

GNU culture is different from Linux culture. They also put very strong emphasis on the compatibility, but they have different focus: they tend to support correctly written stuff only. If something works by accident then they may decide to support existing binaries to help clueless users (see memcpy story) but developers are supposed to fix their own mess (not how memcpy saga ended up with fix which works for old binaries, but does not work for old sources).

Since make by itself is not supposed to be used by non-programmers and since change only breaks Makefiles which were not correct to begin with there are literally nothing to fix.

GNU Make 4.0 released

Posted Oct 10, 2013 21:50 UTC (Thu) by sfeam (subscriber, #2841) [Link] (3 responses)

"since change only breaks Makefiles which were not correct to begin with there are literally nothing to fix"
Rubbish. You have indeed put your finger on a difference in culture, but it highlights the problems that arise when the cultural slant is towards nerdview . The Makefiles in question were not "incorrect"; they were simply making use of a feature that the original developers had not considered. [Or perhaps had considered but not documented?] A better solution would be to document and standardize the unintended feature. It is a bad idea to assume that anyone who uses your product/program/document/whatever has exactly the same expectations and perspective on usability that you do, and that anyone who uses it differently is wrong.

And don't get me started on the topic of annoying error messages that signify only that the GNU developers have indulged in a bit of meaningless API shuffling. What is the point of spewing out messages like

aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'
You know what I wanted, you know it used to be the documented API, you know exactly how to handle it, - why are you spitting out pointless error messages?

GNU Make 4.0 released

Posted Oct 10, 2013 23:35 UTC (Thu) by khim (subscriber, #9252) [Link] (2 responses)

The Makefiles in question were not "incorrect"; they were simply making use of a feature that the original developers had not considered.

They used syntax which was never valid to beging with. It was never documented and only worked in some cases by accident. This is similar to changes in GCC which forbids some constructs which were allowed in old versions by accident. GCC provides -fpermissive flag to allow such constructs, but they are only used for widely-used constructs, not for all of them.

Or perhaps had considered but not documented?

Most likely GNU Make developers had no idea that said construct was ever used by anyone till they introduced new code which disallowed these. When they discovered that such abomination existed they added helpful error message and added a note to the NEWS file. What else should they do?

A better solution would be to document and standardize the unintended feature.

Why? Said unintended feature worked in some cases and not in other cases. To make it work in all cases you'll need to create complete and noncontradictory semantic for that feature, etc. And since it was never documented valid programs don't use it.

It is a bad idea to assume that anyone who uses your product/program/document/whatever has exactly the same expectations and perspective on usability that you do, and that anyone who uses it differently is wrong.

Sure. But it's good idea to assume that anyone who uses your product/program/document/whatever will do that in accordance to the spec. This reduces size of ABI and makes the support work actually manageable. Yes, sometimes people jump through hoops to preserve undocumented semantic in their products but in general if you are using undocumented interfaces you must be prepared to salvage your creation when something goes wrong. That's how Windows is doing thing, that's how Android is doing things and that's how GNU projects are doing things. Linux is somewhat different because they don't actually provide a canonical documentation (that is done by other guys) thus they can not actually talk about “documented” or “undocumented” features.

What is the point of spewing out messages like

aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in'

You know what I wanted, you know it used to be the documented API, you know exactly how to handle it, - why are you spitting out pointless error messages?

You can as well argue against ANSI C which removed K&R-style function declarations from C language. Perhaps it was bad idea to change the name of input file, perhaps it was a good idea, but after decision to change the name was made the obvious solution is to add the transition period and issue a warning—how else do you propose to introduce such a change?

And don't get me started on the topic of annoying error messages that signify only that the GNU developers have indulged in a bit of meaningless API shuffling.

If you don't like the decisions GNU developers did you can always switch to something else. But if you decide to use it then such messages are obviously helpful: they give your change to change your code now when it's just a warning not error.

GNU Make 4.0 released

Posted Oct 11, 2013 4:47 UTC (Fri) by eru (subscriber, #2753) [Link]

You can as well argue against ANSI C which removed K&R-style function declarations from C language.

Actually, it did no such thing. The ANSI C committee was very careful to be compatible with the existing C language whenever possible. Their main problem was that at the time there were multiple divergent C implementations, and they could not make ANSI C be fully compatible with all of them. But they did their best, and the result was in my opinion impressive. Correct programs that stuck to the original K&R language ran as-is in most cases, and yet they managed to standardize useful extensions from PCC and others.

People who want to extend programming languages and tools without inflicting pain on users could do worse than study the ANSI C standardization. The "Rationale" part is fascinating.

GNU Make 4.0 released

Posted Oct 12, 2013 15:14 UTC (Sat) by RCL (guest, #63264) [Link]

> in general if you are using undocumented interfaces you must be prepared to salvage your creation when something goes wrong. That's how Windows is doing thing

You don't read The Old New Thing much, do you? :) Microsoft goes to great lengths to provide backward compatibility, for both binaries and sources.

GNU Make 4.0 released

Posted Oct 10, 2013 22:15 UTC (Thu) by lambda (subscriber, #40735) [Link] (3 responses)

s/working/accidentally working/

Nearly every software system has corner cases that are not documented, that people rely on all the time. Just because something isn't documented to work doesn't mean that people don't rely on it. Sure, you should try not to rely on undocumented behavior, as it may be unsupported and changed in the future, but it can be awfully hard to tell the difference between that and whether it's a supported feature that no one had bothered to document. Have you never been writing code, but found that the documentation was lacking or unclear, and so just written some test code to see how the system actually behaved?

GNU culture is different from Linux culture.

Yes, and there are some people who would like to change this. As torquay quoted Linus above, "I'd really like to see some of the kernel culture spread into userspace."

And as you can see from this thread, there are many others who agree; who have been burned way too often in the past by stupid backwards compatibility breakage which meant that they had to rewrite way too much code which had all changed not for any particularly good reason but just for the sake of being different or being more "correct" according to some sense of aesthetic.

Since make by itself is not supposed to be used by non-programmers and since change only breaks Makefiles which were not correct to begin with there are literally nothing to fix.

In an free software system, saying that "make by itself is not supposed to be used by non-programmers" is ridiculous. There are lots of end users, who are not programmers, who need to compile software. Want to test out if the new release of package X fixes your problem, without waiting for your distro to upgrade it? Bam, you need to compile software. That doesn't make you a developer.

My boss is not a developer; he's probably what you would call a power user. He's pretty good at testing lots of versions of software, tweaking lots of knobs, and so on to eke out the best performance for his use case. He knows how to configure and compile a kernel, and will frequently do so to see if a new kernel offers some performance benefit or fixes some bug that we're encountering. But faced with an error like the Makefile problem described above, he'd have no idea what to do. In fact, he's been building lots of different kernels recently, to try to debug some problems that we've encountered. Luckily we're using an old distro that still uses Make 3.81 or we probably would have run into this issue.

Even when you're a programmer, you are not necessarily an expert in every piece of software you're compiling. I want to get package X installed, so I can do my work. It depends on packages W, Y, and Z, which in turn depends on package Q. Why should I suddenly have to learn Q's build system just because someone decided to remove a feature that they knew people were depending on?

GNU Make 4.0 released

Posted Oct 11, 2013 0:07 UTC (Fri) by khim (subscriber, #9252) [Link] (2 responses)

Nearly every software system has corner cases that are not documented, that people rely on all the time.

Of course. It's very easy to use some undocumented feature by accident and sometimes you can not achieve what you need via documented interface (but can be achieved via undocumented one). But in all cases when you are using undocumented interfaces your code can be broken at any update (even very minor one!)

Just because something isn't documented to work doesn't mean that people don't rely on it.

Sure, but when that happens you really need to consider need of such users with the other considerations (like quality of the code). These users were living on a borrowed time anyway and any act of mercy WRT their needs much be compared with other considerations.

Have you never been writing code, but found that the documentation was lacking or unclear, and so just written some test code to see how the system actually behaved?

Obviously. Sometimes you really need to use such knowledge, but then it's good idea to use it very carefully and document the reasons for why you are doing this and remember that this will be a weak point in your program. Something like this is, unfortunately, sometimes unavoidable. But of course when you've used undocumented behavior by accident there are no need to do something like this: you just need to go and fix the issue.

Want to test out if the new release of package X fixes your problem, without waiting for your distro to upgrade it? Bam, you need to compile software.

And that is the problem.

That doesn't make you a developer.

Yes, it makes you [unwilling] developer. What's the point of compiling anything if you don't know how to write and change the code? And if you do know how to write and change the code then you should know how to write/fix Makefiles, too. That's pretty basic stuff.

Why should I suddenly have to learn Q's build system just because someone decided to remove a feature that they knew people were depending on?

Because developers of Q don't give you prebuild binaries or you don't want to use them?

GNU Make 4.0 released

Posted Oct 11, 2013 0:37 UTC (Fri) by josh (subscriber, #17465) [Link] (1 responses)

People in this thread keep saying this is an "undocumented" bit of GNU make behavior. Let's explore that for a minute.

The GNU make manual documents that you can have multiple targets in one rule.

The GNU make manual documents that you can have pattern rules.

Nothing in the GNU make manual explicitly said you *couldn't* put those two together, and have a specific target and a pattern target in the same rule, with a common build stanza.

In practice, doing so would work, at least in some circumstances.

So, what makes this an "undocumented feature", exactly?

GNU Make 4.0 released

Posted Oct 11, 2013 1:28 UTC (Fri) by khim (subscriber, #9252) [Link]

The GNU make manual documents that you can have multiple targets in one rule.

Yes.

The GNU make manual documents that you can have pattern rules.

Sure.

Nothing in the GNU make manual explicitly said you *couldn't* put those two together.

Of course you could. That is not a problem. Always was allowed and [hopefully] will always be allowed.

But this is not what happens here.

So, what makes this an "undocumented feature", exactly?

Lack of documentation? Before you'll even try to talk about syntax you need to decide just what kind of syntax you are talking about. There are five of these (I'm using documentation from 2003 to make sure it's not something changed after release of Make 3.82):
explicit rule
implicit rule
variable definition
directive
comment

If you say that your abomination is somehow part of the Makefile syntax then you should be able to put it in one five categories listed above. So… what is it? Is it explicit rule, implicit rule, variable definition, directive, or comment? Yes, it looks a little like mix or explicit rule and implicit rule but it's neither explicit rule nor implicit rule and thus not allowed. End of story.

GNU Make 4.0 released

Posted Oct 11, 2013 13:36 UTC (Fri) by Otus (subscriber, #67685) [Link]

> It's not an ABI since it's not used when program is run.

Yes it is. It's used when you run make itself or any other program that calls make (like deb build tools).

It's not used when (many) programs built with make are run, but that's like saying GTK+ has no ABI because it's not used when I run a shell script I wrote with gedit.

GNU Make 4.0 released

Posted Oct 10, 2013 14:50 UTC (Thu) by lambda (subscriber, #40735) [Link] (1 responses)

Why do you list Windows and Linux under the "unhealthy cultures"? Both Microsoft and the Linux kernel go to fairly extreme lengths to preserve compatibility. Raymond Chen at Microsoft has a blog that spends a lot of time describing all the nasty hacks it takes to preserve backwards compatibility with Windows applications that access private functions. And Linus is famous for his "don't break userspace" rule, even when userspace is doing something that it's not supposed to.

Have you ever considered that part of the reason that Mac OS hasn't been more successful in business is because Apple doesn't have the same strong commitment to backwards compatibility? They do a pretty good job when making big architectural changes, like the 68k to PowerPC switch, OS 9 to OS X transition, and PowerPC to x86 transition, adding a compatibility layer that allows old code to continue running for a few years. But after a couple of years they deprecate and then remove that, meaning that you generally don't have much over a 5 year horizon over which code is still likely to work. I know businesses that still have old Mac OS 9 machines because they have a FileMaker database that they don't want to upgrade; how much longer is that really tenable?

Being willing to break backwards compatibility may be fine for trendy consumer devices, where people are going to buy it for the cool new things you can do with it despite the fact that it won't run your old code. But businesses generally don't want to have to keep upgrading old software that works just fine for what they need to do.

GNU Make 4.0 released

Posted Oct 10, 2013 16:31 UTC (Thu) by khim (subscriber, #9252) [Link]

Both Microsoft and the Linux kernel go to fairly extreme lengths to preserve compatibility.

They both only preserve binary-level compatibility. They introduce source-level incompatibilities regularly. Not “often” but “regularly”.

According to torquay this means both are “unhealthy cultures” which will collapse soon.

Raymond Chen at Microsoft has a blog that spends a lot of time describing all the nasty hacks it takes to preserve backwards compatibility with Windows applications that access private functions.

This blog contains many amusing tales, but note how it always discusses Windows (or sometimes MS Office) and rarely discussed MSVC or Windows SDK. That's because Microsoft is extremely vigilant with changes to end-user-facing tools but it's not nearly as paranoid when changes are introduced to development tools. If you are using something like resharper or some other addin then you are supposed to keep it updated.

And Linus is famous for his "don't break userspace" rule, even when userspace is doing something that it's not supposed to.

Again: this is a problem if normal program breaks. Source code is kept compileable when it makes sense but if it makes sense to break it then it's done without any remose.

Have you ever considered that part of the reason that Mac OS hasn't been more successful in business is because Apple doesn't have the same strong commitment to backwards compatibility?

Sure. I wrote about it years ago. But again: this only about system ABI, not about developer's tools.

GNU Make 4.0 released

Posted Oct 10, 2013 11:18 UTC (Thu) by eru (subscriber, #2753) [Link] (11 responses)

Now, software developers are different: their work (or their life passion, does not matter) is code. Writing code, fixing code, that kind of stuff. It's perfectly fine to break stuff which is only ever touched by developers.

Sir, I beg to differ. Just like everyone else, software developers don't want gratuitous work dumped on them. When I'm working on my own application, I just hate it when some tool developer generates extra work for me by making an incompatible change only because he finds it fixes some cosmetic issue. It is especially jarring in a tool that has been stable for years, or decades. (That alone is proof that the change simply wasn't worth it). One is more tolerant of this kind of thing in a fast-developing project.

"Make" in fact is an example of software tool that, on Linux, is used also by other people than what would strictly be considered developers. Relatively unsophisticated users use it to build an application from a "tarball", following step-by-step instructions.

You noted "users ≠ developers". That I think is a false dichotomy. I am a "developer" in some matters, "user" in some others. Everyone is.

(it's not uncommon to see that you need MSVC6 or MSVC2005 to develop something because older versions are not 100% backward compatible)

Not a shining example to emulate. I'm pretty sure the incompatibilities in Microsoft tools are often driven by commercial considerations. Keep the upgrade treadmill rolling. For example, the latest Visual C++ does not produce Windows XP-compatible binaries without some special obscure set-up. They could have easily made it plain, but the agenda is to finally kill XP.

GNU Make 4.0 released

Posted Oct 10, 2013 13:12 UTC (Thu) by khim (subscriber, #9252) [Link] (10 responses)

"Make" in fact is an example of software tool that, on Linux, is used also by other people than what would strictly be considered developers. Relatively unsophisticated users use it to build an application from a "tarball", following step-by-step instructions.

Well, sure, but this is totally separate problem. Make was never designed or developed with “unsophisticated users” in mind. The fact that said users are dealing directly with make is serious failing of GNU/Linux model, but it's totally separate from problems of “make” itself.

You noted "users ≠ developers". That I think is a false dichotomy. I am a "developer" in some matters, "user" in some others. Everyone is.

Sure. I'm user, too and when I need to use “make” just to run some supposedly “user-friendly” program I absolutely hate that. But then, of course, the problem lies with the need to run “make” at all, not in the quality of “make”!

I'm pretty sure the incompatibilities in Microsoft tools are often driven by commercial considerations.

Some of them, yes, but surely not of them!

For example, the latest Visual C++ does not produce Windows XP-compatible binaries without some special obscure set-up. They could have easily made it plain, but the agenda is to finally kill XP.

Well, it's much harder to use latest version of GCC to develop things which you can run on Fedora Core 1 (which is much younger then XP!). Should we conclude that GCC development is dominated by commercial considerations to a much larger degree then Microsoft? Since Microsoft have created the appropriate instructions, while GCC developers don't even explain how to use their creation to develop software for Wheezy as was already discussed!

GNU Make 4.0 released

Posted Oct 10, 2013 16:00 UTC (Thu) by Otus (subscriber, #67685) [Link] (3 responses)

> The fact that said users are dealing directly with make is serious failing of GNU/Linux model, but it's totally separate from problems of “make” itself.

Arguably so, but note your use of the word 'directly'. Even if no end user had to use make directly, the tools they did could use it under the hood. A regression in make would still break their things even if they got a nice GUI error message.

GNU Make 4.0 released

Posted Oct 10, 2013 16:50 UTC (Thu) by khim (subscriber, #9252) [Link] (2 responses)

If some tool uses make “under the hood” then it's responsibility of the tool to cope with changes in make. There are many different versions of make and even metamake tools like automake or cmake which help you cope with all that zoo. You, as developer, should decide if you want to play with fire or not but you should not expect that your abuse if make somehow makes it product for the end user.

GNU Make 4.0 released

Posted Oct 11, 2013 13:27 UTC (Fri) by Otus (subscriber, #67685) [Link] (1 responses)

> If some tool uses make “under the hood” then it's responsibility of the tool to cope with changes in make.

Should it also be the responsibility of e.g. libc to cope with changes to Linux?

GNU Make 4.0 released

Posted Oct 11, 2013 16:13 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

If the kernel sucked at ABI stability where libc cares, yeah, that'd be a killer feature. Since kernel ABI breaks usually come in the form of paths or contents of /sys and /proc, what is libc going to do to help?

A better analogy would be a GUI toolkit abstraction layer. When one of the backends gains a new API for doing something that's more efficient, wouldn't it be nice if the users of the layer didn't have to care when the old way got deprecated and finally removed, transparently? Isn't that the whole point of the layer in the first place?

Think about it this way: if you use make under the hood, it's basically an implementation detail and if you don't cope with changes to make, you're leaking that fact out. And probably breaking your program's "ABI" of "the build button builds the project or tells the user where the user has errors". Your program's failure to keep pace with make is not the user's problem.

GNU Make 4.0 released

Posted Oct 11, 2013 5:08 UTC (Fri) by eru (subscriber, #2753) [Link] (2 responses)

"Make" in fact is an example of software tool that, on Linux, is used also by other people than what would strictly be considered developers. Relatively unsophisticated users use it to build an application from a "tarball", following step-by-step instructions.

Well, sure, but this is totally separate problem. Make was never designed or developed with “unsophisticated users” in mind. The fact that said users are dealing directly with make is serious failing of GNU/Linux model, but it's totally separate from problems of “make” itself.

Er, it's open source, isn't it? That means the "tarball" is in some ways the primary format of a program, and a DEB or ROM is just a convenience. As I see it, it is good that a developer can supply the source and not worry about packaging it for N+1 distributions and CPU architectures. But for this to work, certain basic tools must be have a stable interface.

Well, it's much harder to use latest version of GCC to develop things which you can run on Fedora Core 1 (which is much younger then XP!). Should we conclude that GCC development is dominated by commercial considerations to a much larger degree then Microsoft? Since Microsoft have created the appropriate instructions, while GCC developers don't even explain how to use their creation to develop software for Wheezy as was already discussed!

Windows is a closed system where the idea that users ever would see source code is antithetical. No wonder it pays more attention to binary compatibility. Linux has a totally different philosophy from the start.

I guess my point is this: For the open source system to be useful, it must try to retain maximal source-level backward-compatibility, which also includes key tools like the shell, compilers and standard utilities.

It's good to innovate, but if you make an incompatible change, it's effectively a new tool, and don't fool people by keeping the old name and incrementing just a minor version number.

GNU Make 4.0 released

Posted Oct 11, 2013 9:33 UTC (Fri) by khim (subscriber, #9252) [Link] (1 responses)

Er, it's open source, isn't it?

Sure. Arduino is open-source, too.

That means the "tarball" is in some ways the primary format of a program, and a DEB or ROM is just a convenience.

Do you want to imply that primary format for the Arduino will be a blueprint and that every user if Arduino should be able to make the PCB and solder the elements to it? This is not how it works: people who want to tinker with design can build various Brasuínos and Freeduinos, but of course, typical user is not supposed to tinker with PCB itself.

As I see it, it is good that a developer can supply the source and not worry about packaging it for N+1 distributions and CPU architectures.

You may argue that it's good in some cosmic sense, but it's primary reason for pitiful 1% desktop penetration of GNU/Linux.

But for this to work, certain basic tools must be have a stable interface.

Right. But the alternative is to provide stable ABI—which is simpler. And people don't even do that. Why do you think they will provide higher-level stable interface if they don't want to do it on the lower level?

It's good to innovate, but if you make an incompatible change, it's effectively a new tool, and don't fool people by keeping the old name and incrementing just a minor version number.

Sorry, but you are wrong: if you don't restrict use of the tool then practically any change will be incompatible change! Especially in a tool like make which can call itself recursively and then retry. Do something like “$(MAKE) foo || $(MAKE) bar” (as I already wrote), implement improper use of some functionality in “$(MAKE) foo”—and voila: your Makefile works with old version of “make” and does not work with a new one.

This was just a minor change because they just added better error message to a case which was always forbidden. The fact that people actually used this erroneous combination is unfortunate, but as always when you use undocumented features you are expected to fix breakage when it changes/disappears.

GNU Make 4.0 released

Posted Oct 11, 2013 10:01 UTC (Fri) by eru (subscriber, #2753) [Link]

That means the "tarball" is in some ways the primary format of a program, and a DEB or ROM is just a convenience.

Do you want to imply that primary format for the Arduino will be a blueprint and that every user if Arduino should be able to make the PCB and solder the elements to it?

If everyone had a magic machine that would make an Arduino or some other gadget from blueprints, then I would actually say yes. Of course that is fantasy for hardware. But it exists in a very real sense for software.

About API and ABI compatibility, it is chiefly a matter of discipline. Perfect backwards-compatibility is of course unattainable (perfect anything always is). But it should be the aim. Even to the point of supporting use of undocumented features, if they are reasonable and have come into widespread use.

One reason why virtual machines have become popular as platforms for applications is that the hardware interfaces they emulate are very stable. And why is that? Because making changes in actual hardware is more expensive and slower than in software, forcing the people responsible for them to exercise restraint and take compatibility more seriously.

GNU Make 4.0 released

Posted Oct 11, 2013 9:10 UTC (Fri) by anselm (subscriber, #2796) [Link] (2 responses)

Make was never designed or developed with “unsophisticated users” in mind. The fact that said users are dealing directly with make is serious failing of GNU/Linux model, but it's totally separate from problems of “make” itself.

Make has its shares of idiosyncrasies but even an »unsophisticated user« can gainfully employ it for all sorts of interesting things that have nothing to do with software development. IMHO, a claim that »make was never designed or developed with ›unsophisticated users‹ in mind« insults the intelligence of »unsophisticated users« and could just as well apply to other idiosyncratic Unix tools like, say, awk or sed, which are nevertheless just as useful to a general audience. It also purports having an insight into the mind of Stuart Feldman that I do not think you actually command.

GNU Make 4.0 released

Posted Oct 11, 2013 9:53 UTC (Fri) by khim (subscriber, #9252) [Link] (1 responses)

a claim that »make was never designed or developed with ›unsophisticated users‹ in mind« insults the intelligence of »unsophisticated users« and could just as well apply to other idiosyncratic Unix tools like, say, awk or sed, which are nevertheless just as useful to a general audience.

Nope, they are not useful. Anyone who even knows how to start command-line is a geek from the “Joe Everage” POV. And it's kind of impossible to use awk and/or sed without a command line.

It also purports having an insight into the mind of Stuart Feldman that I do not think you actually command.

The Times They Are a-Changin'. When Stuart Feldman created make “unsophisticated users” were very different and back then use of make was nothing to brag about—but back then people also knew how to fix their Makefiles when they refused to work (they even sometimes fixed their programs when they refused to work!). World was very different back then.

I'm not talking about “unsophisticated users” of 1977 (which lived with Fortran77 compiler made by the very same Stuart Feldman and which was, surprise, surprise, not compatible with Fortran66 compiler!), I'm talking about “unsophisticated users” of today. Make was never intended for these.

GNU Make 4.0 released

Posted Oct 11, 2013 10:12 UTC (Fri) by anselm (subscriber, #2796) [Link]

Using the Linux command line is not exactly rocket science. On the contrary, one of the main advantages of Linux is that you have a powerful command line available in addition to a GUI, and it would be silly not to make use of that where appropriate. I make a living teaching »unsophisticated users« Linux and on the whole they're doing fine with basic command line stuff. It helps if you motivate them using examples for which the GUI really sucks ;^)

It is true that we usually don't get as far as »make« but that is an issue of available time more than anything else. I do cover the basics of sed and awk, though …

GNU Make 4.0 released

Posted Oct 10, 2013 15:58 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

> their work (or their life passion, does not matter) is code

Actually, it probably does[1]. If it's a passion, there's caring there, otherwise they'll probably do the minimal amount of work to avoid doing more (though the shortsightedness isn't factored into the work involved…c'est la vie).

[1]http://symbo1ics.com/blog/?p=2119 (See "All hail the career programmer." in particular)

GNU Make 4.0 released

Posted Oct 16, 2013 21:20 UTC (Wed) by Jandar (subscriber, #85683) [Link]

> Users ≠ developers.

If I use make while developing another project than I'm a user with respect to make.

GNU Make 4.0 released

Posted Oct 12, 2013 8:12 UTC (Sat) by dvdeug (guest, #10998) [Link] (2 responses)

Not being "hobbled" by backwards compatibility means that I can't take some program that does some obscure job I need done, that was last updated in 1999, and expect it to build. It means that we're throwing away code at a steady rate and wasting all the effort that was put into it. That's not a good thing.

GNU Make 4.0 released

Posted Oct 12, 2013 18:30 UTC (Sat) by raven667 (subscriber, #5198) [Link] (1 responses)

A perfect example are Linux games. I have most of the games that Loki published and a few others. I tried to install Alpha Centauri in Ubuntu 12.10, a stable release, and didn't get them to work. The kernel should be capable of running them, why is userspace broken. The same games available from gog.com work fine on modern Windows and Mac systems.

GNU Make 4.0 released

Posted Oct 12, 2013 18:55 UTC (Sat) by dvdeug (guest, #10998) [Link]

That's not an entirely fair comparison, since gog.com releases versions that have been patched work fine on modern systems.

GNU Make 4.0 released

Posted Oct 9, 2013 23:31 UTC (Wed) by madscientist (subscriber, #16861) [Link] (14 responses)

That change in GNU make required a TWO LINE patch to ONE FILE in the kernel source, which was duly applied over three years ago and released in Linux 2.6.34. The idea that anyone is building a Linux kernel at version 2.6.33 or older and yet are so constrained that they cannot add an innocuous patch to the build system (not even the code), and thus are forced to continue to use GNU make version 3.81, is just not credible.

I can only assume you're talking about some other packages, not the kernel. However there haven't been many complaints about packages not working due to this change.

Finally, the syntax under discussion was *never* legal, according to the GNU make manual; it just happened to work due to a quirk in the parser. And in fact, it didn't work reliably (if you wrote the targets in a different order it would fail for example). When the parser was enhanced, this not-valid-anyway quirk was eliminated.

As for Debian's position on 3.82, you'll have to ask Manoj about it. As has been pointed out here many other distributions have updated.

GNU Make 4.0 released

Posted Oct 9, 2013 23:52 UTC (Wed) by josh (subscriber, #17465) [Link] (6 responses)

A quick search through the kernel git log for "\<3.82\>" turned up at least three separate patches, definitely more than two lines in one file. (That's ignoring all the other changes to support make 3.82, and the various regressions that distros have incorporated patches for.) And having to apply (and adapt) any patches at all, no matter how simple, at every checkout of a different version in a git-powered archaeological dig is quite painful.

Also, I have yet to find anything in the manual or elsewhere that *prohibits* that syntax either, and it's a rather natural syntax to use if you use the same stanza to build several different things.

I understand that changes to the parser happened to break this. I also understand that you never intended it as valid syntax. However, to take a lesson from the Linux kernel itself, the ABI is defined by what programs actually use, not by what it would be sane for them to use or what the developers intended. In such a fundamental tool as make, breaking backward compatibility with functionality actually used in real software is a Bad Idea.

On that note: if someone else did the work to write and test it, would you be willing to accept a patch re-adding support for this syntax?

GNU Make 4.0 released

Posted Oct 10, 2013 0:44 UTC (Thu) by khim (subscriber, #9252) [Link] (5 responses)

However, to take a lesson from the Linux kernel itself, the ABI is defined by what programs actually use, not by what it would be sane for them to use or what the developers intended.

Well, yeah. The rule is WE DO NOT BREAK USERSPACE!. And here is the latest example of contortions Linux developers are willing to do to keep this property.

But note. ABI, not API! The very same example shows us the case where new kernel intentionally makes some [incorrect] programs unbuildable!

In such a fundamental tool as make, breaking backward compatibility with functionality actually used in real software is a Bad Idea.

Sure. But can you give us better explanation about what is wrong? I was under impression that everything was done as it should: if you file is incorrect then you get an error message and make does not build incorrect programs. Which means that ABI is preserved and API is not—which is exactly how such changes should be done and which is exactly how serious projects like (Linux kernel, GLibC, Windows, etc) are doing it.

GNU Make 4.0 released

Posted Oct 10, 2013 6:23 UTC (Thu) by josh (subscriber, #17465) [Link] (4 responses)

That perf example is hardly a shining moment for compatibility, and generally the kernel maintains API as well as ABI.

In any case, the API/ABI distinction only applies to source versus binaries, not to makefiles. Utilities like make don't have an API/ABI distinction; make is either backward-compatible with old makefiles or it isn't. In this case, it isn't.

The key point: breaking backward compatibility with code that you consider incorrect (but which the spec never actually proscribed and the code actually allowed) is still breaking backward compatibility.

GNU Make 4.0 released

Posted Oct 10, 2013 9:37 UTC (Thu) by khim (subscriber, #9252) [Link] (3 responses)

That perf example is hardly a shining moment for compatibility, and generally the kernel maintains API as well as ABI.

Why not? IMO it's great example. It may not be typical, but it's not atypical either.

In any case, the API/ABI distinction only applies to source versus binaries, not to makefiles.

Sure, but why they are treated differently?

Utilities like make don't have an API/ABI distinction; make is either backward-compatible with old makefiles or it isn't. In this case, it isn't.

Sure, and this is not a big deal.

I think you are doing wrong conclusion from the lack of ABI. If your project is API-only (e.g. python or make) then it does not automatically means that it should be preserved. You need to take a look on user of said API. If the user of said API is skilled software engineer then you can safely break it if the payoff is large enough: user can (and will!) fix the mess. If the user is someone else then keeping unbroken API is important: since you user does not have skills needed to fix the problem s/he will just stop using your product and that is “bas thing”™.

The key point: breaking backward compatibility with code that you consider incorrect (but which the spec never actually proscribed and the code actually allowed) is still breaking backward compatibility.

Sure, but that's fine and, in fact, unavoidable: the only way to have 100% backward compatibility is to never change anything at all—and you can always do that because old versions are still around. If you want to use something new then you often need to add some small changes to you sources.

GNU Make 4.0 released

Posted Oct 10, 2013 14:53 UTC (Thu) by josh (subscriber, #17465) [Link] (2 responses)

> Sure, but that's fine and, in fact, unavoidable: the only way to have
100% backward compatibility is to never change anything at all

Why would that be? You can most definitely make changes and add new features without breaking old code (e.g. old makefiles). Many projects can and do maintain backward compatibility. Regression test suites exist for that kind of purpose, as well.

GNU Make 4.0 released

Posted Oct 10, 2013 15:40 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

You can make "VM"-level changes (e.g., chunking output by target, autoparallelism, etc.) just fine (though I'm sure *something* cares about stuff like how much CPU `make` takes, so even auto-j would be suspect; chunking would be fine since it could have been valid output before anyways), but adding variables (something like MAKE_OUTPUT_IS_TTY), syntax ("!="), or changing parsing rules isn't allowed because those might have other uses already with different meanings. If you want "100%", you have to abide by that.

GNU Make 4.0 released

Posted Oct 10, 2013 16:39 UTC (Thu) by khim (subscriber, #9252) [Link]

No matter how many “regression test suites” you have you still break compatibility regularly. If you introduce some new syntax and/or new “magic” variable then you break something. Heck, if you introduce some new capability you automatically break some things because it's perfectly possible to write “$(MAKE) foo || $(MAKE) bar” in some rule and if “$(MAKE) foo” will start succeeding then everything will blow up (e.g. if it was written in advance to exploit your feature till it's design stabilized but then was not updated when your feature was actually added to “make”).

Now you can start spouting some nonsense like “you are not supposed to support broken code, we are only talking about code which people wrote in accordance to certain rules”, but then you'll lose again because “certain rules” (GNU Make Manual here) never gave you right to write the code which was broken by “make 3.82”!

GNU Make 4.0 released

Posted Oct 10, 2013 3:12 UTC (Thu) by xtifr (guest, #143) [Link] (6 responses)

As for Debian's position on 3.82, you'll have to ask Manoj about it. As has been pointed out here many other distributions have updated.

I don't know what's in Manoj's mind, but it may just be an excess of caution. Make is integral to Debian's packaging system—even packages that don't use make at all internally will still have their build system called by make when building the Debian package—and thus, Debian may have one of the largest collections of makefiles anywhere. (Except that most of them are named debian/rules, rather than Makefile.) With that much code to test, some care seems reasonable.

GNU Make 4.0 released

Posted Oct 10, 2013 6:14 UTC (Thu) by josh (subscriber, #17465) [Link] (5 responses)

Debian has a release-critical bug open against make 3.82 in experimental: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=635317

If make 3.82 were in unstable, that bug would prevent it from migrating to testing and eventually reaching stable. Thus, it makes sense to keep make 3.82 in experimental and 3.81 in unstable as long as that bug remains open.

GNU Make 4.0 released

Posted Oct 13, 2013 20:37 UTC (Sun) by ploxiln (subscriber, #58395) [Link] (4 responses)

hilarious, it's just the wrapper code for the closed-source nvidia module. this is why we can't have nice things!

GNU Make 4.0 released

Posted Oct 13, 2013 21:06 UTC (Sun) by josh (subscriber, #17465) [Link] (3 responses)

That's where the bug was originally filed, but the makefile in question was actually in the kernel headers.

GNU Make 4.0 released

Posted Oct 13, 2013 21:42 UTC (Sun) by madscientist (subscriber, #16861) [Link] (2 responses)

Yes, this bug should have been re-targeted at the kernel headers package and fixed there. It is absolutely not a release-critical bug against GNU make. This is simply a bug triage failure.

The upstream kernel headers had been fixed in Linux kernel version 2.6.34, and there's no excuse for the Debian linux-headers-3.0.0 package to still be using Makefiles from a release older than that.

For what it's worth, I checked the version of linux-headers in wheezy (linux-headers-3.2.0-4-amd64) and it doesn't appear to have this problem, so this bug should be closed anyway.

GNU Make 4.0 released

Posted Oct 13, 2013 22:03 UTC (Sun) by josh (subscriber, #17465) [Link] (1 responses)

It's a regression in make for not handling makefiles it used to handle. And not only that, it's getting fixed upstream; someone figured out that by changing the error about mixed targets to a warning, the code itself actually still works as well as it used to.

GNU Make 4.0 released

Posted Oct 13, 2013 23:32 UTC (Sun) by madscientist (subscriber, #16861) [Link]

I'm sorry, but that's wrong. The bug referred to here is a bug in the linux-headers package. The linux-headers package for the 3.0.0 kernel should be using the module build system provided with the 3.0.0 kernel. If it were, everything would work regardless of the version of GNU make installed. That's the bug. Full stop. And it appears to have been fixed in the linux-headers 3.2.0 package in wheezy.

You may feel that there is some other bug that should be filed, such as "compiling kernels older than 2.6.34 on my Debian system with GNU make 3.82 fails, while it succeeds with GNU make 3.81". That definitely belongs in the GNU make bug list. Then the question becomes whether that should be release-critical (I would argue "no" but I'm not the Debian maintainer so it's not up to me), and what to do about it if the upstream maintainer does not consider it a bug.

It's not a goal of GNU make, now or ever, to always handle every makefile that it used to handle at any time in the past without change. New releases of GNU make introduce new features. Because of the free-form nature of makefile syntax it's sometimes not feasible to add a feature without _potentially_ causing some makefile, somewhere, to change behavior. These potential changes, along with workarounds, are documented in the NEWS file.

If the requirement for GNU make to avoid being held back by Debian is that it always "handle [all] makefiles it used to handle", then it's unlikely Debian will ever update GNU make to any later version. That's their choice of course. Then when some package somewhere requires a newer version of GNU make in order to build correctly, they'll have to decide how to handle that.

> And not only that, it's getting fixed upstream

In the next version of GNU make there will be a change to make this particular problem a non-fatal error, since that turned out to be not very difficult. That doesn't undo any of the other numerous backward-incompatibilities added in GNU make since 3.81, so it does not mean that GNU make now magically "handles makefiles it used to handle", in general. Further, there's no guarantee that this particular problem won't become a fatal error again in future releases, if makefile syntax changes such that maintaining it becomes too onerous.

But if all you are _really_ concerned about is building versions of Linux older than 2.6.34 with a newer version of GNU make, then the next release should offer relief.

GNU Make 4.0 released

Posted Oct 10, 2013 6:56 UTC (Thu) by zwenna (guest, #64777) [Link]

> This bug is the reason why make 3.82, released in 2011, still sits in Debian experimental and hasn't yet graduated to unstable where it might form part of a release. Make 4.0 seems likely to suffer the same fate.

The bigger problem for Debian is that the package maintainer has been MIA for several years, and this does not seem to change. Thus, I don't even expect Make 4.0 to show up in experimental any time soon.

GNU Make 4.0 released

Posted Oct 9, 2013 17:06 UTC (Wed) by wazoox (subscriber, #69624) [Link]

Slackware 14 has 3.82. I don't see the point of being able to compile ancient kernels. Simply include make-3.81 in the repository and define the proper build dependencies for other stuff.

GNU Make 4.0 released

Posted Oct 10, 2013 11:45 UTC (Thu) by jezz (guest, #59547) [Link] (8 responses)

I am pleased to see GNU Make team choose a popular language for their extensions.

While some young geeks would choose python, GNU Make team has preferred Guile maturity. Guile is really the best answer for all people who complain Makefile grammar is hideous.

In add, Guile is very well integrated with my ed [1] editor.

[1] http://www.gnu.org/fun/jokes/ed.msg.html

GNU Make 4.0 released

Posted Oct 10, 2013 13:16 UTC (Thu) by khim (subscriber, #9252) [Link] (7 responses)

Guile is really the best answer for all people who complain Makefile grammar is hideous.

Are you joking are you referring to the fact that one look on guile grammar will make Makefile grammar look adequate in comparison?

Guile is not so bad, but this choice basically means that nobody will bother with make anymore… people are switching to CMake en masse and while I hate CMake I will probably eventually switch too… because it's better to have one mature and usable tool rather then be stuck with dozen of incompatible tools or with tools which don't have many important features.

GNU Make 4.0 released

Posted Oct 10, 2013 16:17 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (5 responses)

Well, the fact that CMake gets you Visual Studio support on Windows is a great boon (as opposed to "get Cygwin working (and have fun with the slow I/O!)" or "use MinGW (and good luck!)". Yeah, CMake's syntax can suck[1], but there are things you just need to avoid (and it currently takes experience to know what those are unfortunately)…and they stick around for backwards compatibility.

[1]And can be gotten very wrong too…*grumble*OpenCV*grumble*.

GNU Make 4.0 released

Posted Oct 14, 2013 13:23 UTC (Mon) by rghetta (subscriber, #39444) [Link] (4 responses)

imho, a good alternative to CMake here is premake (http://industriousone.com/premake).
It's much nicer than CMake, while being powerful enough to handle even complex projects

GNU Make 4.0 released

Posted Oct 14, 2013 19:44 UTC (Mon) by mathstuf (subscriber, #69389) [Link] (3 responses)

So I have yet to encounter any project which uses premake. The syntax for it does look nicer and more consistent, but the problem seems to be that the community is small and that the "where is project X?" problem isn't solved outside of copy pasta. Poking at the forums for a bit, I found this[1] script which uses CMake to do that work…which seems odd. It also is a little simple and probably does not handle all of the cases that find modules create. Not to mention it was last released in 2010, lacks VS2012 and VS2013 support (without fetching from hg at least). It looks promising, but I would say that it is lacking in the formal maintenance department. I also don't know how backwards compatibility is handled.

[1]http://industriousone.com/topic/cmake-integration

GNU Make 4.0 released

Posted Oct 15, 2013 7:27 UTC (Tue) by rghetta (subscriber, #39444) [Link] (2 responses)

We use premake for over 100 libraries and executables, with various dependences. So far it handled almost everything we needed, and was very simply to extend it for the few things missing.
We looked at CMake, but found premake much better.

GNU Make 4.0 released

Posted Oct 15, 2013 15:04 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (1 responses)

Is this one massive build or is each library (or group thereof) its own build tree? If the latter, how much work is finding things like Boost (find the header, libs are relative to it), Qt (running qmake to get paths), FFmpeg (pkg-config, but one for each library), or your own code on Windows? Do they all need to be installed to the same place (echoing /usr from *nix)?

Usually most of the nicities of CMake come in when dealing with platforms differences for things not directly related to building code (e.g., paths for debug vs. release builds with VS and Xcode, testing and dashboard integration, ccmake for incrementally customizing builds (vs. having to keep giving ./configure the same set of flags if you want to modify one thing), etc.).

GNU Make 4.0 released

Posted Oct 16, 2013 8:55 UTC (Wed) by rghetta (subscriber, #39444) [Link]

every subprojects has its tree, and both libraries and sources are found by searching in a set of directories (different builds, releases, architectures etc can have different search directories)
OTH, we don't need to launch external programs to get paths and configurations; under this aspect, our builds is simpler and more controlled than, say, the average OSS project.

GNU Make 4.0 released

Posted Oct 12, 2013 15:23 UTC (Sat) by nix (subscriber, #2304) [Link]

Guile is not so bad, but this choice basically means that nobody will bother with make anymore
No, it just means that particular feature is unlikely to see heavy use. Just like a whole bunch of other GNU Make features. (I would not be surprised to find at least one feature that is used only by the glibc build system out of the entire universe of free software projects. :} )

GNU Make 4.0 released

Posted Oct 13, 2013 12:21 UTC (Sun) by fishface60 (subscriber, #88700) [Link] (1 responses)

I am more concerned with the bootstrapping implications of requiring Guile to be able to build Make, since currently you can get away with binutils, gcc, gawk, eglibc, busybox and make.

Adding Guile into the mix requires libffi, bdwgc and libunistring, as well. Also I've experienced Guile having build times comparable to gcc, effectively doubling the build times to bootstrap a system.

It appears to be optional for now, but if anything starts needing the Guile extensions, then it becomes mandatory to build new versions.

GNU Make 4.0 released

Posted Oct 15, 2013 1:01 UTC (Tue) by wahern (subscriber, #37304) [Link]

Building against libffi is a gigantic PITA. For one thing, OS X's ffi.h header is broken at line 305--an #elif without a condition. I submitted a bug report 2 years ago and it's still unfixed. But the location of ffi.h is a complete crapshoot anyhow.

I know that the developers prefer that libffi not be a typical third-party library in the manner of OpenSSL or libxml, but it's often packaged that way downstream, and developers depend on it being available. But various Linux distributions and other OS vendors each package it differently. Very annoying.


Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds