Autoconf and m4

Posted Apr 30, 2024 19:40 UTC (Tue) by epa (subscriber, #39769)
Parent article: Security patterns and anti-patterns in embedded development

GNU Autoconf with its combination of shell scripting, m4 macros, odd snippets of code to compile and test, and Makefile generation is indeed obscure to all but the most advanced wizards. There are good reasons for that—it has a messy job to do. Nonetheless, you might want to cut some of the complexity.

There are forces pulling in two opposing directions. On the one hand you want to build from pristine sources, straight out of git, and not from the traditional release tarball. That argues for doing the full autoconf setup each time, rather than relying on a configure script that the maintainer has generated. On the other hand, you want the build to have as few steps as possible and to be understandable in its entirety without having to know m4 and all that.

Just possibly the answer could be to take a step further away from the original sources. Nowadays there is less variety among Unix-like systems, or at least those that 99% of developers care about. Your contributors are not using a mixture of Irix, AIX and old SunOS versions. People using those obscure systems might still run the configure script, but for everyone else why not ship a pre-generated makefile that assumes sensible defaults for a GNU/Linux system? Simple customizations like install root could be set via environment variables. Then Linux distributions could pull the sources, delete the autoconf/ subdirectory just to make sure it’s not used, and build the rest in a predictable way.

Is that at all feasible? Or is it like Microsoft Office, where we agree that only 10% of Autoconf’s functionality is needed, but nobody can agree which 10%?

Autoconf and m4

Posted Apr 30, 2024 20:20 UTC (Tue) by Paf (subscriber, #91811) [Link]

"GNU Autoconf with its combination of shell scripting, m4 macros, odd snippets of code to compile and test, and Makefile generation is indeed obscure to all but the most advanced wizards. There are good reasons for that—it has a messy job to do. Nonetheless, you might want to cut some of the complexity."

Shell-independent shell scripting, babe-y. It's a thing of beauty. Or at least ... a thing.

Autoconf and m4

Posted Apr 30, 2024 20:23 UTC (Tue) by Paf (subscriber, #91811) [Link] (3 responses)

RE: Assume sensible defaults.

The problem - or so it seems to me - is that configure is the main way builds find and report missing dependencies. So I'm building X and the build requires totally-reasonable-but-not-universal library Y (or god forbid it requires obscure library Z). Those dependencies are generally expressed via the configure script. Seems like a non-starter.

Autoconf and m4

Posted Apr 30, 2024 20:50 UTC (Tue) by epa (subscriber, #39769) [Link] (2 responses)

But if the deps are missing the build will fail anyway. The configure script is mostly for configuring optional dependencies. (We saw this used in the xz attack, I believe, where the detection of Landlock was subtly broken so that the build continued without it.)

There will be exceptions, but generally I think you could define a default build that requires most of the dependencies without having to sniff whether they are available. If that’s not flexible enough for everyone, some will stay using the configure script.

Autoconf and m4

Posted May 1, 2024 7:16 UTC (Wed) by epa (subscriber, #39769) [Link] (1 responses)

I guess one improvement would be a "strict mode" for the configure script where everything has to be explicitly specified as either --with-foo or --without-foo. If some new dependency appears then you'd get a message like

This configure script was invoked with --strict. The optional dependency 'libfoo' has not been specified.
You must pass either --with-libfoo or --without-libfoo.
Autodetection found that libfoo is present on this system.

Then build systems for Linux distributions would tend to use the --strict flag and nail down exactly what dependencies they want, leaving nothing to autodetection. That would have stopped the xz attack where the configure script stopped including the Landlock dependency and nobody noticed.

Autoconf and m4

Posted May 3, 2024 1:22 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Yes, I've found that explicit requests for things like that are best. It's really frustrating when you copy a configure line from one machine to another and get a result because somewhere in the middle of the log dependency X's state differed between the two machines and you're left figuring out why behavior Y differs. *Some* things make sense to just inspect and change, but that's for things that you can polyfill (BSD routines on Linux or glibc routines elsewhere) or are just plain impossible on the platform (e.g., v4l support on macOS or Windows).

Autoconf and m4

Posted May 1, 2024 15:35 UTC (Wed) by mb (subscriber, #50428) [Link] (2 responses)

>GNU Autoconf with its combination of shell scripting, m4 macros, odd snippets of code to compile and
>test, and Makefile generation is indeed obscure to all but the most advanced wizards.
>There are good reasons for that—it has a messy job to do.

Today there is pretty much *no* good reason for 95% of the autotools mess.

Almost all project don't care, if the build system works on obscure operating systems of the past. Yet, most code and checks in autotools are there because of some obsolete and obscure operating system that it used to support (or claims to still support). Your app won't work on these operating systems *anyway*, if you have never actually tested that.

Seriously, if you are still using autotools, you need to migrate away from that mess.
Some projects have started to migrate away from it like 20 years ago.

Having autotools in a project is a anti quality indicator.

Autoconf and m4

Posted May 1, 2024 16:07 UTC (Wed) by paulj (subscriber, #341) [Link] (1 responses)

I'd generalise a bit from that... having fought with all kinds of build systems (and seen some crazy crazy things):

- If a project's build system is not generally a series of declarative statements; with no more than a modicum of small, confined and clear, ad-hoc logic (for whatever transforms or tests needed) then that is an anti-pattern.

Is quite possible to have a small, clean, fast, declarative build system in auto*, and it's possible to make mess in pretty much any build system tool. The anti-pattern is the mess. The anti-pattern is the willingness of the people who made the build system to engage in dirty hacks, rather than read the documentation of the tool and do it properly (whether, using the features of the tool properly; or extending it cleanly).

Basically: You can evaluate a project simply on the amount of ad-hoc logic they've stuffed into their build system, and the actual build system doesn't really matter that much to this.

Autoconf and m4

Posted May 6, 2024 10:26 UTC (Mon) by LtWorf (subscriber, #124958) [Link]

Sometimes the tool is limited and there is no proper way.