Modernizing Fedora's C code

Posted Nov 3, 2022 22:00 UTC (Thu) by sam_c (subscriber, #139836)
In reply to: Modernizing Fedora's C code by khim
Parent article: Modernizing Fedora's C code

> Why is it hard? I have done such things before. They are tedious, sure, but “hard”? How? Why?

"Hard" in that there's a lot of work, not a lot of people helping with it right now, and not all packages follow a standard pattern. I'm glad you've done it before and I welcome your help in our efforts! Also, there's a lot of packages in the world, and therefore a lot of logs to sift through and actually fix the problems in.

1. Not everything uses `autotools` so you can't just wrap for e.g. configure. Of course, in Gentoo, it's easy enough for us to wrap anything called in our `src_configure` phase. An example of this is the bug in the article: distutils (https://github.com/pypa/setuptools/issues/3648).

2. Older `autotools` doesn't create a config.log. Related to #1, as different build systems behave differently. They may store their results in a different place.

3. autotools and other build systems may generate implicit function declaration errors intentionally when testing for e.g. BSD-only functions. This is noise and makes it harder to identify "real" breakage.

4. Testing every build combination of every package is impossible. In Gentoo, for example, PHP has 82 or so USE flags (different build options). We therefore can't test all combinations ourselves. Making a test for these issues programmatically so that users are warned if their configuration might be broken is possible (and will likely be done), but reducing false positives is important to avoid user confusion.

5. As noted in another comment, the shipped versions of autoconf (including the latest version 2.71) emits declarations which are broken with `-Werror=strict-prototypes`.

6. Related to #4: suppose we missed a problem in a build of a library. A user then reports a problem in a consumer of that library. It's not always going to be obvious that the library was miscompiled, especially if this is several years down the line.

I'm not claiming this is all intractable, but it's also not easy, and it's also not quick.

Modernizing Fedora's C code

Posted Nov 3, 2022 22:28 UTC (Thu) by khim (subscriber, #9252) [Link] (2 responses)

> I'm not claiming this is all intractable, but it's also not easy, and it's also not quick.

It definitely wouldn't be quick and that's why I think it's good idea to do a bit more infrastructure work before starting looking on logs.

#1: I'm not proposing to wrap configure. Not even close. I propose to make wrapper for gcc (and clang if it's used). You either specify it as compiler or, if even that's hard, rename old gcc to something like gcc.real and put wrapper in that place instead.

The only trouble would be the fact that Gentoo wouldn't allow wrapper to just store logs in some random file, but that's not a big deal: you can send them by connecting to 127.0.0.1 and talking to some server, e.g.

#2 and #3 become irrelevant, because you are not even comparing logs. You are just comparing behavior of two gcc calls. And if they lead to the same result (both attempts compiled conftest succefully or both failed) then you don't even need to do anything about that case. But if one succeeded and one failed then you need to look and see what happens there.

#4 is indeed a big problem, but that's Gentoo-specific issue, that I have no idea how to fix. I only needed to ensure that after switch from glibc to musl in some pretty convoluted embedded image builder there would be no changes in behavior (or at least we would know what have changed). It's a bit easier than what you attempted, because there are no 82 use flags.

#5 you would, probably, require some changes in autotools, but is it really that problematic? I guess that's something I haven't actually needed, but without working fix it's hard to even start.

#6 is, again, same problem as #4. You can not ensure there would be not changes in all possible combinations of use flags, but at least you can ensure that tested combinations work fine.

Modernizing Fedora's C code

Posted Nov 3, 2022 22:36 UTC (Thu) by sam_c (subscriber, #139836) [Link] (1 responses)

1. I said "wrap for". The point being, if you want to just find 'configure' or configure-ish failures, and avoid this affecting package builds (which makes things awkward because you then can build things which depend on it), it can be a problem. We do sandbox builds including network but yes, it'll be doable to store in a file somewhere.

2/3. Right, it's a good idea. You indeed don't have to worry about diffing later, you can import if something simply doesn't compile with the second compiler. I'll go play with it.

4. It's really just the nature of the large amount of software out there. Some of these bugs will end up hitting other distros anyway.

5. Yes, it is, because you have to go around running autoreconf in tonnes of packages when the configure scripts in the release tarball are stale (and it's not fixed in an autoconf release, so new releases of software will continue to be broken).

Would it perhaps have been more acceptable to you if I'd said "complex" rather than "hard", as there's many factors? :)

Modernizing Fedora's C code

Posted Nov 4, 2022 15:52 UTC (Fri) by mbunkus (subscriber, #87248) [Link]

What's even worse is that autoconf versions aren't always compatible. You cannot simply take a years-old autoconf.ac source file & process it with a the newest, bug-fixed autoconf release. There are several breaking changes.