LWN: Comments on "Some nonstring turbulence"

GCC pre-releases and venerable traditions

sammythesnake — Thu, 22 May 2025 15:25:37 +0000

Debian uses a tilde for version numbers that should reliability sort before, rather than after, an upcoming version. You'll see this with the "backports" section (i.e. versions from testing adapted to be installed on the stable release of Debian)

E.g. there's foo-1.8 in stable, foo-2.0~bp1 in backports, and foo-2.0 in testing. When the next stable release of Debian happens, foo-2.0 will be considered a "newer" version than foo-2.0~bp1 and installed preferentially.

Typically, the last backports version is identical with the new stable version except for being compiled against (and declaring dependencies on) the older libraries available in stable.

Conceptually, it's like having a patch version number of "-1" which naturally sorts earlier than a patch version of "0" (i.e. foo-2.0~bp1 sorts like 2 point 0 point -1 point (bp)1 < 2 point 0 point 0)

I like the semantics of this approach, but it does require that whatever is sorting version numbers understands what the tilde means and isn't really a widespread enough idiom for that be be something one can assume in general.

No safeguards?

Wol — Fri, 02 May 2025 15:34:53 +0000

> Now think about all the people who do not software full time and think: How hard could software be? If it were hard, it wouldn't be called "soft"ware :-)

Job security? If software is hard, you have to leave it to the professionals?

I've only once worked in a pure software environment - it drove me almost suicidal. Pretty much every job I've had has been a small DP team supporting end users. There's no reason why software should be hard. If you have a mixed team of professional end users who can program, professional programmers who can end-user, AND EASY-TO-USE SOFTWARE, then doing things "right" isn't hard. That's why I'm a Pickie!!!

(And I don't call Excel, SQL, BQ/Oracle/etc easy to use.)

Cheers,
Wol

No safeguards?

marcH — Fri, 02 May 2025 14:56:18 +0000

> does any programming education teach you anything about test coverage and trying to break your own code? Or version control, or code reviews, or CI, or any quality topic,...

How could I forget the "ugliest" child of them all: build systems :-D

No safeguards?

marcH — Fri, 02 May 2025 14:44:45 +0000

Like all people in a "privileged" work environment, I'm not sure you realize how ridiculously little testing some code gets before being submitted. Discussing "mutation testing" with the corresponding submitters is like trying to discuss literature with children learning to read. Please wait until they've reached middle school? In the mean time, "manually" point at a couple lines in their submission and ask them if any test fails when they break them. You might rarely ever interact with any such people so you can't notice but I can promise you there are some who never bother to try anything like that. Learning to do that does not require _any_ search which is already too much for people absolutely not interested in spending any time finding any bug with their code[*]. The very first ("baby") step is fixing that _mindset_ and that's already difficult enough. Don't scare these people with textbooks, at least not at first.

I've used an education analogy which makes me wonder: does any programming education teach you anything about test coverage and trying to break your own code? Or version control, or code reviews, or CI, or any quality topic,... I don't remember any at all but it was a while ago. I learned it all on the job. But these were full time software jobs. Now think about all the people who do not software full time and think: How hard could software be? If it were hard, it wouldn't be called "soft"ware :-)

[*] that's the job of the "validation team". Their precious time should be spent writing new bugs^H code.

No safeguards?

NYKevin — Fri, 02 May 2025 07:53:37 +0000

It's not that easy. Saying "mutation testing" to a developer who has never heard of it is probably not the best way to evangelize mutation testing... but it *is* the best search term to find tooling that enables you to actually do it (without having to reinvent everything from scratch). So we're stuck using fancy ten dollar words for these things at least some of the time.

No safeguards?

pizza — Wed, 30 Apr 2025 11:21:09 +0000

>> I'm in the MAINTAINERS file.
>That explains it.

Explains what, exactly?

No safeguards?

Avamander — Wed, 30 Apr 2025 09:43:29 +0000

> I'm in the MAINTAINERS file.

That explains it.

No safeguards?

pizza — Tue, 29 Apr 2025 20:06:22 +0000

> I bet you're not the only one in such a difficult situation, maybe there are other people who can help?

I'm afraid the difference between "can help" and "will help" is effectively insurmountable -- at least when there is no money involved.

No safeguards?

marcH — Tue, 29 Apr 2025 16:11:09 +0000

I'm really sorry that have you been hurt by ISO 9000 in the past and that this tangent of mine acted as a "trigger" for you. But this was just a tangent, I know nothing about ISO 9000 specifically and there is not much I can do to help with your healing process. On the contrary, my key message was: baby steps, don't even use fancy words like "mutation" to avoid scaring people.

I bet you're not the only one in such a difficult situation, maybe there are other people who can help?

No safeguards?

marcH — Tue, 29 Apr 2025 16:01:25 +0000

> versus "I created a situation that resulted in a failure but the existing tests didn't catch it, and if you don't fix this coverage gap immediately you're clearly lying about everything and can't be trusted blablabla"

Well beyond a strawman: it's elevated to an art form! :-)

Open source upstreams aren't suppliers

farnz — Tue, 29 Apr 2025 15:31:44 +0000

They're either an upstream offering you a project, or a downstream offering you a patch. They are not a supplier in either case, and expecting them to behave like a supplier is going to lead to problems and misunderstandings, precisely because that's the wrong sort of relationship to imagine.

Upstream and downstream also gets interesting because, unlike supplier and customer, it's one where the same entities dealing with the same project can change roles; Linus Torvalds is upstream of me in the Linux kernel fork he runs (and that we generally accept as the mainline), but if I chose to run my own Linux fork, Linus could choose to be downstream of me and submitting patches to me, or pulling patches from my upstream project into his downstream project. And, for added fun, Linus and I can swap roles - I can treat Linus as my upstream when I pull in 6.16, but then treat him as my downstream if he notices that I have a useful change that he'd like in his kernel.

No safeguards?

kleptog — Tue, 29 Apr 2025 15:15:31 +0000

We solve this by having a "make quick-test" target which looks the the last commit and based on the directories of the files modified runs some specific subset of the tests. So if you're modifying the Makefile or some shared repo it's still going to take a while. But for the majority of patches you end up only running the tests for a single subsystem which reduces the turnaround time significantly.

Sure, it occasionally happens that a patch does actually break a test in another subsystem that you didn't expect, but that's pretty uncommon.

Like you, I wanted to automated this dependency detection, but a hand-maintained list gave all almost all the bang for very little buck.

Open source upstreams aren't suppliers

Wol — Tue, 29 Apr 2025 15:12:12 +0000

> In the end, an upstream is not a supplier; there's even a different word to describe the relationship, since you have supplier/customer relationships in business, and upstream/downstream relationships in open source. Expecting an upstream to act as a supplier is opening yourself up to a world of pain every time the upstream's priorities and yours don't coincide.

I thought we were talking about a DOWNSTREAM SUPPLYING faulty patches.

So we're blacklisting people who can't be arsed to supply properly working, tested code.

I have to be careful here, as I've done exactly that, but I've done my best to provide stuff that works (in a language, Scheme, that I find very difficult to work with), and I just have to accept that if nobody else sees value in what I've done they won't take it on. But if I behaved "entitled" and expected somebody to finish the work for me, then they have every right not to want to do business with me.

It all boils down, once again, to the "entitled" mentality a lot of people seem to have about other people doing work for "free" ...

Cheers,
Wol

Open source upstreams aren't suppliers

farnz — Tue, 29 Apr 2025 14:50:25 +0000

The question then becomes where you're going to go for open source software if you blacklist all upstreams that refuse to act as suppliers. You can, of course, pay Red Hat, SUSE, Canonical, CIQ or others to act as your supplier based on top of open source upstreams, but then you're not going straight to the person who writes the code.

In the end, an upstream is not a supplier; there's even a different word to describe the relationship, since you have supplier/customer relationships in business, and upstream/downstream relationships in open source. Expecting an upstream to act as a supplier is opening yourself up to a world of pain every time the upstream's priorities and yours don't coincide.

No safeguards?

pizza — Tue, 29 Apr 2025 14:36:40 +0000

> Clearly not:

I've been part of the team dragging an organization through ISO9000 certification.
I've also worked for organizations in highly regulated spaces where ISO9000 was just the first of many, many steps.

But hey, if you're all about that process, guess what? You can report the test coverage bug through official channels, it will be triaged and prioritized based on the documented process, and it it meets the actionable threshold (ie it's the supplier responsibility to fix, as opposed to "new work" which may itself require further negotiations) it will be added to the development backlog. Eventually, it gets handed to a developer, and has to be QA'd and signed off by whatever else the process requires, and _eventually_ will land in some some future release.

So if you want that level of assurance and process from your suppliers? It's going to cost you... a _lot_. Not just in money, but time as well.

What's that, you have no formal contract that specifies the deliverables, compensation, and processes for reporting problems, plus the SLA for responses? Then they're not your supplier, and you have precisely *zero* legal (or moral) right to demand anything from them. Enjoy the "AS IS, NO WARRANTY WHATSOEVER" terms of the software you didn't pay for.

No safeguards?

jezuch — Tue, 29 Apr 2025 14:32:10 +0000

Bah, should've read the entire thread before answering :)

No safeguards?

jezuch — Tue, 29 Apr 2025 14:30:55 +0000

I think it's called mutation testing?

No safeguards?

pizza — Tue, 29 Apr 2025 14:20:12 +0000

> I think you're missing the point that that person cannot be trusted. If you've got any sense, you're going to blacklist him as a supplier ... especially if you're paying him in credibility!

How many times do I have to point out that this person is *NOT* your supplier?

You're going to get better resuts when starting with "hey, your test coverage is missing something, here's a patch that fills the gap" versus "I created a situation that resulted in a failure but the existing tests didn't catch it, and if you don't fix this coverage gap immediately you're clearly lying about everything and can't be trusted blablabla"

No safeguards?

Wol — Tue, 29 Apr 2025 08:29:24 +0000

> But even if it's true, so what? That person is not your supplier [1].

I think you're missing the point that that person cannot be trusted. If you've got any sense, you're going to blacklist him as a supplier ... especially if you're paying him in credibility!

"Honesty is the best policy" - I do my best to test my code, but I still regularly get bitten by quirks of the language, things I've forgotten, etc etc. And I try and make sure that my "customers" know what is tested and what isn't. The fact that half the time they don't listen, and the other half they don't understand, isn't my problem. Well it is, I have to fix the mess, but that's another incentive for me to try and get it right.

Cheers,
Wol

No safeguards?

marcH — Tue, 29 Apr 2025 04:31:55 +0000

> Seriously?

Clearly not:

> If you want ISO9000 compliance from me...

No safeguards?

pizza — Tue, 29 Apr 2025 03:02:58 +0000

> Take a look at this very short section of [ISO9000]

Seriously?

If you want ISO9000 compliance from me, you had damn well better be paying me.

If you're not, I repeat: I AM NOT YOUR SUPPLIER.

No safeguards?

marcH — Tue, 29 Apr 2025 01:17:38 +0000

> That's a nice sequence of what-ifs.

This is indeed a pretty specific and hypothetical path that we followed... _together_. Until now?

> But even if it's true, so what?

Then there are two possibilities:

1. The maintainer of that subsystem does not care and merges code anyway.
2. He cares and does not merge.

In _either_ case, everyone can draw very clear, evidence-based and useful conclusions about the quality of that subsystem.

The most important thing in quality is not the quality level itself. That level does matter of course, but what is even more important is not being ignorant and having some mere idea of where quality stand.

Take a look at this very short section
https://en.wikipedia.org/wiki/ISO_9000_family#ISO_9000_se...
It's all about processes, evidence and transparency. It's not concerned about defining what's "good" or "bad" quality, it's more about having some metrics in the first place - which unfortunately cannot be taken for granted.

When testing is "underrated" and mostly ignored in code reviews, that quality information is not even available, no one knows! Maybe the engineer who submits the code has been following some strict but private company QA process? Or maybe he just winged the whole thing due to unreasonable deadlines. Who knows - anyone with a bit of experience in this industry has already seen both. So, even a very basic "did you test this?" discussion already goes a long way.

Quality information is critical and actionable: it lets a company that sells some actual Linux-based product decide whether they should rewrite the Bluetooth subsystem or implement their own sound daemon versus stress-testing the existing one and participating upstream. Just some random examples; this sort of decisions happens all the time because open-source is "not a supplier".

Note an evidence-based, testing discussion is also (and in many cases: has been) very useful to reduce maintainer overload. FAIL | UNTESTED -> NACK. Done! Next (assuming that subsystem is interested in landing in some products)

No safeguards?

Paf — Tue, 29 Apr 2025 00:53:31 +0000

The merge process is “Linus hits a button on a git command”, how could it be gated on anything without a more complex infra?

No safeguards?

pizza — Tue, 29 Apr 2025 00:34:05 +0000

> If someone submitting changes is seriously asking someone ELSE to fix their _own_ lack of test coverage, right after being caught red-handed lying about said coverage, well now there is very simple, clear and compelling evidence that this person cannot be trusted with testing.

That's a nice sequence of what-ifs.

But even if it's true, so what? That person is not your supplier [1].

[1] https://www.softwaremaxims.com/blog/not-a-supplier

No safeguards?

marcH — Tue, 29 Apr 2025 00:02:59 +0000

If someone submitting changes is seriously asking someone ELSE
to fix their _own_ lack of test coverage, right after being caught red-handed lying about said coverage, well now there is very simple, clear and compelling evidence that this person cannot be trusted with testing.

That's pretty far from not wanting to perform "mutation testing" or some other fancy word that most people won't even bother googling.

No safeguards?

pizza — Mon, 28 Apr 2025 19:55:10 +0000

> On the other hand, if you tell them "Did you realize breaking this or that line passes the tests anyway?" after they proudly claimed to have tested their changes, then there is a small chance it will make some difference.

...or respond with "patches with improved tests welcome!"

No safeguards?

marcH — Mon, 28 Apr 2025 19:16:40 +0000

> > The simplest and most underrated technique is: manually _test the tests_ by temporarily breaking the product code. I keep being amazed at how few developers do or even know that technique.

> As stated in another reply, this is mutation testing.

Yes and no.

If you ask a developer who is not interested in finding problems "Could you please perform some mutation testing?" then guess what will happen: nothing at all (Mutation what?)

On the other hand, if you tell them "Did you realize breaking this or that line passes the tests anyway?" after they proudly claimed to have tested their changes, then there is a small chance it will make some difference.

Who knows; the younger ones who don't think they know everything yet might even feel like they just tasted something useful and "upgrade" to more advanced and extensive mutation testing in the longer term.

Baby steps!

No safeguards?

mathstuf — Mon, 28 Apr 2025 16:34:44 +0000

> The simplest and most underrated technique is: manually _test the tests_ by temporarily breaking the product code. I keep being amazed at how few developers do or even know that technique.

As stated in another reply, this is mutation testing. One of my ideas is to use code and mutation testing to discover "this code affects this test" relations so that one could take a diff and run just the tests that "care" about it. This would help with test turnaround time during patch review (while still doing full runs prior to merging). One of our bottlenecks in our regular CI is that testing always runs everything even for "obviously cannot be affected by the diff" changes. It'd be much nicer to cycle just the relevant tests to green before moving onto the full test suite.

No safeguards?

daroc — Mon, 28 Apr 2025 14:49:08 +0000

LWN covered one such tool for Rust code in October. I've tried it in some of my personal projects since then and found it somewhat useful for expanding my test suites.

Imagine: the future

mathstuf — Mon, 28 Apr 2025 12:16:34 +0000

And…? `strcat` is a function and not some operator, so it's not like NUL-terminated strings really "win" there either. IMO, removing direct writes via pointers to strings would be a further benefit.

GCC pre-releases and venerable traditions

mathstuf — Mon, 28 Apr 2025 12:10:56 +0000

FWIW, I was answering "what else", not giving suggestions for GCC :) .

In any case, I like it and GCC's more than the even/odd flipping that used to be way more prevalent. AFAIK, HDF5 is a holdout on that front, but I struggle to think of others I come across these days.

Orders of magnitude and exhaustive testing

farnz — Sun, 27 Apr 2025 18:54:50 +0000

The combination of property testing and order of magnitude thinking can help you find your way here, if you weren't already thinking about it.

Property testing randomly generates a test case, and confirms that a property holds (like your "must round trip from f32 to realistic::Real to f32 correctly"); there's libraries that can help with this, but it's also reasonable to write your own property-based tester if the problem space doesn't benefit from shrinking test inputs. Once you have a property tester, it's not hard to use it to find out how much of the input domain you can test in one second - you adjust the number of random inputs you generate until your test case takes about a second to run.

From there, you need to know that an hour is about three times 10³ seconds, and a day is about 80% of 10⁵ seconds, while your input space size is roughly 10^{number of bits * 3 / 10}; a 32 bit space is thus about 10^9.6 items (which you can round to 10¹⁰ - approximation is the name of the game here), and a 64 bit space is about 10^19.2 (which you can round to 10²⁰).

You then put the two together to work out how long your test would take if you stopped randomly generating test cases, and instead just went exhaustive; if your property test can test around 10⁶ items in a second, then a day's run will cover 10¹¹ possibilities. This is more than the number of possibilities in 32 bits, so you can exhaustively test the 32 bit space in a day. Similarly, if you can get your random tester to test around 10¹³ possibilities in a second on the available hardware, you know that a day will cover about 10¹⁸ tests, and so you need 10² (or 100) days to exhaustively test a 64 bit space (which, while slow, is fast enough that you might leave it running and pick up all the cases it finds for manual testing into the future, at least on significant releases).

And, of course, you can short-circuit this if, in your judgement, a single data item can be tested in few enough clock cycles; you know that 1 GHz is 10⁹ clock cycles per second, so if you judge that your test takes under 1,000 clock cycles per item, that's 10⁶ tests per second, which can exhaust a 32 bit space in a day.

No safeguards?

tialaramex — Sun, 27 Apr 2025 13:55:43 +0000

I like all of what's mentioned about testing, but I think it's worth mentioning one radical option that people need to have in the back of their heads when thinking about tests: Exhaustive testing.

256 seems like lots to us, and so we instinctively don't want want to try all 256 possible inputs to a function which takes a single byte. But 256 is nothing to a machine, so exhaustive tests are an effective choice here and might catch bugs in weird cases you hadn't considered.

Obviously you can't always do this, for a variety of reasons, and when you can it may be too slow and need to run overnight or something - but it's worth having the idea in your mind because when you can just try everything that's it, you're done, all inputs were tested.

I was testing my impl TryFrom<f32> for realistic::Real and impl From<Real> for f32 to check that they round trip when non-NaN and finite. I quickly discovered a one epsilon problem for some values, not a big deal given the relatively low precision of 32-bit floats but good to know and worth fixing - however a million into the exhaustive testing it found huge deviations because my previous "I know what to test" testing hadn't hit upon some important cases and the exhaustive testing had stumbled onto these - we're talking an order of magnitude size error like oops 0.07 isn't 0.0067

Imagine: the future

wahern — Sun, 27 Apr 2025 10:29:40 +0000

There were two competing proposals, Tom MacDonald's Variable Length Arrays (http://jclt.iecc.com/Jct13.pdf#page=67), and Ritchie's Variable-Size Arrays (http://jclt.iecc.com/Jct22.pdf#page=5). The committee eventually went with MacDonald's, apparently. Both proposals seemed to emphasize the utility for numerical computing a la Fortran, not for automatic bounds checking. Also, neither resolved the function parameter syntax problem--i.e. array parameter decay. In Ritchie's proposal, same as in MacDonald's, you were (syntactically) passed a pointer to an array, which required (syntactic) dereferencing. Ritchie's example prototype: foo(int (*a)[?][?]). MacDonald's (and VLAs since C99): foo(int n, int m, int (*a)[n][m]). In both cases sizeof (*a) worked as expected, but resolving at runtime.

Notably, Ritchie's proposal lacked variable size automatic storage arrays. With the VLA proposal, you can declare arrays on the stack: int a[n][m];. In Ritchie's proposal you were required to use malloc (or alloca?): int (*a)[?][?] = (int (*)[n][m])
malloc(n * m * sizeof (int)));. I don't think this was an intrinsic limitation; I think Ritchie just objected to the ambiguity of how and when sizeof evaluated the integral expression(s) used in the type definition (i.e. n and m above). MacDonald's paper mentions that the compiler would have to cache the evaluated value so it reflected the value of the type at it's declaration; subsequently modifying n or m wouldn't change the result of sizeof. Except when using VLAs in parameters, or with variable length structures (part of the original proposal), this wasn't quite true (or true but irrelevant), something fat pointers avoids.

It would have been better if we ended up with Ritchie's proposal. But I surmise (based on those papers alone) that MacDonald's won the day because 1) it was easier to implement--no ABI changes nor introduction of fat pointers into the compiler architecture; 2) GCC seemed to already have most of the implementation already, albeit with a slightly different syntax; 3) Ritchie gave short shrift to declaring automatic storage variable size arrays; 4) MacDonald was working at Cray so presumably was deemed to speak for pressing industry demands. #3 and #4 seem especially pivotal given everybody's preoccupation with numerical computing rather than bounds safety per se.

Attribute on type vs variable

jwakely — Sun, 27 Apr 2025 09:34:43 +0000

>Seems what's really wanted here is the ability to use string literal syntax, but without an implicit trailing NUL, obviating the need to warn about it being dropped

I agree that some special literal syntax to exclude the nul would be useful, but I think there's also value in marking the variable as "not a string". That could allow the compiler to warn it you pass it to strlen, strcpy etc.

That might not be very relevant to the kernel but seems useful in general.

But outside the kernel I would just use C++ which has better ways to do all this anyway (user-defined literals, string views, std::array, ...)

Imagine: the future

wahern — Sun, 27 Apr 2025 07:56:33 +0000

In Ritchie's proposal[1] the fat pointers weren't just a 2-tuple like Rust's str. They were an N-tuple, according to the dimension of the array. char[][] would require a 3-word fat pointer, which is actually more space efficient compared to, say, an array of Rust str's, while preserving the ability to cast between fat and non-fat multidimensional arrays without copying. Also, technically, Rust therefore doesn't actually have fat pointers in the sense of Ritchie's proposal.

I don't know why his approach was rejected. Perhaps it was the same reason Rust's fat-pointer-like data types aren't as general as what Ritchie proposed? But C99 ended up adopting the alternative approach he argued against--VLAs. And proposals for addressing some of the criticisms he pointed out seem poised for adoption, though they unfortunately missed the C23 window.

[1] See https://web.archive.org/web/20151226050349/https://www.be...

Attribute on type vs variable

wahern — Sun, 27 Apr 2025 06:55:08 +0000

> The __nonstring__ attribute applies to variables, not types, so it must be used in every place where a char array is used without trailing NUL bytes. He would rather annotate the type, indicating that every instance of that type holds bytes rather than a character string, and avoid the need to mark rather larger numbers of variable declarations. But that is not how the attribute works, so the kernel will have to include __nonstring markers for every char array that is used in that way.

I don't think it could work well as an attribute on the type. The relevant type is a char array, not just a char, nor (in this scenario) pointer-to-char. While you can typedef a char array, only to either an incomplete array type (typedef char foo[]) or an array of a specific size (typedef char foo[42]). I don't think there's a way to use the typedef while also being able to set the size at the variable declaration, as required by the example (cachefiles_charmap[64]). And you want to be able to define the size at the declaration, not the definition, in order to be able to use a string literal for initialization. You can use an initializer list ({ 'A', 'B', ...}), but that's not only more cumbersome, it entirely defeats the purpose as there would be no diagnostic to suppress in that case. You could in theory just set the attribute on the scalar typedef (type char foo) and make it work in the compiler, as Linus clearly had in mind, but it's ugly and arguably non-sensical--you're setting an attribute on a type that's only meaningful when the type is used derivatively. This kind of attribute exposes the gaps in C array semantics.

Seems what's really wanted here is the ability to use string literal syntax, but without an implicit trailing NUL, obviating the need to warn about it being dropped. But that's not the feature GCC has. :( This problem is kind of par for the course for alot of C extensions, especially ones involving arrays--leaky abstractions predicated on internal compiler semantics and conspicuously highlighting gaps in C array semantics.

No safeguards?

roc — Sun, 27 Apr 2025 01:51:47 +0000

This is called mutation testing. There are a lot of existing tools for it, some of which are C-syntax-aware. Also there are mutation-testing tools that work by patching binary code.

Fedora not stable distro.

shemminger — Sun, 27 Apr 2025 00:08:02 +0000

Agree, the point of being an upstream developer is to be a scout and clear the path.
It is much worse when some enterprise customer reports an issue when a new widget comes out.

No safeguards?

gmaxwell — Sun, 27 Apr 2025 00:03:52 +0000

I have a set of shell scripts that will go through source code and make varrious mutations one at a time such as change + to -, replace && with ||, replace 0 with ~0, change 1 to 2 or 2 to 1, swaps <, >, and =, inserts negations, blanks a line entirely, etc. Then it attempts to compile the code with optimizations. If it compiles, the script checks the sha256sum of the resulting binary against all the hashes it's seen before and if it's a new hash it runs the tests. If the tests pass the source is saved off for my manual inspection later.

The single point changes tend to not make errors which self cancel out, and usually if an error does cancel out or the change is in code that doesn't do anything the binary will not change. In code where tests have good condition/decision branch coverage most things this procedure catches are test omissions or bugs.

This approach is super slow and kludgy, I've been repeatedly surprised and frustrated that no one has made a C-syntax aware tool to do similar testing without wasting tons of time on stuff that won't compile or won't make a difference (e.g. mutating comments.. though sometimes I've addressed this by first running the code through something that removes all the comments).

But it's worked well enough for me and parsing C syntax is far enough away from the kind of programming I enjoy that I haven't bothered trying to close this gap myself.

LWN: Comments on "Some __nonstring__ turbulence"

GCC pre-releases and venerable traditions

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

Open source upstreams aren't suppliers

No safeguards?

Open source upstreams aren't suppliers

Open source upstreams aren't suppliers

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

No safeguards?

Imagine: the future

GCC pre-releases and venerable traditions

Orders of magnitude and exhaustive testing

No safeguards?

Imagine: the future

Attribute on type vs variable

Imagine: the future

Attribute on type vs variable

No safeguards?

Fedora not stable distro.

No safeguards?

LWN: Comments on "Some nonstring turbulence"