Undefined Behaviour as usual
Undefined Behaviour as usual
Posted Feb 24, 2024 1:03 UTC (Sat) by jwarnica (subscriber, #27492)In reply to: Undefined Behaviour as usual by mb
Parent article: Stenberg: DISPUTED, not REJECTED
Posted Feb 24, 2024 11:53 UTC (Sat)
by mb (subscriber, #50428)
[Link] (12 responses)
UB bugs must be documented as such and must get fixed right away.
Posted Feb 24, 2024 12:07 UTC (Sat)
by jwarnica (subscriber, #27492)
[Link] (11 responses)
DB is DB in the sepc. It is further defined (possibly incorrectly) in the implementation.
System v+1 might change both of these scenarios.
The only difference if v+1 changes your application is how righteous your bug report to the system is.
Posted Feb 24, 2024 12:33 UTC (Sat)
by mb (subscriber, #50428)
[Link] (1 responses)
Correct.
Also: The defined-behavior check is being mostly done by the release tests of the system components.
> UB is UB in the spec. It is defined in the implementation.
That is not true.
You are talking about Implementation-Defined and just assume that UB is the same thing.
Posted Mar 1, 2024 15:48 UTC (Fri)
by mathstuf (subscriber, #69389)
[Link]
The concrete behavior is non-deterministic, sure. However, the language spec can say things like "must treat the value as valid" even if its value is not what is expected (or even naïvely possible given the code). Instead, we have "if a data race happens, the bits in the memory must not be interpreted". The JVM has the former (you get *some* behavior of the code you wrote even if it's not what you expect). C has the latter (if the compiler notices a data race, it can instead assume it never happens and optimize accordingly; this may be "no behavior" after optimization passes complete). Note that C compilers don't stick in "if we notice UB at runtime, do_bad_thing()" code (unless requested via UBSAN and friends). Instead, their noticing of UB ends up affecting how optimizations treat the code which *then* transform it into unexpected logic flows based on the raw syntax of the code available.
In this sense, I think that overflowing on a command line argument parse is unlikely to have any real effect as the optimizer is just going to assume it stays in range and "means something". However, instead we have "the timeout parameter takes an integer, multiplication makes an integer…maybe it overflows, but who cares as 'it doesn't happen'". The code is the same whether overflow happens or not…it's just that the value is not interpreted as 1000*input but as 1000*input % UINT_MAX (or whatever type it is). Given the stratospheric values needed to overflow, I dare say that anyone flirting with these values already had a DoS on their plate in case the intended timeout actually ended up having to expire. It's only a real problem for those running with UBSAN on…but they're asking for DoS crashes by doing so anyways.
IMO, should UB be patched and guarded against? Yes. Does *this* instance warrant a CVE? No. `curl | sh` with bogus timeout parameters is more easily handled with replacing a URL with a download from malicious.site instead.
Posted Feb 27, 2024 11:21 UTC (Tue)
by danielthompson (subscriber, #97243)
[Link] (8 responses)
I think this confuses undefined behavior (e.g. reading past the end of an array) with implementation defined behavior (e.g. signess of char, size of int, etc). In some cases (such as reading past the end of an array whose length is not known at compile time) then compiler implementation cannot define what happens!
Signed integer overflow is interesting because it is an undefined behaviour (according to the C spec) and many implementations really do make it's behavior undefined. For example the behavior on overflow can and does change based on the optimization level. Nevertheless it is possible for an implementation to fully define what happens on overflow if the application is willing to accept missed optimization opportunities (hence -fwrapv).
It is also interesting because one of the common undefined behaviors linked to signed overflow is the removal of security bounds checks that (incorrectly) assumed integer overflow would wrap... and this certainly did lead to vulnerabilities.
However, despite the above, I'm unconvinced by the what-about-ism in the "every overflow is a ticking bomb" argument": the set of plausible optimizations on a time conversion during command line processing is relatively small.
Posted Feb 29, 2024 8:35 UTC (Thu)
by anton (subscriber, #25547)
[Link]
The removal of bounds checks is another such "optimization"; however, it is often based on the assumption that pointer arithmetic (not signed integer arithmetic) does not wrap around. Therefore, to avoid this kind of miscompilation, -fwrapv is insufficient. The better option is -fno-strict-overflow (which combines -fwrapv and -fwrapv-pointer).
Posted Feb 29, 2024 11:47 UTC (Thu)
by SLi (subscriber, #53131)
[Link] (6 responses)
While I'm all for making code robust and even moving to safer languages, I think most people are more interested in vulnerabilities in actual binaries running on actual computers than in ones where a theoretical, possibly evil compiler could read the spec and perform a theoretically valid compilation where the output does something bad.
Posted Feb 29, 2024 14:31 UTC (Thu)
by pizza (subscriber, #46)
[Link] (5 responses)
Yeah, that's my take on this too.
It's "undefined" in the spec, but the actual compiled binary (+runtime environment) exhibits highly consistent (albeit unexpected/unintended) behavior. After all, script kiddies couldn't exploit those bugs into a privilege escalation without the binary behaving in this implementation-specific manner.
Posted Feb 29, 2024 19:10 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (4 responses)
Am I right in thinking that you'd agree that it's the output binary that's usually deterministic in its environment (modulo things with clearly-defined non-determinism such as the RDRAND instruction, or runtime data races between multiple threads), and not the combination of compiler (including flags) and source code?
In other words, while this would be a nasty surprise, you wouldn't be surprised if recompiling the same UB-containing source resulted in a different binary, but you would be surprised if using the same binary in the same environment with the same inputs had a different output, unless it was doing something that is clearly specified to have non-deterministic behaviour by the hardware (like a data race, or RDRAND).
Posted Mar 1, 2024 1:37 UTC (Fri)
by pizza (subscriber, #46)
[Link] (3 responses)
Yes, I'd agree with this, and I don't think that it's a terribly controversial opinion.
> In other words, while this would be a nasty surprise, you wouldn't be surprised if recompiling the same UB-containing source resulted in a different binary
If you're recompiling with the same toolchain+options I'd expect the resulting binaries to behave damn near identically from one build to the next [1]. (Indeed, as software supply-chain attacks become more widespread, 100% binary reproducibility is a goal many folks are working towards)
> but you would be surprised if using the same binary in the same environment with the same inputs had a different output, unless it was doing something that is clearly specified to have non-deterministic behaviour by the hardware (like a data race, or RDRAND).
Yep. After all, the script kiddies wouldn't be able to do their thing unless a given binary on a given platform demonstrated pretty consistent behavior.
[1] The main difference would the linker possibly putting things in different places (especially if multiple build threads are involved) but that doesn't change the fundamental attack vector -- eg a buffer overflow that smashes your stack on one build (and/or platform) will still smash your stack on another, but since the binary layout is different, you'll likely need to adjust your attack payload to achieve the results you want. Similarly, data-leakage-type attacks (eg Heartbleed) usually rely on being able to repeat the attack with impunity until something "interesting" is eventually found.
Posted Mar 1, 2024 10:04 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (2 responses)
This is a potentially dangerous expectation in the presence of UB in the source code; there are optimizations that work by searching for a local maximum, and where for fully-defined code (even where it's unspecified behaviour, where there are multiple permissible outcomes), we know that there is only one maximum they can find. We use non-determinism in that search to speed it up, and for UB we run into the problem that there can be multiple maxima, all of which are locally the best option.
Because the search is non-deterministic, exactly which maximum we end up in for some UB cases is also non-deterministic. This does mean that 100% binary reproducibility has the nice side-effect of wanting to reduce UB - by removing UB, you make the search type optimizations find the one and only one optimal stable state every time, instead of choosing a different one each time).
And I'd agree that it's not terribly controversial to believe that a binary running in user mode has no UB - there's still non-deterministic behaviour (like the result of a data race between threads, or the output of RDRAND), and if your binary's behaviour is defined by something non-deterministic, it could end up in what the standards call unspecified behaviour. This is not universally true once you're in supervisor mode (since you can do things on some platforms like change power rails to be out-of-spec, which results in CPUs having UB since the logic no longer behaves digitally, and thus it's possible for software UB to turn into binary defined behaviour of changing platform state such that the platform's behaviour is now unpredictable).
Posted Mar 1, 2024 14:48 UTC (Fri)
by pizza (subscriber, #46)
[Link] (1 responses)
FWIW I've seen my fair share of shenanigans caused by nondeterminsitic compiler/linking behavior. To this day, there's one family of targets in the CI system that yields a final binary that varies by about 3KB from one build to the next depending on which nonidentical build host was used to cross-compile it. I suspect that is entirely due to the number of cores used in the highly parallel build; I've never seen any variance from back-to-back builds on the same machine (binaries are identical except for the intentionally-embedded buildinfo)
But I do understand what you're saying, and even agree -- but IME compilers are already very capable of loudly warning about the UB scenarios that can trigger what you described. Of course, folks are free to ignore/disable warnings, but I have no professional sympathy for them, or the consequences.
I've spent most of my career working in bare-metal/supervisory land, and yeah, even a off-by-one *read* could have some nasty consequences depending on which bus address that happens to hit. OTOH, while the behavior of that off-by-one read is truly unknowable from C's perspective, if the developer "corrects" the bug by incrementing the array size by one (therefore making it too large for the hw resource) the program is now "correct" from C's perspective, but will trigger the same nasty consequences on the actual hardware.
Posted Mar 1, 2024 16:34 UTC (Fri)
by farnz (subscriber, #17727)
[Link]
I spend a lot of time in the land of qualified compilers, where the compiler promises that as long as you stick to the subset of the language that's qualified, you can look only at the behaviours evident in the source code to determine what the binary will do. You're expected, if you're working in this land, to have proper code review and separate code audit processes so that a fix of the type you describe never makes it to production, since it's obvious from the source code that the program, while "correct" from C's perspective, is incorrect from a higher level perspective.
And a lot of the problems I see with the way UB is handled feel like people expect all compilers to behave like qualified compilers, not just on a subset of the language, but on everything, including UB.
Undefined Behaviour as usual
Because re-assessing known UB bugs is completely unnecessary, if they are understood as ticking time bombs that can trigger undefined behavior (including security issues) at any point in time if the ever so slightest bit of environment changes.
Rejecting a CVE is the opposite of that. It is not a responsible reaction to an UB bug. It is hiding your head in the sand.
Documenting and fixing the UB *solves* the problem once and for all. No re-assessment needed.
Undefined Behaviour as usual
Undefined Behaviour as usual
Upon system change: Having to check two things or having to check only one out of the two. What is better?
Data race UB is not defined in the implementation anywhere. It cannot be defined. It is non-deterministic. That's just one example.
It is not.
Undefined Behaviour as usual
Undefined Behaviour as usual
There is no way to "really make it's behaviour undefined", except by compiling to code where the architecture makes the behaviour undefined. Sane computer architects don't, and even insane architects don't do it for anything that a compiler is likely to output for signed addition. When reading past the end of an array, the code produced by the compiler usually just reads past the end of the array; it takes special effort by the compiler writer to do something different; but yes, compiler writers have taken this special effort, with the result being that there are cases of "optimizing" a loop with a read past the end of an array into an infinite loop.
Undefined Behaviour as usual
Undefined Behaviour as usual
Undefined Behaviour as usual
Undefined Behaviour as usual
Undefined Behaviour as usual
Undefined Behaviour as usual
If you're recompiling with the same toolchain+options I'd expect the resulting binaries to behave damn near identically from one build to the next [1]. (Indeed, as software supply-chain attacks become more widespread, 100% binary reproducibility is a goal many folks are working towards)
Undefined Behaviour as usual
> This is not universally true once you're in supervisor mode (since you can do things on some platforms like change power rails to be out-of-spec [...]
Undefined Behaviour as usual
OTOH, while the behavior of that off-by-one read is truly unknowable from C's perspective, if the developer "corrects" the bug by incrementing the array size by one (therefore making it too large for the hw resource) the program is now "correct" from C's perspective, but will trigger the same nasty consequences on the actual hardware.