|
|
Log in / Subscribe / Register

My advice on implementing stuff in C:

My advice on implementing stuff in C:

Posted Oct 23, 2010 18:29 UTC (Sat) by paulj (subscriber, #341)
In reply to: My advice on implementing stuff in C: by wahern
Parent article: Russell: On C Library Implementation

Why make excuses for poor design?

Nix isn't making excuses, he's pointing out reality. Which, sadly, is always far from perfect. A programme which is designed to cope with failure *despite* the suckiness of reality should do better than one that depends on perfection underneath it...


to post comments

My advice on implementing stuff in C:

Posted Oct 23, 2010 19:39 UTC (Sat) by wahern (subscriber, #37304) [Link]

Robustness, like security, should be applied in-depth. Of course I use monitor processes and dead man switches to restart processes. But I don't rely on one to the exclusion of another.

My advice on implementing stuff in C:

Posted Oct 24, 2010 15:17 UTC (Sun) by nix (subscriber, #2304) [Link] (4 responses)

Indeed. It is simply reality that nobody ever tests malloc() failure paths -- at least, they do not and cannot test every combination of malloc-fails-and-then-it-doesn't, because there is an exponential explosion of them. People do not armour most programs, even important ones, to survive malloc() failure, because it would make the code unreadable and because available memory continues to shoot upwards so most people prefer to assume that reasonably sized allocations will not fail unless something is seriously wrong with the machine. And, guess what? They're right nearly all the time.

The suggestion to avoid stack-OOM by converting recursive algorithms to iterative ones is just another example of this, because while deep recursion is more likely to stack-OOM than the function calls involved in an iterative algorithm, the latter will still happen now and then. The only way to avoid *that* is to do a deep recursion first, and then ensure that you never call functions further down in the call stack than you have already allocated, neither in your code nor in any library you may call. I know of no tools to make this painful maintenance burden less painful. So nobody at all armours against this case, either.

I think it *is* important to trap malloc() failure so that you can *log which malloc() failed* before you die (and that means your logging functions *do* have to be malloc()-failure-proof: I normally do this by having them take their allocations out of a separate, pre-mmap()ed emergency pool). Obviously this doesn't work if you are stack-OOMed, nor if the OOM-killer zaps you. Note that this *is* an argument against memory overcommit: that overcommit makes it harder to detect which of many allocations in a program is buggy and running away allocating unlimited storage. But 'we want to recover from malloc() failure' is not a good reason to not use overcommmitment, because very few programs even try, and of those that try, most are surely lethally buggy in this area in any case: and fixing this is completely impractical.

Regarding my examples above: glib always aborts on malloc() failure, so so do all programs that use it. glibc does not, but its attempts to handle malloc() failure are buggy and leaky at best, and of course it (like everything else) remains vulnerable to stack- or CoW-OOM.

My advice on implementing stuff in C:

Posted Oct 25, 2010 10:05 UTC (Mon) by hppnq (guest, #14462) [Link] (3 responses)

The only way to avoid *that* [stack-OOM] is to do a deep recursion first, and then ensure that you never call functions further down in the call stack than you have already allocated, neither in your code nor in any library you may call.

You would have to know in advance how deep you can recurse, or you should be able to handle SIGSEGV. The maximum stack size can be tuned through rlimits, and that should solve wahern's problem of some other process draining out all available memory. This problem is not the result of bad programming, but of bad systems management.

(That said, rlimits are horribly broken. Just add more memory. ;-)

My advice on implementing stuff in C:

Posted Oct 25, 2010 22:28 UTC (Mon) by paulj (subscriber, #341) [Link] (2 responses)

FWIW, it's not defined what happens if you overflow the stack. You can't rely on getting a SEGV (isn't that a very recent addition to Linux, thanks to that Xorg security hole)?

My advice on implementing stuff in C:

Posted Oct 25, 2010 22:36 UTC (Mon) by nix (subscriber, #2304) [Link] (1 responses)

Even if you do get SIGSEGV from a stack-OOM, well, you'd better hope the system supports sigaltstack() as well, or you'll not be able to call the signal handler... oh, and, btw, it is (even now) easier to make a list of the systems on which sigaltstack() works properly than the systems on which it does not :(

My advice on implementing stuff in C:

Posted Oct 26, 2010 7:55 UTC (Tue) by hppnq (guest, #14462) [Link]

The point is, you can't safely expand the stack by recursing deeply in order to prevent running out of stack.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds