LWN.net Logo

CERT C Secure Coding Standard: last call for reviewers

CERT C Secure Coding Standard: last call for reviewers

Posted Mar 16, 2008 20:19 UTC (Sun) by nix (subscriber, #2304)
In reply to: CERT C Secure Coding Standard: last call for reviewers by wahern
Parent article: CERT C Secure Coding Standard: last call for reviewers

`Gracefully' is an interesting question. In C, without the option of 
throwing exceptions and letting the unwinder handle deallocation, it's 
often very annoying to clean up after malloc()s. obstacks make it easier 
but make so many other things harder that it's often not worth it; 
Apache-style mempools are good as well... but often I arrange to partition 
the system into processes which leave notes in some shared communication 
medium, then simply die, such that some controlling process can detect 
when a process has OOMed or suffered some other transient failure, and can 
clean up and restart them appropriately. I'm not sure if 'just die' counts 
as 'graceful', though. (All this is probably easiest if the processes 
spend most of their time talking to transactioned things like databases, 
because the 'clean up' phase is trivial.)


(Log in to post comments)

CERT C Secure Coding Standard: last call for reviewers

Posted Mar 16, 2008 21:53 UTC (Sun) by wahern (subscriber, #37304) [Link]

There are certainly many different ways to handle this. Like many things, much of it comes
down to choice of data structures, code flow, etc. You learn to balance these things, and to
program in a way which minimizes the intrusion of error control in the logic.

Of course, many things in C are tedious--compared to so many scripting and so-called "managed"
languages--so "graceful" is relative. But those people who think its too tedious to handle it
shouldn't be using C. There are many other wonderful languages and application environments to
use, and I use them often myself. If someone thinks bailing on malloc() should be S.O.P., then
maybe they ought to use a Web Application paradigm, where operations--and failures--are
inherently isolated by whatever means the web server uses. There's nothing wrong with
leveraging those tools, and those authors' investments. That's what Free Software is all
about. Unfortunately, too often people want to get down and dirty and write "efficient" code
in C, but then complain when they have to manage the chores, too.

Of course, most of this goes for other languages, like Java, C#, Perl, etc when developing
applications of equivalent complexity; but I can't help it if this lackadaisical attitude has
already gripped those spheres. Exceptions and GC only go so far when a programmer is more
interested in doing the fun stuff, and couldn't care less about the tedious chores. People use
software to get things done; they want software writers' to make the _problems_ disappear
(i.e., to be handled automatically, not dropped on the floor). This is a contract which
shouldn't be broken lightly.

In my experience, though, handling malloc() is no different than handling a malformed HTTP
request, or any other exceptional condition that occurs when an application handles external
resources and data. More often than not, where one finds a function that "gracefully" handles
malloc() failure it also handles other errors. malloc() isn't any different than fopen()
failing, or read() failing, etc. No reason to treat it differently; there's no better way to
"fix" a read()--as alluded to by the OP--than malloc(). If a client disconnects abruptly, I
can't introspect his process stack to "fix" the problem. I just clean up and move along. This
is simply the general pattern of error handling, and it always requires releasing any
intermediate resources. (I've written allocator pools and whatnot, but after a few years I
realized it doesn't actually help that much in terms of effort; if anything it's better just
as a performance boost for small string allocations, or for finer-grained memory limits and
accounting. (See libarena)). 

Granted, there are some failures that can't really be handled well, like a stack overrun
(though I've seen ingenious code which goes pretty far). But malloc() returns NULL for a
reason, and its not because the sanest thing to do is merely exit. Its no excuse, however, to
say that, because C is helpless in some respects to memory management, its futile to manage
all memory management.

CERT C Secure Coding Standard: last call for reviewers

Posted Mar 16, 2008 22:43 UTC (Sun) by nix (subscriber, #2304) [Link]

There is a reason why malloc() should be treated differently from, say, 
fopen() failing. A failing fopen() is unlikely to be transient and 
unlikely to be something that can be fixed without human intervention, nor 
is it likely to make other unrelated things fail (other fopen()s will 
probably work, as will, say, fwrite()). malloc() has neither of these 
properties.

CERT C Secure Coding Standard: last call for reviewers

Posted Mar 17, 2008 1:31 UTC (Mon) by wahern (subscriber, #37304) [Link]

What do you mean by "unlikely"? If you mean to say that the scenarios where one fails and the
other doesn't, then of course there are differences. But, I fail to see how that means they
should be treated different when they fail.

Noting this distinction is part of the original faulty logic, which is that a prerequisite
condition of managing malloc() failure is that the why's and how's need be somehow discernible
by the application, otherwise its a futile endeavour. Ostensibly, this is so the application
can "fix" them, or alter its behavior according to the specific reasons. I don't... it's hard
to discern the motivations or intentions in that line of argument. I can't see how the
requirement of "keep running" is conditioned on whether MySQL has a memory leak, because a
user just loaded a 200MB tiff image in Firefox, or because the /tmp partition filled up.

A robust application, by definition, should handle any _potentially_ transient failure
regardless of the underlying cause, generally speaking (there are myriad exceptions, of
course). A robust application can and should strive to remain internally consistent and
stable. Arguing that handling malloc() different from fopen() because one condition holds true
for malloc() 90% of the time, but only 10% of the time for fopen(), makes absolutely no sense
in a general context. And general context is what we're talking about: the proper practice for
handling malloc() failure.

What might be 10% or 90% generally might be 1% or 100% for any particular user or environment.
Different users experiencing wildly different behavior from software is one of the stains on
the industry, and its specifically because of these dependencies on unwarranted assumptions
and distinctions. Bugs will happen, and reasonable people can argue about what's easy or
difficult, but its ludicrous when the intentional practice is defended.

Once an application has hit a steady state (if it's that sort of application, like a desktop
application, server daemon, or even a really long lived data processor, etc), malloc() failure
should never cause it to exit. Even if it has a watchdog to restart it, these are the types of
applications which juggle multiple tasks, and which the unnecessary loss of state creates
gigantic headaches; they demand internal isolation of all reasonable errors. And arguing that
handling a malloc() failure is unreasonable doesn't hold water, particularly after the
application has already enough committed resources to subsist. This almost never requires
keeping "emergency" memory buffers or any other tricks; I've never had to resort to such
things. It's the proverbial case of throwing the baby out with the bath water.

Imagine, for instance (with the infamous exception of glib) libraries exiting like this? If
OpenSSL crashed your application because it couldn't initialize an internal resource?  OpenSSL
is an incredibly complex piece of software; probably too complex, with lots of cruft. But they
manage to do this "impossible" task, and don't resort to making distinctions between this
resource or that resource. It propagates the condition to the appropriate level, which is the
portion of code which decides whether its remainder can proceed with the failure of the
_logical_ task, and (similar to the issue above), not based on why, specifically, it failed.
And because of the miracle of componentization and abstraction, the problem of handling
malloc() failure becomes identical to handling any of the other potential failure conditions.
And unless you can make a case in a particular instance for bailing on all such errors, why
bail on only some of them?



CERT C Secure Coding Standard: last call for reviewers

Posted Mar 20, 2008 15:13 UTC (Thu) by paulj (subscriber, #341) [Link]

The simplest and most reliable error-strategy often is to restart, surely? Optimising restart
has benefits for more than just error-handling too.

The obvious example would be malloc() failing because the application itself is leaking
memory. No amount of retrying malloc() will help there. Restarting will however at least
mitigate the problem, allowing the user to continue to use the system, until the software can
be fixed to address the leak.

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds