|
|
Log in / Subscribe / Register

My advice on implementing stuff in C:

My advice on implementing stuff in C:

Posted Oct 14, 2010 21:34 UTC (Thu) by butlerm (subscriber, #13312)
In reply to: My advice on implementing stuff in C: by HelloWorld
Parent article: Russell: On C Library Implementation

If you want your compiler / parser / runtime library / image / compression / encryption / numeric processing code to run at a competitive speed there aren't many alternatives. Virtually every other language is implemented in C and uses a number of runtime libraries written in C for a reason.

We would all be better off if that could all be done in a comparably efficient statically compiled language with some semblance of pointer safety of course. Not too many people developing those these days, unfortunately.


to post comments

My advice: implement stuff in D:

Posted Oct 14, 2010 22:49 UTC (Thu) by Ed_L. (guest, #24287) [Link]

Perhaps not too many people. But it only takes two, and the D community is by now far larger than that, including a gdc, a front-end for gcc. See The D Programming Language by Andrei Alexandrescu. D seems extremely well thought-out including "extern (C)" which (ahem) does exactly what one would hope, and expect...

My advice on implementing stuff in C:

Posted Oct 15, 2010 13:38 UTC (Fri) by HelloWorld (guest, #56129) [Link] (78 responses)

C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C. Templates is one, destructors (which facilitate resource management immensely) is another, and the list doesn't end there.
There are also numerous other languages, like Go, Rust or D, which are all comparable to C in terms of performance (and miles ahead in most other respects).

My advice on implementing stuff in C:

Posted Oct 15, 2010 14:00 UTC (Fri) by mjthayer (guest, #39183) [Link] (76 responses)

> C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C.

Perhaps it is just me, but C++ always seems to me like a poisoned chalice. It has lots of useful and powerful features, but they all seem to come with lots of fine print, so that by the time you have made them work the way you want, and debugged all the hidden edge cases, you need more time than you would have needed with C. Like -

C++ classes with their automatic destruction when they get out of range (if they are on the stack, which one quickly gets into the habit of overusing) - very useful, but you also have to get your head around exceptions and copy constructors which are firmly embedded in them.

Templates are also very powerful but have a tendency to start taking over all your programming time before you get them right.

Exceptions themselves would also be great, except that they are also horribly hard to do right, witness the number of long papers telling you that "exceptions are easy, just follow this methodology to use them".

Or typecast operators which are very hard to predict, especially as a compiler may decide to cast an object to an intermediary form (without telling you which) before casting it to its final incarnation.

Anyone feel like adding to the list?

My advice on implementing stuff in C:

Posted Oct 15, 2010 14:09 UTC (Fri) by dskoll (subscriber, #1630) [Link] (39 responses)

I agree with the comments about C++. It's a horrible, horrible language that doesn't know if it wants to be object-oriented or not. Other brokenness besides that mentioned in the parent:

Virtual Base Classes. A real WTF if ever there was one.

Insanely complex rules about how overloaded functions are picked, what you can and can't do in constructors/destructors, etc. that mostly come about because of compiler design constraints rather than intentional language design.

RTTI. Doesn't that go against OO design?

My advice on implementing stuff in C:

Posted Oct 15, 2010 15:16 UTC (Fri) by nix (subscriber, #2304) [Link]

I think I agree with michaeljt's comments more than yours. Most of the things you complain about are either very rare (virtual base classes) or provided by every other OO language (RTTI) *and* rare in C++ code.

The overloaded function resolution rules aren't all that bad... however, the name lookup rules all added together are fiendish: add 'export' and I can understand people's brains dribbling out of their ears.

My advice on implementing stuff in C:

Posted Oct 15, 2010 16:57 UTC (Fri) by HelloWorld (guest, #56129) [Link] (2 responses)

A language doesn't need to choose if it's object-oriented or not, it can support many different programming styles. I think that this is a good thing, and many other people seem to think so too. For example, Python is also a multi-paradigm language. It allows you to do imperative, object-oriented and functional programming, yet nobody seems to complain about it.

Also, I don't see your point about virtual inheritance. It was added to the language in order to solve the diamond inheritance problem, which occurs very rarely anyway, and it does solve that problem for me. If you absolutely can't stand the feature, then just avoid it by avoiding diamond inheritance.

My advice on implementing stuff in C:

Posted Oct 15, 2010 19:20 UTC (Fri) by mjthayer (guest, #39183) [Link] (1 responses)

> Also, I don't see your point about virtual inheritance. It was added to the language in order to solve the diamond inheritance problem, which occurs very rarely anyway, and it does solve that problem for me.

I'm wondering if I understood dskoll correctly, but pure virtual classes are actually one of the things in C++ that I find good without having to qualify. I find that inheriting behaviour often makes code harder to understand without a good reason.

I do like the way go handles this with its semi-implicit interfaces (from reading about it, not actually programming in it). But of course go is still too young and little used for people to have discovered its warts.

My advice on implementing stuff in C:

Posted Oct 18, 2010 0:10 UTC (Mon) by HelloWorld (guest, #56129) [Link]

I'm wondering if I understood dskoll correctly,
I don't think so. I think that dskoll was talking about virtual inheritance. Say, you have code such as this:
struct A { int a; };
struct D1 : A { int d1; };
struct D2 : A { int d2; };
struct B : D1, D2 { int b; };
Then, B will inherit A::a twice, once from D1 and once from D2. So, inside B, you'll actually have two fields named a, and if you want to use one of them, you always have to use full qualification, that is, you have to write D1::a or D2::a instead of just plain a. Virtual inheritance solves this problem. If you change the code as follows,
struct A { int a; };
struct D1 : virtual A { int d1; };
struct D2 : virtual A { int d2; };
struct B : D1, D2 { int b; };
then the two A sub-objects in B will be collapsed to a single one. It's also explained in the C++ FAQ lite: http://www.parashift.com/c++-faq-lite/multiple-inheritance.html

My advice on implementing stuff in C:

Posted Oct 15, 2010 17:19 UTC (Fri) by HelloWorld (guest, #56129) [Link] (33 responses)

Oh, and by the way, if you honestly think that rules such as "Don't call virtual functions in constructors or destructors. Don't throw exceptions in a destructor." are "insanely complex", then perhaps you shouldn't be programming at all (at least not low level programming, which is what C++ is about). There are very good reasons for these rules, and you probably wouldn't be complaining if you had understood them.

You do have a point about the name lookup rules, but at least some of those problems ultimately come from C, for example the "implicit int" rule from C89 assigns a meaning to some constructs that could otherwise be made illegal (and will be made illegal in the next C++ standard). Also the rule that a function declaration must come before the first call to the function comes from C. It's actually funny to see how almost every design mistake in C++ can be traced back to some kind of fuckup in C.

My advice on implementing stuff in C:

Posted Oct 15, 2010 17:28 UTC (Fri) by dskoll (subscriber, #1630) [Link] (18 responses)

There are very good reasons for these rules, and you probably wouldn't be complaining if you had understood them.

I understand the rules perfectly. They came about because Stroustrup et. al. threw everything + the kitchen sink into C++. Then when the dust settled, they discovered all kinds of corner-cases that needed clarification, or weird combinations that just don't work so need to be forbidden, or silly rules that came about because of how C++ compilers must be implemented.

I've programmed since 1982 (professionally since 1990) and used C and C++ extensively. While it's obvious that C was designed carefully and thoughtfully and the C standardization committees have done a stellar job, it's also obvious that C++ was a "yeah, throw in that feature!" design followed by "Oh crap... now we have to document the weird corner-cases."

My advice on implementing stuff in C:

Posted Oct 15, 2010 18:25 UTC (Fri) by HelloWorld (guest, #56129) [Link] (17 responses)

Oh, another victim of the C hacker syndrome. Please get well soon :)

My advice on implementing stuff in C:

Posted Oct 15, 2010 21:33 UTC (Fri) by dskoll (subscriber, #1630) [Link] (1 responses)

What did I write (in grandparent to this comment) that isn't factual?

My advice on implementing stuff in C:

Posted Oct 15, 2010 22:36 UTC (Fri) by HelloWorld (guest, #56129) [Link]

Err, like, everything? Aside from the statement that you've been programming since 1982, all of what you've written is not a fact but your personal opinion, and it's also so vague that it's basically impossible to disprove it, making any discussion pointless. The good thing is that I don't need to convince you, which is why this comment ends here :)

My advice on implementing stuff in C:

Posted Oct 15, 2010 23:21 UTC (Fri) by chad.netzer (subscriber, #4257) [Link] (9 responses)

Why does the author of this essay not sign his name to it? It took some hunting to find the presumed author.

In any case, C++ has had 25 years to convince the "C hackers" that it is the logical and valid step up from C. It should stand on its own merits, and not need a bunch of evangelists *still* trying to convince people from year to year. It isn't just stubborness; even after all this time, there are many valid reasons not to use it. I mean, if you set out to design a language from scratch, today, who in their right mind would come up with C++?

It would be much more convincing (IMO) to simply tell the C hackers to skip C++, and start using one of the languages that has learned the lessons of C++, and is trying to replace it. And in fact, many organizations have been essentially doing that exact thing throughout the lifetime of C++. It's quite telling...

My advice on implementing stuff in C:

Posted Oct 15, 2010 23:34 UTC (Fri) by dlang (guest, #313) [Link] (8 responses)

what language 'learned the lessons of C++' and is so clearly superior?

the fact that C++ is still being used so heavily (in spite of people saying for 25 years that it is junked and should be skipped and people instead use a 'real' language ;-) makes me believe that all of these next-generation languages are missing something.

companies keep trying to ignore C++, but they aren't succeeding so spectacularly that C++ is dieing away.

My advice on implementing stuff in C:

Posted Oct 16, 2010 1:25 UTC (Sat) by dskoll (subscriber, #1630) [Link] (6 responses)

what language 'learned the lessons of C++' and is so clearly superior?

C++ is "good enough" for many purposes, and it's certainly possible to write decent C++. The problem is that you have to restrict yourself to a subset of the language and have a very disciplined programming team with strict style guidelines.

Once you start going mad with esoteric C++ features (or even not-so-esoteric ones like templates), you code can become unreadable and unmaintainable.

Too bad Objective-C didn't catch on more than C++. I think it is a much better language if your goal is "C with objects".

My advice on implementing stuff in C:

Posted Oct 16, 2010 10:59 UTC (Sat) by marcH (subscriber, #57642) [Link]

> C++ is "good enough" for many purposes, and it's certainly possible to write decent C++. The problem is that you have to restrict yourself to a subset of the language and have a very disciplined programming team with strict style guidelines.

and the result of such discipline is called... Java.

OK, maybe too much discipline :-)

My advice on implementing stuff in C:

Posted Oct 16, 2010 13:15 UTC (Sat) by HelloWorld (guest, #56129) [Link] (3 responses)

If your code is unmaintainable, it's usually not because of language features, but because you have misused a language feature or you have chosen the wrong one for the problem.

My advice on implementing stuff in C:

Posted Oct 16, 2010 14:31 UTC (Sat) by Baylink (guest, #755) [Link]

And I believe that the assertion being made here by the anti-C++ faction is that *the fundamental design of the language and it's library/template environment is such* that this is much much harder than seems warranted, given the competition.

My advice on implementing stuff in C:

Posted Oct 16, 2010 19:50 UTC (Sat) by dskoll (subscriber, #1630) [Link] (1 responses)

If your code is unmaintainable, it's usually not because of language features, but because you have misused a language feature or you have chosen the wrong one for the problem.

What I assert is that there are many dangerous features in C++ that are easy to misuse. This is what I mean when I write that C++ is a horrible language; there are much better-designed languages that take a lot more effort to misuse. :) (C, Tcl, Lisp spring to mind immediately...)

My advice on implementing stuff in C:

Posted Oct 19, 2010 10:23 UTC (Tue) by nix (subscriber, #2304) [Link]

Tcl and Lisp are so flexible that they can be very easy to misuse in the wrong hands. This is mostly because of the very feature that gives them expressivity: the macro system (for Lisp) or ability to redefine every word in the language (for Tcl and for that matter Forth).

Subsets of C++

Posted Oct 17, 2010 4:53 UTC (Sun) by CChittleborough (subscriber, #60775) [Link]

dskoll is right: if you select an appropriate subset of C++, you can use it to write perfectly good code, even very large systems. Several teams have done this.

The problem is that the other teams using C++ are very unlikely to be using the same subset. Worse still, the team that wrote a library that your team would like to use probably selected a different subset than your team ...

My advice on implementing stuff in C:

Posted Oct 16, 2010 3:24 UTC (Sat) by chad.netzer (subscriber, #4257) [Link]

Note that the context is convincing diehard "C hackers", not necessarily C++ programmers, to migrate. Although, for the latter case, certainly Java is an example of a language directly intended to replace many of C++'s use cases, and both it and C# have been quite successful overall. C++ obviously has a long road ahead of it, and some exciting changes are on the horizon.

That said, of the languages meant to be "a better C++ for C hackers", several have been mentioned, and I don't claim any of will ever be "superior", in the sense of mindshare, marketshare, etc. Just that convincing those C hackers who have repeatedly objected to C++ (pardon the pun), is perhaps a fruitless battle. Personally, I'd kind of like "ooc" to become popularĀ…

As for the "lessons learned", its no surprise that many of the newer languages deliberately make an effort to ease the burden of implemention, speed of compilation, etc. Waiting for non-buggy implementations of C++'s newer features over the years left quite an impression on peopleĀ…

Ignorance at work

Posted Oct 16, 2010 11:07 UTC (Sat) by marcH (subscriber, #57642) [Link] (3 responses)

Oh, another victim of the C hacker syndrome.
I have read until the 5th line:
"[Linus'] opposition to any programming paradigms and concepts related to those paradigms which are not possible or very awkward to use in C. These include things like object-oriented design, abstraction, etc."
... which immediately helped me stop wasting my time.

Ignorance at work

Posted Oct 19, 2010 10:32 UTC (Tue) by nix (subscriber, #2304) [Link] (2 responses)

There's no object-oriented design or abstraction evident in the kernel until you look really deep into it, like the directory structure or the header files.

(oops)

Ignorance at work

Posted Oct 25, 2010 1:00 UTC (Mon) by vonbrand (guest, #4458) [Link] (1 responses)

Au contraire, it is very evident each time you take a peek a device drivers, filesystems, ...

It is in operating systems (and then simulation) where OOP was first used...

Ignorance at work

Posted Oct 25, 2010 6:54 UTC (Mon) by nix (subscriber, #2304) [Link]

Exactly my point. :)

My advice on implementing stuff in C:

Posted Oct 16, 2010 14:28 UTC (Sat) by Baylink (guest, #755) [Link]

Yes, I've read the first third of that page, and the C++ partisan is clearly purposefully failing to interpret Linus' words in a reasonable context, so as to have something to attack.

Nope, sorry.

My advice on implementing stuff in C:

Posted Oct 15, 2010 18:22 UTC (Fri) by chad.netzer (subscriber, #4257) [Link] (5 responses)

> then perhaps you shouldn't be programming at all

That suggestion seems overly condescending.

> (at least not low level programming, which is what C++ is about).

I assert that "low level programming" is *not* what C++ is about. Modern C++ style recommends that you use smart pointers, rather than C pointers, for example. Nor should you be using C strings, C arrays, C structures, C stdlib functions, C-like error handling, C macros, etc. Basically, modern C++ encourages using full high-level abstractions for data structures and algorithms, and is thus, fundamentally, high-level. And when all these new features and abstractions are used properly, it can be a beautiful, elegant thing IMO (that takes a lot of time and memory to compile). But it's not low-level.

The fact that many people still want to use C++ as only "a better C" (ie. no exceptions, multiple inheritance, namespaces, virtual functions, the stdlib, RTTI, or even templates and RAII), and thus *not* use the new features added in the last 15 years, is a direct consequence of many of those features being "insanely complex".

But the C++ FAQ, and C++ FQA make both ends of this argument in a more elegant fashion than I can:

http://www.parashift.com/c++-faq-lite/index.html
http://yosefk.com/c++fqa/

Note how rarely the FAQ mentions pointers, btw.

My advice on implementing stuff in C:

Posted Oct 15, 2010 19:40 UTC (Fri) by Ed_L. (guest, #24287) [Link] (1 responses)

"But its not low level."
To a certain extent its a circular argument. As others have observed, if you want to do system level (low level) programming on *nix, then you will ultimately end up calling libc, which libraries like glibmm admittedly do a wonderful job of wrapping. For the most part. But for that small part they don't, I've yet to find a substitute for just calling libc (or a syscall) directly. And for me that's one of the beautiful things about C++: its not dogmatic, and allows one to write grotty Fortran when nothing else will do.

:-)

My advice on implementing stuff in C:

Posted Oct 16, 2010 14:32 UTC (Sat) by Baylink (guest, #755) [Link]

> And for me that's one of the beautiful things about C++: its not dogmatic, and allows one to write grotty Fortran when nothing else will do.

How come that's not one of the Quotes of the Week?

My advice on implementing stuff in C:

Posted Oct 15, 2010 20:29 UTC (Fri) by HelloWorld (guest, #56129) [Link] (2 responses)

I assert that "low level programming" is *not* what C++ is about. Modern C++ style recommends that you use smart pointers, rather than C pointers, for example.

How does that make the language any less "low level"? A smart pointer just automates stuff you'd normally do by hand (i. e. free resources or decrement a reference counter). It's just as efficient

Nor should you be using C strings, C arrays, C structures, C stdlib functions, C-like error handling, C macros, etc. Basically, modern C++ encourages using full high-level abstractions for data structures and algorithms, and is thus, fundamentally, high-level.

The use of C arrays isn't discouraged because they're "low-level", but because there are better alternatives. std::tr1::array is just as low-level-ish as a C array, it does't do bounds checking or anything fancy, and it's just as efficient. The difference is that it offers the interface of an STL container, allowing you to use STL algorithms with it.

The same basically applies to C macros. The MAX(x,y) macro kind of works, but std::max(x,y) works better. It'll complain if x and y are not of the same type, and it won't evaluate its arguments more than once. std::max isn't somehow higher-level than MAX, it just sucks less.

Some things in C++ actually raise the level of abstraction, for example with std::string you don't have to worry about memory allocation any longer, since the class will do it for you when needed. If you can't afford that, nobody is going to blame you for not using it. C++ was deliberately designed not to force some style of programming on the user, be it a high or a low level one (unlike C, which forces you to program on a low level of abstraction all the time).

My advice on implementing stuff in C:

Posted Oct 15, 2010 22:49 UTC (Fri) by chad.netzer (subscriber, #4257) [Link] (1 responses)

> unlike C, which forces you to program on a low level of abstraction all the time

And so why did you claim above (while admonishing others) that: "low level programming [...] is what C++ is about"? My claim is that it is about much more than that. Agree?

My advice on implementing stuff in C:

Posted Oct 15, 2010 22:55 UTC (Fri) by HelloWorld (guest, #56129) [Link]

Yes, perhaps I should have made it more clear that C++ is also about low level programming.

My advice on implementing stuff in C:

Posted Oct 15, 2010 22:23 UTC (Fri) by wahern (subscriber, #37304) [Link] (4 responses)

the "implicit int" rule from C89 assigns a meaning to some constructs that could otherwise be made illegal (and will be made illegal in the next C++ standard). Also the rule that a function declaration must come before the first call to the function comes from C. It's actually funny to see how almost every design mistake in C++ can be traced back to some kind of fuckup in C.

Both of those were features at the time (and the latter at least arguably still). They made writing a compiler and linker significantly easier. Given that ease of implementation was evidently at the very, very, very bottom of C++'s list of priorities, you should blame C++, not C, if those were carried forward.

Compatibility is no excuse because C++ is not compatible with C at the source level. C++ people always seem to demur on this issue, arguing that they're only incompatible at the fringes. As a primarily C developer who occasionally has to muck around w/ C++, getting real C code to compile in "extern C" mode is a nightmare. In my experience, I prefer to view C++ and C as not compatible at all; this makes for fewer headaches. In practice compatibility really stems from shared ABIs, and languages like Go and D make little pretense about this reality.

I don't have any real gripes with C++. I choose not to use it for very idiosyncratic reasons; namely that it dropped implicit conversion of void pointers. I also eschew Java largely because it has no unsigned integers, and also because Java is incredibly unportable outside of Windows and Linux.

My advice on implementing stuff in C:

Posted Oct 16, 2010 21:40 UTC (Sat) by jzbiciak (guest, #5246) [Link] (3 responses)

Compatibility is no excuse because C++ is not compatible with C at the source level. C++ people always seem to demur on this issue, arguing that they're only incompatible at the fringes. As a primarily C developer who occasionally has to muck around w/ C++, getting real C code to compile in "extern C" mode is a nightmare. In my experience, I prefer to view C++ and C as not compatible at all; this makes for fewer headaches.

Hmmm... I haven't had too much trouble moving C code to C++. If your point is that the experience isn't edit-free, I'll give you that though. C++ is generally much pickier. My main issues have been that the C++ compiler is much more righteously indignant about const-abuse1, and it wants me to explicitly cast pointers to void * (which you also mentioned).

I never thought I'd get into C++ much, but I have totally gotten hooked on templates and the stricter type checking. Modern compilers do a fantastic amount of work at compile time, and I love bringing that force to bear on programming problems. I also actually like that I have to propagate const around more proactively: it exposes thinkos in my design earlier. And I especially like the new reinterpret_cast vs. static_cast vs. dynamic_cast vs. const_cast. It's like having a torque wrench, flat head screwdriver, Philips head screwdriver and a hammer, rather than just having a 20lb sledge.

All that said, I've written way more C than I have C++ and still find C my default go-to language when writing in a compiled language. And lately, I've been writing a lot more Perl. You won't catch me CamelCasing in C or C++, although I will name classes Like::This in Perl. When in Rome...

What I don't understand is all the language hate between C and C++. Save your ire for Python. ;-)

(Just kidding on the Python part!)


1 Yes, I know you can get the same with C code if you use a good compiler and crank up the compiler warnings. And believe me, I do crank them up.

My advice on implementing stuff in C:

Posted Oct 17, 2010 19:14 UTC (Sun) by wahern (subscriber, #37304) [Link] (2 responses)

C99 has diverged considerably from C89, the forking point of "extern C". The biggest headaches for me are named initializers and compound literals, both of which are used in headers and macros of newish C code.

While the different kinds of casts are nice in C++, casting is frowned upon in both C and C++. Bjarne says that he purposefully made casting in C++ ugly to dissuade people from casting, and that part of the design criteria of C++ was to reduce the need to cast. And yet in actual code I see casting as far more prevalent in C++ than in C, maybe because people see the feature and feel it was put there to use freely; I dunno. Much complexity was added to replace the loss of implicit void pointer conversions, and I'm not sure there was any net gain. In any event, it's a PITA at the boundary of C and C++

My advice on implementing stuff in C:

Posted Oct 17, 2010 19:50 UTC (Sun) by jzbiciak (guest, #5246) [Link]

I hear you on the missing named initializers. I had forgotten about that. I guess, other than using restrict generously, I haven't started using too many C99-specific features.

I didn't know about the compound literals.... nifty!

I don't use a lot of casting, personally, but where I do, I like the ability to specify what exactly I'm trying to accomplish. Where I use casting most is in embedded programming, where I need to cast between a pointer type and an unsigned int. That comes up a lot when talking to peripherals. reinterpret_cast makes it so much clearer what I'm trying to do, IMHO.

My advice on implementing stuff in C:

Posted Oct 17, 2010 23:47 UTC (Sun) by foom (subscriber, #14868) [Link]

> And yet in actual code I see casting as far more prevalent in C++ than in C

I suspect this is simply because you *can* see the casting in C++: a "static_cast<Whatever *>(x)" sticks out like a sore thumb, vs the almost-invisible C-style parenthesized type expression.

My advice on implementing stuff in C:

Posted Oct 19, 2010 9:54 UTC (Tue) by nix (subscriber, #2304) [Link] (2 responses)

What? Implicit int doesn't have anything to do with the complexity of name lookup. I was thinking of things like Koenig lookup, the effect of templates on name lookup in general, and what happened to it when 'export' came in. All the rules in isolation are sensible, but in combination it's fearsome.

My advice on implementing stuff in C:

Posted Oct 19, 2010 14:46 UTC (Tue) by HelloWorld (guest, #56129) [Link] (1 responses)

It does have to do with the name lookup rules in very non-obvious ways. I quote from "Design and Evolution of C++", page 141/142:
typedef int P();
typedef int Q();
class X {
  static P(Q); // define Q to be a P. 
               // equivalent to ''static int Q()''
               // the parentheses around Q are redundant

               // Q is no longer a type in this scope

  static Q(P); // define Q to be a function taking an argument of type P
               // and returning an int.
               // equivalent to ''static int Q(int());
};

Declaring two functions with the same name in the same scope is fine as long as their argument types differ sufficiently. Reverse the order of member declarations, and we define twi functions called P instead. Remove the typedef for either P or Q from the context, and we get yet other meanings.

This example ought to convince anybody that standards work is dangerous to your mental health. The rules we finally adopted makes[sic] this example undefined.

Note that this example -- like many others -- is based on the unfortunate ''implicit int'' rule inherited from C.

My advice on implementing stuff in C:

Posted Oct 19, 2010 15:32 UTC (Tue) by nix (subscriber, #2304) [Link]

Ah yes, I forgot that ingenious example. However, my point stands: this is not a name lookup problem, it is a particularly ingenious use of the parsing rules around implicit int to produce radically different parse trees from nearly identical input (and by no means the only example: see Alexandrescu's wonderful code in _Modern C++ Design_ to execute arbitrary code at compile time via abuse of sizeof().)

But, yes, this sort of example is probably an indictment of C++. Clarity in coding this is not!

My advice on implementing stuff in C:

Posted Oct 21, 2010 19:02 UTC (Thu) by ccurtis (guest, #49713) [Link]

Virtual Base Classes. A real WTF if ever there was one.

I've used a virtual base class to serialize access to a piece of hardware with software fallback. Allows for easy rate-limiting when the amount of queries might exceed the hardware precision and reading the device is (potentially) slow ...

My advice on implementing stuff in C:

Posted Oct 15, 2010 16:03 UTC (Fri) by Ed_L. (guest, #24287) [Link] (35 responses)

Its not just you, but for the sake of argument it may as well be :) If you feel you personally are a better, more productive programmer using C rather than C++, by all means use C. Aside from corner cases, its a subset :) :)

Me, I've been productive with C++ for over twenty years, and really like it. I'll grant there are more modern languages, but for HPC purposes I haven't found any more powerful, until I recently stumbled across D. (You know, the language C++ always wanted to be but was too rushed.) And that trip is too recent for me to draw a firm conclusion.

Some will ague that Java is just as good at HPC, and for them they are probably right. (Insert obligatory Fortran dereference here.) I also dabble in system programming, and just personally prefer one language that does it all. Others prefer to mix and match. And surely there must be places for Perl and its ilk -- provided they are kept brief and to the point.

"Although programmers dream of a small, simple languages, it seems when they wake up what they really want is more modelling power." -- Andrei Alexandrescu

My advice on implementing stuff in C:

Posted Oct 15, 2010 16:21 UTC (Fri) by mjthayer (guest, #39183) [Link] (34 responses)

> Its not just you, but for the sake of argument it may as well be :) If you feel you personally are a better, more productive programmer using C rather than C++, by all means use C.

I do now prefer to use C for that reason. But I still find C++ tantalisingly tempting, as it can do so many things that are just painful in C. I do know from experience though that it will come back to haunt me if I give in to the temptation. And I am experimenting to find ways to do those things more easily in C. The two that I miss most are automatic destruction of local objects (which is actually just a poor man's garbage collection) and STL containers.

Oh yes, add binary compatibility with other things to my list of complaints above; dvdeug's comment below is one example of the problem. That is something that has hurt me more often than I expected.

My advice on implementing stuff in C:

Posted Oct 15, 2010 20:25 UTC (Fri) by mpr22 (subscriber, #60784) [Link]

I gave up on C for recreational programming for one very simple reason: It is impossible to write vector arithmetic in a civilized syntax in C.

My advice on implementing stuff in C:

Posted Oct 16, 2010 10:18 UTC (Sat) by paulj (subscriber, #341) [Link] (32 responses)

Have you looked at Vala? Modern OOP language that builds on GLib and spits out C. Seems reasonably sane, certainly compared to C++...

My advice on implementing stuff in C:

Posted Oct 18, 2010 8:41 UTC (Mon) by marcH (subscriber, #57642) [Link] (4 responses)

> Have you looked at Vala? Modern OOP language that builds on GLib and spits out C.

Compiling to a lower-level yet still "human-writable" language is an interesting approach that can be successful to some extend. However it always has this major drawback: debugging & profiling becomes incredibly more difficult. It also gives a really hard time to fancy IDEs. All these need tight integration and the additional layer of indirection breaks that. So handing maintenance of average/poor quality code over to other developers becomes nearly impossible.

My advice on implementing stuff in C:

Posted Oct 18, 2010 9:09 UTC (Mon) by mjthayer (guest, #39183) [Link] (3 responses)

> Compiling to a lower-level yet still "human-writable" language is an interesting approach that can be successful to some extend. However it always has this major drawback: debugging & profiling becomes incredibly more difficult.

Without having looked at Vala, I don't see why this has to be the case. C itself is implemented as a pipeline, this would just add one stage onto the end. The main problem to solve that I can see is how to pass information down to the lower levels about what C code corresponds to which Vala code.

My advice on implementing stuff in C:

Posted Oct 18, 2010 9:28 UTC (Mon) by cladisch (✭ supporter ✭, #50193) [Link] (2 responses)

> The main problem to solve that I can see is how to pass information down to the lower levels about what C code corresponds to which Vala code.

C has the #line directive for that (GCC doc); AFAIK Vala generates it when in debug mode.

My advice on implementing stuff in C:

Posted Oct 18, 2010 10:58 UTC (Mon) by mjthayer (guest, #39183) [Link]

> C has the #line directive for that (GCC doc); AFAIK Vala generates it when in debug mode.
Sounds reasonable as long as they skip the pre-processor stage, otherwise things might get rather confused. I assume that their variables map one-to-one to C variables to simplify debugging.

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:23 UTC (Tue) by nix (subscriber, #2304) [Link]

I don't entirely understand why they don't generate it always. GCC's own preprocessor does. If you don't, even compiler error messages will be wrong, and you want them to be right even if you're not in debug mode.

My advice on implementing stuff in C:

Posted Oct 18, 2010 9:14 UTC (Mon) by mjthayer (guest, #39183) [Link] (26 responses)

> Have you looked at Vala? Modern OOP language that builds on GLib and spits out C.
I haven't looked at GLib that closely though. Is it used anywhere other than user space/desktop programming? If you are careful about what language features you use - and to disable exceptions! - C++ can be used very close to the bone (or iron or whatever).

My advice on implementing stuff in C:

Posted Oct 18, 2010 18:01 UTC (Mon) by paulj (subscriber, #341) [Link]

You can make Vala not use GLib if you wish, on a class by class basis by makring them "Compact". You lose some things, like the automatically refcounted classes, inheritance.

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:24 UTC (Tue) by nix (subscriber, #2304) [Link] (24 responses)

Yes, glib is used all over the place these days. syslog-ng and dbus aren't desktop programs by any means.

My advice on implementing stuff in C:

Posted Oct 19, 2010 15:19 UTC (Tue) by mjthayer (guest, #39183) [Link]

> Yes, glib is used all over the place these days. syslog-ng and dbus aren't desktop programs by any means.

They are still definitely user space though. If you are careful, C++ can be used for driver or even kernel code (e.g. the TU-Dresden implementation of the L4 micro-kernel with its unfortunate name was implemented in C++). Perhaps GLib would be too with a bit of work on it, I haven't used it enough to know.

My advice on implementing stuff in C:

Posted Oct 21, 2010 2:35 UTC (Thu) by wahern (subscriber, #37304) [Link] (22 responses)

Perfect. Core system daemons using a library that aborts on malloc() failure.

Geez.

This is why I never use Linux on multi-user systems.

My advice on implementing stuff in C:

Posted Oct 21, 2010 3:00 UTC (Thu) by foom (subscriber, #14868) [Link] (21 responses)

With the default settings on many distros, you're much more likely to just get a random process on your box forcibly killed when you run out of memory than for malloc to fail. So, there's really not much point in being able to gracefully handle malloc failure...

Just so long as pid 1 can deal with malloc failure, that's pretty much good enough: it can just respawn any other daemon that gets forcibly killed or aborts due to malloc failure.

My advice on implementing stuff in C:

Posted Oct 21, 2010 19:55 UTC (Thu) by nix (subscriber, #2304) [Link] (20 responses)

Quite so. Note that things like bash also abort on malloc() failure.

My advice on implementing stuff in C:

Posted Oct 21, 2010 20:15 UTC (Thu) by mjthayer (guest, #39183) [Link] (19 responses)

> Quite so. Note that things like bash also abort on malloc() failure.

Isn't that the FSF's standard recommendation (/requirement)? I find the thought amusing that if you subdivide your application well into different processes and make sure that you set atexit() functions for those resources that won't be freed by the system that isn't so far away from throwing an exception in C++.

My advice on implementing stuff in C:

Posted Oct 21, 2010 22:01 UTC (Thu) by nix (subscriber, #2304) [Link] (18 responses)

Yes, it is. It's really the only sensible thing to do. If you want to do something more complex than die on malloc() failure, do it in a parent monitor process: anything else is too likely to be unable to do whatever the recovery process is, because, well, you're still out of memory. (Bonus: overcommit and the OOM killer work fine with this model, as long as your monitor process is much smaller than the OOMing one, which is very likely. It's even more certain to work if the monitor oom_adj/oom_score's itself away from being OOM-killed.)

My advice on implementing stuff in C:

Posted Oct 22, 2010 21:42 UTC (Fri) by wahern (subscriber, #37304) [Link] (17 responses)

That's horrible design for a system, especially a server system. All of my daemons handle malloc failure. If I'm streaming a video feed to 2,000 clients and get a failure on the 2,001st (descriptor failure, malloc failure, any other failure), why would I destroy all 2,000 contexts when I can just fail one!? The practice is called graceful failure for a reason.

The first thing I do on any of my server systems is to disable overcommit. Even w/ it disabled I believe the kernel will still overcommit in some places (fork, perhaps), but at least I don't need to worry about some broken application 'causing some other critical service to be terminated.

If an engineer can't handle malloc failure how can he be expected to handle any other myriad possible failure modes? Handling malloc failure is hardly any more difficult, if at all, than handling other types of failures (disk full, descriptor limit, shared memory segment limit, thread limit, invalid input, etc, etc, etc). With proper design all those errors should share the same failure path; if you can't handle one you probably aren't handling any of them properly.

Plus, it's a security nightmare. If the 2,001st client can cause adverse results to the other 2,000 clients... that's a fundamentally broken design. Yes, there are other issues (bandwidth, etc), but those are problems to be addressed, not justifications for skirking responsibility.

And of course, on embedded system's memory (RAM and swap) isn't the virtually limitless resource as on desktops or servers.

Bailing on malloc is categorically wrong for any daemon, and most user-interactive applications. Bailing on malloc failure really only makes sense for batch jobs, where a process is doing one thing, and so exiting the process is equivalent to signaling inability to complete that particular job. Once you start juggling multiple jobs internally, bailing on malloc failure is a bug, plain and simple.

My advice on implementing stuff in C:

Posted Oct 22, 2010 22:18 UTC (Fri) by nix (subscriber, #2304) [Link] (14 responses)

Well, you still do need to worry about that. Not because of fork(): because of the stack. Unless your programs carefully start with a huge deep recursion to blow the stack out, you're risking an OOM kill every single time you make a function call. So you do need to deal with it anyway.

I don't know of any programs (other than certain network servers doing simple highly decoupled jobs, and sqlite, whose testing framework is astonishingly good) where malloc() failure is usefully handled. Even when they try, a memory allocation easily slips in there, and how often are those code paths tested? Oops, you die. From a brief inspection glibc has a number of places where it kills you on malloc() failure too (mostly due to trying to handle errors and failing), and a number of places where the error handling is there but is obviously leaky or leads to the internal state of things getting messed up. And if glibc can't get it right, who can? In practice this is not a problem because glibc also calls functions so can OOM-kill you just by doing that.

(And having one process doing only one job? That's called good design for the vast majority of Unix programs. Massive internal multithreading is a model you move to because you are *forced* to, and one consequence of it is indeed much worse consequences on malloc() failure.)

Even Apache calls malloc() here and there instead of using memory pools. Most of these handle errors by aborting (such as some MPM worker calls) or don't even check (pretty much all of the calls in the NT service-specific worker, but maybe NT malloc() never returns NULL, I dunno).

In an ideal world I would agree with you... but in practice handling all memory errors as gracefully as you suggest would result in our programs disappearing under a mass of almost-untestable massively bitrotten error-handling code. Better to isolate things into independently-failable units. (Not that anyone does that anyway, and with memory as cheap as it is now, I can't see anyone's handling of OOM improving in any non-safety-critical system for some time. Hell, I was at the local hospital a while back and their *MRI scanner* sprayed out-of-memory errors on the screen and needed restarting. Now *that* scared me...)

My advice on implementing stuff in C:

Posted Oct 23, 2010 1:14 UTC (Sat) by wahern (subscriber, #37304) [Link] (13 responses)

glib or glibc? Those are completely different libraries. If glibc is aborting on allocation error than it's non-conforming and it should be reported as a bug. There's a reason C and POSIX define ENOMEM.

As for the stack, the solution there is easy, don't recurse. Any recursive algorithm can be re-written as an iterative algorithm. Of course, if you use a language that optimizes tail-calls then you're already set. C doesn't, and therefore writing recursive algorithms is a bad idea, and it's why it's quite uncommon in C code.

As for testing error paths: if somebody isn't testing error paths than they're not testing error paths. What difference does it matter whether they're not testing malloc failure or they're not testing invalid input? It's poor design; it creates buggy code. And if you use good design habits, like RAII (not just a C++ pattern), then the places for malloc failure to occur are well isolated. It's not a very good argument to point out that most engineers write crappy code. We all know this; we all do it ourselves; but it's ridiculous to make excuses for it. If you can't handle the responsibility, then don't write applications in C or for its typical domain. If I'm writing non-critical or throw-away code, I'll use Perl or something else. Why invest the effort in using a language with features--explicit memory management--that I'm not going to use?

Using a per-process context design is in many circumstances a solid choice (not for me because I write HP embedded network server software, though I do prefer processes instead of threads for concurrency, so I might have 2 processes per cpu each handling hundreds of connections). But here's another problem w/ default Linux--because of overcommit, it's not always--perhaps not even often--that the offending process gets killed; it's the next guy paging in a small amount of memory that gets killed. It's retarded. It's a security problem. Can you imagine your SSH session getting OOMd because someone was fuzzing your website? It happens.

Why make excuses for poor design?

My advice on implementing stuff in C:

Posted Oct 23, 2010 3:18 UTC (Sat) by foom (subscriber, #14868) [Link]

> it's not always--perhaps not even often--that the offending process gets killed; it's the next guy paging in a small amount of memory that gets killed.

Actually, the OOM-killer tries *very* hard to not simply kill the next guy paging in a small amount of memory, but to determine what the real problem process is and kill that instead. It doesn't always find the correct culprit, but it often does, and at least it tends not to kill your ssh session.

My advice on implementing stuff in C:

Posted Oct 23, 2010 18:29 UTC (Sat) by paulj (subscriber, #341) [Link] (6 responses)

Why make excuses for poor design?

Nix isn't making excuses, he's pointing out reality. Which, sadly, is always far from perfect. A programme which is designed to cope with failure *despite* the suckiness of reality should do better than one that depends on perfection underneath it...

My advice on implementing stuff in C:

Posted Oct 23, 2010 19:39 UTC (Sat) by wahern (subscriber, #37304) [Link]

Robustness, like security, should be applied in-depth. Of course I use monitor processes and dead man switches to restart processes. But I don't rely on one to the exclusion of another.

My advice on implementing stuff in C:

Posted Oct 24, 2010 15:17 UTC (Sun) by nix (subscriber, #2304) [Link] (4 responses)

Indeed. It is simply reality that nobody ever tests malloc() failure paths -- at least, they do not and cannot test every combination of malloc-fails-and-then-it-doesn't, because there is an exponential explosion of them. People do not armour most programs, even important ones, to survive malloc() failure, because it would make the code unreadable and because available memory continues to shoot upwards so most people prefer to assume that reasonably sized allocations will not fail unless something is seriously wrong with the machine. And, guess what? They're right nearly all the time.

The suggestion to avoid stack-OOM by converting recursive algorithms to iterative ones is just another example of this, because while deep recursion is more likely to stack-OOM than the function calls involved in an iterative algorithm, the latter will still happen now and then. The only way to avoid *that* is to do a deep recursion first, and then ensure that you never call functions further down in the call stack than you have already allocated, neither in your code nor in any library you may call. I know of no tools to make this painful maintenance burden less painful. So nobody at all armours against this case, either.

I think it *is* important to trap malloc() failure so that you can *log which malloc() failed* before you die (and that means your logging functions *do* have to be malloc()-failure-proof: I normally do this by having them take their allocations out of a separate, pre-mmap()ed emergency pool). Obviously this doesn't work if you are stack-OOMed, nor if the OOM-killer zaps you. Note that this *is* an argument against memory overcommit: that overcommit makes it harder to detect which of many allocations in a program is buggy and running away allocating unlimited storage. But 'we want to recover from malloc() failure' is not a good reason to not use overcommmitment, because very few programs even try, and of those that try, most are surely lethally buggy in this area in any case: and fixing this is completely impractical.

Regarding my examples above: glib always aborts on malloc() failure, so so do all programs that use it. glibc does not, but its attempts to handle malloc() failure are buggy and leaky at best, and of course it (like everything else) remains vulnerable to stack- or CoW-OOM.

My advice on implementing stuff in C:

Posted Oct 25, 2010 10:05 UTC (Mon) by hppnq (guest, #14462) [Link] (3 responses)

The only way to avoid *that* [stack-OOM] is to do a deep recursion first, and then ensure that you never call functions further down in the call stack than you have already allocated, neither in your code nor in any library you may call.

You would have to know in advance how deep you can recurse, or you should be able to handle SIGSEGV. The maximum stack size can be tuned through rlimits, and that should solve wahern's problem of some other process draining out all available memory. This problem is not the result of bad programming, but of bad systems management.

(That said, rlimits are horribly broken. Just add more memory. ;-)

My advice on implementing stuff in C:

Posted Oct 25, 2010 22:28 UTC (Mon) by paulj (subscriber, #341) [Link] (2 responses)

FWIW, it's not defined what happens if you overflow the stack. You can't rely on getting a SEGV (isn't that a very recent addition to Linux, thanks to that Xorg security hole)?

My advice on implementing stuff in C:

Posted Oct 25, 2010 22:36 UTC (Mon) by nix (subscriber, #2304) [Link] (1 responses)

Even if you do get SIGSEGV from a stack-OOM, well, you'd better hope the system supports sigaltstack() as well, or you'll not be able to call the signal handler... oh, and, btw, it is (even now) easier to make a list of the systems on which sigaltstack() works properly than the systems on which it does not :(

My advice on implementing stuff in C:

Posted Oct 26, 2010 7:55 UTC (Tue) by hppnq (guest, #14462) [Link]

The point is, you can't safely expand the stack by recursing deeply in order to prevent running out of stack.

My advice on implementing stuff in C:

Posted Oct 25, 2010 11:04 UTC (Mon) by mjthayer (guest, #39183) [Link] (4 responses)

> As for the stack, the solution there is easy, don't recurse.

Just out of interest, are there really no simple ways (as nix suggested) to allocate a fixed-size stack at programme begin in Linux userland? I can't see any theoretical reasons why it should be a problem.

> And if you use good design habits, like RAII (not just a C++ pattern), then the places for malloc failure to occur are well isolated.

Again, I am interested in how you do RAII in C. I know the (in my opinion ugly and error-prone) goto way, and I could think of ways to do at run time what C++ does at compile time (doesn't have to be a bad thing, although more manual steps would be needed). Do you have any other insights?

My advice on implementing stuff in C:

Posted Oct 25, 2010 11:52 UTC (Mon) by hppnq (guest, #14462) [Link]

Just out of interest, are there really no simple ways (as nix suggested) to allocate a fixed-size stack at programme begin in Linux userland?

ld --stack or something similar?

My advice on implementing stuff in C:

Posted Oct 25, 2010 22:41 UTC (Mon) by nix (subscriber, #2304) [Link] (2 responses)

You do RAII in C by wrapping everything up in opaque structures allocated by dedicated allocators and freed either by dedicated freers or by APR-style pool destructors. If you're using mempools, you can even get close to the automagic destructor calls of C++ (you still have to free a mempool, but if you free the pool the free cascades down all contained pools and all their destructors.)

My advice on implementing stuff in C:

Posted Oct 26, 2010 8:06 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

> You do RAII in C by wrapping everything up in opaque structures allocated by dedicated allocators and freed either by dedicated freers or by APR-style pool destructors.

Right, roughly what I was thinking of. Thanks for the concrete pointers!

My advice on implementing stuff in C:

Posted Oct 26, 2010 8:18 UTC (Tue) by mjthayer (guest, #39183) [Link]

> Right, roughly what I was thinking of.

Except of course that there is no overriding need to use memory pools. You can also keep track of multiple allocations (possibly also with destructors) in some structure and free them all at one go when you are done. Freeing many allocations in one go rather than freeing each as soon as it is no longer needed might also be more efficient cache-wise.

My advice on implementing stuff in C:

Posted Oct 25, 2010 1:56 UTC (Mon) by vonbrand (guest, #4458) [Link]

No overcommit makes OOM kills much more likely (even in cases which would work fine otherwise). You've got your logic seriously backwards...

My advice on implementing stuff in C:

Posted Oct 25, 2010 16:10 UTC (Mon) by bronson (subscriber, #4806) [Link]

> Bailing on malloc is categorically wrong for any daemon, and most user-interactive applications. Bailing on malloc failure really only makes sense for batch jobs

OK, let's say your interactive application has just received a malloc failure. What should it do? Display an error dialog? Bzzt, that takes memory. Free up some buffers? There's good chance that any memory you free will just get sucked up by a rogue process and your next malloc attempt will fail too. And the next one. And the next one. And be careful with your error-handling code paths because, if you cause more data to get paged in from disk (say, a page of string constants that are only accessed in OOM conditions), you're now in even deeper trouble.

Bailing out is about the only thing ANY process can reliably do. If you try to do anything more imaginative, you are almost guaranteed to get it wrong and make things worse.

The days of cooperative multitasking and deterministic memory behavior are long gone (or, more accurately, restricted to a tiny sliver of embedded environments that no general purpose toolchain would ever consider a primary target). And good riddance! Programming is so much nicer these days that, even though this seems heinous, I'd never want to go back.

I can virtually guarantee you've never actually tested your apps in OOM situations or you would have discovered this for yourself. Try it! Once you fix all the bugs in your untested code, I think you'll be surprised at how few options you actually have.

My advice on implementing stuff in C:

Posted Oct 15, 2010 14:45 UTC (Fri) by dvdeug (guest, #10998) [Link]

The problem is, for a standard Unix library, you have to offer an interface usable from C, Python, Perl, Fortran, Ada, Java and the rest of the bunch. Which means if you're not C, you have to fake it, and that in particular means you can't use any feature that might not work if the main or the calling code is written in C. Templates are unusable in an interface that has to be called by <i>any</i> other language, and objects won't work with any non-OO language like C.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds