My advice on implementing stuff in C: [LWN.net]

My advice on implementing stuff in C:

Posted Oct 18, 2010 16:52 UTC (Mon) by HelloWorld (guest, #56129)
In reply to: My advice on implementing stuff in C: by cmccabe
Parent article: Russell: On C Library Implementation

C has a few implicit conversions that might be confusing to novices. That hardly qualifies it as "a mess."

C has value-destroying conversions (double -> float, int -> char, hell, even float -> int), it has cycles in the graph of conversions (int -> char, char -> int), and the void* -> T* conversion frequently leads to bugs. enums aren't really a type but a means to define integer constants. 'a' isn't a char literal but an integer literal (check it, sizeof 'a' == 4 on most compilers). If all this isn't a mess, what is? Even Bjarne Stroustrup called C's implicit conversions "rather chaotic" (Design & Evolution of C++, page 224).

In C and C++, functions and variables that are declared "file static" can't be referenced outside that file. If you combine that with a sane policy about how to split up functionality between multiple files, it does of the things that module systems do in other languages.

Actually, it's a rather lame workaround. You have to put header guards everywhere, which is just useless clutter. When a header file changes, you have to recompile every file that includes it, even if the change is totally irrelevant for some files, for example when you removed an unused function prototype or added a comment. If you forget to include a header file into its implementation file, the compiler won't check that the prototypes match the implementation. Also, the issue of name collisions isn't resolved that way. I learned about this when I worked on a project that typedefed its own fixed-size integer types (u64, u32 etc.), and one day they realized that some system header file also typedefed those names.

Any language that doesn't have eval() probably needs a macro system.

I'm not so sure about that, but let's just assume its true. Then why did they build such a crappy one for C? Proper macro systems, such as those from LISP, work on the AST level, while the C preprocessor works on the token level. The result of that is that you can't write #define square(x) x*x, you have to write #define square(x) ((x)*(x)). Also, the C preprocessor doesn't really know anything about the C language. You can't do #if sizeof (int) >= 4, and you can't do stuff like declare a function only if it's not declared already.

Look at what happened with Java. In their arrogance, the designers believed that they were beyond the need for a macro system. But the language wasn't expressive enough to describe a lot of the things that people needed to do. The result was that an ugly mass of automatic code generators got written to do the things that a macro system would have done. Now, in addition to learning the core language, you have to read huge tomes describing these automatic code generators. Usually they come with clunky XML interfaces and specialized tools.

The C preprocessor is just as extralinguistic as those code generators, the only difference is that it's called by the compiler driver, while other such tools aren't. Also, the C preprocessor clearly isn't powerful enough to replace code generators even for C. Just look at the numerous code generators in use every day, like flex, bison or ecpg, or those things that generate "config.h" for a C program. On the other hand, Java has annotations and annotation processors, which can replace many uses of code generators.

to post comments

My advice on implementing stuff in C:

Posted Oct 18, 2010 18:23 UTC (Mon) by cmccabe (guest, #60281) [Link] (6 responses)

> Actually, it's a rather lame workaround. You have to put header guards
> everywhere, which is just useless clutter. When a header file changes, you
> have to recompile every file that includes it, even if the change is
> totally irrelevant for some files, for example when you removed an unused
> function prototype or added a comment

Well, it sounds like you should be advocating Google Go then, rather than C++.

> You can't do #if sizeof (int) >= 4, and you can't do stuff like declare a
> function only if it's not declared already

Actually, you can. Check out static_assert in boost and BUILD_BUG_ON in the kernel.

> The C preprocessor is just as extralinguistic as those code generators,
> the only difference is that it's called by the compiler driver, while
> other such tools aren't.

This is a case where what seems like a small detail to academics is actually a huge deal for practical programmers. Having one preprocessor instead of hundreds really simplifies a programmer's life.

> Also, the C preprocessor clearly isn't powerful
> enough to replace code generators even for C. Just look at the numerous
> code generators in use every day, like flex, bison or ecpg, or those
> things that generate "config.h" for a C program. On the other hand, Java
> has annotations and annotation processors, which can replace many uses of
> code generators.

I haven't thought about it deeply, but I don't think that any macro preprocessor would be powerful enough to replace flex or bison. If you want to define your own domain-specific language without using code generators, just use a high-level language like Ruby.

Also, my understanding is that Java annotations are usually used in conjunction with code generators and static analyzers, rather than apart from them.

My advice on implementing stuff in C:

Posted Oct 18, 2010 21:15 UTC (Mon) by HelloWorld (guest, #56129) [Link] (5 responses)

> Well, it sounds like you should be advocating Google Go then, rather than C++.

I'm not advocating C++. If you read my comments carefully, you'll see that I said that C++ is not a beautiful language, and I also admitted that the name lookup rules are too complex. And of course, some of my criticisms of C (lack of a module system, the C preprocessor) apply to C++ as well; it also takes too long to compile.
I merely defend C++ when I feel it's being criticized unfairly, that doesn't mean I'm a fan of the language. I also mentioned other languages than C++ (specifically, Go, Rust and D) in my second comment; I just don't know them as well as C++.

> Actually, you can. Check out static_assert in boost and BUILD_BUG_ON in the kernel.

You're missing the point. The point is that the C preprocessor doesn't know anything about the C language. When you use BUILD_BUG_ON(x), it's not the preprocessor that complains when x is true, but the C compiler; the preprocessor is merely used to generate a C program fragment that is illegal if x is true. It's the same with BOOST_STATIC_ASSERT. There are plenty of other things that can't be done with the C preprocessor due to this, for example you can't check whether two arguments of a macro are of the same type.

> This is a case where what seems like a small detail to academics is actually a huge deal for practical programmers. Having one preprocessor instead of hundreds really simplifies a programmer's life.

Actually, the C preprocessor is a wonderful example of how having a sucky hack that is thought the be "good enough" by many people prevents progress. If the preprocessor hadn't been there, somebody would have invented proper facilities for generic programming and modules, which would have simplified programmer's lives much more than the preprocessor did.
Btw, I'm still not convinced that any language lacking eval needs a macro facility, could you elaborate on that?

> I haven't thought about it deeply, but I don't think that any macro preprocessor would be powerful enough to replace flex or bison.

Then go and learn about Lisp macros. With those, you have the full power of Lisp available to do source code transformations. Writing a Lisp macro that generates parsers isn't harder than writing a Lisp program that does.

> Also, my understanding is that Java annotations are usually used in conjunction with code generators and static analyzers, rather than apart from them.

I only recently learned about what annotations can do. Projekt Lombok uses them get rid of all those trivial functions for Java classes (getters, setters, hashCode etc.). This is clearly a case where code generators are actively being made redundant with annotations.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:01 UTC (Mon) by cmccabe (guest, #60281) [Link] (4 responses)

> There are plenty of other things that can't be done with the C
> preprocessor due to this, for example you can't check whether two
> arguments of a macro are of the same type

From kernel.h:

> /*
> * min()/max()/clamp() macros that also do
> * strict type-checking.. See the
> * "unnecessary" pointer comparison.
> */
> #define min(x, y) ({ \
> typeof(x) _min1 = (x); \
> typeof(y) _min2 = (y); \
> (void) (&_min1 == &_min2); \
> _min1 < _min2 ? _min1 : _min2; })

I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.

From the Project Lombok web page:

> Data is nice, but its certainly not the only boilerplate buster that
> lombok has to offer. If you need more fine grained control, there's
> @Getter and @Setter, and to help you in correctly cleaning up your
> resources, @Cleanup can automatically and without cluttering your source
> files generate try/finally blocks to safely call close() on your resource
> objects. That's not all, but for the complete list you'll need to head
> over to the feature overview

So in other words, it's just yet another code generator.

> Writing a Lisp macro that generates parsers isn't harder than writing a
> Lisp program that does.

And writing a domain-specific language in Ruby is easier still. Apparently high-level languages do high level things better than low-level languages do. Film at 11.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:43 UTC (Mon) by foom (subscriber, #14868) [Link]

> From kernel.h:
The linux kernel is not written in C: it is written in an extended GCC-proprietary C variant.

>> #define min(x, y) ({ \
({ is a GCC extension (statement-expression), it is not in C.

>> typeof(x) _min1 = (x); \
>> typeof(y) _min2 = (y); \
typeof is a GCC extension, it is not in C.

>> (void) (&_min1 == &_min2); \
Comparison of distinct pointer types is not an error in C.

>> _min1 < _min2 ? _min1 : _min2; })
>I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.
Just because you can do it in GCC, doesn't mean you can do it in C.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:45 UTC (Mon) by HelloWorld (guest, #56129) [Link] (2 responses)

> I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.

That code uses two GNU extensions, statement expressions and typeof. It's not possible to do this in C89 or C99.

Also, you're still missing the point. The point is that the preprocessor doesn't understand the language, which is a _fundamental_ shortcoming of that facility.

> So in other words, it's just yet another code generator.

Wake up.
The preprocessor, cpp, is "just yet another code generator", it generates C code from a (pretty dumb) macro language.
The C compiler, cc1, is "just yet another code generator", it generates assembly from C code.
The assembler, gas, is "just yet another code generator", it generates machine code from assembly.

Just because all this stuff is hidden from you by the compiler driver, gcc, doesn't mean it doesn't exist. However, unlike gcc's compiler driver, the java compiler allows you to hook up arbitrary stuff, like lombok. Being flexible is an advantage in my book.

> And writing a domain-specific language in Ruby is easier still. Apparently high-level languages do high level things better than low-level languages do.

You clearly have _no_ idea what Lisp is about. Here's a hint: it's not a low level language, and it's _at least_ as suitable for writing domain-specific languages as Ruby. Actually, it had most of what ruby has before ruby even existed.

My advice on implementing stuff in C:

Posted Oct 19, 2010 20:13 UTC (Tue) by cmccabe (guest, #60281) [Link] (1 responses)

My point isn't what you can do with ISO-standard C. My point is what you can do with C, period. Most good extensions to the language get rolled into the standard eventually.

Yes, I am aware that the C pre-processor is a code generator. My point is that if code generation is going to be required, it ought to be part of the language rather than separate.

> However, unlike gcc's compiler driver, the java compiler allows you to
> hook up arbitrary stuff, like lombok. Being flexible is an advantage in my
> book.

C/C++ has annotations too. Look up doxygen. It's not that hard to put an at-sign in a comment and run a macro generator over it later. The fact that Java has chosen to give this construct the pompous name of an "annotation" doesn't make it any better. Before annotations were added to Java, the code generators would match method names against regular expressions. For example, test_FOO would be used to generate a test of the FOO class.

> You clearly have _no_ idea what Lisp is about. Here's a hint: it's not a
> low level language, and it's _at least_ as suitable for writing
> domain-specific languages as Ruby. Actually, it had most of what ruby has
> before ruby even existed

What part of my statement makes you think I don't know what Lisp is? I'm well aware of Lisp's history as the first functional language.

I'm also aware of standard ML and OCaml. I have written compilers in these languages, and I'm well aware of their features.

You see, I have used many different languages, and I'm aware of what they're good at. Java works pretty well to write high-level apps on Android. Ruby works pretty well to write web applications. C works well in the kernel. And in some cases, C++ is the right choice (its best feature is templates.)

> I'm not advocating C++. If you read my comments carefully, you'll see that
> I said that C++ is not a beautiful language, and I also admitted that the
> name lookup rules are too complex. And of course, some of my criticisms of
> C (lack of a module system, the C preprocessor) apply to C++ as well; it
> also takes too long to compile.
> I merely defend C++ when I feel it's being criticized unfairly, that
> doesn't mean I'm a fan of the language. I also mentioned other languages
> than C++ (specifically, Go, Rust and D) in my second comment; I just don't
> know them as well as C++.

When I read back through this thread, I'm not at all sure what you're advocating. You start off by condemning C and casually commenting "C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C." Then, when people challenge you to explain these "huge benefits," you air a bunch of minor grievances about small things like implicit conversions in the type system and lack of features present in Standard ML (Seriously? We're comparing a low-level language to Standard ML?) Pretty much all of these gripes could equally well be raised against C++, because almost every valid C program is also a valid C++ program. You go on to raise a bunch of "but I can't do X" whines, which I then counter with "here is how you do X in C."

Probably the only valid point you raised at all is that generic programming would make a good addition to C. In fact, I think it's probably C++'s best feature. I suspect that a lot of people write very C-like code in C++ and use g++ just to make use of the STL.

I'm sorry that you feel so negatively about C. And really, I'm sorry that you feel so negatively about programming languages in general. If you could offer a constructive suggestion rather than "everything sucks," people might actually listen to you. Language extensions get made all the time. The truth is, though, a lot of the comments you have made come across as bikeshedding. Perhaps if you had been Kernigan or Ritchie, you would have put an "e" in the creat() function. Or changed C so that shorts did not get implicitly promoted to int. But you weren't, and they didn't, so we're just going to have to get on with our lives.

PS. I probably can't continue replying to this thread. Cheers.

My advice on implementing stuff in C:

Posted Oct 19, 2010 22:02 UTC (Tue) by HelloWorld (guest, #56129) [Link]

My point isn't what you can do with ISO-standard C. My point is what you can do with C, period.

Oh, so C isn't defined by a standard any more, but by what you think C is? I guess "you know it when you see it", right?

Yes, I am aware that the C pre-processor is a code generator. My point is that if code generation is going to be required, it ought to be part of the language rather than separate.

And my point is that cpp is pretty much the worst way to make it part of the language.

C/C++ has annotations too. Look up doxygen.

The Java equivalent to Doxygen is Javadoc. Annotations are something completely different; they are not (special) comments. And actually, C++0x will be supporting annotations, though they'll be called attributes there.

What part of my statement makes you think I don't know what Lisp is?

First, you didn't know that domain specific languages can be implemented with Lisp macros. Then you made the completely unfounded claim that Ruby is more suitable for implementing DSLs than Lisp. The following remark that "high-level languages do high level things better than low-level languages do." looked like a justification for that claim, and in the context it looks as if you're thinking of Lisp as a low-level language which it isn't.

When I read back through this thread, I'm not at all sure what you're advocating. You start off by condemning C and casually commenting "C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C." Then, when people challenge you to explain these "huge benefits," you air a bunch of minor grievances about small things like implicit conversions in the type system and lack of features present in Standard ML (Seriously? We're comparing a low-level language to Standard ML?)

I explicitly pointed out that some ideas from Standard ML, such as having strong typing, polymorphism and a module system, have nothing at all to do with whether it's a high level language or a low level one. Please, just read carefully

You go on to raise a bunch of "but I can't do X" whines, which I then counter with "here is how you do X in C."

All you've shown is that you're missing the point. The point is that cpp doesn't understand the C language. You then pointed out that it is possible to use cpp to generate a C program that fails to compile under certain circumstances. That is something completely and utterly different, and the fact that you still didn't understand it doesn't speak for you.

Probably the only valid point you raised at all is that generic programming would make a good addition to C.

Yeah right, because I didn't point out how con- and destructors (or RAII for short) massively simplifies resource management at the very beginning.

I'm sorry that you feel so negatively about C. And really, I'm sorry that you feel so negatively about programming languages in general. If you could offer a constructive suggestion rather than "everything sucks," people might actually listen to you.

I've pointed out what I think the worst problems with C are (sucky preprocessor, lack of a module system, lack of support for generic programming, lack of support for resource management), and I honestly think that these problems are so bad that nobody should be using C, except if no compiler for a better language is available for your target platform(s).
Also, contrary to your claim, I did point out alternatives: D, C++, Rust and Go, at the very beginning of this thread. I don't want to advocate any particular one, because I think that people should make up their own minds about what language they want. And also because I think that it'd obscure my point which is not "Language X is better than C", but "get rid of C if you can". If you don't like that -- tough.

Perhaps if you had been Kernigan or Ritchie, you would have put an "e" in the creat() function. Or changed C so that shorts did not get implicitly promoted to int. But you weren't, and they didn't, so we're just going to have to get on with our lives.

Unlike you, I'd rather try to fix mistakes from the past, rather than carrying them around forever and trying to cope with them forever.

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:40 UTC (Tue) by nix (subscriber, #2304) [Link]

The C preprocessor is just as extralinguistic as those code generators

While it seems so from cpp's quite remarkable lack of expressive power, it actually isn't possible to write a C preprocessor that is truly extralinguistic, because (as you yourself said), it operates on a token stream, not a character stream. It really is integrated into the language: the problem is that it's mostly (but not entirely) bolted onto the front end of it, as befits something that at one point was an extralinguistic addon that operated on character streams.