My advice on implementing stuff in C: [LWN.net]

My advice on implementing stuff in C:

Posted Oct 15, 2010 23:03 UTC (Fri) by HelloWorld (guest, #56129)
In reply to: My advice on implementing stuff in C: by cmccabe
Parent article: Russell: On C Library Implementation

If people could just accept that different programming languages are good for different things, 99% of these discussions would be over before they began.

I do accept that different languages are good at different things, I just happen to think that C is bad even at what it's supposed to be good at. There is no support for generic programming (read: type-safe container classes). The type system is generally a mess, as early C didn't have a cast operator and therefore needed all kinds of crazy implicit conversions. There is no module system. The preprocessor is a sucky workaround used to emulate features that ought to be in the language proper (genericity and modules), and it makes it much harder to build good tools for the C language.

None of the above conflict with the language being a systems programming language, so that's not an excuse for C sucking as hard as it does.

to post comments

My advice on implementing stuff in C:

Posted Oct 16, 2010 22:46 UTC (Sat) by marcH (subscriber, #57642) [Link] (1 responses)

The real excuse is age. To be fair with C and put it in perspective you have to compare it with languages just as old. This will not solve anything but explain a lot.

My advice on implementing stuff in C:

Posted Oct 17, 2010 22:20 UTC (Sun) by HelloWorld (guest, #56129) [Link]

OK, so C was invented in 1972, ML was invented in 1973. ML had a revolutionary type system (including polymorphism, type inference and algebraic data types), it had a powerful module system, and it didn't have a preprocessor because it didn't need one.
At least some of these innovations could have been copied by C without compromising the suitability for systems programming, for example the idea of strong typing and the module system; they just didn't do it.

Also, even if they hadn't known about the necessity of modules, generic algorithms and data structures etc., they could have added them later on when it became apparent; for example Ada had generic programming features in '83.

My advice on implementing stuff in C:

Posted Oct 18, 2010 5:10 UTC (Mon) by cmccabe (guest, #60281) [Link] (13 responses)

> The type system is generally a mess, as early C didn't have a cast
> operator and therefore needed all kinds of crazy implicit conversions

C has a few implicit conversions that might be confusing to novices. That hardly qualifies it as "a mess." I think C's type system is fine for low-level programming, where you're playing with bits and bytes.

> There is no module system.

In C and C++, functions and variables that are declared "file static" can't be referenced outside that file. If you combine that with a sane policy about how to split up functionality between multiple files, it does of the things that module systems do in other languages.

C also has a built-in way of loading code at runtime in the form of shared libraries. C even has the ability to run code when a shared library is loaded, and when it is unloaded.

As Rusty would no doubt say: modularity isn't a programming language feature. It's a programmer feature.

> The preprocessor is a sucky workaround used to emulate features that ought
> to be in the language proper (genericity and modules), and it makes it
> much harder to build good tools for the C language.

Any language that doesn't have eval() probably needs a macro system.

Look at what happened with Java. In their arrogance, the designers believed that they were beyond the need for a macro system. But the language wasn't expressive enough to describe a lot of the things that people needed to do. The result was that an ugly mass of automatic code generators got written to do the things that a macro system would have done. Now, in addition to learning the core language, you have to read huge tomes describing these automatic code generators. Usually they come with clunky XML interfaces and specialized tools.

> There is no support for generic programming (read: type-safe container
> classes).

Ok, I agree with you here. Generic programming support would be a big improvement. Templates are probably C++'s best feature.

My advice on implementing stuff in C:

Posted Oct 18, 2010 8:51 UTC (Mon) by marcH (subscriber, #57642) [Link] (2 responses)

> In C and C++, functions and variables that are declared "file static" can't be referenced outside that file. If you combine that with a sane policy about how to split up functionality between multiple files, it does of the things that module systems do in other languages.

Sorry but this is not good enough: it does not scale. Whereas most module systems allow nesting, "static" in C gives you only one level. And this puts severe constraints on how you split your project into files.

> Java designers believed that they were beyond the need for a macro system. The result was that an ugly mass of automatic code generators got written to do the things that a macro system would have done.

While I share the hate for these code generators, a macro system does not seem like the solution to me. Examples?

My advice on implementing stuff in C:

Posted Oct 18, 2010 19:17 UTC (Mon) by cmccabe (guest, #60281) [Link] (1 responses)

> Sorry but this is not good enough: it does not scale. Whereas most module
> systems allow nesting, "static" in C gives you only one level. And this
> puts severe constraints on how you split your project into files.

It's easy enough to split your project into multiple libraries. Then each library can do its own self-contained thing. Hopefully the graph of dependencies is a tree. I have programmed C in this style before. Build systems like CMake make it easy.

> While I share the hate for these code generators, a macro system does not
> seem like the solution to me. Examples?

Macros can be used for:

* Initializing structs in a very repetitive way.

* Generating short functions that do something trivial, like accessors. High level languages like ruby have :attr_accessor, but that relies on metaprogramming which isn't available in a low-level language like C.

* Generic programming

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:36 UTC (Tue) by nix (subscriber, #2304) [Link]

A better alternative for short functions that do something trivial is static inline functions. You can't make something a macro unless it's just shortening something the user can do anyway, and for *that* you generally want static inlines in a header file. A decent compiler will inline them for you if appropriate.

What macros really can do that nothing else can is anything involving stringization or token pasting, e.g. filling up a structure statically with information describing other C identifiers (fields in some other structure, that sort of thing). Thanks to stringization you can fill in the *name* of the identifier and its size without worrying about skew.

My advice on implementing stuff in C:

Posted Oct 18, 2010 16:52 UTC (Mon) by HelloWorld (guest, #56129) [Link] (8 responses)

C has a few implicit conversions that might be confusing to novices. That hardly qualifies it as "a mess."

C has value-destroying conversions (double -> float, int -> char, hell, even float -> int), it has cycles in the graph of conversions (int -> char, char -> int), and the void* -> T* conversion frequently leads to bugs. enums aren't really a type but a means to define integer constants. 'a' isn't a char literal but an integer literal (check it, sizeof 'a' == 4 on most compilers). If all this isn't a mess, what is? Even Bjarne Stroustrup called C's implicit conversions "rather chaotic" (Design & Evolution of C++, page 224).

In C and C++, functions and variables that are declared "file static" can't be referenced outside that file. If you combine that with a sane policy about how to split up functionality between multiple files, it does of the things that module systems do in other languages.

Actually, it's a rather lame workaround. You have to put header guards everywhere, which is just useless clutter. When a header file changes, you have to recompile every file that includes it, even if the change is totally irrelevant for some files, for example when you removed an unused function prototype or added a comment. If you forget to include a header file into its implementation file, the compiler won't check that the prototypes match the implementation. Also, the issue of name collisions isn't resolved that way. I learned about this when I worked on a project that typedefed its own fixed-size integer types (u64, u32 etc.), and one day they realized that some system header file also typedefed those names.

Any language that doesn't have eval() probably needs a macro system.

I'm not so sure about that, but let's just assume its true. Then why did they build such a crappy one for C? Proper macro systems, such as those from LISP, work on the AST level, while the C preprocessor works on the token level. The result of that is that you can't write #define square(x) x*x, you have to write #define square(x) ((x)*(x)). Also, the C preprocessor doesn't really know anything about the C language. You can't do #if sizeof (int) >= 4, and you can't do stuff like declare a function only if it's not declared already.

Look at what happened with Java. In their arrogance, the designers believed that they were beyond the need for a macro system. But the language wasn't expressive enough to describe a lot of the things that people needed to do. The result was that an ugly mass of automatic code generators got written to do the things that a macro system would have done. Now, in addition to learning the core language, you have to read huge tomes describing these automatic code generators. Usually they come with clunky XML interfaces and specialized tools.

The C preprocessor is just as extralinguistic as those code generators, the only difference is that it's called by the compiler driver, while other such tools aren't. Also, the C preprocessor clearly isn't powerful enough to replace code generators even for C. Just look at the numerous code generators in use every day, like flex, bison or ecpg, or those things that generate "config.h" for a C program. On the other hand, Java has annotations and annotation processors, which can replace many uses of code generators.

My advice on implementing stuff in C:

Posted Oct 18, 2010 18:23 UTC (Mon) by cmccabe (guest, #60281) [Link] (6 responses)

> Actually, it's a rather lame workaround. You have to put header guards
> everywhere, which is just useless clutter. When a header file changes, you
> have to recompile every file that includes it, even if the change is
> totally irrelevant for some files, for example when you removed an unused
> function prototype or added a comment

Well, it sounds like you should be advocating Google Go then, rather than C++.

> You can't do #if sizeof (int) >= 4, and you can't do stuff like declare a
> function only if it's not declared already

Actually, you can. Check out static_assert in boost and BUILD_BUG_ON in the kernel.

> The C preprocessor is just as extralinguistic as those code generators,
> the only difference is that it's called by the compiler driver, while
> other such tools aren't.

This is a case where what seems like a small detail to academics is actually a huge deal for practical programmers. Having one preprocessor instead of hundreds really simplifies a programmer's life.

> Also, the C preprocessor clearly isn't powerful
> enough to replace code generators even for C. Just look at the numerous
> code generators in use every day, like flex, bison or ecpg, or those
> things that generate "config.h" for a C program. On the other hand, Java
> has annotations and annotation processors, which can replace many uses of
> code generators.

I haven't thought about it deeply, but I don't think that any macro preprocessor would be powerful enough to replace flex or bison. If you want to define your own domain-specific language without using code generators, just use a high-level language like Ruby.

Also, my understanding is that Java annotations are usually used in conjunction with code generators and static analyzers, rather than apart from them.

My advice on implementing stuff in C:

Posted Oct 18, 2010 21:15 UTC (Mon) by HelloWorld (guest, #56129) [Link] (5 responses)

> Well, it sounds like you should be advocating Google Go then, rather than C++.

I'm not advocating C++. If you read my comments carefully, you'll see that I said that C++ is not a beautiful language, and I also admitted that the name lookup rules are too complex. And of course, some of my criticisms of C (lack of a module system, the C preprocessor) apply to C++ as well; it also takes too long to compile.
I merely defend C++ when I feel it's being criticized unfairly, that doesn't mean I'm a fan of the language. I also mentioned other languages than C++ (specifically, Go, Rust and D) in my second comment; I just don't know them as well as C++.

> Actually, you can. Check out static_assert in boost and BUILD_BUG_ON in the kernel.

You're missing the point. The point is that the C preprocessor doesn't know anything about the C language. When you use BUILD_BUG_ON(x), it's not the preprocessor that complains when x is true, but the C compiler; the preprocessor is merely used to generate a C program fragment that is illegal if x is true. It's the same with BOOST_STATIC_ASSERT. There are plenty of other things that can't be done with the C preprocessor due to this, for example you can't check whether two arguments of a macro are of the same type.

> This is a case where what seems like a small detail to academics is actually a huge deal for practical programmers. Having one preprocessor instead of hundreds really simplifies a programmer's life.

Actually, the C preprocessor is a wonderful example of how having a sucky hack that is thought the be "good enough" by many people prevents progress. If the preprocessor hadn't been there, somebody would have invented proper facilities for generic programming and modules, which would have simplified programmer's lives much more than the preprocessor did.
Btw, I'm still not convinced that any language lacking eval needs a macro facility, could you elaborate on that?

> I haven't thought about it deeply, but I don't think that any macro preprocessor would be powerful enough to replace flex or bison.

Then go and learn about Lisp macros. With those, you have the full power of Lisp available to do source code transformations. Writing a Lisp macro that generates parsers isn't harder than writing a Lisp program that does.

> Also, my understanding is that Java annotations are usually used in conjunction with code generators and static analyzers, rather than apart from them.

I only recently learned about what annotations can do. Projekt Lombok uses them get rid of all those trivial functions for Java classes (getters, setters, hashCode etc.). This is clearly a case where code generators are actively being made redundant with annotations.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:01 UTC (Mon) by cmccabe (guest, #60281) [Link] (4 responses)

> There are plenty of other things that can't be done with the C
> preprocessor due to this, for example you can't check whether two
> arguments of a macro are of the same type

From kernel.h:

> /*
> * min()/max()/clamp() macros that also do
> * strict type-checking.. See the
> * "unnecessary" pointer comparison.
> */
> #define min(x, y) ({ \
> typeof(x) _min1 = (x); \
> typeof(y) _min2 = (y); \
> (void) (&_min1 == &_min2); \
> _min1 < _min2 ? _min1 : _min2; })

I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.

From the Project Lombok web page:

> Data is nice, but its certainly not the only boilerplate buster that
> lombok has to offer. If you need more fine grained control, there's
> @Getter and @Setter, and to help you in correctly cleaning up your
> resources, @Cleanup can automatically and without cluttering your source
> files generate try/finally blocks to safely call close() on your resource
> objects. That's not all, but for the complete list you'll need to head
> over to the feature overview

So in other words, it's just yet another code generator.

> Writing a Lisp macro that generates parsers isn't harder than writing a
> Lisp program that does.

And writing a domain-specific language in Ruby is easier still. Apparently high-level languages do high level things better than low-level languages do. Film at 11.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:43 UTC (Mon) by foom (subscriber, #14868) [Link]

> From kernel.h:
The linux kernel is not written in C: it is written in an extended GCC-proprietary C variant.

>> #define min(x, y) ({ \
({ is a GCC extension (statement-expression), it is not in C.

>> typeof(x) _min1 = (x); \
>> typeof(y) _min2 = (y); \
typeof is a GCC extension, it is not in C.

>> (void) (&_min1 == &_min2); \
Comparison of distinct pointer types is not an error in C.

>> _min1 < _min2 ? _min1 : _min2; })
>I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.
Just because you can do it in GCC, doesn't mean you can do it in C.

My advice on implementing stuff in C:

Posted Oct 18, 2010 22:45 UTC (Mon) by HelloWorld (guest, #56129) [Link] (2 responses)

> I'm getting a little tired of being told what I can't do, especially when it turns out that I can do it after all.

That code uses two GNU extensions, statement expressions and typeof. It's not possible to do this in C89 or C99.

Also, you're still missing the point. The point is that the preprocessor doesn't understand the language, which is a _fundamental_ shortcoming of that facility.

> So in other words, it's just yet another code generator.

Wake up.
The preprocessor, cpp, is "just yet another code generator", it generates C code from a (pretty dumb) macro language.
The C compiler, cc1, is "just yet another code generator", it generates assembly from C code.
The assembler, gas, is "just yet another code generator", it generates machine code from assembly.

Just because all this stuff is hidden from you by the compiler driver, gcc, doesn't mean it doesn't exist. However, unlike gcc's compiler driver, the java compiler allows you to hook up arbitrary stuff, like lombok. Being flexible is an advantage in my book.

> And writing a domain-specific language in Ruby is easier still. Apparently high-level languages do high level things better than low-level languages do.

You clearly have _no_ idea what Lisp is about. Here's a hint: it's not a low level language, and it's _at least_ as suitable for writing domain-specific languages as Ruby. Actually, it had most of what ruby has before ruby even existed.

My advice on implementing stuff in C:

Posted Oct 19, 2010 20:13 UTC (Tue) by cmccabe (guest, #60281) [Link] (1 responses)

My point isn't what you can do with ISO-standard C. My point is what you can do with C, period. Most good extensions to the language get rolled into the standard eventually.

Yes, I am aware that the C pre-processor is a code generator. My point is that if code generation is going to be required, it ought to be part of the language rather than separate.

> However, unlike gcc's compiler driver, the java compiler allows you to
> hook up arbitrary stuff, like lombok. Being flexible is an advantage in my
> book.

C/C++ has annotations too. Look up doxygen. It's not that hard to put an at-sign in a comment and run a macro generator over it later. The fact that Java has chosen to give this construct the pompous name of an "annotation" doesn't make it any better. Before annotations were added to Java, the code generators would match method names against regular expressions. For example, test_FOO would be used to generate a test of the FOO class.

> You clearly have _no_ idea what Lisp is about. Here's a hint: it's not a
> low level language, and it's _at least_ as suitable for writing
> domain-specific languages as Ruby. Actually, it had most of what ruby has
> before ruby even existed

What part of my statement makes you think I don't know what Lisp is? I'm well aware of Lisp's history as the first functional language.

I'm also aware of standard ML and OCaml. I have written compilers in these languages, and I'm well aware of their features.

You see, I have used many different languages, and I'm aware of what they're good at. Java works pretty well to write high-level apps on Android. Ruby works pretty well to write web applications. C works well in the kernel. And in some cases, C++ is the right choice (its best feature is templates.)

> I'm not advocating C++. If you read my comments carefully, you'll see that
> I said that C++ is not a beautiful language, and I also admitted that the
> name lookup rules are too complex. And of course, some of my criticisms of
> C (lack of a module system, the C preprocessor) apply to C++ as well; it
> also takes too long to compile.
> I merely defend C++ when I feel it's being criticized unfairly, that
> doesn't mean I'm a fan of the language. I also mentioned other languages
> than C++ (specifically, Go, Rust and D) in my second comment; I just don't
> know them as well as C++.

When I read back through this thread, I'm not at all sure what you're advocating. You start off by condemning C and casually commenting "C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C." Then, when people challenge you to explain these "huge benefits," you air a bunch of minor grievances about small things like implicit conversions in the type system and lack of features present in Standard ML (Seriously? We're comparing a low-level language to Standard ML?) Pretty much all of these gripes could equally well be raised against C++, because almost every valid C program is also a valid C++ program. You go on to raise a bunch of "but I can't do X" whines, which I then counter with "here is how you do X in C."

Probably the only valid point you raised at all is that generic programming would make a good addition to C. In fact, I think it's probably C++'s best feature. I suspect that a lot of people write very C-like code in C++ and use g++ just to make use of the STL.

I'm sorry that you feel so negatively about C. And really, I'm sorry that you feel so negatively about programming languages in general. If you could offer a constructive suggestion rather than "everything sucks," people might actually listen to you. Language extensions get made all the time. The truth is, though, a lot of the comments you have made come across as bikeshedding. Perhaps if you had been Kernigan or Ritchie, you would have put an "e" in the creat() function. Or changed C so that shorts did not get implicitly promoted to int. But you weren't, and they didn't, so we're just going to have to get on with our lives.

PS. I probably can't continue replying to this thread. Cheers.

My advice on implementing stuff in C:

Posted Oct 19, 2010 22:02 UTC (Tue) by HelloWorld (guest, #56129) [Link]

My point isn't what you can do with ISO-standard C. My point is what you can do with C, period.

Oh, so C isn't defined by a standard any more, but by what you think C is? I guess "you know it when you see it", right?

Yes, I am aware that the C pre-processor is a code generator. My point is that if code generation is going to be required, it ought to be part of the language rather than separate.

And my point is that cpp is pretty much the worst way to make it part of the language.

C/C++ has annotations too. Look up doxygen.

The Java equivalent to Doxygen is Javadoc. Annotations are something completely different; they are not (special) comments. And actually, C++0x will be supporting annotations, though they'll be called attributes there.

What part of my statement makes you think I don't know what Lisp is?

First, you didn't know that domain specific languages can be implemented with Lisp macros. Then you made the completely unfounded claim that Ruby is more suitable for implementing DSLs than Lisp. The following remark that "high-level languages do high level things better than low-level languages do." looked like a justification for that claim, and in the context it looks as if you're thinking of Lisp as a low-level language which it isn't.

When I read back through this thread, I'm not at all sure what you're advocating. You start off by condemning C and casually commenting "C++ has been around for a long time, and while it's not exactly a beautiful language, it does provide huge benefits over C." Then, when people challenge you to explain these "huge benefits," you air a bunch of minor grievances about small things like implicit conversions in the type system and lack of features present in Standard ML (Seriously? We're comparing a low-level language to Standard ML?)

I explicitly pointed out that some ideas from Standard ML, such as having strong typing, polymorphism and a module system, have nothing at all to do with whether it's a high level language or a low level one. Please, just read carefully

You go on to raise a bunch of "but I can't do X" whines, which I then counter with "here is how you do X in C."

All you've shown is that you're missing the point. The point is that cpp doesn't understand the C language. You then pointed out that it is possible to use cpp to generate a C program that fails to compile under certain circumstances. That is something completely and utterly different, and the fact that you still didn't understand it doesn't speak for you.

Probably the only valid point you raised at all is that generic programming would make a good addition to C.

Yeah right, because I didn't point out how con- and destructors (or RAII for short) massively simplifies resource management at the very beginning.

I'm sorry that you feel so negatively about C. And really, I'm sorry that you feel so negatively about programming languages in general. If you could offer a constructive suggestion rather than "everything sucks," people might actually listen to you.

I've pointed out what I think the worst problems with C are (sucky preprocessor, lack of a module system, lack of support for generic programming, lack of support for resource management), and I honestly think that these problems are so bad that nobody should be using C, except if no compiler for a better language is available for your target platform(s).
Also, contrary to your claim, I did point out alternatives: D, C++, Rust and Go, at the very beginning of this thread. I don't want to advocate any particular one, because I think that people should make up their own minds about what language they want. And also because I think that it'd obscure my point which is not "Language X is better than C", but "get rid of C if you can". If you don't like that -- tough.

Perhaps if you had been Kernigan or Ritchie, you would have put an "e" in the creat() function. Or changed C so that shorts did not get implicitly promoted to int. But you weren't, and they didn't, so we're just going to have to get on with our lives.

Unlike you, I'd rather try to fix mistakes from the past, rather than carrying them around forever and trying to cope with them forever.

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:40 UTC (Tue) by nix (subscriber, #2304) [Link]

The C preprocessor is just as extralinguistic as those code generators

While it seems so from cpp's quite remarkable lack of expressive power, it actually isn't possible to write a C preprocessor that is truly extralinguistic, because (as you yourself said), it operates on a token stream, not a character stream. It really is integrated into the language: the problem is that it's mostly (but not entirely) bolted onto the front end of it, as befits something that at one point was an extralinguistic addon that operated on character streams.

My advice on implementing stuff in C:

Posted Oct 19, 2010 11:30 UTC (Tue) by nix (subscriber, #2304) [Link]

C also has a built-in way of loading code at runtime in the form of shared libraries. C even has the ability to run code when a shared library is loaded, and when it is unloaded.

No it doesn't. All C has is atexit(). The features you mention are POSIX, not C.