Apache resigns from the Java Community Process executive committee [LWN.net]

Apache resigns from the Java Community Process executive committee

Posted Dec 14, 2010 0:29 UTC (Tue) by HelloWorld (guest, #56129)
In reply to: Apache resigns from the Java Community Process executive committee by cmccabe
Parent article: Apache resigns from the Java Community Process executive committee

Yes, the problem with this approach is documented here:
http://www.boost.org/doc/libs/1_36_0/libs/smart_ptr/share...

I agree that this is a very nasty problem, and I'm not sure I would have spotted it in a code review.

What makes it worse is that there is no good way to avoid this problem. One could make all constructors private and create objects through static member functions which immediately wrap every new-ed pointer in a shared_ptr. But that requires boilerplate code, and a reusable solution can't be done well in C++98 (because of the perfect forwarding problem and because there are no variadic templates). It's really pretty bad, thanks for pointing that out.

to post comments

Apache resigns from the Java Community Process executive committee

Posted Dec 14, 2010 5:05 UTC (Tue) by cmccabe (guest, #60281) [Link] (22 responses)

There is only one way to avoid the problem: experience. You must not allow people to use anonymous shared_ptr objects; it's too dangerous.

Here's another traditional memory leak, which I don't think you'll need teh Google to spot:

> #include <iostream>
> #include <tr1/memory>
>
> using std::tr1::shared_ptr;
> using std::cout;
>
> class Foo {
> public:
> Foo() {}
> };
>
> class Bar : public Foo {
> public:
> Bar(int b_) : b(b_) { }
> private:
> int b;
> };
>
> int main(void) {
> Foo *my_foo = new Bar(5);
> shared_ptr<Foo> my_shared_foo(my_foo);
>
> return 0;
> }

Apache resigns from the Java Community Process executive committee

Posted Dec 14, 2010 12:53 UTC (Tue) by HelloWorld (guest, #56129) [Link] (21 responses)

> There is only one way to avoid the problem: experience. You must not allow people to use anonymous shared_ptr objects; it's too dangerous.
It's hard to enforce this kind of rule though. These things are easy to overlook in a code review.

> Here's another traditional memory leak, which I don't think you'll need teh Google to spot:
I didn't use "teh google" for the other one, I just knew that pitfall already. I have no idea what might be wrong with the second one though.

Apache resigns from the Java Community Process executive committee

Posted Dec 14, 2010 18:07 UTC (Tue) by cmccabe (guest, #60281) [Link] (20 responses)

> I didn't use "teh google" for the other one, I just knew that pitfall
> already. I have no idea what might be wrong with the second one though.

The implicit non-virtual destructor causes slicing.

It's possible to write correct code in C++, but it requires a *lot* of self-discipline. You can make all the same mistakes a C programmer can, plus a whole lot more.

Apache resigns from the Java Community Process executive committee

Posted Dec 14, 2010 22:14 UTC (Tue) by foom (subscriber, #14868) [Link] (6 responses)

While of course you're exactly right that calling the destructor on Foo when the instance is of Bar is a bad mistake to make, your *particular* example, at least with gcc, will not cause a memory leak. That is simply because Bar's destructor doesn't do anything. The only action taken, in the end, is to call free(my_foo), which works perfectly well and still releases all the memory when my_foo is a "Bar" object rather than the Foo object it is declared to be.

Now, if, instead of "int b;", you'd put "std::string b;" as the member of Bar, then you'd have an actual leak. :)

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 0:51 UTC (Wed) by paulj (subscriber, #341) [Link] (5 responses)

Serious? The destructor that gets called is the base class, not that of instantiated type of the object, cause the shared_ptr uses the base class type? Is that a flaw in shared_ptr or in the underlying language? That's just amazing...

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 2:03 UTC (Wed) by nix (subscriber, #2304) [Link] (1 responses)

That's because the destructor isn't virtual. That's what the absence of virtual *means*. (Now perhaps the fact that virtual wasn't default, with lack-of-virtual an option, *is* a language flaw.)

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 7:08 UTC (Wed) by paulj (subscriber, #341) [Link]

Ah yes, of course. Keep forgetting about C++'s selective inheritance.

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 2:14 UTC (Wed) by foom (subscriber, #14868) [Link] (1 responses)

No no, you misunderstand. It's not as bad as that. What destructor shared_ptr calls depends on the declared type of the argument in its constructor, not the type of the shared_ptr itself. So, in an idiomatic spelling:
> shared_ptr<Foo> foo(new Bar(5));
everything works fine, because the argument is of type "Bar *" (that being the return type of the new expression).

It's only in the case of:
> Foo *foo_rawptr = new Bar(5);
> shared_ptr<Foo> foo(foo_rawptr);

that you get wrong behavior.

That is a feature of C++ the language: it lets you decide whether you want to keep track of the runtime type of the object or not. "Not" is a very important use-case: it allows you to use many of the features of C++ even in situations where adding a class-pointer word to every instance would be excessively expensive, or impossible (e.g. mmaping a file full of structs from disk).

So anyhow, the *default* is to not track the runtime-type. However, if you mark the destructor "virtual", then it *will* add a word to the size of the object instance to keep track of the actual type, and then delete will call the destructor for "Bar", even in the second case.

Taking shared_ptr out of the picture for simplification, the thing that doesn't work without a virtual destructor is:
> Bar *bar = new Bar(5);
> Foo *foo = bar;
> delete foo;

(that is: where the static type of the argument to delete is different from the dynamic type of the object).

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 3:06 UTC (Wed) by HelloWorld (guest, #56129) [Link]

No no, you misunderstand. It's not as bad as that. What destructor shared_ptr calls depends on the declared type of the argument in its constructor, not the type of the shared_ptr itself.

Whoa! That is certainly a behaviour I didn't expect. Other smart pointer classes like boost::scoped_ptr or std::auto_ptr don't do this, and I really can't think of a reason for making shared_ptr behave that way. Do you know of any rationale for this behaviour?

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 2:24 UTC (Wed) by HelloWorld (guest, #56129) [Link]

> Is that a flaw in shared_ptr or in the underlying language?
shared_ptr has nothing to do with it, the same thing would happen if you'd delete the pointer by hand. The problem stems from the fact Bar's destructor isn't virtual.

I guess the reason for not making destructors virtual by default is that this would be inconsistent with the rest of the language, as other methods aren't virtual by default either. In the very early days of C++, member functions existed but virtual member functions were introduced only later. At that point, Bjarne chose not to make methods virtual by default, but you had to use the virtual keyword to make them. I probably would have done it the other way around, as the worst thing a virtual method call can do is make your program a little slower, while making a non-virtual method call can cause severe bugs.

So, whether this is a flaw in the language depends on whom you ask.

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 1:41 UTC (Wed) by HelloWorld (guest, #56129) [Link] (12 responses)

It's possible to write correct code in C++, but it requires a *lot* of self-discipline. You can make all the same mistakes a C programmer can, plus a whole lot more.

I believe that this argument is a red herring for two reasons. The first is that it doesn't take into account how likely you are to make a certain typical C mistake in C++. For example, it's very easy to make a mess of dynamically allocated arrays in C, and while this is also possible in C++, you're much more unlikely to, as you'd probably just use a container class. The second is that if you do the C equivalent of the things that may cause bugs in a C++ program, you're at least as (and probably more) likely to make the same mistake. Your example would look something like this in C:

#include <string.h>
#include <stdlib.h>
struct Foo_vtbl {
  void (*destroy)();
};
struct Foo {
  struct Foo_vtbl *vtbl;
};
void destroy_Foo(struct Foo *f) {
}
struct Foo_vtbl Foo_vtbl = { &destroy_Foo };
void delete_Foo(struct Foo *f) {
  f->vtbl->destroy(f);
  free(f);
}
void construct_Foo(struct Foo *f) {
  f->vtbl = &Foo_vtbl;
}
struct Foo *new_Foo() {
  struct Foo *f = malloc(sizeof(struct Foo));
  construct_Foo(f);
  return f;
}

struct Bar_vtbl {
  struct Foo_vtbl foo_vtbl;
};
struct Bar {
  struct Foo foo;
  char *str;
};
void destroy_Bar(struct Bar *b) {
  free(b->str);
  b->foo.vtbl = &Foo_vtbl;
  destroy_Foo(&b->foo);
}
struct Bar_vtbl Bar_vtbl = { { &destroy_Bar} };
void delete_Bar(struct Bar *b) {
  b->foo.vtbl->destroy(b);
  free(b);
}
void construct_Bar(struct Bar *b) {
  construct_Foo((struct Foo *)b);
  b->foo.vtbl = (struct Foo_vtbl*) &Bar_vtbl;
  b->str = malloc(6);
  strcpy(b->str, "Hello");
}
struct Bar *new_Bar() {
  struct Bar *b = malloc(sizeof(struct Bar));
  construct_Bar(b);
  return b;
}

int main(void) {
  struct Foo *f = (struct Foo *)new_Bar();
  delete_Foo(f);
}

This is the kind of crap you need to write in order to do trivial object-oriented stuff in C. Note that making a destructor non-virtual is just as easy as in C++, just change

  f->vtbl->destroy(f);

  destroy_Foo(f);

in delete_Foo - it's still perfectly innocent-looking! And there are dozens of other opportunities for making silly mistakes in the above code that aren't there in the C++ version. So, while C++ certainly has its problems, I'd choose it over C any day of the week.

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 17:55 UTC (Wed) by cmccabe (guest, #60281) [Link] (11 responses)

Aaaaragh! It burns, it burns!

How about you read the kernel style guide and rewrite it as this:

> #include <stdlib.h>
> #include <string.h>
> 
> struct Foo {
>         /* foo stuff */
> };
> 
> struct Bar {
>         struct Foo f;
>         int b;
> };
> 
> int main(void) {
>         struct Foo *my_foo = malloc(sizeof(struct Bar));
>         memset(my_foo, 0, sizeof(struct Bar));
>         /* do stuff ... */
>         free(my_foo);
>         return 0;
> }

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 19:33 UTC (Wed) by HelloWorld (guest, #56129) [Link] (10 responses)

You're totally missing the point. As foom pointed out earlier, the case that you gave initially doesn't actually leak memory, as an int member doesn't require any cleanup to be performed. In order for it to leak memory, it needs to be an object of a class that requires cleanup, like std::string. And if you add such a member to your struct Bar, you need all the plumbing I've shown.

(formally, using delete with a base class pointer that points to an object of a derived class yields undefined behaviour (UB) unless the base class's destructor is virtual, but the memory is correctly freed in any implementation I know of, it's just that the destructor for the additional member in Bar isn't run -- which doesn't matter as int doesn't have a destructor. Also, your example also exhibits UB as soon as you want to use my_foo as a struct Bar* (which ought to be possible since it really does point to a struct Bar))

Apache resigns from the Java Community Process executive committee

Posted Dec 15, 2010 23:50 UTC (Wed) by cmccabe (guest, #60281) [Link] (9 responses)

> You're totally missing the point. As foom pointed out earlier, the case
> that you gave initially doesn't actually leak memory, as an int member
> doesn't require any cleanup to be performed. In order for it to leak
> memory, it needs to be an object of a class that requires cleanup, like
> std::string. And if you add such a member to your struct Bar, you need all
> the plumbing I've shown.

*You* are missing the point. Programming in both C and C++ requires strict adherence to certain conventions to make it practical. In C the convention is to have an alloc_foo() and a free_foo(). In C++, the conventions are more complicated (and numerous), but one of them involves using a base class with a virtual destructor.

> (formally, using delete with a base class pointer that points to an object
> of a derived class yields undefined behaviour (UB) unless the base class's
> destructor is virtual, but the memory is correctly freed in any
> implementation I know of, it's just that the destructor for the additional
> member in Bar isn't run -- which doesn't matter as int doesn't have a
> destructor

Yes, several things need to come together for the bad behavior to bubble to the surface-- putting "new" on a different line than the declaration of shared_ptr, a non-trivial destructor for Bar, a low-memory condition, and your most important customer attending the demo. Does that make you feel better, or worse?

> Also, your example also exhibits UB as soon as you want to use my_foo as a
> struct Bar* (which ought to be possible since it really does point to a
> struct Bar))

Wrong. There's nothing undefined about the behavior of typecasts in C.

In C, the address of the first data member is always equal to the address of the containing struct. So using my_foo as a struct Foo is fine. If you want to use it as a Bar (downcasting) you must use a typecast.

On a more fundamental level, inheritance is an overused technique. C++ experts like Scott Meyers advocate more use of composition, and less of inheritance. Unfortunately, languages like Java and C++ have the opposite bias, giving you all kinds of fancy syntax for inheritance, but almost none for composition.

For example, composition is the only safe way to extend the standard containers in the STL. (Partly this is because they have non-virtual destructors, and partly because the methods they contain may be implementation-dependent.) But if you try to use composition in this way, you'll have to write a wrapper class by hand that passes through every function call to the contained object. It's a tedious and manual process.

Apache resigns from the Java Community Process executive committee

Posted Dec 16, 2010 1:28 UTC (Thu) by HelloWorld (guest, #56129) [Link] (8 responses)

> *You* are missing the point. Programming in both C and C++ requires strict adherence to certain conventions to make it practical. In C the convention is to have an alloc_foo() and a free_foo(). In C++, the conventions are more complicated (and numerous),

No, they're not really. The cases you can deal with in such a simple manner are just as simple in C++ - you allocate with new and you free with delete, no need for virtual destructors. However, if you actually *need* a virtual destructor, then writing an equivalent program in C requires you to write all of that boilerplate code I've shown earlier.
The difference is not that C is somehow simpler than C++, it's that C++ provides solutions (perhaps not perfect ones) to problems that C doesn't deal with at all.

> Wrong. There's nothing undefined about the behavior of typecasts in C.
You're right, I was wrong in that respect.

> On a more fundamental level, inheritance is an overused technique. C++ experts like Scott Meyers advocate more use of composition, and less of inheritance. Unfortunately, languages like Java and C++ have the opposite bias, giving you all kinds of fancy syntax for inheritance, but almost none for composition.
What kind of "fancy syntax" for composition do you have in mind?

Apache resigns from the Java Community Process executive committee

Posted Dec 16, 2010 4:39 UTC (Thu) by cmccabe (guest, #60281) [Link] (7 responses)

Just so people don't think I only criticize C++, here are some subtle errors in C code:

> char buf[80];
> fgets(user_str, sizeof(buf), stdin);
> printf(user_str);

> extern int *my_array;
> /* "zero the array" */
> memset(my_array, 0, sizeof(my_array));

Anyway, to reiterate. C++ was never designed to be safer than C. Every time the designers of C++ had to choose between execution speed and safety, they chose execution speed. C++ is almost like a rapid application development kit for C.

The problem is, when you select execution speed and short development time as your two highest priorities, readability and code quality suffer. For a lot of applications (for example, the Linux kernel), that's simply not acceptable. C++ is still appropriate for many applications. For example, the computer games industry loves C++, despite all the new languages that have come out in the last few years.

> What kind of "fancy syntax" for composition do you have in mind?

At a minimum, C++ should offer something like Ruby does: the ability to create accessors for elements by adding a quick attribute in the variable declaration. For example:

> class Foo {
>  int bar : attr_reader
>  int baz : attr_reader, attr_writer
> }
>
> cout << my_foo.bar() << endl; // look ma, no boilerplate!

Personally, I'm going to explore Google Go a little bit more. It doesn't even have the concept of inheritance. Go just has "specifications" that say what methods a type must provide to meet the specification. Any type that has such methods can be used. Sort of like duck typing, but enforced at compile time and safe! And there is no "static type" vs. "dynamic type" misery, and no base classes teleporting random variables into the middle of your code.

Apache resigns from the Java Community Process executive committee

Posted Dec 21, 2010 0:54 UTC (Tue) by HelloWorld (guest, #56129) [Link] (6 responses)

Anyway, to reiterate. C++ was never designed to be safer than C. Every time the designers of C++ had to choose between execution speed and safety, they chose execution speed.

When you design a systems programming language, then that seems like a sensible thing to do.

The problem is, when you select execution speed and short development time as your two highest priorities, readability and code quality suffer.

Perhaps it's just me, but I think that putting "virtual" in front of a destructor is a lot more readable than implementing vtables by hand which looks like this.

At a minimum, C++ should offer something like Ruby does: the ability to create accessors for elements by adding a quick attribute in the variable declaration.

There is no special syntax for it, but on the other hand, returning a reference isn't very verbose (and you can use a macro to make it shorter).

Personally, I'm going to explore Google Go a little bit more. It doesn't even have the concept of inheritance. Go just has "specifications" that say what methods a type must provide to meet the specification. Any type that has such methods can be used. Sort of like duck typing, but enforced at compile time and safe! And there is no "static type" vs. "dynamic type" misery, and no base classes teleporting random variables into the middle of your code.

Well, Go may be a nice language but sometimes you don't want a garbage collector.

Apache resigns from the Java Community Process executive committee

Posted Dec 21, 2010 21:19 UTC (Tue) by cmccabe (guest, #60281) [Link] (5 responses)

Perhaps it's just me, but I think that putting "virtual" in front of a destructor is a lot more readable than implementing vtables by hand which looks like this.

You're missing the point. vtables can be useful, but you just don't need them that often. The fact that there is a special syntax in C++ encourages people to write bad code.

There are numerous examples of vtables in the Linux kernel. Usually it's just as simple as a struct with function pointers and a registration function that's called on that struct.

C++'s special syntax for vtables makes it quicker to create them, but does it make it easier to do it correctly? In my experience, most C++ programmers don't know the hidden gotchas that lurk everywhere once you first type "virtual."

For example, what's the problem here?

class Foo {
public:
  virtual ~Foo() { log("destroying Foo."); }
protected:
  virtual void log(const std::string & foo) { }
};

class LoggedFoo : public Foo {
public:
protected:
  virtual void log(const std::string & str) {
    std::cout << "logging important message " << str << std::endl;
  }
};

There is no special syntax for it, but on the other hand, returning a reference isn't very verbose (and you can use a macro to make it shorter).

The minimum accessor length I can think of is something like this:

const Foo & foo() const { return _foo; }

That's a lot of typing just for one data member. The fact is, they really blew it by not having a better syntax for this-- in my opinion. Also, Java has the same problem but even worse.

Well, Go may be a nice language but sometimes you don't want a garbage collector.

If you don't want a garbage collector, why not just write it in C? C++ represents the awkward, error-prone adolescence of higher-level languages, not their future.

Apache resigns from the Java Community Process executive committee

Posted Dec 21, 2010 23:07 UTC (Tue) by HelloWorld (guest, #56129) [Link] (4 responses)

You're missing the point. vtables can be useful, but you just don't need them that often.

The funny thing is that you wrote the rebuttal to this statement yourself only a few lines later: "There are numerous examples of vtables in the Linux kernel".

The fact that there is a special syntax in C++ encourages people to write bad code.

Limiting the language in order to prevent programmers from writing bad code never worked and it never will, since bad programmers will find other ways to write a bad program. On the other hand, you're making life harder for the people who are able to use advanced features in the language sensibly. This looks like a very bad tradeoff to me.

C++'s special syntax for vtables makes it quicker to create them, but does it make it easier to do it correctly? In my experience, most C++ programmers don't know the hidden gotchas that lurk everywhere once you first type "virtual." For example, what's the problem here?
class Foo {
public:
  virtual ~Foo() { log("destroying Foo."); }
protected:
  virtual void log(const std::string & foo) { }
};

class LoggedFoo : public Foo {
public:
protected:
  virtual void log(const std::string & str) {
    std::cout << "logging important message " << str << std::endl;
  }
};

I don't know what you expected the program to do, so how am I supposed to know what you consider to be a problem? Perhaps you meant that ~Foo will call Foo::log and not LoggedFoo::log. I believe that this is a Good Thing. LoggedFoo may have introduced a new meber variable x, and if LoggedFoo::log would use x somewhere, it'd blow up when called by ~Foo, as x would already have been destroyed at that point.

If you don't want a garbage collector, why not just write it in C?

Because C doesn't offer me the features I want. I want type-safe containers (read: templates), I want OOP, I want exceptions and I want destructors. C doesn't have those, and I actively hate C for this (amongst other things).

C++ represents the awkward, error-prone adolescence of higher-level languages, not their future.

No, C++ was never meant to rival high level languages. When it was designed, high-level languages like Smalltalk existed, and Bjarne deliberately designed C++ differently, because he wanted to do low-level work.
Of course, C++ has its share of problems, many of which are either due to its low-level nature or inherited by C (weak type system, bad syntax, horrible preprocessor), so I'd certainly pick a higher-level language over C++ when that is possible. But if the choice is between C and C++, I know what to choose.

Apache resigns from the Java Community Process executive committee

Posted Dec 22, 2010 11:43 UTC (Wed) by cmccabe (guest, #60281) [Link] (3 responses)

> The funny thing is that you wrote the rebuttal to this statement yourself
> only a few lines later: "There are numerous examples of vtables in the
> Linux kernel".

I think we both know that the real question is not how many X the Linux kernel source has, but how many X per lines of code. By that measure, vtables are used sparingly and tastefully.

> Limiting the language in order to prevent programmers from writing bad
> code never worked and it never will, since bad programmers will find other
> ways to write a bad program. On the other hand, you're making life harder
> for the people who are able to use advanced features in the language
> sensibly. This looks like a very bad tradeoff to me.

My point wasn't about limiting the language. I think we all know that C can be used to do anything C++ can do, and vice versa. The question is, what does the feature set of the language encourage you to do? Is it in good taste? Does it encourage you to write things in a readable way, or in an unnecessarily complex and formal way? Do the features work well together, or are there some that can't safely be used in combination (or even separately?)

> [snip discussion of virtual function calls in base constructor/destructor]

Congrats, you know the solution to this one. Now be careful, because the behavior is different in Java and C#.

Apache resigns from the Java Community Process executive committee

Posted Dec 22, 2010 14:43 UTC (Wed) by paulj (subscriber, #341) [Link] (2 responses)

To be fair, non-virtual methods are, we now know, extremely dangerous and inconsistent with the type-safety of inheritance (some of which was developed with and even after the development of C++). That C++ even allows them is perhaps a historical curiosity. One of a number of misfeatures in C++, where new language features were added before they were fully understood, leaving vestigial dangers waiting to trip up programmers. I like the other commentator's description of C++ being "the awkward, error-prone adolescence of higher-level languages".

Apache resigns from the Java Community Process executive committee

Posted Dec 25, 2010 4:46 UTC (Sat) by foom (subscriber, #14868) [Link] (1 responses)

> To be fair, non-virtual methods are, we now know, extremely dangerous [...]. That C++ even allows them is perhaps a historical curiosity.

Absolutely not a historical curiosity! I can't imagine using a C++-like language that didn't allow non-virtual members. Being able to do OOP with *zero* runtime overhead is a killer feature!

There's certainly place in the world for languages like Python too (which I do like, despite its horrific slowness), but C++ without non-virtual methods would just be sad.

Apache resigns from the Java Community Process executive committee

Posted Dec 25, 2010 10:50 UTC (Sat) by paulj (subscriber, #341) [Link]

It's a killer feature in terms of weird bugs, yes. If non-virtual methods were also not overridable, then perhaps it'd be a useful thing...