See my comment below. Basically, private data members of a class also need to be #included in that class' header file. So you *cannot* "avoid parsing the vast majority of headers more than once."
Under ideal conditions, C++ compilation is slow. If you add even a few non-ideal conditions, like programmers who love to define functions in header files "for performance", extensive use of templates, auto-generated anything, or unecessary cross-module dependencies, it becomes positively glacial.
Unfortunately real-world projects tend to have some or all of these conditions. I'm too lazy to find the reference now, but Google's C++ compile times are said to be measured in hours. And those guys read Effective C++ and know their stuff.
Precompiled headers sound helpful, but only for headers you are including from external libraries. Maybe they would be useful for something like QT? I haven't used precompiled headers.