Interview with Donald Knuth (InformIT)

If you read the interview again, you might note that he doesn't write that test suites are bad
in general -- he writes that *his* development style doesn't fit with unit tests a la JUnit or
so.

Please remember: TeX was probably the very first open source program that ever came with a
test suite, the trip test, and where every port must pass this torture test, otherwise it
can't be named TeX. And the trip test was already there in 1978. I'd like if half of the other
open source software would have test suites that are 10% as good as those of TeX and friends.

And I don't know what you mean with "weird restrictions". If you mean the overall usage of
global variables, reluctance to use structures, and other stuff that's not up-to-par with our
current software methods -- that's caused by the restrictions of Pascal compilers at the time
of TeX's creation. Back in 1981/1982, when we did our first TeX port, we actually had to
change the Pascal compiler to be able to get TeX running.

True. Lightspeed means that at 1Ghz, no signal can travel more than about 30cm. At 3Ghz, no
signal can travel more than 10cm. And switching-times aren't instantaneous, so the practical
limits on die-size are smaller.

I'd like to point out though, that the *physical* limit on serially dependant computation is a
*volume* and not an *area*.

If lightspeed allows a 100mm^2 (10mm by 10mm) then lightspeed also allows a cube of 10mm by
10mm by 10mm, or 1000mm^3.

There are reasons we don't do it that way, both construction and cooling would be a real
bitch. It's hard enough dealing with the multiple layers we have on todays CPUs, making an
awful LOT more layers would -not- simplify design.

Consumer systems aren't doing single linear tasks any more. There's generally a bunch of
intermittent stuff going on throughout the session, and a single task that the user's
attention is on, whose reaction time determines the quality of the user experience. For this
sort of workload, adding cores helps up to the maximum number of possible
simultaneously-running tasks. That is, people will watch movies while waiting for their
spreadsheets to recalculate, and they notice how long it takes and whether the movie is
smooth.

That means that, while O(N) cores isn't yet useful for most people, the small constant number
of useful cores is more than one. I expect that there will be demand for about 8 cores when
software developers get used to being able to do CPU-hungry background tasks without impacting
the interactive thread at all.

From what I understand, it's to improve productivity.  He has not used email since 1990, after
using it for fifteen years.  On his web page, he wrote that he does his work in "batch mode",
and responds to correspondence the same way.  So I'd gather he doesn't use a net-connected
computer because it limits distractions and lets him do things in batches.

One thing that I think is missed here is that if you look at what the benefits of Moore's law
have gone to in the past, they've been split between making the *computer* more capable by
exploiting the increased power for more sophisticated tasks (like real-time video decoding)
and making *programming* the computer easier, by permitting programmers to write less
efficient programs in higher level languages.

Now hardware manufacturers are saying "we're going to produce chips where the only way to
exploit the benefits of the increased power is to adopt significantly more complex and
difficult programming models, and we're going to expect programmers to recode the applications
to do that." 

Well, so basically what they're saying is that they're not going to (be able to) continue to
make computers easier to program. That sounds like Moore's Law hitting a wall to me. Sure by
going massively parallel we could perhaps continue to reduce time to compute in those rare
situations where it's worth it to do so, but that's only half of the benefit we've
traditionally gotten from Moore's Law.

Instead of saying "rewrite your programs to take advantage of multiple cores" they could *not*
give us more cores and say "rewrite your programs to be more efficient with the hardware you
have," but that would be blatantly admitting that Moore's Law is dead.

You can probably go into the kernel and disable support for more then one or two cores, then
enable them later and see if there is any difference. On a quad core machine, of course. Which
is pretty affordable nowadays, in terms of development machines.

For what it's worth, I regularly run testsuites and build jobs that are embarrassingly
parallel, so when I purchased a quad-core for work this fall, make -j made a nearly linear
improvement on this particular bottleneck in my work cycle. Didn't make me any smarter or
quicker with emacs, but made a half hour cycle between major edits collapse to about 7
minutes. That was very noticed. I'll happily support buying 256 cores on the same line of
reasoning, when they're available, because 7 seconds is much better than 7 minutes!

(And this is a tolerably fast job; I've had embarrassingly parallel workloads that take
*weeks* on the provided hardware. That's when you start snooping around for cross-organization
job queues and whatnot, to steal idle time from other machines.)

I'll grant that developers are probably not who the multicore "revolution" is aimed at. But
the numbers I've seen suggest that transaction-bound network servers of various forms see a
good win too: particularly those that were built around some sort of process-pool or
thread-pool model. More of the parallel thingies run at the same time. QED.