I don't know what Linus' problem is, but the problem I have with big version numbers is that the differences become too small - I lose track of where we are up to.
The difference between 38 and 39 is simply a lot smaller than the difference between 8 and 9 (is some cognitive sense which I believe psychologist can measure) and I have trouble remembering which of those two is 'next'.
It felt really good when 2.6.39 was done because then the "next" version would be 2.6.40, and the difference between 40 and 39 is MUCH bigger than the difference between 39 and 38. And then 2.6.42 wasn't far away and the difference between 42 and any other number is clearly very big too. So we were coming to a time when I would not be confused about version numbers for at least a year or so.
So it seemed like a strange time to change numberings, just when the up coming numbers would be so much easier to work with. But I definitely understand how numbers can be "too big" - it is when their differentiation becomes too small.