I understand, and I think we actually do agree on this if we could just put it into the right terms. My use of "difficulty" was wrong; that is a term with a very specific meaning in Bitcoin, but I was using it in a generic sense to refer to the time it takes to generate a block (the target rate), not the number of hashes or upper limit for acceptable solutions (which are derived from that target rate and the actual rate).
Obviously you don't want the rate at which blocks are generated to increase sharply, since that would make it relatively easy to invalidate some or all of the work that went into creating the existing block chain. If it takes a year to get to block 1000, but only another week to get to block 2000, then you have a problem--a concerted effort could supplant the original block chain, or at least a significant suffix of it, with a new one of the attacker's choice, causing the system to break down (double-spending, etc.). For that to happen to the last day's transactions is one thing, but with a large rate of change a few day's effort could invalidate a much longer period of the chain history.
To avoid that the network has to regulate the rate at which new blocks are generated such that acceleration of that rate, if any, is extremely gradual; currently that means no average change at all, although there is some variation above and below the goal rate between difficulty adjustments.
What I'm saying, however, is that this goal rate doesn't have to be six blocks per hour to prevent so-called "brute force" attacks; provided the clients remain synchronized, it could just as easily be six blocks per *second*. I refer, of course, to a constant rate of six blocks per second since the network was formed, not a sudden 3600x drop in the difficulty from an existing six-blocks-per-hour chain. If there was to be a transition from one rate to the other it would have to be extremely gradual.