Huh, the tact filter point is a good one. I'd read that essay before, but never connected it. I don't think it would be a particularly inaccurate summary of the diversity conversation to say that the community should be open to people who weren't picked on as kids as well as people who are (and that one easy way to do that is to change the assumption of the default filter -- you don't insult anyone by having too much "tact", but you do insult them by having too little).
I do personally know dozens of people (mostly women, as it turns out, but several men) who are all quite competent programmers, but have gotten the impression somehow that the community they want to participate in does, in fact, look down on them, or that they're not "hardcore" or knowledgeable or accomplished enough to participate in that community. I agree with you that in almost all cases, this isn't true. But it only takes one person saying "yes, I actually look down on you" to make someone think this is representative of the community. Going back to the tact filters, it's very easy to convince someone that "they don't really mean it" is untrue. And then the tact filter crumbles, and everyone else is interpreted as if they do really mean it.
There is arguably something of a correlation with the tact filters, come to think of it. People who have the tact filter installed in the picked-on-nerd direction will just go ahead and do or say something, assuming that other people will ignore them at their own discretion. People who have the tact filter installed "correctly" will feel presumptuous participating somewhere where they think they shouldn't, absent a clear statement that they should.