I read Dave Neary's comment as saying that it all depends on the context _and_ that it's irrelevant what we're all individually happy with or not: If we're aware that the potential exists to irritate someone then it's best to take that into account and avoid potential offense.
I'm not arguing against your own assessment of the annoyance. I have, personally, been both annoyed and also amused by Paddy jokes. It all depends on context: my friends can make them, others can't. I can imagine that it's much worse for someone with a more immediate source of unease.
I personally enjoyed Pesce's talk and wasn't in the least offended by it and was a bit surprised at the furore after I'd read the notes and viewed the slides, so probably it's useful for me that an explicit policy exists to let me be aware that some other people feel very differently.
That said, I do wonder where is the logical endpoint of avoiding offense once it's spelt out as a quasi-legal policy instead of being a set of cultural norms? The subjective nature of determining whether offense has been given (or was intended to be given) makes this very grey territory and the language of the generic anti-harrasment policy enters boldly into this at several points, e.g. "Booth staff (including volunteers) should not use sexualized clothing/uniforms/costumes, or otherwise create a sexualized environment." Who exactly determines what is "sexualized clothing"? Is this according to Western norms? Do we accomodate the viewpoints of members of various Judaeo-Christian religions of a more conservative type?
A glib answer that it's the conference organizers who make the determination does nothing to answer the core question. On the whole this seems like a poor candidate to choose in order to exercise a policy which was initially debated in the context of stopping the problem of women being "stalked, leered at, and physically assaulted at conferences".