First, let me agree that "please stop discussing whether this was bad because I feel harassed by that meta-discussion" is Right Out; such conversations are a Non-Maskable Interrupt (or, if you prefer Robert's, a Point Of Personal Privilege), and by their nature immune from such things, just as, for example, discussions about ... well, let me not start a meta-meta-argument.
In any event: clearly some of the material was offensive to some people.
Clearly, that wasn't accidental; the speaker chose his material *precisely because* he expected it to evoke a response.
So, the question becomes, should it be reasonable to consider that "harassment"?
While harassment is *generally* targeted at some small number of specific people, it certainly doesn't need to be; it can be targeted at a group, such as homosexuals.
Harassment, though, is generally used to describe a *pattern* of behavior, with what a reasonable man would say was no other specific goal in mind except the discomfort of the targeted person or group.
Neither part of that description applies here: the talk was going to end at some point; there was no reason any specific person had to stay; the discomfort wasn't necessarily *aimed* at any specific group, and there was a completely separate goal: to identify as bad specific situations, and motivate attendees to do something to fix them.
This is not the first time we'll have had a debate here about a topic like this, and this one will likely end poorly as well, with people quoting Derailing For Dummies, and other such tactics which are, themselves, derailing from the actual topic.
My personal reaction, in a one off case like this, is "if you're uncomfortable, leave the damn room". But then, I've always been as much a libertarian as a liberal. :-)