I think the idea of a "conference blacklist" of people who are known to cause these kind of troubles would be a good idea, but very hard to setup.
That said, I agree with direct action. If sexually assaulted in any way (such as gropes, grabbing and kissing, etc) then physical self defense seems like the best thing to do. After a couple of women yell "rape" really loud and punch or kick the guy harassing them it will be an immediate and unmistakable message. If police are called, all the better. I'd say call them yourself.
Of course, this won't work for simple cat calls and lewd remarks. Reporting people to the conference and seeing if it's taken seriously is the best you can do there.
Lawsuits do take a while, but you can always decide not to file. Just getting the guy a visit from the police will send a pretty strong message too.
I realize it's tough and scary to have to physically defend yourself, but short of a large scale boycott/walkout I'm not sure what else could be done (especially by an individual) to get the message across REAL fast.
(Note: I'm assuming the US or similar treatment of such sexual assaults. If the country wouldn't help or would actually go after you for defending yourself, your only choice besides status quo would be not going)