No, I'm saying any conference willing to contractually guarantee good behavior of its attendees is foolish.
If you want to require that conference organizers vet presentations for offensive content, promptly eject attendees for inappropriate comments and behavior, and encourage women to participate, then that's fine. There's plenty of precedent for that.
If you want the ability to unilaterally terminate your obligations and recover previously-made payments because an attendee alleged that another attendee groped her in a pub at 1am, then you're going to run into some resistance, and rightly so.