*Not* ejecting people also opens organizers up to lawsuits from the people being harassed. (Especially when the harasser is a repeat offender that everyone knows about but ignores anyway. Even if we ignore the *ethical* aspects here, that'd be a *far* more difficult lawsuit to defend against.)
And, for that matter, pretty much everything else involved in running a conference *also* opens one up to lawsuits -- e.g., guess who's on the hook if some attendees trash the venue.
So we have standard ways to deal with this -- written policies (if the form when you signed up said "attendance may be revoked on whim of organizers" then you hardly have a legal leg to stand up if your attendance did get revoked), and running the conference under the auspices of a limited liability corporation with a civil liability insurance policy.
This is all so standard that when I see the liability argument I always feel like the person advancing it is just trying to find some logical justification to back up their gut reaction. Sorry if that's not the case here, but that's how I feel.