Effects much worse for other distributions than expected

Actually I wouldn't say you're entirely safe if the server is vulnerable and you're not.  
There's still the issue of the host key, which is used to prevent the bad guys from 
pretending to be the server.  If that host key is compromised, then someone can pretend 
to be the server.  Then you're in a little trouble if they can also get your public key (it's 
treated as public, shouldn't be horribly hard), and more trouble if you're using password 
authentication.