1) You need a separate ip-address for a https-server, this is a problem because shared-ip hosting is extremely common, even for fairly high-profile sites. Yes SNI solves that, but that's a recent development (and thus a solution that'd mean blocking out all older browsers)
2) The situation where self-signed-https causes scary warnings, whereas no-encryption-http does NOT. I don't know what the browser-makers are smoking, but the practical result is that if I make my site MORE secure, the users get hassled with warnings about me being UNSECURE. This is totally batshit crazy. The certificate-signing business, is a fraud. Lots of money, for essentially nothing. Why the hell doesn't new domain-names come with signed wildcard certs for the domain, in the first place ?
3) The initial handshake, means https is slower. (the encryption and decryption is basically irrelevant) It causes several more round-trips, and thus is a huge problem, particularily if there's many-small-requests that can't or aren't all using keep-alive.
And don't give me "it's cheap". Not it's not cheap. If I should have valid certs for just the domains I use for various hobby/experimental stuff, It'd be a week or so of full-time work and costly enough to more than double my hosting-costs, and by the time you've added in separate ips for each server, you're talking of multiplying my hosting-costs by like 5.