I have been wishing for such a feature for year. The way OpenSSH does it is perfect. I do trust my personnal certificate hash storage more than CAs, their search for profit prevents accurate verification of identities.
The probability that the site I visit was not showing the right certificate the first time I connected to it is very low and comparing the hash is adding a really useful security layer. Defense in depth.