Still a few caveats...
I used to think this way but then I spent more time pondering it. If you do have electric
heating it will usually be optimised to heat spaces where people live and work. Heating any
part of the house when it's empty (ignoring frost protection), or the closet with a household
server in it, or a spare bedroom, or even a living room when everyone's asleep upstairs, is
still mostly useless because no people benefit.
It's true in principle that you can find situations where the power isn't "wasted" by turning
it into heat, but they're rarer than might first appear. Maybe $100 spent running a PC
translates to $10 of "free" heating for some people, and even more for a few, but that's still
a poor reason to leave all that hardware running when energy prices are rising.