"Philosophically speaking, however, I don't see why anyone should expect to earn interest on a risk free transaction."
I don't think there's any philosophy involved here, just economics.
If you assume (as most of the world does) that progress is not an illusion, then making idle resources active in pursuit of progress is a gain. Remember that to make these transactions "risk free" to the saver we need only in fact make them less risky in total than keeping your money under the mattress. The tricky bit (which is handled by things like FDIC) is smearing the uneven risks so that ordinary citizens don't get a nasty surprise.
"Earn interest today, pay for it in increased taxes (or inflation) tomorrow."
You assume a zero sum where there is no evidence for one. The permanent building societies make a mockery of this claim. Since the early 19th century they've been using Peter's money to buy Paul a house, then charging Paul interest and paying it to Peter, they've put vast numbers of people into homes, given back vast sums in interest, and they're cheap to run. It's just organised sharing, the kind of thing we try to teach kids in kindergarten.
However mutual societies don't make people obscenely rich. And so in the 1980s and 1990s this created an enthusiasm for demutualising the societies (people trying to capitalise on this were called "carpetbaggers") and "unlocking" their assets by converting them into banks. The surviving building societies (which fought off carpet baggers with a variety of tactics) are still going today, their worst exposure to the financial crisis being an increase in defaulters which is easily covered by their mandatory long-term holdings. In contrast many of those which demutualised are now in ruins, being rescued by the government, or by government deals struck with larger foreign banks. The carpetbaggers (ordinary people, albeit greedy ones) keep their loot and the tax payer picks up the bill.