I meant there is no need for manual intervention at every startup. So you can install Linux on your server without worrying about it being stuck at a menu every time it reboots.
Clearly, if you can plug in a USB key then you have physical access to the machine. The criterion for defeating malware is surely that you can't change the bootloader without physical access. Somebody with that access could equally well install a keylogger or (in principle) just replace the motherboard with a trojaned one.
In fact, you could argue that physically plugging something in is how it should have worked from the beginning. Like an old Nintendo console, your PC or tablet device could come with a Windows cartridge installed, and if you want to boot something else you have to remove that and plug in a different cartridge (which may still allow booting Windows if you wish). Unfortunately that would make the devices a couple of dollars more expensive, so we have these shenanigans with signed bootloaders instead.