Calibre and setuid "works on every single linux distro"

The idea of fine-grain control may sound attractive in theory, but it is completely impractical. In theory, you can create a general-purpose kernel with fine-grain access, but in practice Coyotos is arguably the best attempt to implement this idea, and it is far from being finished, let alone to being widely adopted.

The real problem is not limitations of hardware (it is relatively easy to get around some hardware limitations by sacrificing some performance). If you want to a fine-grain system, you have to address the confused deputy problem, and it is extremely difficult to deal with in a general purpose OS. You can completely eliminate 'root', but then you will have something like 'System' on Windows, which being compromised as bad as 'root'.

Many people do not realize that granting any additional permission to regular users has a very serious security impact on the whole system. It is exactly what happened the Calibre's lead developer. He thought that granting an unprivileged user mount/unmount/eject anything is just a nice feature, but it is also turned out to be a security flaw.

I do not say that LSM or other methods to limit what process can do are not useful. However, typically they do not provide real isolation as it would be impractical, so it is more about mitigation certain attacks or at least making them more difficult. On the other hand, virtual machines provide usually very good isolation, but there is no fine-grain control. BTW, the fact that VMs turned out to be so useful for many users prompted hardware manufacture to provide "VT-support", which includes improvements to MMU as well. So once software engineers find something useful and practical, hardware will be improved to support this functionality better.