"Smaller attack surface"?

Certainly it would be possible for the software case, where the isolation is process isolation or similar rather than potentially separate hardware - still a significant obstacle for most realistic attacks.

Also you could do something like RFC 9345 "Delegated Credentials", there the idea is you get a certificate but the signatures from the certified key are used to delegate a further credential, so maybe you make one delegation every 30 minutes, and the delegated credential is only good for an hour, but clearly a hardware HSM can keep up with that rate even if the web server is using that credential millions of times per second. Stealing a delegated credential that you can use for at most one hour isn't good, but it's extremely time limited, if you have an attack for this in the morning but I fix it at lunch time then everything is fine before closing.

There are practical problems today but nothing which can't be addressed. RFC 9345 points out that clock skew is an issue, a lot of machines will be wrong by exactly one hour due to "Daylight saving" nonsense and others won't have accurate clocks, humans tend to treat 5-10 minutes as acceptable unless they're weird like me. In some applications you don't care - you can guarantee accurate clocks in intended clients, but general web serving won't be one of those.