if you have processes/threads sharing something, then you should take that into account when scheduling them to reduce the cost of the sharing.
but if you are talking about processes (which is the more common case), then you don't gain anything by having them share a cache, and in fact you are less likely to allow them to run out of cache if they share it because they will be contending for the space.
if you have a problem of entering C states reducing your performance, then the answer should be to change the controller that is causing you to enter the C states to hurt you less.
this doesn't require you to tune the scheduler for every workload, it simply takes accepting the fact that what is best for power is not going to be best for performance, and therefor not insisting that 'everyone cares about power' which implies that the power saving mode is the only one that should matter.
going back up the thread a few posts, the heuristic that if you are on battery power you are probably willing to sacrifice a bit of performance for significant reductions in power use, but if you are on line power you are probably not does represent the real world. It's not perfect, which is why it is a default, not a hard-coded mode, but it's a pretty accurate heuristic.