I think you're mistaken in thinking such activity bursts only impact the CPU.
Memory is equally impacted (going between fully active and self refresh), as are other parts of the chipset. And cpu + memory + chipset do make up a significant part of system power. Sure the screen takes a lot too.. but to get reasonable power you need to look across everything that's significant.