If the started power management work would stop at USB, it would be rather
pointless.
And for battery powered devices even that's a huge amount. If you want
the device to be able to idle for weeks, all this kind of extra power
usage needs to be fixed.