And then there's the fun AMD's new Bulldozer platform brings with its core/module structure. It's not exactly hyperthreading, but not exactly different either. You also get the excitement of being able to run some modules at a higher clock if others are powered down.
Speaking as an outsider that's not really familiar with the guts here, it does seem like the governor might be a good place to decide what CPUs the scheduler should play with, and maybe what order it should fill them in. It seems like that would potentially allow you to factor out the CPU / system specific heuristics for hyperthreading, core/module, NUMA, etc. from the actual process of scheduling.