I already tried home nodes in 2.4. They didn't work.
There was also another implementation from NEC. They saw some success
on very large systems -- with large numa factors -- but they did
poorly on the more common low NUMA factor two and four socket servers.
The reason is that on these systems not using a core is always much
worse than using remote memory. And if you give the scheduler too
many conflicting inputs it will become schizo and schedule poorly and
not use all cores well anymore.
This is worst on dynamic workloads, for more static workloads it's
not quite as bad.
A better approach is some form of automatic migration, e.g. as
implemented by Lee Schermerhorn: http://permalink.gmane.org/gmane.linux.kernel.numa/590
This can actually fix up imbalances and also allow some other
optimizations. Unfortunately it also doesn't work for all workloads,
so it would need to be an optional knob.