Our esteemed editor said that compaction should lead to less fragmentation but this is not necessarily the case. Anti-fragmentation (preferably with min_free_kbytes adjusted as recommended by hugeadm or tuned automatically within the transparent hugepage tree) still is the key to handling fragmentation (i.e. not necessarily reducing but but giving the system options to handle it). Anti-frag aims to keep pages of different mobility within the same huge page boundaries. Compaction doesn't affect this directly although there would be some minor benfits when the linear scanner would take movable pages from an unmovable range of superpages and put them somewhere else.
They key benefits it really brings are
a) one can avoid lumpy reclaim so a contiguous page can be created faster by copying pages instead of dumping them with IO
b) it can move pages that are mlocked so if locked pages are a major factor of the workload, they will have less impact on large allocations
c) On swapless systems, the pages can be moved where as without compaction there is very little page reclaim can do about anonymous pages. This is probably the "big" use-case I am expecting in 2.6.35 because it's something that has been whinged about in clusters.
d) if/when transparent hugepage support gets merged, it'll be cheaper to promote base pages to huge pages than if lumpy reclaim was used.
A side-benefit is that it avoids some bad trashing from lumpy reclaim. It wasn't such a big deal before but it can cause trashing in the system. Waiting to writeback dirty pages is *slow* and its waiting logic can lead to large stalls as well. It's been on my TODO list to investigate this closely for some time and see if there is a better option which I'm hoping to get around to some time next month. It's been put on the long finger because there were other issues that were a lot more urgent over the last 6-12 months.