Posted Mar 19, 2012 4:21 UTC (Mon) by khim
In reply to: Hardware?
Parent article: Toward better NUMA scheduling
PCI and PCIe are definitely cache coherent (or more precisely, they support it, although you can tell devices to not snoop caches).
PCI does not support it at all (initially it was optional part of the standard but since nobody bothered to implement it later versions just removed it completely) and PCIe does not recommend to use it on large and/or busy systems (and NUMA systems tends to be both large and busy).
Which the hardware already does where IOMMUs are present...
Nope. IOMMU only hides physical addresses from hardware devices. OS kernel is in charge and must keep everything in sync. IOMMU presence is quite visible for device drivers. Since you want to make something not visible in kernel at all you need yet another level of indirection.
> Success. “Cache-coherency mechanism used for the L3 level” is part of OS kernel
This is totally false, and is simply a ridiculous claim.
The cache coherence of L3 caches in x86 SMP systems certainly isn't managed by the kernel!
See above. If your system includes some hardware which does not care about L3 cache coherence (contemporary system tend to include few PCI devices at least and on busy systems you don't want to use built-in PCIe cache snooping because it sucks significant amount of inter-CPU bandwidth which is scarce on such systems) then your kernel is charge of keeping L3 cache and main memory in sync.
to post comments)