User: Password:
Subscribe / Log in / New account

Related to MSI-X?

Related to MSI-X?

Posted Nov 21, 2009 17:20 UTC (Sat) by kleptog (subscriber, #1183)
Parent article: Receive packet steering

How is this related to MSI-X, the system whereby network cards can assert different MSI interrupts based on a checksum in the header. This allows the load to be spread across CPUs in much the same way as suggested above.

I'm also wondering how this interacts with PCAP. If you have a machine with a dozen processes attached to an interface, then the packet needs to be copied to several different places in userspace (assuming MMAP ring-buffers). These are all going to be running on different CPUs so I don't think the above processing will help. But perhaps the actual BPF filtering can be spread out over multiple CPUs?

I ran into a problem this week, where IO-APIC round-robin interrupt routing is disabled on machines with >= 8 CPUs, which means if you don't have MSI-X you have >50% of one CPU dedicated to interrupt processing. The scheduler doesn't know this, leading to some odd effects. So if the above system works on ordinary MSI network cards this could be a solution,

(Log in to post comments)

Related to MSI-X?

Posted Dec 2, 2009 21:37 UTC (Wed) by therbert (guest, #61498) [Link]

I think maybe you are referring to networking cards that provide multiple receive queues where each one can have a separate interrupt. In some sense, RPS is a an emulation of this which is useful for "legacy" NICs that have only one queue. Even so, we found that certain combinations if HW multiqueue and RPS actually can have better performance than just using HW multiqueue alone.

With regards to PCAP, it's possible it may not help. However, the technique of moving packets between CPUs might be applicable at a higher layer.

This solution should definitely help if you don't have round robin interrupts (it's actually better because parallelism in the receive path).

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds