Device-to-device memory-transfer offload with P2PDMA [LWN.net]

Device-to-device memory-transfer offload with P2PDMA

Posted Oct 3, 2018 16:25 UTC (Wed) by mrybczyn (subscriber, #81776)
In reply to: Device-to-device memory-transfer offload with P2PDMA by dullfire
Parent article: Device-to-device memory-transfer offload with P2PDMA

Yes, you're right. It would be more accurate to say "behind a host bridge". You will find more about it in the last part of the article when it talks about the use cases.

to post comments

Device-to-device memory-transfer offload with P2PDMA

Posted Oct 4, 2018 21:13 UTC (Thu) by jgg (subscriber, #55211) [Link] (1 responses)

'behind the same bridge' is the right language, if not a little confusing. It doesn't mean 'behind the last bridge' but simply any bridge. Ie the upstream bridge of a switch is sufficient to satisfy the condition, even though there are later bridges before reaching the device.

Behind the same root port (for PCI-E) is not quite the same thing, ie two functions on the same device cannot do P2P DMA with this patch series if they are plugged directly into the root port.

All that aside, this series does have the requirement that the devices be behind a switch. You can't use it on a GPU and a NVMe drive plugged directly into root ports on your CPU, for instance. This greatly limits the utility, and hopefully will go away eventually when people can figure out how to white list root complexes and BIOSs that support this functionality.

Device-to-device memory-transfer offload with P2PDMA

Posted Dec 7, 2024 11:51 UTC (Sat) by sammythesnake (guest, #17693) [Link]

> two functions on the same device cannot do P2P DMA with this patch series if they are plugged directly into the root port

That seems like a fairly likely use case - passing off some data from one stage of processing to another, so hopefully this restriction is lifted soon. I imagine that's a direction in the developers' sights, though - I'm happy to assume that my negligible level of domain knowledge is outdone by theirs ;-)

A couple of possible factors that might make it less of an urgent need occur to me, though, how likely are these, I wonder...?

1. How common would it be for these related functions to be plugged into the root, rather than sharing a (device internal?) bridge?

2. I imagine such devices might simply share the memory between the stages and not need DMA at all for this kind of stage-to-stage handover...?