Shared memory mappings for devices
In a short filesystem-only discussion at the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Jérôme Glisse wanted to talk about some (more) changes to support GPUs, FPGAs, and RDMA devices. In other talks at LSFMM, he discussed changes to struct page in support of these kinds of devices, but here he was looking to discuss other changes to support mapping a device's memory into multiple processes. It should be noted that I had a hard time following the discussion in this session, so there may be significant gaps in what follows.
![[Jérôme Glisse]](https://static.lwn.net/images/2018/lsf-glisse-sm.jpg "Jérôme Glisse")
A device driver stores the device context in the private_data field of the struct file, Glisse said, which has worked well, but is now becoming a problem. There are new devices that developers want to be able to attach to an mm_struct. In addition, though, those devices are still being used by a legacy API that needs to be preserved.
Glisse said that his first idea was to associate the device context with an mm_struct. That led to various developers to try to better understand the use case. Ted Ts'o summarized what came out of that. He suggested that what Glisse wanted was for every mm_struct to have a unique ID associated with it and to store that unique ID in the device context. Any ioctl() that tried to access the device would only work if the unique ID in the mm_struct of the caller is the same as that buried in the device context. Glisse agreed that would do what he was aiming for.
Ts'o noted that you can't use the address of the mm_struct because that would vary between processes. It wouldn't necessarily be implemented as a unique ID, he said, but that is conceptually how it would work. Kent Overstreet suggested a simple global sequence number for mm_struct. Processes that shared them would have the same sequence number, so the ioctl() enforcement could be done.
After questions about where the changes might lie, Glisse said that he had not written any code yet, but that he did not think changes to the virtual filesystem (VFS) layer would be required. VFS maintainer Al Viro did not really think it mattered where the changes would be made, his question was whether the behavior is needed. Glisse said that it is; it will allow the legacy code to continue running on GPUs, while allowing for more modern uses of the devices going forward.
| Index entries for this article | |
|---|---|
| Kernel | Device drivers/Support APIs |
| Conference | Storage, Filesystem, and Memory-Management Summit/2018 |
Posted May 7, 2018 20:31 UTC (Mon)
by willy (subscriber, #9762)
[Link] (10 responses)
I have a preliminary patch to use an IDR to assign each mm_struct a u32 ID. There are various details around how quickly those can be reused. It actually saves memory (replacing the list_head with a u32). I'll try to get that finished up soon.
Posted May 7, 2018 20:42 UTC (Mon)
by luto (guest, #39314)
[Link] (2 responses)
Posted May 7, 2018 21:02 UTC (Mon)
by willy (subscriber, #9762)
[Link] (1 responses)
Posted May 7, 2018 21:37 UTC (Mon)
by luto (guest, #39314)
[Link]
Posted May 8, 2018 21:29 UTC (Tue)
by mirabilos (subscriber, #84359)
[Link] (6 responses)
Asking for my acronyms database…
Posted May 8, 2018 21:33 UTC (Tue)
by willy (subscriber, #9762)
[Link] (3 responses)
Posted May 11, 2018 7:37 UTC (Fri)
by jem (subscriber, #24231)
[Link] (2 responses)
I miss Byte magazine, which meticulously spelled out an acronym the first time it appeared in an article, no matter how common. For example: "Random Access Memory (RAM)".
Posted May 11, 2018 14:16 UTC (Fri)
by willy (subscriber, #9762)
[Link]
Posted May 15, 2018 23:52 UTC (Tue)
by giraffedata (guest, #1954)
[Link]
IBM used to do that in all its manuals as well, and it's an irritating style because acronyms are not abbreviations - they're words in their own right that are better known and have more meaning than the words from which they are derived. Sometimes the acquired meanings even contradict those words. The etymology of the acronym is still useful, of course, but the right way to do it is RAM (Random Access Memory), not Random Access Memory (RAM).
RAM is an interesting example, by the way, because words used illogically was another peeve Byte editors had and the magazine never used RAM to mean what most people meant by it: read/write memory. The term in Byte for that was "programmable memory." RAM would appear in Byte only in contrast with sequential access memory.
Posted May 11, 2018 16:44 UTC (Fri)
by cladisch (✭ supporter ✭, #50193)
[Link] (1 responses)
IDR is an ID allocator; the "R" stands for "radix tree", which is just an implementation detail. IDA is an ID allocator that does not store a data poiner for each ID.
Posted May 14, 2018 13:16 UTC (Mon)
by dezgeg (subscriber, #92243)
[Link]
Posted May 7, 2018 21:27 UTC (Mon)
by spopuri (guest, #47635)
[Link]
Posted May 8, 2018 2:39 UTC (Tue)
by ebiederm (subscriber, #35028)
[Link]
So be very careful and don't even consider making something like this visible to userspace unless you intend to make it globally unique and even then hesitate and make absolutely certain that is what you want.
Posted May 8, 2018 14:34 UTC (Tue)
by Sesse (subscriber, #53779)
[Link]
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Shared memory mappings for devices
Don't make this visible to userspace
and then it will need to get namespaced and then it's utility for device drivers will be gone.
Shared memory mappings for devices