Free user space for non-graphics drivers [LWN.net]

By Jake Edge
June 3, 2020

In the kernel graphics world, there has been a longstanding "line in the sand" that disallows merging kernel drivers without a corresponding free-software user-space driver. The idea is that not having a way to test the full functionality means that the kernel developers cannot verify the proper functioning and security of the driver; changes to the kernel driver may lead to unforeseen (and untestable) problems on the user-space side. More recently, though, we have seen other types of devices with complex drivers, but no useful free user-space piece, that have been proposed for inclusion into the kernel; at least one was merged, but the tide has perhaps turned against those types of drivers at this point—or some of them, anyway.

In mid-May, Jeffrey Hugo posted an RFC patch for the "Qualcomm Cloud AI 100" device, which is a PCIe card with an application-specific integrated circuit (ASIC) that targets "deep learning" workloads. The device is also referred to as a QAIC device; it presents a modem host interface (MHI) control path and a DMA engine for the data path. These are exposed in the driver as a Linux character device with ioctl() commands to access the data path.

Dave Airlie, who drew the line in the sand back in 2010, wondered how the QAIC driver was really any different. It has a user-facing API (which he calls the "uapi") with no open-source users and no tests for it.

Although this isn't a graphics driver, and Greg [Kroah-Hartman] will likely merge anything to the kernel you throw at him, I do wonder how to validate the uapi from a security perspective. It's always interesting when someone wraps a DMA engine with user ioctls, and without enough information to decide if the DMA engine is secure against userspace misprogramming it.

Hugo said that he would like to put one of the devices "into the hands of Linaro, so that it can be put into KernelCI". In order to do that, he needs to get permission from others at Qualcomm. Beyond that, though, it will be difficult to get anything particularly useful on the user-space side:

Regarding what the community could do on its own, everything but the Linux driver is considered proprietary - that includes the on device firmware and the entire userspace stack. This is a decision above my pay grade.

I've asked for authorization to develop and publish a simple userspace application that might enable the community to do such testing, but obtaining that authorization has been slow.

Perhaps to Airlie's surprise, though, Kroah-Hartman said that he would not review the driver any further without that kind of information (and code); furthermore, he would not merge it and he would actively oppose anyone else merging it, as well.

Ok, that's a decision you are going to have to push upward on, as we really can't take this without a working, open, userspace.

Especially given the copyright owner of this code, that would be just crazy and foolish to not have open userspace code as well. Firmware would also be wonderful as well, go poke your lawyers about derivative work issues and the like for fun conversations :)

Hugo said that he would try, but did not seem to have a lot of hope that he would be able to deliver what the kernel developers are looking for. On the other hand, Daniel Vetter was surprised by Kroah-Hartman's position:

So the merge criteria for drivers/accel (atm still drivers/misc but I thought that was interim until more drivers showed up) isn't actually "totally-not-a-gpu accel driver without open source userspace".

Instead it's "totally-not-a-gpu accel driver without open source userspace" _and_ you have to be best buddies with Greg. Or at least not be on the naughty company list. Since for habanalabs all you wanted is a few test cases to exercise the ioctls. Not the entire userspace.

Vetter is referring to the Habana Labs kernel driver that was merged over his, Airlie's, and others' objections. The problems that were pointed out at the time are much the same as those raised for the QAIC driver, but it would seem the outcome is going to be different this time. Kroah-Hartman said that he was particularly concerned with "the copyright owner of this code", which must be referring to Qualcomm. The code is actually copyrighted by the Linux Foundation (Kroah-Hartman's employer) and comes from the Code Aurora Forum, which is a Linux Foundation collaborative project; the Qualcomm Innovation Center (QuIC) is a principal member of that organization.

He noted that Habana Labs has released its library code as open source since the merge, so that situation has resolved. But he did admit that he was wrong about these types of drivers:

Also, to be fair, I have changed my mind after seeing the mess of complexity that these "ioctls for everyone!" type of pass-through these kinds of drivers are creating. You were right, we need open userspace code in order to be able to properly evaluate and figure out what they are doing is right or not and be able to maintain things over time correctly.

So I was wrong, and you were right, my apologies for my previous stubbornness.

The recent submission of a DirectX driver from Microsoft is in a similar position. Sasha Levin posted an RFC patch on May 19:

The driver exposes a paravirtualized GPU to user mode applications running in a virtual machine on a Windows host. This enables hardware acceleration in environment such as WSL (Windows Subsystem for Linux) where the Linux virtual machine is able to share the GPU with the Windows host.

The use case, at least for now, is not about graphics at all; it is meant to provide a means for a Linux guest on a Windows system to access the GPU for compute purposes. But the blog post announcing the feature seems to indicate that graphics support is at least on the horizon. Vetter said that, as documented, a graphics driver needs a free user-space before it can go upstream, but that does not seem to be what is happening here:

From the blog it sounds like the userspace is all closed. That includes the hw specific part and compiler chunks, all stuff we've generally expected to be able to look [at] in the past for any kind of other driver.

He suggested that since the driver is not targeting graphics it could move elsewhere in the tree (such as drivers/hyperv). Airlie thought similarly:

This is a driver that connects a binary blob interface in the Windows kernel drivers to a binary blob that you run inside a Linux guest. It's a binary transport between two binary pieces. Personally this holds little of interest to me, I can see why it might be nice to have this upstream, but I don't [foresee] any other Linux distributor ever enabling it or having to ship it, it's purely a WSL2 pipe. I'm not saying I'd be happy to see this in the tree, since I don't see the value of maintaining it upstream, but it probably should just [exist] in a drivers/hyperv type area.

He is concerned, though, that merging it in a Hyper-V-specific part of the tree may lead to trouble down the road if graphics functionality gets added to the driver. But Microsoft's Steve Pronovost said that, while graphics support may be in the future, the driver being proposed would not be used; it is strictly meant for the GPU-compute case. There is work going on to support Linux GUI applications on WSL, but that would be done using the Remote Desktop Protocol (RDP) and a Wayland compositor; that work would be fully open source, he said.

Levin noted that he would personally prefer that the user-space pieces needed for the proposed driver were available as open source "and wish I could do something about it". He is amenable to moving the driver outside of the graphics tree, which would seem to be the path forward. He also pointed out that the RFC "is not a case of 'we want it upstream NOW' but rather 'let's work together to figure out how to do it right' :)".

It would seem, then, that the QAIC driver is going to languish, while the Microsoft driver may be able to find its way into the mainline, even though neither has any significant free user-space code that can be used to test it. In part, that may be due to the difficult relationship Qualcomm has had with the kernel development community over the years. Beyond that, the QAIC device seems to be a more complicated beast, with more ways that things can go wrong—invisibly. But the Microsoft driver has not been merged yet; we will have to wait to see whether its lack of a free user space holds it back—or not.

These are undoubtedly not the last drivers of this sort we will see. There is a trend toward devices that are programmable from user space and the kernel is simply used as a conduit to carry proprietary code and data between user space and the device; only the device maker has any real view into what is actually happening inside it. There is clearly an advantage for device makers to get their drivers into the mainline, but is there any real gain for the community by doing so? As Airlie said about the DirectX driver: "[...] I don't see the value this adds to the Linux ecosystem at all, and I think it's important when putting a burden on upstream that you provide some value."

Users who have these devices may benefit from finding the driver in their distribution kernel, and the device makers get "free maintenance" by the kernel developers, but the community is left with a pile of code that is not particularly useful—and could be fragile in ways that could cause problems in the future. It does not really sound like the usual free-software bargain. Luckily, the kernel developers provide a highly functioning platform that the device makers can use to sell and run their devices; they may just have to maintain and distribute their own drivers in order to do so.

Index entries for this article
Kernel	Development model/User-space ABI
Security	Device drivers/Accelerators

to post comments

Free user space for non-graphics drivers

Posted Jun 3, 2020 21:05 UTC (Wed) by roc (subscriber, #30627) [Link] (17 responses)

I hope these drivers at least expose enough information to know exactly which userspace memory is read or written by each ioctl (and in between ioctls as well). That information is essential for fuzzing, and also for rr.

Free user space for non-graphics drivers

Posted Jun 3, 2020 21:44 UTC (Wed) by blackwood (guest, #44174) [Link] (16 responses)

Not even close, which is the entire problem. These compute/gpu things run touring complete code with hw security features you have to take on faith, and between the ioctl stuff and the actual things you run there's generally a few layers of abstraction. E.g. on most gpus you just tell the kernel to run a buffer, which contains a cmdlist, which contains state packets, which contains pointers to more stuff, most of these containing the data, but some of it also containing the actual shaders/kernels you run on the gpu/compute hw. And this entire hairball is directly execute by hw.

Reverse-engineering this with humans and specialized fuzzing/discover tools takes a few years. Forget doing any kind of security audit without that work being done, or having proper access to both the runtime and all the compilers in userspace. Ideally also hw specs.

And all the kernel<->userspace uapi is entirely custom to the hw, and the most leaky abstraction you can think of. Stuff like which automatic clock gating bits the kernel sets impacts which instruction scheduling workarounds the compiler needs to apply to prevent the entire thing from keeling over and crashing the compute workloads userspace submits.

Free user space for non-graphics drivers

Posted Jun 3, 2020 21:49 UTC (Wed) by roc (subscriber, #30627) [Link] (3 responses)

Too bad. I guess that means when one of these drivers is accessible you may as well assume complete kernel compromise.

Being able to bound the user-space-visible effects of each system call seems like a good place to draw a line in the sand.

Free user space for non-graphics drivers

Posted Jun 3, 2020 22:08 UTC (Wed) by blackwood (guest, #44174) [Link] (2 responses)

Well this is why drivers/gpu requires fully open stack in the userspace side of things. Allows us to review the entire thing, figure out whether the security model has a reasonable chance of working. Backstop it with kernel-side validation if necessary, which has huge uapi impact since you can't validate Turing complete stuff, so you need to figure out a way to limit the Turing completeness to where it doesn't matter for security, without impacting the functionality exposed to userspace in a way that users care about.

Plus when (this isn't an if in a post smeltdown world) the hw security features turn out to be more leaky than presumed, we have a fighting chance to fix up the mess going forward.

But without all that it's indeed hopeless, and you might as well assume your kernel is compromised.

Free user space for non-graphics drivers

Posted Jun 4, 2020 15:48 UTC (Thu) by nivedita76 (subscriber, #121790) [Link] (1 responses)

I'm not really close to this stuff, but I don't see how even getting open-source userspace is really enough for security. That tells you if the vendor-provided user-space library can be compromised, but you really need to get specifications for what the hardware is capable of doing, not just what the vendor-provided userspace uses it for, since a real attacker could do something completely different with the provided ioctl's.

Free user space for non-graphics drivers

Posted Jun 4, 2020 21:26 UTC (Thu) by blackwood (guest, #44174) [Link]

Even the hw spec doesn't give you security, the hw might not live up to it, see smeltdown.

What an open stack allows you is to fix up the mess once you are aware of a security hole. Worst case you might end up with something like CONFIG_ENABLE_ROOTHOLES to not break existing uapi if the old one is unfixable. But going forward you can change both kernel and userspace driver parts to come up with something to work around the security holes. But often the breakage is really minor (like more gpu hangs in some corner case), so good enough if you just ship bugfixes on both sides. If one part is an opaque blob you can't really reason about, much less change, then that's all impossible. And vendors abandon their closed source stacks real fast, better if they sell you new hw with new drivers.

Free user space for non-graphics drivers

Posted Jun 7, 2020 8:26 UTC (Sun) by marcH (subscriber, #57642) [Link] (11 responses)

> These compute/gpu things run touring complete code with hw security features you have to take on faith, ...

So this is basically Asymetric Multi Processing (CPU and [GP]GPU) with one side running open-source (Linux) and the other side running random closed source code with little or no security boundary between the two processors. Correct?

To be honest I'm not sure how that compares from a security perspective to loading a dozen /lib/firmwares without even a debug statement in the firmware loader showing every file loaded. Which is worse? There are has been for a long time more than two processors running closed source in any modern [D]SoC; how much isolation between them? Can the kernel even tell?

[D] Distributed

Free user space for non-graphics drivers

Posted Jun 7, 2020 21:49 UTC (Sun) by marcH (subscriber, #57642) [Link]

> loading a dozen /lib/firmwares without even a debug statement in the firmware loader showing every file loaded.

Of course /lib/firmware doesn't include a number of other closed-source bits loaded even before the kernel (generally because they play a role in... booting the kernel).

https://www.ptsecurity.com/ww-en/analytics/disabling-inte...

Now the security of these is different because they can't be unsigned code, unlike most stuff in /lib/firmware I guess or the GPU/AI workloads mentioned above. But they're not open-source either.

Free user space for non-graphics drivers

Posted Jun 7, 2020 22:11 UTC (Sun) by mfuzzey (subscriber, #57966) [Link] (2 responses)

>I'm not sure how that compares from a security perspective to loading a dozen /lib/firmwares

That depends what is done with the stuff loaded from lib/firmware.

Normally it will be loaded onto some other device so the risk depends on the level of hardware access that device has to memory and the degree to which Linux can restrict it (by an IOMMU for example).

So for example firmware loaded into an I2C connected touchscreen controller is unlikely to be able to do much harm since that bus provides no DMA capacity, worst case it could simulate clicks to enter false data (assuming it knows what is on the screen).

On the other hand firmare for hardware with bus mastering DMA capabilities not restricted by an IOMMU basically has full system access.

Free user space for non-graphics drivers

Posted Jun 8, 2020 19:05 UTC (Mon) by marcH (subscriber, #57642) [Link] (1 responses)

> Normally it will be loaded onto some other device so the risk depends on the level of hardware access that device has to memory and the degree to which Linux can restrict it (by an IOMMU for example).

IOMMU is apparently hit-and-miss: https://lwn.net/Articles/782381/

> On the other hand firmare for hardware with bus mastering DMA capabilities not restricted by an IOMMU basically has full system access.

Thanks, so let's stop pretending the kernel cares about attack vectors from co-processors when it doesn't even log the filenames loaded from /lib/firmware/ by default [*]

These attack vectors and pseudo-security concerns being out of the picture, that leaves only the maintenance (troubleshooting and refactoring) questions. I suspect the kernel <-> GPU interfaces are generally much more complex than the interfaces between the kernel and stuff in /lib/firmware, hence the reluctance. Is
that higher GPU complexity why there is a "line in the sand" somewhere in the middle, do I get that right? Or because /lib/firmware has been causing fewer maintenance headaches in the past? Or both.

[*] As the distributed system bootloader that it is, in an ideal world request_firmware() should check signatures. I digress.

Free user space for non-graphics drivers

Posted Jun 8, 2020 19:43 UTC (Mon) by mfuzzey (subscriber, #57966) [Link]

> I suspect the kernel <-> GPU interfaces are generally much more complex than the interfaces between the kernel and stuff in /lib/firmware, hence the reluctance.

I think it's also a question of exposure of the interface to an attacker.

On the general /lib/firmware side the issue is more potentially malicious firmware.

Unpriveleged users can't (or shouldn't) be able to change the firmware files. Most of the time the firmware won't be receiving commands directly from userspace without the kernel sanitizing them first. In fact normally the uapi will be something standard on Linux for that subsystem and the commands sent to the firmware will be completely determined by the kernel driver.

On the GPU side though it's a bit different as unpriveleged applications can submit commands to the GPU via /dev/dri (otherwise stuff like games wouldn't work). The kernel is mostly a pipe. So a malicious actor who has knowledge of a GPU exploit could submit crafted buffers to compromise the system.

The kernel can't assume that userspace is Mesa or something trusted.

But the existence of an open source userspace, even if it is bypassed by an attacker, means that the knowledge is available for the kernel to sanitize the commands. Up to a point at least because there could still exist undiscovered holes. But it's way better than an open source kernel driver accepting opaque command buffers with no idea what they do or the security implications.

Free user space for non-graphics drivers

Posted Jun 8, 2020 2:32 UTC (Mon) by pabs (subscriber, #43278) [Link] (6 responses)

Is anyone working on reverse engineering /lib/firmware? There is precious little open firmware out there and it is mainly for increasingly obsolete hardware.

https://wiki.debian.org/Firmware/Open

Free user space for non-graphics drivers

Posted Jun 8, 2020 12:40 UTC (Mon) by pizza (subscriber, #46) [Link] (5 responses)

As a general response, nobody is working on reverse-engineering the stuff in /lib/firmware, but there are some folks working on specific devices with varying degrees of success.

The thing is, each device has to be independently reverse-engineered, a process so involved that by the time the effort yields something usable, the device will have been obsolete for a decade.

(At least these days the embedded CPUs are usually an ARM variant, or some other licensed/documented core...)

Free user space for non-graphics drivers

Posted Jun 8, 2020 12:42 UTC (Mon) by pabs (subscriber, #43278) [Link] (4 responses)

Do you have some examples of the projects you mentioned?

Free user space for non-graphics drivers

Posted Jun 8, 2020 14:26 UTC (Mon) by pizza (subscriber, #46) [Link] (3 responses)

Offhand I can only think of three that saw success:

* Broadcom wifi f(openfwwf)
* Cypress FX/FX2 (fx2lib)
* EMU10K1 (Sound Blaster Live/Audigy)

But there are others out there in varying stages of functionality.

Free user space for non-graphics drivers

Posted Jun 9, 2020 2:09 UTC (Tue) by pabs (subscriber, #43278) [Link] (2 responses)

openfwwf was already on the wiki, along with Prism54 FreeMAC, and the carl9170fw and open-ath9k-htc-firmware projects released by Atheros.

http://netweb.ing.unibs.it/~openfwwf/
https://perso.m-labs.hk/sb/prism54/freemac.html
https://perso.m-labs.hk/sb/prism54/snapshots/prism54_free...
https://github.com/qca/open-ath9k-htc-firmware
https://github.com/chunkeey/carl9170fw

I added fx2lib (and sigrok fx2lafw) to the wiki.

https://github.com/djmuhlestein/fx2lib/
https://sigrok.org/wiki/Fx2lafw

I wasn't able to find anything about the EMU10K1 work you mentioned, only about the GPLed but sourceless blobs (sigh) in the alsa-firmware git repository:

https://github.com/alsa-project/alsa-firmware/tree/master...

If you have any links to the EMU10K1 work or the others, that would be welcome.

Free user space for non-graphics drivers

Posted Jun 9, 2020 2:40 UTC (Tue) by pizza (subscriber, #46) [Link] (1 responses)

> https://perso.m-labs.hk/sb/prism54/freemac.html

I'm not sure this really belongs on the list, as they don't know how to control anything beyond the basic ARM9 core and other off-the-shelf peripherals (eg PCI and USB cores). Hardware timers and the radio itself are complete unknowns, So while technically there is "open firmware" it's good for little more than blinking LEDs.

(In a former life, I had access to the firmware source code and low-level hardware documentation for the entire p54 "Arm MAC" family..)

> I wasn't able to find anything about the EMU10K1 work you mentioned, only about the GPLed but sourceless blobs (sigh) in the alsa-firmware git repository

That firmware appears to be FPGA bitstreams for certain pro-grade E-MU hardware, and isn't needed for the prosumer SBLive/Audigy cards built on the EMU10K1/10K2 chips.

(IIRC, if you plug in a SBLIve/Audigy card, it does not rely on any external firmware, though the driver uploads little DSP snippets to implement driver-exposed functionality such as multichannel mixing and other such things. Those DSP programs were written using the 'as10k1' assembler, users could upload arbitrary programs if they so chose..)

Free user space for non-graphics drivers

Posted Jun 9, 2020 3:18 UTC (Tue) by pabs (subscriber, #43278) [Link]

Hmm, I can't find anything that uses as10k1. It seems instead the linux.git sound/pci/emu10k1/emufx.c driver has a couple of functions called by the OP/A_OP defines that use instruction set defines in the include/uapi/sound/emu10k1.h header.

Free user space for non-graphics drivers

Posted Jun 3, 2020 21:28 UTC (Wed) by iabervon (subscriber, #722) [Link] (2 responses)

It seems like Qualcomm ought to release open source user space code that programs the DMA engine and does some useless example computation. That wouldn't let people do deep learning techniques without using Qualcomm's proprietary code, but would let people verify that the driver and device are harmless.

Free user space for non-graphics drivers

Posted Jun 4, 2020 13:44 UTC (Thu) by brigdh (subscriber, #137272) [Link] (1 responses)

I proposed that, and was rejected. GregKH wants the actual library we use. I'm running it up the chain to figure out what we can do, but I expect it to take time.

Free user space for non-graphics drivers

Posted Jun 9, 2020 20:22 UTC (Tue) by marcH (subscriber, #57642) [Link]

Has such dummy user space code already been released? I guess it could still be useful for people using kernel code not merged by Greg KH. I heard rumours that not everyone uses "pure upstream" kernel code all the time (unbelievable, I know)

Maybe that limited but non-zero experience could even provide some useful lessons for all parties involved, who knows.

Free user space for non-graphics drivers

Posted Jun 4, 2020 10:22 UTC (Thu) by daniels (subscriber, #16193) [Link]

The Microsoft /dev/dxg interface is not really a 'DirectX driver' in the way that most people would interpret it. It is a direct port of their Windows DirectX kernel interfaces, but in analogue to Linux, this is DRM rather than OpenGL/Vulkan. Those interfaces allow you to allocate GPU hardware buffers (as in DRM), submit pre-compiled hardware bytecode shaders for execution (as in DRM), and synchronise against that execution (also as in DRM).

It doesn't stream DirectX client API commands over the device boundary (as VirGL does for GL/GLES). WSL2 will use a proprietary libdirectx.so, and accompanying hardware-vendor drivers, to implement the DirectX graphics API, translate that into hardware-specific commands, and then submit those commands to the kernel interface.

Pushing the comparison further, it would be entirely possible to port any of the Mesa hardware drivers - say Intel's ANV or AMD's RADV Vulkan drivers, or any of the Gallium OpenGL/GLES drivers - to run on top of /dev/dxg instead of on top of DRM kernel support.

Free user space for non-graphics drivers

Posted Jun 4, 2020 17:41 UTC (Thu) by kpfleming (subscriber, #23250) [Link] (2 responses)

Since nothing else in tree interacts with this driver, and that situation seems unlikely to change in the foreseeable future, requiring users to obtain the driver separately and install it using DKMS seems completely reasonable to me.

Free user space for non-graphics drivers

Posted Jun 6, 2020 18:51 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

Couldn't you create a subdirectory for "vendorpipedrivers", and it's made abundantly clear that anything in that directory is (a) the vendor's problem, and (b) will not be enabled by default.

Hopefully stuff will be upgraded as kernels change around it, but it can be freely redistributable GPL code, just not the kernel dev's problem - it's down to users and vendors.

Or could that be setting a dangerous precedent ...

Cheers,
Wol

Free user space for non-graphics drivers

Posted Jun 6, 2020 19:47 UTC (Sat) by jiiksteri (subscriber, #75247) [Link]

> Couldn't you create a subdirectory for "vendorpipedrivers", and it's made abundantly clear that anything in that directory is (a) the vendor's problem, and (b) will not be enabled by default.

People will ignore such abundant clarity and enable it anyway. And if _anything_ breaks they will report the breakage to their distribution / upstream, only to be asked to reproduce the problem without the offending driver. Which they can't, and since nobody apart from the vendor can debug the problem, time is wasted and nobody's better off after the exercise.

Isn't this what already happens / happened with the binary nvidia drivers and those weren't even shipped with the kernel :)

Free user space for non-graphics drivers

Posted Jun 5, 2020 4:54 UTC (Fri) by flussence (guest, #85566) [Link] (7 responses)

I just can't see a use case for a proprietary DirectX API inside a Linux VM running on a Windows 10 box. We already have Vulkan, OpenGL 4.6, OpenCL, CUDA, and an *open* D3D stack in Mesa and Wine that regularly outperforms the one microserfs play with their toys on.

Why would anyone waste the effort to write apps for a frankensteined-together mishmash of OSes that exists at the sole whim of a company infamous for its architectural ADHD? This'll be quietly left to bitrot in 3 years and all five downstream users will end up with a hard dep on Ubuntu 20.04 long after it's out of support.

Free user space for non-graphics drivers

Posted Jun 5, 2020 6:37 UTC (Fri) by tzafrir (subscriber, #11501) [Link] (6 responses)

Because Microsoft DRM would be even more confusing. See comment above: https://lwn.net/Articles/822207/

Free user space for non-graphics drivers

Posted Jun 5, 2020 7:28 UTC (Fri) by mfuzzey (subscriber, #57966) [Link] (5 responses)

Sure but in that case why implement /dev/drx at all?

Microsoft could, if the aim were GPU support in Linux guests running on a Windows host (though apparently it's not about that but just compute for the moment) implement a real Linux DRM kernel driver that talks to the Windows host kernel instead of hardware directly.

That way the linux guest could use existing Mesa userspace drivers

Free user space for non-graphics drivers

Posted Jun 5, 2020 13:39 UTC (Fri) by blackwood (guest, #44174) [Link]

The main use-case is to run the nvidia blob for CUDA and that stuff. And to run that they dig a direct path from the windows kernel gpu driver to linux, but the code that runs in linux isn't a linux driver at all, it's a windows driver. Simply recompiled so it's an ELF .so instead of PE .dll.

Even the mesa gl driver that could run on top of this is the brand new windows gl mesa driver that runs on top of directx. So really this is all about running the windows graphics stack, in linux, on top of windows. It'll never run on bare metal, and it won't look like a linux graphics stack. Hence why the wayland compositor is completely custom, and will use RDP to shovel the buffers over the wire to windows. Essentially microsoft is working on a wayland compositor for windows, using the windows directX stack and windows paravirt RDP desktop integration stuff, but then accidentally run that on linux. But still a windows graphics stack (just now also with wayland protocol support) end to end.

Whether drivers/hyperv wants that or not in upstream is entirely different question.

Free user space for non-graphics drivers

Posted Jun 5, 2020 14:31 UTC (Fri) by excors (subscriber, #95769) [Link] (2 responses)

Using DRM sounds like a lot of extra work for little benefit. The user-mode components which are being ported from Windows-only to cross-platform would have to support both DRM and D3DKMT kernel APIs, which I guess will have a lot of subtle differences and probably some major differences as they evolve and new GPU features are supported in one but not the other; then the Windows side would need to translate DRM back into D3DKMT. By using D3DKMT throughout, it's a simpler and much less bug-prone task of just proxying commands between the two kernels and recompiling the existing user-mode components for Linux with minimal API changes.

One of the major use cases seems to be CUDA (e.g. for developing and testing CUDA applications on Windows with WSL, before deploying to a Linux cloud), which is only supported on Linux by the proprietary NVIDIA drivers, so Mesa is irrelevant there. For 3D graphics, I assume there's normally some tight integration between user-mode and kernel-mode drivers, and the real kernel-mode driver (in the Windows kernel) is going to be the proprietary NVIDIA/AMD/Intel one, so you'll need the matching proprietary user-mode driver on Linux - you wouldn't be able to use Nouveau with it even if they all supported DRM.

D3D12 support on WSL doesn't sound directly useful for applications, because WSL seems like a development platform rather than a deployment platform, so games on Windows will use the native D3D12 API and games on native Linux can't use D3D12 at all (or at least Microsoft hasn't announced any plans for that yet). But D3D12 support means Microsoft can port their OpenGL-to-D3D12 translation layer (https://devblogs.microsoft.com/directx/in-the-works-openc...) from Windows to Linux with little extra effort, meaning OpenGL-based applications can be developed on WSL then deployed on Linux. (Apparently that translation layer does use the higher-level parts of Mesa, with a driver that talks to libd3d12 instead of libdrm.)

Free user space for non-graphics drivers

Posted Jun 11, 2020 16:58 UTC (Thu) by nye (guest, #51576) [Link] (1 responses)

> One of the major use cases seems to be CUDA (e.g. for developing and testing CUDA applications on Windows with WSL, before deploying to a Linux cloud), which is only supported on Linux by the proprietary NVIDIA drivers, so Mesa is irrelevant there. For 3D graphics, I assume there's normally some tight integration between user-mode and kernel-mode drivers, and the real kernel-mode driver (in the Windows kernel) is going to be the proprietary NVIDIA/AMD/Intel one, so you'll need the matching proprietary user-mode driver on Linux - you wouldn't be able to use Nouveau with it even if they all supported DRM.

> D3D12 support on WSL doesn't sound directly useful for applications, because WSL seems like a development platform rather than a deployment platform

The initial motivation for WSL appears to be just to turn Windows into a viable development platform (very successfully IMO), but I rather suspect that this specific feature is intended primarily for running real production workloads in Docker Linux containers on the same servers that are running Docker Windows containers, and with minimal (plausibly really could be negligible) performance impact.

I'm not certain what applications people are running in Docker Windows containers, or why you would actually want to do that, but it's definitely a thing so clearly there are some use cases out there.

Free user space for non-graphics drivers

Posted Jun 11, 2020 18:15 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

> I'm not certain what applications people are running in Docker Windows containers

CI processes :) . It's really nice to have a known-clean state to start from for CI. Without the unbearable slowdown/latency VMs tend to have.

Free user space for non-graphics drivers

Posted Jun 5, 2020 19:43 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

> That way the linux guest could use existing Mesa userspace drivers
That's not how DRM works. DRM basically is a "dumb pipe" for commands from userspace to the end device. So you still need a Mesa driver for that device.

Free user space for non-graphics drivers

Posted Jun 7, 2020 8:10 UTC (Sun) by marcH (subscriber, #57642) [Link] (1 responses)

> There is work going on to support Linux GUI applications on WSL, but that would be done using the Remote Desktop Protocol (RDP) and a Wayland compositor;

BTW Xming worked with WSL1 last time I checked (< 1 year ago). Just in case this sentence gave someone a wrong impression.

Free user space for non-graphics drivers

Posted Jun 8, 2020 16:19 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

Yes, TCP/IP works from WSL1 so you can use GUI apps with Cygwin/X if you're OK with no hardware acceleration or even XSHM. Presumably the focus of the new development is on making it possible to accelerate the rendering of GUIs within WSL and efficiently composite the results onto the Windows desktop.