DVB, header files, and user-space regressions

A regression that was recently reported for 5.14 in the media subsystem is a bit of a strange beast. The kernel's user-space binary interface (ABI) was not changed, which is the usual test for a patch to get reverted, but the report still led to a reversion. The change did lead to problems building a user-space application because it moved some header files to staging/ as part of a cleanup for a deprecated—though apparently still functioning—driver for a Digital Video Broadcasting (DVB) device. There are a few different issues tangled together here, but the reversion of a regression in the user-space API (and not ABI) is a new wrinkle.

Soeren Moch reported the regression in a lengthy message on August 11. Parts of the message were aimed at Linus Torvalds and requested the reversion of three patches committed by media subsystem maintainer Mauro Carvalho Chehab. Those patches moved the av7110 driver to staging and followed up by moving some documentation and header files there as well. But that broke the ability to build parts of the Video Disk Recorder (VDR) application, which is shipped with multiple Linux distributions, Moch said.

He pointed to a Red Hat bug that had been filed by another member of the VDR community as evidence of the problem. VDR uses header files from the uapi/linux/dvb directory; when those got moved, VDR would no longer build on the kernel used for the upcoming Fedora 35. Beyond that, Moch disagreed with moving "the long existing and working av7110 driver" to staging/.

Much of the email was addressed to Chehab; it complained about the deprecation of the existing DVB API [Update: As noted in the comments, this is the "full-featured" DVB API implemented by av7110 and used by VDR.], the removal of av7110 support, and Chehab's unwillingness to merge another driver that also implements the API. There are, Moch said, working devices and existing DVB streams that can be accessed using those drivers. He took issue with Chehab's commit message on the patch moving av7110:

Your commit message "the decoder supports only MPEG2, with is not compatible with several modern DVB streams" is at least misleading. The most popular satellite TV provider in Germany (Astra) still transmits most of the interesting programs MPEG-2 encoded, so also this is actively used and no reason to retire this card.

But Chehab sees things differently; the av7110 hardware stopped being produced 15 years ago and the DVB API that the driver implements is not useful for today's hardware:

The API that got removed was written to control the av7110 MPEG-2 decoder, and was never integrated at the DVB core: the av7110 had a driver-specific implementation inside its code.

Besides that, the API was never fully documented: there are several ioctls from the now removed API that never had any in-kernel implementation, nor had and descriptions at the specs. None of the current upstream maintainers have any [clue] about what such ioctls are supposed to do, nor do we have any av7110 hardware to test it.

There is a clear disagreement about some of that, and Moch repeatedly offers to maintain the av7110 driver going forward. He would also like to see his saa716x driver, which also implements the DVB API, get merged and said he would maintain that as well. According to Moch, there is an active community that is using the hardware with VDR and the DVB API fits its needs well. But Chehab said:

Keeping av7110 in-kernel has been a waste of limited upstream development resources. A couple of years ago, we needed to fix the av7110 API due to year-2038 issues. From time to time, we get bugs affecting it (even security ones), as the code has been bit-rotting for a long time. The most recent one probably broke the driver without nobody noticing it for a couple of Kernel reviews, as mentioned above.

Moch disputes that characterization and had some strongly worded complaints about Chehab's handling of the media subsystem, part of which he addressed to Torvalds. There is clearly a long-running dispute being aired in the thread; Torvalds stepped in to clarify a few things. He said that the regression policy is solely aimed at the ABI of the kernel, which did not change: "old programs continue to work". He also lamented that VDR was using the kernel headers directly; that is not how it is supposed to work:

We very much try to discourage user space applications from using the kernel header files directly - even projects like glibc etc are supposed to _copy_ them, not include the kernel headers.

Exactly because re-organization and changes to the kernel tree shouldn't be something that then causes random problems elsewhere that are so hard to test - and synchronize - from the kernel standpoint (or from the standpoint of the other end).

On the other hand, though, the header files in question were in the uapi directory: "while it is annoying how user space programs used them this way, I think it's also not entirely unreasonable". So he reverted the header-file move, while leaving the other changes (moving av7110 and the DVB API documentation to staging/) intact. He suggested that VDR and any other application using the header files copy them into their tree. He agreed with Chehab that moving av7110 out of the main tree was the right move:

I'm not convinced that it makes sense to move the av7110 driver back from staging - it may continue to work, but it _is_ old and there is no maintenance - and I would certainly suggest that any other out-of-tree driver that uses these old interfaces that nothing else implements shouldn't do so, considering that nothing else implements them.

Both Torvalds and Chehab seem to ignore or discount Moch's offer to maintain the driver, though he presumably can do so in staging/. Keeping the driver working is a good way to ensure that it does not get removed altogether. Moch's questions about the eventual fate of av7110 remain unanswered, however:

How long can this driver stay in staging? Would you move the driver back from staging when I do proper maintenance for it? Is it normal linux policy to remove drivers after a certain period of time, even if a driver still has users and someone that volunteers to maintain it?

So, for now at least, users of VDR will be able to build it for recent kernels; moving forward, copying the header files should ensure that it continues to build. If the driver gets removed, though, users will also need to build the driver in order to make things work. Chehab said that he is open to discussing an API for DVB, "but such API should support modern embedded hardware, and should be designed to allow it to be extended to support to future needs". That could lead to a reworked saa716x driver that would be acceptable for inclusion in the mainline, but even that hardware is quite old and it is not clear that the VDR community is interested in making those kinds of changes.

It is somewhat surprising to see a working driver with an active community getting this treatment, but Chehab is adamant that the DVB API is deprecated as part of the media subsystem. In the thread, there is a strong undercurrent of displeasure with the treatment of the DVB API over the years and with Chehab's handling of it, but Linux maintainers have a wide remit over their subsystems. The two "sides" seem pretty dug in, at this point, so it may take some other interested parties stepping up to find some kind of workable solution—or the VDR community will need to continue using drivers that are either in staging/ or out of the mainline entirely.

This event does demonstrate a bit of an expansion of the regression policy for the kernel. Even though he was not happy about the use of the header files, Torvalds did not want to break an existing user-space application, thus the revert. One wonders how many other applications or libraries have not gotten the memo about not using the kernel user-space API header files directly, however. That could lead to problems with reorganizations and cleanups down the road.