Kernel Summit: Video Drivers [LWN.net]

This article is part of LWN's 2004 Kernel Summit coverage.

Keith Packard took time out of the Desktop Developers' Conference to talk with the kernel hackers about video drivers. He described the current situation as a nightmare, mostly as a result of the fact that video hardware is currently managed from three different places:

The kernel's framebuffer layer handles things like mode selection, and is in charge when the console is in text mode.
If DRI is being used, it feels that it is in charge of the hardware. It deals with interrupts, DMA operations, and memory management.
The X server also feels that it is in charge. It, too, does mode selection; the server also gets involved in the suspend and resume operations.

The end result is a mess where things often do not work right, the video hardware is not used to its full capability, and confusion abounds.

According to Keith, developers have not yet realized what they are dealing with. A modern video card can carry 512MB of memory, a processor which is faster than the system's CPU, and a high-speed interconnect between the two. It is, in other words, a high-performance computing system in its own right, and it needs an operating system to run it. Somebody needs to schedule the processor, and it needs a suitably smart memory manager. Users of those 512MB of onboard memory need to tracked, and some (but probably far less than half) of that memory needs to be saved when the system is suspended.

There is also the small issue of what happens when the X server tries to map all 512MB of memory into user space. The Linux kernel currently does not handle that well, and, on 32-bit systems at least, probably never will.

Video mode selection is a big problem. Video cards can be incredibly complex devices, requiring a lengthy setup process. The code to set video modes is, often, far too large to be rationally included in the kernel. So it needs to happen in user space. But that is problematic too; somebody needs to set up the console at boot time. That task cannot be handled in user space.

Just as importantly, the kernel needs to be able to get text out to the display when something goes very wrong. Currently, if the system panics while the console is in graphics mode, the only user-visible sign is that everything just stops. Most users are uninclined to do things like setting up serial consoles, so no useful bug reports get generated. It would be far better if the system could, somehow, put an oops message where the user will see it. Options include somehow forcing the card back into text mode (which can be problematic if it is in the middle of an operation), or somehow painting the message onto the display graphically.

The general consensus is that these problems can be solved, it will just require a fair amount of work. Keith is determined to rework the X server and bring modern graphics capabilities to Linux; with luck, the 2.7 kernel series will provide the support he needs to get that job done.

>> Next: Desktop performance.

(Log in to post comments)

No of course not!

Posted Jul 20, 2004 18:44 UTC (Tue) by dwheeler (guest, #1216) [Link]

You may be confusing the term "user space". By "user space" they just mean "not kernel space". For example, the X server is in user space, yet it's specially privileged. Already an ordinary user program can't access modes, etc.

No of course not!

Posted Jul 20, 2004 21:49 UTC (Tue) by dmantione (guest, #4640) [Link]

No. The X-server is not specially privileged other than that it runs as root. By
running as root it is capable of using the ioperm and other low level system
calls to get access to the hardware.

However, even though the X-server runs in user space, it's malfunction can
make the system unusable. Therefore the X-server is just as dangerous code
and must obey to the same stability standards as i.e. in-kernel network drivers.

This is bad design. All dangerous code should be run in kernel space. So,
switching a graphics card into a video mode, should not be done by the X
server. Oh and why should a program be set suid root to be able to do some
basic operation like change a video mode?

Also, being limited to user space, certain features of the graphics hardware
remain unused. For example currently, most graphics drivers resort to polling
methods to check is a device has finished its drawing operation. This is a
waste of time, GPUs can generate an interrupt when drawing is complete.

Need to transfer large amounts of data from system memory to graphics
memory? X wastes your processor cycles. Unnecessary. The GPU can do it
for you using a bus mastering operation. Besides interrupts, you need to work
with physical memory addresses etc. You don't want to do this from userspace.

However, doing this right requires very fast kernel <-> userspace interaction.
And that's the challenge for OS designers.

ROTFL

Posted Jul 21, 2004 1:37 UTC (Wed) by stuart (subscriber, #623) [Link]

"All dangerous code should be run in kernel space."

Yeah, what a plan.

Stu.

ROTFL

Posted Jul 21, 2004 4:43 UTC (Wed) by dmantione (guest, #4640) [Link]

..which of course should be interpeted as code that does stuff that can crash
the system. And not as code that is dangerous because it does contain bugs.

ROTFL

Posted Jul 21, 2004 13:43 UTC (Wed) by alonz (subscriber, #815) [Link]

Hmm... Are you proposing then that the kernel hackers stop working on initramfs, hotplug, and on moving all system initialization to user space?

Because, you know, these are dangerous... A hotplug script may reconfigure your system so it won't work anymore :-P

Seriously--there is no difference whatsoever between doing things in kernel space, and doing them in a properly-architected user space. You just have to have the correct interfaces. That's all. Kernel space is not a "magic bullet" that will cause things to be more robust.

ROTFL

Posted Jul 22, 2004 13:57 UTC (Thu) by dmantione (guest, #4640) [Link]

No, I'm not proposing that. The idea of initramfs is to move certain features to user space that are better handled in user space.

I.e. the DHCP client, the search for the root filesystem etc. Implementing a specific protocol is typically a user-space task. Many protocols exist you you don't want to have them all-in kernel, that would introduce a lot of code to maintain and any bug in kernel space is crash.

Accessing hardware in user-space is not necessarily bad. In a micro-kernel approach, hardware is accessed mostly by user space processes. Such operating systems do have a well architected user space to do hardware access, allthough features like bus mastering transfers and interrupt handled are allways troublesome from user space.

We both know that Linux is not a micro-kernel operating system. Currently Linux has nor a total userspace solution (the framebuffer device is in kernel space), nor a total kernelspace solution, since the X-server does its own initialization and mode switching.

Anyway, in a full user-space solution you would need to communicate to some privileged daemon to switch video mode etc., which is not the current situation in Linux either; everything is packed together into the X-server; there is no separation between the Windowing system and the graphics access API. I.e. a bug in the X-server, which is megabytes of resident code, can make the system unusable. Kernels are normally designed to prevent user-space programs to crash the system.

Some applications (games, competing windowing systems) don't want X. You can argue that anyone should use X, but fact is the desire is there to bypass X.

To display a console, the kernel already does need to initialize the graphics hardware. Most hardware is pretty easy to initialize, like the standard VGA. Some is more complex, like the Mach64 family; I have some code for that family in the kernel.

So, in kernel graphics is nothing revolutionary and I don't think the kernel can do without graphics support. It just needs to be developed in a fine graphics solution. Some parts of that solution belong in user space, other parts in kernel space.

What is revolutionary is that the X-server does things it shouldn't do. Allhtough the XFree86 project needs credit for the stability of their X-server, it does crash seldomly, bringing down the entire system. This is not necessary.

X crashing the system

Posted Jul 23, 2004 4:16 UTC (Fri) by giraffedata (subscriber, #1954) [Link]

I have only a passing acquaintance with video, but privilege separation always interests me.

How can X in user space crash the system? Is it something actually related to driving the video card, or just due to the fact that a superuser process can crash the system in unrelated ways?

I always thought the reason X required superuser privilege was simply that Joe User logged in from some remote terminal shouldn't have the right to mess with the system's display -- it's not his to play with.

Also, what exactly is the privilege required by X that people achieve by running it at root? Is it just CAP_SYS_RAWIO (ioperm())? Permission to a device file?

X crashing the system

Posted Jul 23, 2004 11:30 UTC (Fri) by dmantione (guest, #4640) [Link]

There are several things that can go wrong.

The first is quite obvious, if the X-server crashes, the video hardware is in an
unknown state. I.e. the console subsystem of the kernel is unable to draw
anything on the display. A good framebuffer driver might help here by the way,
but the matter of the fact is that the framebuffer devices do not reset all
graphics card registers the X-server uses. The kernel then still thinks the
console is in graphics mode, the keyboard is in full raw mode and therefore
both your display and keyboard are unusable. The system might not have
crashed in the literal sense of the word, but it is unusable.

Second, the X-server mmaps /dev/mem. Therefore the X-server has access to
physical memory. If can of a bug, the X-server can write into random memory.
Even kernel memory is possible.

The X-server also messes with clocks. Personal experience is that clocks are
very dangerous. Graphics chips block the PCI/AGP-bus if their clock signal
malfunctions, causing a lock up of the entire system. My own experience is
that you can get a crash even if you program completely valid values into the
clock registers, because the clock signal can get unstable for a few cycles.

The X-server also does a lot of tweaking on the PCI-bus, which is not very
safe either.

And there are propably still a few more ways the X-server can crash the
computer.

X crashing the system

Posted Jul 23, 2004 11:36 UTC (Fri) by dmantione (guest, #4640) [Link]

By the way, if Joe User logs in from remote, he can start extra X-servers and
terrorize the user behind the console this way, it is set suid root after all. This
is a normal system-security issue which has little to do with how X is
implemented that the Linux community hasn't really solved well yet.

X crashing the system

Posted Jul 23, 2004 15:50 UTC (Fri) by giraffedata (subscriber, #1954) [Link]

Joe User being able to start an X server against the system's console is a problem at a much higher level -- the level at which someone decided to make the X server setuid superuser. On my Linux system, it is not.

On Red Hat Linux systems I've seen, there is a setuid Xwrapper, but my understanding is that Xwrapper won't let Joe User start an X server. It has a concept of a uid that "owns" the system console and lets only that uid (or probably uid 0) start up an X server.

IIRC the Red Hat "console owner" is the first uid to log into a virtual console since boot.