An update on input

Peter Hutterer gave an update on the input stack at the 2016 X.Org Developers Conference (XDC). A lot has been accomplished, but there is, naturally, more to do—especially as more and more quirky (or buggy) input hardware is released. But, overall Hutterer painted a picture of a mature subsystem that is largely feature-complete at this point.

Touchpads

Hutterer started with touchpads, saying that too many of them are still using the PS/2 protocol in the kernel. That protocol is limited to two touches and has limited bandwidth for reporting events. Over time the PS/2 firmware in those devices has gotten less and less testing as well.

The way forward is the RMI4 [129-page PDF] protocol for Synaptics touchpads for hardware that supports it. RMI4 is roughly six years old, so Linux is a bit behind the curve by not supporting it yet. The protocol allows for multiple fingers and has more accurate axis ranges and resolutions. The RMI4 core is in the mainline for I2C and SPI transport layers. RMI4 over HID devices is a work in progress that is just waiting for review, though some are already using RMI4 on HID over I2C. SMBus support is slated for 4.10.

Someone in the audience wondered how you would know if you need RMI4 support. Hutterer said that if you have a Synaptics touchpad, you will likely want RMI4 support. One side effect of these changes is that the touchpad device names will change. Anything that matches on the touchpad name in user space will end up with a "mute touchpad", he said.

Some new udev properties have been added for touchpads; one to identify trackball/touchpad combinations and another to identify touchpads that are integrated with the keyboard (as in a laptop), which are called "internal", versus those that are external. One use of the property is that only the internal touchpads will be disabled while the user is typing.

There have also been updates to the udev hardware database to make axis range corrections for some devices. The resolution in the database sometimes "does not match the real world". RMI4 will invalidate some of the database entries, as well, which he hopes is not a problem since RMI4 is much better protocol.

The goal for touchpad support in libinput is for it to be better than the xf86-input-synaptics driver. This is not universally true yet. One of the areas that needs work is pointer acceleration. Another is to make libinput responsive without using SIGIO, as had been done in the server in the past; more details about that would come later in the talk, he said.

The libinput developers tried to remove the touchpad hysteresis feature, which requires that a finger move a minimum distance before deciding that it actually is a movement, but that led to a whack-a-mole exercise trying to eliminate wobbly cursors for various devices. So hysteresis was eventually turned on again by default.

Another problem was that libinput would have cursor jumps that did not occur with the Synaptics driver. It turns out that changing between gestures (from two fingers to one for example) would cause the hardware to terminate the wrong finger "slot" so it would appear that the remaining finger jumped. The Synaptics driver is single-touch, so it did not experience the problem. Libinput was effectively penalized for trying to do something fancier.

Hutterer listed some quirky hardware that libinput has had to work around. Some Lenovo trackpoints send random events that trigger palm detection. Other Lenevo hardware randomly filters slower motions, which causes the cursor to stall. That was fixed by setting the acceleration slower on those devices so that users move their fingers faster and don't hit the firmware problem. It is one way to "train the users", an audience member said.

Tablets

Wacom still provides good support for its new tablets, he said. Other tablet support is "more random". It will not help that the DIGImend project, which targets better graphics tablet support for Linux, is shutting down because the maintainer doesn't have time to work on it.

The Wacom kernel driver (wacom.ko) has undergone a massive cleanup, Hutterer said. The biggest change that is visible to user space is that the LEDs are now controlled by the kernel through the LED subsystem. The driver used to require changes to support new devices, but it has switched to using HID descriptors. That allows integrated touch devices (screens and pads) to work out of the box. With luck, it will also allow future Wacom tablets to simply be plugged in and work, without needing an updated kernel for driver changes.

For libinput, tablet support is pretty much feature-complete these days. There is support for the tools, pads, and pad modes (including reporting the modes via the LEDs). Touch support has been changed so that the tablet is either treated as a giant touchpad or touchscreen. Tablet tools (e.g. stylus) are tracked with a serial number, which would allow them to be used on multiple tablets if the application supports it. The pads get their own event nodes, as do the various "rings", "strips", and buttons that are available on tablets.

For Wayland, the tablet protocol is now up to version 2. It is large, just because there is a lot of different kinds of data that is received from the tablets, but it is fairly straightforward, Hutterer said. The GTK+ tablet support is feature-complete at this point. There are still some GNOME configuration pieces that are partially missing, but otherwise there is nearly a full stack for tablets on GNOME.

Mice

He then turned to mice. The battery status handling for Logitech unifying receiver devices is being moved into the kernel. That will standardize the interface for GUIs. It is a work in progress at this point.

Libinput now provides the physical angle for mouse wheels as part of the event data. There is at least one mouse out there that has a different click angle for the horizontal wheel than it does for the vertical one—the horizontal wheel does not even have a way to do a click, but still sends those events at times. As with other devices, mice require some fixes for hardware quirks.

For Wayland, relative pointer information has been added to the protocol that provides both accelerated and unaccelerated motion data. It is something that was wanted for games, particularly "shooter games". There is also a protocol addition for pointer confinement—restricting the pointer to a particular surface (window). Both of those are not yet stable and need more testing. For XWarpPointer() support, which can't be done using the core Wayland protocol, there is a proposal to add it into XWayland.

The libratbag library, which was introduced at XDC 2015 as a library to configure gaming mice, has taken a back seat over the last year, he said. It supports quite a few different devices, with profile support so that configurations can be saved on the devices. As a follow-up to a question from last year, Hutterer told Keith Packard that the most expensive mouse the project has worked with cost $159 (up from roughly $100 last year, which Packard was incredulous about).

There is now a D-Bus daemon for libratbag (ratbagd), which will allow applications to configure these mice through the library. There is a GUI application called Piper, which is a standalone program, not part of GNOME Control Center or any other settings application. It is a rather specialized function that shouldn't be required frequently, he said. It doesn't store the configuration locally, it only changes the settings on the device itself.

One side effect of the libratbag work is a library to access Logitech unifying receivers called liblur. The idea is to hopefully replace the various home-cooked solutions out there, Hutterer said. There are "five, six, or even ten or more" of those available and there is really only a need for one library to handle operations like pairing a mouse or keyboard with a receiver.

Keyboards

The slide said "page intentionally left blank". Hutterer added: "We didn't do anything for keyboards because nothing needed to be done."

X server

Up through X server 1.18, all input events were handled using SIGIO. The user-space drivers would register a file descriptor with the server and install a SIGIO handler. Events would then trigger the signal, which would be processed in the signal handler. But there are lots of operations that cannot be done from a signal handler, memory allocation in particular.

This signal approach would not work for libinput as it needs to be able to allocate memory at times. For example, graphics tablet drivers cannot preallocate memory for all of the tools that might appear. Handling cursor movement, pointer acceleration, and other operations in the signal handler made event processing seem responsive. Not using that approach meant that libinput event handling would sometimes stall when the server was busy, which was especially noticeable when scrolling.

Thanks to Packard, there is a new input thread in the server. It simply waits for one of the input file descriptors to be ready, which makes libinput feel much more responsive. It did lead to some race conditions that needed to be tracked down, but most have been fixed at this point. The libinput test suite helped to find many of those, Hutterer said.

There is a plan to retire various specialized input drivers (e.g. Synaptics, Wacom) in favor of libinput. Traditionally, the evdev driver has been the catch-all driver and there have been some specialized drivers loaded for specific types of devices. Now, though, libinput is also a catch-all driver, but should be preferred over the specialized drivers. The X.Org configuration mechanism (xorg.conf.d) is not flexible enough to support this, however. It is based solely on the sort order of the files in the directory.

So, in server 1.19, there is now support for a new NoMatch directive that will only match a specialized driver if libinput has not already been attached to the device. In addition, the X server now has a built-in fallback driver to use if the driver module chosen is not available. In hindsight, he said, that should have been added in "2007 or so".

There has been some reshuffling of the entries in xorg.conf.d so they sort differently, as well. Instead of being near the bottom priority-wise (i.e. 98), libinput has been moved to 40, which puts it just below evdev and above all of the other specialized drivers. He noted that caused problems for desktop environments that do not support libinput (e.g. Cinnamon and MATE) and he strongly recommended that desktops move to support the library.

In summary, Hutterer said, input developers "are reaching peak boredom". There are mostly just little tweaks for hardware quirks and so on; there are no major features on the horizon. "That's a good thing", he said, since most "sane people" don't want their touchpads, mice, and other input devices to be exciting. For more details, interested readers can consult the slides or YouTube video from the talk.

[I would like to thank the X.Org Foundation for sponsoring my travel to Helsinki for XDC.]