LWN: Comments on "Supporting virtual reality displays in Linux"

Supporting virtual reality displays in Linux

zlynx — Tue, 20 Mar 2018 16:43:51 +0000

Well, actually, wiring the Nvidia card to the external ports makes a lot of sense.

Not great for Linux currently, but in a world where the drivers worked for everyone it would be great.

By connecting the high power discrete card to the external port, it removes the copy latency Optimus has and does it in the case where the laptop is most likely plugged in. It also allows the Nvidia card to directly control the port, which means G-Sync is possible. I don't know if they enabled it, but it is possible.

Supporting virtual reality displays in Linux

cortana — Tue, 20 Mar 2018 12:43:21 +0000

I've used it. It was hell. Mostly I blame Lenovo for wiring up the P50 in this entirely idiotic way, that makes the external display ports only visible to the NVIDIA graphics card; and NVIDIA, for well documented reasons. Nonetheless, the seemingly endless series of poorly documented software named after children's toys has left me shaking my head in disappointment at what the world has come to; and I fled back into the arms of a disabled Intel graphics card and Xorg, which has left me with problems that are merely massively irritating (awful performance, constant 22 W power consumption and only semi-regular lockups), rather than those that left my laptop entirely unusable.

Supporting virtual reality displays in Linux

madscientist — Sat, 17 Mar 2018 18:41:15 +0000

That's a great page... but it sure would be nice if there were some sort of date or other indication of how "up to date" it is. Note for primary selection it does say "Proper wayland protocol upstreaming is still pending" but I don't know what that means for the average user.

After primary/secondary selection, for me the missing parts are per-app (not full screen) remote window display and NVidia support (when I tried to use GNOME on Wayland in Ubuntu 17.10 with my NVidia GeForce 8400GS (I know it's old but all I do is programming, not gaming--it's good enough to run two monitors) the video seemed jerky and I had problems with some apps--of course now I can't remember what they were. Remmina? Videos in Firefox? Something I needed).

I know it wasn't just my imagination because I honestly didn't realize that my desktop had been set to use wayland by default when I upgraded from Ubuntu GNOME 17.04. It wasn't until I noticed these issues and went poking around that I found this setting; changing back to GNOME on X solved my problems.

Supporting virtual reality displays in Linux

zdzichu — Sat, 17 Mar 2018 17:35:45 +0000

This was implemented couple years ago: https://fedoraproject.org/wiki/Wayland_features#BLOCKER:_...
Any real blockers?

Supporting virtual reality displays in Linux

Hi-Angel — Sat, 17 Mar 2018 17:20:55 +0000

yeah, for me the show-stopper being lack of primary clipboard.

Supporting virtual reality displays in Linux

mpr22 — Fri, 16 Mar 2018 07:54:36 +0000

As far as I can tell, my wired XBox 360 controller works perfectly with every controller-supporting game I've tried it with on Linux (up to the limits of the control scheme; Skullgirls is really designed for an arcade-style joystick, not a standard console controller). I never got killed by controller latency when playing They Bleed Pixels, for instance. Brain and muscle latency? Sure, TBP is pretty merciless even on the added-after-release easy mode. Controller or display latency? Nope.

Supporting virtual reality displays in Linux

Kamilion — Fri, 16 Mar 2018 06:23:46 +0000

Finally.

Finallllllly.

They get it. It took. YEARS.

https://lists.freedesktop.org/archives/wayland-devel/2015...

wacom tablet joystick buttons? yes! Real joysticks and gamepads? NO!

This has been going on for ages, even before I chimed in.

https://lists.freedesktop.org/archives/wayland-devel/2015...

In 2017 it was still a huge problem.

https://www.reddit.com/r/linux_gaming/comments/5xics8/a_b...

As far as I know, it's *still* a problem even outside of VR. And Packard and Arlie are still focused on the display side of things.

Please please please give input some love while you're fixing the kernel up for VR, guys...

This is one of the things you seriously need to get right for gamers, even the most casual 2D platform jumper is unplayable with terrible controls and random latency.

https://www.youtube.com/watch?v=YZ8GuuCwFAw

See how unplayable even something like mario is with poor touch control.

Input is even more important in VR; so the whole pipeline has to do it right.

Supporting virtual reality displays in Linux

gwg — Thu, 15 Mar 2018 04:40:56 +0000

> Wayland works very well now. And for the last several years.

For some peoples value of "very well". For those who need accurate color on the other hand, not so well at all ...

Supporting virtual reality displays in Linux

excors — Wed, 14 Mar 2018 01:35:43 +0000

The lenses in the headset are set up so the displays appear to be roughly 1 meter away, so it should be about the same as focusing on a regular monitor. You can wear the headset on top of regular glasses too (though you might need to move it slightly away from your face, which reduces FOV).

(I get minor problems if I pick up a virtual object and hold it close to my face (like <20cm), because the parallax is telling my brain that the object is close and so my eyes should focus nearer, but then the displays are still at effectively ~1m and are no longer in focus. If I close one eye then it removes the conflicting signals and I can focus fine again.)

Supporting virtual reality displays in Linux

JFlorian — Tue, 13 Mar 2018 14:33:29 +0000

Great article! I love reading about how processes become more "formalized", as with the device leasing here. Nothing seems as profound to me as what USB did to RS-232/422/485.

Note to Keith, you're not the last person in the universe; I still don't have a VR display nor have I even ever used one. I wear glasses, primarily for astigmatism so going w/o isn't an option for me (that I know of). I'm still utterly baffled how anybody can focus on something so close to their eyes. Anyone with pointers to interesting reading about how this works would be most welcome.

Oh and for some hi-tech nostalgia, I still have a 20" Hitachi CRT that can do 1600x1200 @ 85 Hz. I had to author my own modelines for the first 3~5 years. It's stunning for what it is, why is why I can't let it go, though it's wasted in my micro-datacenter for KVM use. I'd never even considered that anyone (e.g., gamers) might covet such a thing. I'd hate to see the bill for shipping it though!

Supporting virtual reality displays in Linux

hkario — Mon, 12 Mar 2018 19:43:04 +0000

the issue is with input-lag, not refresh rate (yes, they are similar, but they are different issues)

Supporting virtual reality displays in Linux

fratti — Sat, 10 Mar 2018 19:06:02 +0000

Looks like I need to make some corrections to this comment:

Idle inhibition is now supported by kwin.
The drag&drop on GNOME now seems to be properly working.
mpv's video-sync=display-resample can be fixed by the presentation-time protocol, if someone implements it into mpv's code.
There has been a global hotkey protocol proposal been posted a few weeks ago, which may gain adoption.

Phew, sounds like I need to try wayland again soon.

Supporting virtual reality displays in Linux

fratti — Sat, 10 Mar 2018 18:58:29 +0000

I apologise, I did not mean to incite a flamewar. I'm merely voicing my frustration at the current situation, as I'd very much like to switch away from X11 but real-world Wayland usability is seemingly being harmed by various groups exploiting the current need for new protocols as a way to play some political 4D chess.

Supporting virtual reality displays in Linux

jem — Sat, 10 Mar 2018 18:38:50 +0000

>And there's the "infinite resolution" myth. Nope. Display masks and dot pitch set very firm limits on where and how many pixels a CRT has.

In theory, a color CRT has an uninterrupted phosphor layer just like a monochrome tube. In a color tube the phospor layer alternates between red, green and blue areas, which limits the resolution of the color component. The chrominance resolution is not limited by dot size: the signal can vary within a color dot, limited by the sharpness of the electron beamand the bandwidth of the video signal.

In practice, the dots have borders which affect picure quality, and the vertical resolution is of course limited by the discrete scan lines.

Supporting virtual reality displays in Linux

tdz — Sat, 10 Mar 2018 17:09:23 +0000

I had an honest question about the status of Wayland compared to X. Let's not turn this into a flame war.

Supporting virtual reality displays in Linux

zlynx — Fri, 09 Mar 2018 22:53:27 +0000

There's a lot of superstition about CRTs out there these days too. Too many people have never used one and those that did have nostalgia.

For one thing, if a CRT wasn't designed for 75 Hz refresh, its phosphors will have too much persistence and it'll smear. On the other hand, with too short persistence and too low a refresh it'll give you a headache from flicker.

And there's the "infinite resolution" myth. Nope. Display masks and dot pitch set very firm limits on where and how many pixels a CRT has.

And the "one frame delay" of LCD and the "super low latency" of CRT thing relies on using the CRT with no vertical sync. Which yes, some gamers did, but it was and still is ugly. And anyway, it only really applied to LCDs with a VGA connection where it had to process the analog signal into pixels. Most LCD gaming displays will set pixels as they come in over DP or HDMI. There really isn't a processing delay at all.

Supporting virtual reality displays in Linux

flussence — Fri, 09 Mar 2018 22:49:54 +0000

I can already imagine another good use for this mechanism - screenlockers that actually lock the screen by switching to an isolated desktop, instead of painting a black rectangle over it and hoping someone doesn't hold down the wrong keyboard button…

Supporting virtual reality displays in Linux

fratti — Fri, 09 Mar 2018 22:34:11 +0000

Idle inhibition, server-side decoration protocol on GNOME, standardised protocol for registering global hotkeys and gestures (because voice chat applications do not want to implement compositor-specific protocols for a push-to-talk button just because nobody can agree on a standard), GNOME actually implementing the standard drag&drop protocol instead of NIH'ing their own.

Also, the last time I've tried KDE Plasma 5.12 with Wayland, my first experience was that the touchpad behaved all wrong, and the DPI was wrong as well, and the only option to "change" it was so bitmap scale the entire output by 2x which is unacceptable. Then something crashed when I tried to set the scaling back to 1x and change the font sizes instead.

Plus I don't know whether wayland even has an equivalent to xrandr to switch refresh rates during runtime, and as far as I know they still don't inform the clients about when a frame was presented so mpv's video-sync=display-resample doesn't work on it as far as I know.

If I actually used Wayland compositors regularly I'd probably find a lot more wrong with them, but all criticism of Wayland is deflected with the excuse that it's clearly not part of the core protocol and therefore not at all a wayland problem, and if applications such as games wanted title bars they should just relinquish the control over presentation to a toolkit such as Gtk, and that atomic scanout is going to be a thing Real Soon™.

Supporting virtual reality displays in Linux

flussence — Fri, 09 Mar 2018 22:27:47 +0000

Valid point, though AFAIK another reason CRTs are popular in that demographic is that they can usually go well above 60Hz - even the original Doom was designed for a 70Hz monitor.

Supporting virtual reality displays in Linux

excors — Fri, 09 Mar 2018 22:20:42 +0000

I assume the delay of an LCD is going to depend on the monitor's refresh rate, not the game's framerate, so just get a standard 144Hz gaming monitor and it'll be negligible (and perhaps even better than a 60Hz CRT).

Supporting virtual reality displays in Linux

k8to — Fri, 09 Mar 2018 21:07:44 +0000

I ordered a pre-installed debian laptop. It came with wayland by default. I didn't even realize for quite some time.

Everything worked fine, and I did my job in an officeplace with a variety of applications, though a lot of the stuff was of course browser based, as things are these days. But slack, various IDEs etc all just worked.

What problems remain?

Supporting virtual reality displays in Linux

mathstuf — Fri, 09 Mar 2018 19:48:38 +0000

> Hmm, I'm not sure about the "evidently" - I think a lot of competitive gamers' hardware choices are based more on superstition than evidence.

While I do tend to agree in general, not all games run at 60 Hz. A one-frame delay may be (more realistically) noticeable on a 24 Hz or maybe even a 30 Hz game.

Supporting virtual reality displays in Linux

zlynx — Fri, 09 Mar 2018 19:44:18 +0000

Wayland works very well now. And for the last several years.

Give Gnome on Fedora a try. If you have an AMD or Intel GPU Wayland will work great. I'm not sure where their binary Nvidia support is at, and I haven't tried it with Nouveau.

Supporting virtual reality displays in Linux

rahulsundaram — Fri, 09 Mar 2018 18:32:33 +0000

> Wayland will approximately be ready for desktop use once GNU/Hurd is.

Depends on how you use your desktop.

Supporting virtual reality displays in Linux

fratti — Fri, 09 Mar 2018 16:00:24 +0000

Wayland will approximately be ready for desktop use once GNU/Hurd is.

Supporting virtual reality displays in Linux

excors — Fri, 09 Mar 2018 15:44:33 +0000

> Those displays have an inertial measurement unit (IMU) for position and orientation tracking and a display with some optics.

Not really relevant to the article but I feel like mentioning it anyway: Good VR headsets don't just rely on the IMU for tracking. Vive requires you to put two 'lighthouse' boxes in high corners of your room, which emit IR pulses and beams that sweep across the room horizontally and vertically. Multiple IR sensors on the headset record the times when the beams reach them. The timing differences between each sensor, combined with the fixed position of each sensor relative to the headset, and some initial calibration, let you determine the absolute position and orientation of the headset. If I remember correctly they say it's accurate to about 1mm at a range of up to 5m, and it updates at 60Hz, which seems pretty decent.

> the one-frame latency introduced by using an LCD display evidently interferes with optimal game playing.

Hmm, I'm not sure about the "evidently" - I think a lot of competitive gamers' hardware choices are based more on superstition than evidence.

Supporting virtual reality displays in Linux

Plagman — Fri, 09 Mar 2018 06:32:10 +0000

Unfortunately no; you'd need a native implementation of their tracking technology and all the services provided by the Oculus Runtime on Windows; SteamVR is an interface to it when using that headset on that platform.

Supporting virtual reality displays in Linux

CCrob — Thu, 08 Mar 2018 20:22:10 +0000

Is this good for the Oculus Rift as well? I'm just rendering to a window on its display atm.

Supporting virtual reality displays in Linux

keithp — Thu, 08 Mar 2018 19:56:21 +0000

This work was sponsored by Valve; their Steam platform runs on Linux using X, so of course we did an X implementation. Plus, there's already a Vulkan extension specified which does this under X which we were able to take advantage of.

Supporting virtual reality displays in Linux

mki_n — Thu, 08 Mar 2018 19:21:27 +0000

This is awesome. Been waiting for this for ages it feels like

Supporting virtual reality displays in Linux

daniels — Thu, 08 Mar 2018 15:11:30 +0000

There's also an interface and implementation for Wayland.

Supporting virtual reality displays in Linux

tdz — Thu, 08 Mar 2018 09:31:06 +0000

That was very interesting to read. Thanks!

I'm surprised that this work was done on X. I somehow expected that X was on its way out and that all new features would be implemented on Wayland first.

Open IMUs?

kalvdans — Thu, 08 Mar 2018 06:50:47 +0000

My company looked for Augmented Reality hardware, and found the Metavision Meta2 headset. Its HDMI connection works fine, but sadly they refuse to tell you the programming interface for the IMU[1]. Can someone recommend a VR or AR headset that has a Linux-friendly IMU?

[1] https://community.metavision.com/t/usb-hardware-documenta...