A look at terminal emulators, part 2 [LWN.net]

A look at terminal emulators, part 2

Posted Apr 16, 2018 9:08 UTC (Mon) by farnz (subscriber, #17727)
In reply to: A look at terminal emulators, part 2 by jhoblitt
Parent article: A look at terminal emulators, part 2

Visual perception is a deep and complex field; it's a long time since I studied this, so please excuse the lack of references. Underlying this is that there is an in-brain model of what you "expect" to see; your visual input is used to update that model, and things about the model that are either unexpected or an area of focus then bubble up to conscious perception. There are three numbers of interest for terminal use cases:

The minimum display time needed to trigger object recognition.
The acceptable time lag between action and response - e.g. between pressing a key, and seeing the screen update with the consequences of that keypress.
The maximum time for animation to count as "smooth" to the eye; i.e. the fusion threshold.

These numbers are all separately measurable with the right equipment, and when I studied this (around 2002), the first was single digit milliseconds in skilled object recognisers (the paper I recall, but cannot find, said that military pilots could accurately identify aircraft that appeared in their field of view for under 4ms; going lower was not possible with the equipment available to the experimenters). The second is around 100ms, but note that you have to allow for input latency, processing latency and output latency in this number, so with 4ms lag on keypresses, and 1ms processing time, you're down to 95ms for display time. The final one is also around 100ms, but is an inter-frame delay; as long as you can output a frame every 100ms or less, you can trick the visual system into seeing smooth movement.

Note, too, that there is a resolution/frame rate tradeoff available with the last one - if you have very high resolution, and use that resolution to accurately simulate the "expected" motion effects (blur etc), then you can get away with the full 100ms. If you have lower resolution, you need a lower frame time to compensate.

So, even a 16ms delay could be noticeable, if what's going on is object recognition rather than animation.

to post comments

A look at terminal emulators, part 2

Posted Apr 17, 2018 17:39 UTC (Tue) by tartley (subscriber, #96301) [Link] (5 responses)

This is not just a visual perception issue. The visuals are coupled to expectations based on the typing being done. Gamers all know that lag of these magnitude is very perceptible during gameplay, and has very substantial effect on both the effectiveness of a player, and the enjoyability of a game. It's really really annoying to play a game with a little lag injected! Personally when I played a lot of online FPS games, I would reject servers with more than 60ms lag as simply unplayable, but could strongly detect influences of lag down to about 20ms. I'm old (47), maybe younger folks would have even more discernment?

This is even with games doing lots of clever stuff to minimize the effect of lag like predicting where objects will be before receiving the confirmatory network message, so the delta's are much smaller.

I don't know if the figures I'm quoting are one way client to server lag, or round trip.

A look at terminal emulators, part 2

Posted Apr 17, 2018 17:55 UTC (Tue) by farnz (subscriber, #17727) [Link] (3 responses)

The lag you're referring to is normally round trip time between the two network endpoints. However, this comes into the 100ms "input to response" lag - the game has 100ms minus the lag you're referring to to detect your input, decide what it's doing, run physics to catch up, output a frame to the screen and have the screen output that as light to you. Input detection is typically 2ms, frame output has typically 33ms delay on a good engine, monitor adds its own delay etc.

Thus, of the 100ms input to response lag budget, you're eating 35ms in pure hardware delays (triple buffering, input latency), plus whatever lag your monitor has, plus the game's working time between input and rendering a new frame, plus the network lag. At 60ms network lag, you're saying that the game and your monitor have just 5ms from input to displaying light - even at 20ms network lag, you're only giving the game and monitor 45ms to react to your actions and display light in response.

Plus, for network games, you get added fun with the need to maintain globally synchronised state - if your opponents are getting light in response to their actions before you, they have an advantage; if this advantage is more than a small number of ms (4ms object recognition delay upper bound, remember), then they are able to recognise your player and start reacting before you have even acted on their presence.

Basically, this is very, very complex, and an utter pain to get right on a human scale.

A look at terminal emulators, part 2

Posted Apr 17, 2018 19:37 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Modern gaming monitors are 144Hz or even 244Hz on the high end, so the delay is less than 10ms for the frame display lag.

A look at terminal emulators, part 2

Posted May 21, 2018 2:46 UTC (Mon) by JanC_ (subscriber, #34940) [Link] (1 responses)

Assuming it doesn't have a processing pipeline (it might take longer than the display time of 1 frame to decode the image coming from HDMI and then apply whatever processing it has built-in to make it look "better" or "sharper" or whatever).

A look at terminal emulators, part 2

Posted May 21, 2018 11:32 UTC (Mon) by excors (subscriber, #95769) [Link]

That seems common on TVs, but then they often have a 'game mode' that disables the expensive image processing to get back to acceptable latency. Something explicitly advertised as a gaming monitor should always be low-latency, because latency is pretty much the defining characteristic of a gaming monitor. (Well, latency plus ridiculously ugly styling). E.g. the PG258Q (commonly used by pro gamers) apparently has a "display lag" of under 5ms (signal processing plus pixel response time, relative to a CRT on the same graphics card). (It also has a light on the bottom that can project a stupid logo onto your desk). And since it's 240Hz, a few frames of latency in the GPU won't add up to many msecs.

A look at terminal emulators, part 2

Posted Apr 19, 2018 8:39 UTC (Thu) by jond (subscriber, #37669) [Link]

Ah, memories of FPS in the 90s... I was using dial-up for longer than I would have liked; I had to learn to be comfortable with lag around 400ms. When I finally did upgrade to cable, it took quite some re-training (I had learned to lead my shots accounting for the lag I was used to, and similar such coping strategies)

Critical fusion frequency

Posted Apr 19, 2018 10:46 UTC (Thu) by michaelkjohnson (subscriber, #41438) [Link]

Critical fusion frequency varies substantially between people and situations. Those few of us who purchased long-phosphor monochrome VGA monitors back in the day because we found standard color VGA monitors uncomfortable were (painfully!) aware of this.

Wikipedia lists a fairly old citation: James Davis (1986), "Humans perceive flicker artefacts at 500 Hz", Sci Rep, Wiley, 5: 7861, doi:10.1038/srep07861, PMC 4314649 Freely accessible, PMID 25644611 — https://www.nature.com/articles/srep07861 "A separate line of research has reported that fast eye movements known as saccades allow simple modulated LEDs to be observed at very high rates. Here we show that humans perceive visual flicker artifacts at rates over 500 Hz when a display includes high frequency spatial edges. This rate is many times higher than previously reported." (This is why I typically drive LED PWM at 2+ kHz.)