X: 25 years and still lacking vsync'ed double buffering [LWN.net]

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 11:20 UTC (Mon) by dion (guest, #2764)
Parent article: LPC: 25 years of X

For all its glory X is still lacking the most important feature for doing flickerfree animations.

Way back when I started dabbling in graphics with DOS and VGA the learning curve was something like this:
1) Switch to graphics mode (int 0x13)
2) Putpixel (mess about in memory around 0xA0000)
3) Set pallette (port 0x3C8 & 0x3C9)
4) Draw in an off screen buffer to make the animation look good.
5) Sync to vtrace before copying the buffer to screen to avoid tearing.
6) Start doing useful work.

Now, true color displays has dispensed with #3, but #5 is still impossible to do with X and it's really sad because that's the main reason video and animations SUCK on X.

I realize that something like this DOS code would never fly on a multitasking system:
void vsync() {
/* wait until any previous retrace has ended */
do { } while (inportb(0x3DA) & 8);

/* wait until a new retrace has just begun */
do { } while (!(inportb(0x3DA) & 8));
}

... but come on! It has to be possible to do smooth full screen graphics, even with the added complexities of multitasking.

No application should ever run any other framerate than the 60-80 FPS that the display runs at, because if it does then it looks like ass, even if you do get to do proper double or tripple buffering that's synchronized to vsync.

I'm guessing the difficulty is that to know if the display is ready for a new frame you need support for the retrace interrupt and as nobody who writes drivers for X or X developers seem to care about tearing and smooth animations that little thing never gets implemented.

to post comments

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 12:31 UTC (Mon) by jdahlin (subscriber, #14990) [Link] (1 responses)

The Intel driver is automatically synced to vblank, at least for GL drawing: https://bugs.freedesktop.org/show_bug.cgi?id=20664

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 14:56 UTC (Mon) by drag (guest, #31333) [Link]

Its probably something that will be much easier to do when the X/Linux video
driver model goes into non-broke mode.

That is when we get away from X having it's own particular set of video
drivers and having multiple drivers trying to drive the same display and the
same hardware then a lot of the very hard issues like this one should be much
easier to fix. (and worth fixing)

No more X DDX, Gallium ftw.

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 13:25 UTC (Mon) by mjthayer (guest, #39183) [Link]

I haven't followed as closely as I should, but how to solve that is (was?) currently being hotly debated on xorg-devel, so it may yet appear for X.Org 1.9 or so.

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 14:11 UTC (Mon) by cortana (subscriber, #24596) [Link] (3 responses)

Mmm, what happens when you have more than one display on the same server?

If I have two TFTs, will they both read from video memory at the same time?

What about two displays running at different rates?

Of course, this shouldn't mean that the common case of a single monitor is affected by these
problems. At least you can use OpenGL to wait for a vertical retrace (modulo problems on
old/unsupported hardware)...

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 17:26 UTC (Mon) by dion (guest, #2764) [Link] (2 responses)

Well, good questions and I suspect that it's good questions like these have delayed proper vsync support in X:)

I guess every display should have its own vsynced double or triple buffer and the clients that care (most don't) should be able to get a maximum of one call-back per frame to do rendering for the next frame.

If the displays have different update rates and the window spans both displays, then I guess you're SOL:)

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 18:01 UTC (Mon) by jzbiciak (guest, #5246) [Link]

If you want the same performance as triple buffering has on a single display (ie. the renderer is never blocked by vsync), for a window straddling N displays you would need N+2 buffers. With that, you can achieve the same non-blocking performance with an arbitrary number of displays.

On a flip, the application lets X know that the buffer just completed is the "most recent" and X can switch to an undisplayed buffer. Since there are N+1 buffers other than the one just completed, there is guaranteed to be at least one available buffer.

When each of the N displays reaches its vertical retrace, it pulls the "most recent" buffer at the time of its vertical retrace for the portion of the window covering that display. In the worst case (ie. a very fast application with an absurdly high frame rate displaying on N displays whose refresh rates are effectively relatively prime to each other, which could happen if they're all driven from separate crystals), all N displays could in fact be displaying their portion from different source frames. But, you won't get any shearing within each display.

You can decrease the buffering from there, sliding down a continuum of likelihood to block the application. The minimum buffering case is still the double buffer, but you have to wait for all displays to hit vertical retrace before the application can resume. Between 2 and N+2, you can have varying levels of "likelihood to block" while the application waits for at least one buffer to be undisplayed. Chances are, even with a largish N (such as, perhaps, a 25 display system arranged as a 5x5, with nominally the same refresh rate but driven from different crystals), performance shouldn't be too bad with as few as 4 buffers, since the displays are all flipping at about the same rate, particularly if your rendering rate isn't dramatically faster than your refresh rate.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 7, 2009 4:47 UTC (Wed) by pphaneuf (guest, #23480) [Link]

I think the reason why there's no good vsync support in X is that you can't hook interrupts from user-space, and they didn't want to poll, because it would have made everything else suck.

Note that there's nothing in the X11 API that precludes using vsync to avoid tearing without exposing it in the API. What you need is just for a drawing command not to be in effect while the retrace is over the affected area.

X could implement its drawing so that this doesn't happen in the middle of one of its drawing commands, and you wouldn't get tearing. Now, you might not want a partially drawn image, even if it does so without any tearing. In that case, you can just do the double buffering yourself, by drawing into a pixmap instead of directly in the window, and XCopyArea the pixmap to the window when you're done (which is a single operation, so could be done without tearing).

The only thing that would need an extension for is throttling, so that you don't draw more than one image to your pixmap when only one copy can be made to the window. What you'd want is a XCopyArea that would send an event back to the client when it has been done. That can even support two displays with different update rates, you'd just get the event for different copies at a different rate, depending on where you put it.

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 19:14 UTC (Mon) by alankila (guest, #47141) [Link] (12 responses)

It will work with OpenGL. You just post a new frame and the hardware switches to it during the frame
sync. Since there are only two buffers in doublebuffering, it follows that GLSwapBuffers() must sleep
until the swap is possible. Looking at the wider topic: perhaps it would be better to forget about X
and instead rewrite toolkits on top of OpenGL and then have OpenGL server instead of X server...

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 20:54 UTC (Mon) by drag (guest, #31333) [Link] (9 responses)

Ya.. That was the original idea behind the now-oldschool XGL and whatnot.

Rewrite everything to run on OpenGL and then use OpenGL drivers to drive everything.

That effort is what lead to Kernel-Mode-Setting (OpenGL has no API for display control) and proper in-Kernel memory management. (having lots of applications open will use more video RAM then there is video RAM at one point or another).

Of course driving 2D displays on OpenGL is a huge pain in the ass. So that is why we have Gallium were you have 1 driver to drive multiple APIs.

API State Tracker --> Gallium Winsys --> DRI2 --> Linux-DRM --> Hardware

Instead of:
EXA/XAA/Xv --> X DDX --> Hardware
+
OpenGL --> DRI --> Linux-DRM --> Hardware
+
Codec decoding accel --> NULL
+
OpenVR --> NULL
+
OpenCL --> NULL

etc etc.

Anyways this is a better design since the world has moved on from OpenGL acceleration to much more general purpose GPUs with programmable shading languages and other things. This is leading to a full-featured GPGPUs which we will see in a year or two. This is were the GPU is just a co-processor of sorts and 'video acceleration' is just software that is compiled/designed/optimized to use the coprocessor as well as the GPU.

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 28, 2009 20:56 UTC (Mon) by drag (guest, #31333) [Link]

er..
not "coprocessor as well as the GPU"

but: GPU coprocessor as well as the CPU.

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 30, 2009 15:39 UTC (Wed) by dgm (subscriber, #49227) [Link] (7 responses)

Well, if I understood it right, XGL was about implementing X over OpenGL. What dion suggested instead was bypassing X, and make toolkits talk directly with the OpenGL driver. It has been suggested many times in GTK, as explained in this article, that also enumerates most of the reason why it's not a good idea: http://blogs.gnome.org/timj/2007/07/17/17072007-opengl-fo...

Also, network transparency and multiple displays make synchronizing with the vertical retrace a complex matter. What's more, current LCD monitors and other display technologies like e-paper do not do any vertical retrace (AFAIK).

In my opinion, what would be useful is some means to avoid unnecessary work in applications. What's the point of repainting the screen at 1200 fps when only 60 can be displayed due to limitations in the display? Wouldn't it be better to save all that wasted power?

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 7:49 UTC (Fri) by hrydgard (guest, #54085) [Link] (6 responses)

LCD monitors absolutely do vertical retrace, it's just not as visible, but
tearing remains a problem. How else would the signal get to the display, did
you think there were individual wires from each cell of framebuffer memory
to each pixel on the display?

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 17:02 UTC (Fri) by dlang (guest, #313) [Link] (4 responses)

vertical retrace interval is an artifact of the CRT and the fact that it takes time to steer the electron beam from the bottom of the display back up to the top of the display.

on an LCD it takes no more time to go from the bottom of the display to the top of the display than it takes to go from the first row to the second row.

in any case, the cause of tearing was caused by disrupting the sync signals (most commonly the horizontal sync at the beginning of each line) causing the display to 'tear' horizontally across the screen. by waiting to update the video ram until the vertical retrace period (when the CRT isn't displaying anything anyway)you avoided causing this problem.

with dual-port ram or a digital feed to the display this problem just doesn't exist

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 17:39 UTC (Fri) by mjg59 (subscriber, #23239) [Link] (3 responses)

LCDs aren't (in the relevant cases) pixel addressable devices. The framebuffer is still scanned out at the vertical refresh rate and the entire image sent to the LCD each time - starting at the top left pixel and finishing at the bottom right. There's even an interval between the end of a frame and the beginning of the next one.

Whatever the original meaning of the word "tearing" as relating to video, its use to describe the effect caused by framebuffer updates not being synchronised with scanout and the associated discontinuity in the image is well established.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 17:59 UTC (Fri) by dlang (guest, #313) [Link] (2 responses)

if a LCD is connected via a VGA interface include the vertical retrace delay, ones connected via digital interfaces do not.

having an image change halfway through an update from one frame to the next with no other distortion of the screen (and cleaned up on the next frame refresh) is a _very_ minor issue compared to loosing sync which scrambles the screen for a significant amount of time (several seconds on a LCD)

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 18:02 UTC (Fri) by mjg59 (subscriber, #23239) [Link] (1 responses)

"if a LCD is connected via a VGA interface include the vertical retrace delay, ones connected via digital interfaces do not."

They do. A *lot* depends on having that interval available.

"having an image change halfway through an update from one frame to the next with no other distortion of the screen (and cleaned up on the next frame refresh) is a _very_ minor issue compared to loosing sync which scrambles the screen for a significant amount of time (several seconds on a LCD)"

The common case is watching videos or playing games, where it's not cleared up on the next frame refresh because another update has happened in the meantime. It manifests itself as a consistent discontinuity in the image. It's greatly irritating.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 11, 2009 22:24 UTC (Sun) by Ross (guest, #4065) [Link]

> They do. A *lot* depends on having that interval available.

Yes, but the delay between updates can be very small in comparison to what are needed for CRTs. This can even be taken advantage of with normal analog VGA connections to get a higher refresh rate and/or resolution with the same pixel clock (see reduced blanking modes).

> The common case is watching videos or playing games, where it's not cleared up on the next frame refresh because another update has happened in the meantime. It manifests itself as a consistent discontinuity in the image. It's greatly irritating.

Or it may slowly move up or down the screen. It can be very distracting in the right situation.

-Ross

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 11, 2009 22:18 UTC (Sun) by Ross (guest, #4065) [Link]

But usually they retrace many strips of the screen in parallel (otherwise the refresh rate would be too slow). So it's not exactly a vertical retrace that you want to know about, but a way to avoid drawing operations during display updates.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 6:45 UTC (Fri) by Tuxie (guest, #47191) [Link] (1 responses)

This is effectively fixed by enable triple buffering in xorg.conf.

Without triple buffering, playing videos with a similar but not exact multiple of the screen Hz (like 29.97 FPS video on a 60 Hz display) can cause problems. MPlayer can come to a situation where each frame is finished just after the vblank so it will stall waiting for vblank most of the time, leaving little time left to do actual decoding, resulting in very choppy playback. This is especially visible on low Hz modes like "24p" (23.976 Hz) and HD video. Enabling triple buffering solves this.

It doesn't however fix the problem with drifting audio/video sync when video/screen Hz only almost match. For that you need either hard frameskips or syncing audio to video with resampling or AC3/DTS packet dropping/duplication, which AFAIK only recent SVN snapshots of XBMC can do on Linux systems. MPlayer and stable XBMC sync video to audio.

For some reason (Nvidia driver bug?) though, triple buffering cause problems with Nvidia hardware decoding/rendering (VDPAU) so if you rely on VDPAU to do your video playback you will want to disable it.

It will also cause a display lag of one frame so you may want to disable it for fast paced FPS games.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 8, 2009 7:52 UTC (Thu) by kragil (guest, #34373) [Link]

Triple buffering, yeah!
It is fun to see that a concept that I used in the Amiga arcade games I wrote as a kid is now a solution to X problems.

It probably is true what they say, that most of hard CS problems were solved in the 60s (I just takes a few generations to implement them everywhere)

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 29, 2009 1:24 UTC (Tue) by BenHutchings (subscriber, #37955) [Link] (4 responses)

This was actually covered in another talk on the Graphics track. Jim Gettys, also in attendance, pointed out that there has been an XSync extension for over 20 years, though XFree86/Xorg doesn't implement it. (Apparently this was introduced to support video-conferencing before there was more extensive video acceleration exposed through the XVideo extension.)

X: 25 years and still lacking vsync'ed double buffering

Posted Sep 30, 2009 18:44 UTC (Wed) by hrydgard (guest, #54085) [Link] (3 responses)

That's interesting - any idea where to get the slides from that (and other)
talk(s) from the conference?

Sorry for the almost content-less comment.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 1, 2009 1:52 UTC (Thu) by BenHutchings (subscriber, #37955) [Link]

No slides, but the notes are at http://dri.freedesktop.org/wiki/CompositeSwap

LPC Slides/Videos (including Keith's Keynote video)

Posted Oct 2, 2009 3:10 UTC (Fri) by niv (guest, #8656) [Link] (1 responses)

Most of the slides are available on our LPC Program page already, and the videos of the keynotes (and some sessions) will be up there too as soon as we get them (hoping in a week or two). Please see our call for material and feedback on the LPC blog as well. If you were at LPC (or even if you weren't), you can send us feedback on the content of the sessions and the conferenceas a whole. We'd appreciate hearing from you!

LPC Slides/Videos (including Keith's Keynote video)

Posted Oct 2, 2009 21:00 UTC (Fri) by hrydgard (guest, #54085) [Link]

Thanks for the links!

No, I didn't attend, but I'm really interested in the nice improvements that
are coming down the pipeline to Linux's video and audio systems, which both
have sadly long been messy and lagging far behind what's available on
competing platforms - from a pure desktop, not multi-user client/server,
perspective.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 1, 2009 19:10 UTC (Thu) by oak (guest, #2786) [Link] (5 responses)

Well, I think vsynching the X screen is the trivial issue as that's only a
small piece of the tear-free graphics puzzle. You need to sync every
thing in this chain with each other: application / GUI toolkit-> app
window composite buffer -> composite manager -> X screen -> monitor.

Things could get more complicated to implement when some windows are using
OpenGL for updating the content (i.e. use something else than CPU to
update it), some use Xvideo, some different GUI toolkits. These should
then be suppored by different composite managers (or window manager if one
isn't using composite manager) one can be using. And somebody above
mentioned also xinerama...

At least it's going to eat truckloads of extra memory.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 1, 2009 20:25 UTC (Thu) by dion (guest, #2764) [Link] (4 responses)

Well, tearing refers to the problem you see when someone updates the on-screen image while it's being sent to the screen, thus showing some of the old and the new frame for a while.

If all drawing to the screen happens to a back buffer and that buffer is only flipped into view during retrace then tearing cannot happen.

You are entirely correct that the entire rendering chain has to be synchronized and driven by the retrace, if you want the best possible result, but it would help a great deal if X could simply stay away from the on-screen buffer until there was a retrace.

Tearing is by far the most nasty artifact that comes from sloppy graphics work, but an application that jumps between full framerate and half framerate (say 70 and 35 Hz) looks almost as bad, so setting up a proper synchronized rendering chain would certainly be worth doing.

I don't think any of this would have to eat truck loads of memory, though, you'd certainly need to have a number (3 or more) frames buffered, but that's nothing compared to the memory on common graphics cards these days.

The buffering might not have to happen for the entire screen, you could easily have a single application (a video player, perhaps) that's the only thing that really needs to sync to vtrace and run at a fixed framerate to look good, that single app. could have a deep set of buffers, everything else could get by with the global tripple bufer.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 1, 2009 20:32 UTC (Thu) by dlang (guest, #313) [Link] (3 responses)

actually, I thought that tearing referred to the problem that (especially on older, cheaper video cards) if you updated the screen while it was being drawn it corrupted the video output so that the screen sync failed. The better video cards used dual-ported ram that could be accessed by two chips at the same time so that it could be updated and read without causing tearing.

just displaying part of the old image and part of the new image is not the same thing.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 18:26 UTC (Fri) by dion (guest, #2764) [Link]

That's a new definition for me.

It doesn't matter what it's called though, doing updates to the screen at any other time that during vsync looks like ass.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 21:03 UTC (Fri) by hrydgard (guest, #54085) [Link]

That's a new one - never heard of that problem, unless it's related to
"snow", which would happen if you changed the palette during parts of the
frame under various EGA and VGA graphics modes and has similar causes.

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 2, 2009 23:09 UTC (Fri) by cortana (subscriber, #24596) [Link]

Here's a picture illustrating the effects of tearing:

http://forum.i3d.net/attachments/technische-hulp-cod4/943...

X: 25 years and still lacking vsync'ed double buffering

Posted Oct 11, 2009 22:09 UTC (Sun) by Ross (guest, #4065) [Link]

I thought X drawing operations were are always supposed to be synched to screen refreshes. I guess that might be impractical if they take too long, but even so, you can draw into a pixmap and update the visible window with a copy operation. This should not flicker (though it may have tearing if there is constant motion and I'm wrong a about vsync).