Why do they need 3D? [LWN.net]

Why do they need 3D?

Posted Nov 4, 2011 19:14 UTC (Fri) by raven667 (subscriber, #5198)
In reply to: Why do they need 3D? by slashdot
Parent article: Rawhide gets GNOME Shell for all display types

This question gets asked a lot and the answer is that modern video devices don't have any 2D acceleration or real 2D APIs anymore. One can emulate an old VGA or VESA interface in software and treat the card as a simple framebuffer, modern CPUs make that fast enough to not totally suck but that's certainly not ideal. It doesn't make sense to have a highly capable GPU attached to your system and not use it to process graphics data. Any graphics API is going to be backed by OpenGL as the standard way to access the hardware.

Why do they need 3D?

Posted Nov 5, 2011 8:59 UTC (Sat) by ikm (guest, #493) [Link] (19 responses)

> modern video devices don't have any 2D acceleration or real 2D APIs anymore

They don't. But software APIs do. So what you talk about is interesting at driver-level only. Meanwhile, users of older cards which do have 2D acceleration but limited 3D capabilities suffer.

> treat the card as a simple framebuffer, modern CPUs make that fast enough to not totally suck

I have never seen a completely unaccelerated framebuffer to perform anywhere near good. Especially when the underlying hardware can do accelerated blits and the like.

> It doesn't make sense to have a highly capable GPU attached to your system and not use it to process graphics data.

It does. It's called "conserving power". That's why we have highly capable CPUs which are idle and throttled most of the time.

Carbon dioxide aside, what if you don't have that highly capable GPU to begin with?

The same discussion again and again...

Posted Nov 5, 2011 9:58 UTC (Sat) by khim (subscriber, #9252) [Link] (18 responses)

Carbon dioxide aside, what if you don't have that highly capable GPU to begin with?

They you are screwed. End of story.

Seriously, this ship sailed long ago. One instructive example is "Android vs iOS". iOS was designed from the ground up to use OpenGL. Android was designed to support dumb devices. The end result: it took years to reach the state comparable to iOS (only Ice Cream Sanditch added GPU acceleration) and in general Android requires more power to do the same things.

> It doesn't make sense to have a highly capable GPU attached to your system and not use it to process graphics data.

It does. It's called "conserving power".

s/conserving power/wasting power/

Difference between CPU and GPU is striking here: it's not percents. It's times. For a lot of operations we are talking about 10x-20x savings. You can do 3-5 times more "work" and still save power.

> modern video devices don't have any 2D acceleration or real 2D APIs anymore

They don't. But software APIs do.

Not anymore. Intel dropped XAA, Nouveau never included it at all, Radeon drivers are still supporting it but slowly start to go in this direction too (XAA Render is not suppored on R/RV/RS[456]xx). You must support EXA, UXA, etc - and all this is kinda pointless because it does not match the underlying hardware anyway.

Why introduce additional energy-sucking complexity where it's not needed? Why not use 3D hardware as 3D hardware?

The same discussion again and again...

Posted Nov 5, 2011 10:11 UTC (Sat) by ikm (guest, #493) [Link] (4 responses)

> For a lot of operations we are talking about 10x-20x savings. You can do 3-5 times more "work" and still save power.

I miss the times when showing doodads on screen wasn't considered "work".

> Then you are screwed. End of story.

I think this summarizes this whole topic quite nicely.

The same discussion again and again...

Posted Nov 5, 2011 14:34 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (3 responses)

I never liked trying to interpret the rows of little bulbs. Everything after that has involved doodads on screens, so I think I have to disagree there.

A lot of people seem obsessed with the idea that OpenGL is specifically about 3D, as if that was somehow in contrast to 2D rather than a superset. But it isn't. These devices accelerate arbitrary vector drawing operations, to them 2D is merely a specific chosen value in one matrix. It's as though we got upset because someone introduced CPUs which do floating point, bemoaning the loss of the "good old days" when CPUs only did integer arithmetic.

[ Somewhere between when I first used it, and today, Linux got rid of its 8087 floating point emulation, implementing all the crazy features of the genuine Intel part using integer operations. I can't say I miss it ]

What is being referred to here as "2D accelerations" was actually more or less the Windows GDI operations embedded into hardware. For every operation that you might still have some use for today (say, a blit) there were a dozen more that are useless (e.g. render Windows bitmap font glyph). None of it generalises, unlike OpenGL. So we definitely don't want that "back" in the sense that we ever had it on non-Windows systems.

The same discussion again and again...

Posted Nov 5, 2011 18:54 UTC (Sat) by drag (guest, #31333) [Link] (1 responses)

> A lot of people seem obsessed with the idea that OpenGL is specifically about 3D, as if that was somehow in contrast to 2D rather than a superset. But it isn't. These devices accelerate arbitrary vector drawing operations, to them 2D is merely a specific chosen value in one matrix. It's as though we got upset because someone introduced CPUs which do floating point, bemoaning the loss of the "good old days" when CPUs only did integer arithmetic.

Pretty much.

To Elaborate:

...in terms of 'consumer desktop' hardware:

Back in the 'bad old days' you only had 2D acceleration. People wanted fast text drawing and games were all 2D. So video card providers created chipsets that accelerated 2D operations.

The next step was when 3D games first started coming out and people wanted those to be faster. At that point pretty much all effort to improve 2D only performance ceased completely.

Years after the first generation 3D acceleration video hardware came out you would still be able to find 2D acceleration engines. These 2D chipsets were just about the same as they were 2-3 hardware generations ago. All development halted years ago and this hardware hung around purely for the sake of old software compatibility. It was 'fast enough' and 2D performance was no longer something that customers gave any thought to.

Fast forward to today:

Nowadays there is no such thing as 2D chipsets. All that hardware is completely almost thing of the past. Some retrograde hardware may still have some vestiges of 2D-only acceleration, but they just really obsolete in the worst way (ie. Via chipsets).

3D acceleration is increasingly becoming a misnomer. It's effectively gone 365 degrees and now we are back to software.

What we are left with is GPUs. General purpose processor units designed for accelerating the sort of fuctions commonly used by graphics programmers. All 'hardware acceleration' means is that instead of just using the CPU to process your applications (software acceleration) your software now runs on both your GPU and your CPU.

So.....

What this means is that the amount of work required to get good 2D acceleration out of GPUs is just about the same amount of work required to get 3D acceleration out of GPUs.

That is you are not saving yourself any effort on the lower stacks by denying yourself the use of 3D APIs.

And since OpenGL acceleration is something that needs to be stable and functionally present on a modern desktop then... there is really no good reason NOT to use OpenGL, except that maybe you prefer other APIs for personal reasons or whatever.

The same discussion again and again...

Posted Nov 5, 2011 19:46 UTC (Sat) by ovitters (guest, #27950) [Link]

In addition: I do still see some issues with drivers (nvidia one). But just a bug that should be fixed in nvidia/gnome-shell/mutter/X/elsewhere. Not a design bug.

The same discussion again and again...

Posted Nov 9, 2011 13:37 UTC (Wed) by nix (subscriber, #2304) [Link]

[ Somewhere between when I first used it, and today, Linux got rid of its 8087 floating point emulation, implementing all the crazy features of the genuine Intel part using integer operations. I can't say I miss it ]

I was just digging through arch/x86/math-emu yesterday. It's still there, as is CONFIG_MATH_EMULATION.

The same discussion again and again...

Posted Nov 5, 2011 10:20 UTC (Sat) by slashdot (guest, #22014) [Link] (12 responses)

Well, because 3D hardware and APIs are primarily designed to render photorealistic 3D scenes, and in modern times to be a generic SIMD processor.

Using them directly for 2D work is going to result in very inelegant and duplicated code (especially because OpenGL is a mess); but if you use an intermediate API, then you can just add a backend for it that uses more traditional 2D APIs and doesn't require a working OpenGL driver.

The same discussion again and again...

Posted Nov 5, 2011 10:30 UTC (Sat) by slashdot (guest, #22014) [Link] (5 responses)

Or to be more specific, it seems that GNOME Shell uses Clutter.

So, just write a Cairo backend for (a subset of?) Clutter, and make sure GNOME Shell can detect that it is running that way and disable anything with strictly requires OpenGL.

The same discussion again and again...

Posted Nov 5, 2011 19:15 UTC (Sat) by drago01 (subscriber, #50715) [Link]

Or just buy a GPU ...

The same discussion again and again...

Posted Nov 7, 2011 0:03 UTC (Mon) by raven667 (subscriber, #5198) [Link] (3 responses)

Drag has it right, it doesn't make sense to implement the graphics layer twice, once using the API that the hardware implements (OpenGL) and once using a "2d" API that is not accelerated. It makes more sense to do what they are doing which is using an implementation of the graphics API which can run on the main CPU so the graphics can run either on the gpu or CPU interchangeably as fast as possible in either case

The same discussion again and again...

Posted Nov 7, 2011 6:44 UTC (Mon) by slashdot (guest, #22014) [Link] (2 responses)

2D APIs such as Cairo are normally accelerated either using OpenGL or directly accessing the 3D hardware via XRender and an EXA driver, of course.

And also they have a fast software rendering framework (Pixman) that doesn't require running a full optimizing compiler at runtime (LLVM) and is relatively simple.

Modern hardware definitely does not "implement OpenGL": OpenGL is actually an ungodly mess of extensions that requires an incredible amount of code to map to hardware.

The same discussion again and again...

Posted Nov 7, 2011 16:50 UTC (Mon) by raven667 (subscriber, #5198) [Link]

that doesn't require running a full optimizing compiler at runtime (LLVM)

I'm not an expert but I thought that graphics acceleration required a JIT compiler to translate the standard bytecode to the underlying instruction set and capabilities of the hardware. nVidia and ATI don't have the same instruction sets or system design for their GPUs. Having this process also able to target the host CPU with fully optimized fast code just makes sense.

Modern hardware definitely does not "implement OpenGL": OpenGL is actually an ungodly mess of extensions that requires an incredible amount of code to map to hardware.

You may not like aesthetically how the OpenGL API has grown organically over the years but it is the standard API for creating and running programs on the GPU. Having an optimized compiler so you can build software which targets both the GPU or CPU so graphics programs can run even when there is not dedicated hardware for them seems like a good idea.

I don't think it follows that having an incompatible "2d" backend is going to be any faster than optimizing the existing graphics API

The same discussion again and again...

Posted Nov 7, 2011 17:09 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

> And also they have a fast software rendering framework (Pixman) that doesn't require running a full optimizing compiler at runtime (LLVM) and is relatively simple.

The thing is, the future of accelerated graphics on Linux (Gallium3D) already requires something like LLVM to compile hardware-agnostic shader code into binaries which can run on the GPU. It also requires CPU-based fallbacks for operations (shader code) which the GPU doesn't support efficiently. The llvmpipe software renderer is basically just a plain Gallium3D driver which relies exclusively on the fallbacks.

"Simpler", in this case, means taking advantage of the common CPU fallback code to provide a full-featured graphics driver, not relying on a completely separate software-only rendering framework with fixed primitive operations which can only be accessed via Cairo.

Due to the modularity of LLVM, much of the effort expended on optimized code generation for the x86 backend can probably be reused for various GPU backends down the road.

The same discussion again and again...

Posted Nov 5, 2011 19:08 UTC (Sat) by drag (guest, #31333) [Link] (5 responses)

> Well, because 3D hardware

There is increasingly no longer any such thing as '2D hardware' or '3D hardware'. 2D hardware in modern devices is completely and entirely gone.

Anyways even 2D hardware that existed on older cards had zero development resources. They are just copy and pasted designs from the earliest days of the first 3D acceleration video cards.

The amount of work required to get good 2D acceleration is just about the same as having good 3D acceleration.

So no matter what API you choose you will always end up with the pretty much the same destination if it's 'accelerated'.

To simplify your question instead of trying to compare Cairo vs OpenGL you should of just asked:

"Why didn't Gnome developers decide to limit themselves to 2D acceleration hardware?"

The answer is going to be:

"Because there is no longer any such thing as 2D acceleration hardware."
(for the most part)

The same discussion again and again...

Posted Nov 6, 2011 1:36 UTC (Sun) by slashdot (guest, #22014) [Link] (4 responses)

Well, of course there is no 2D hardware at the moment, and in fact I'm not even sure if there ever was any 2D-only hardware capable of fully accelerating Render/Cairo (I guess not).

BUT the thing is that you might not have a working 3D driver (maybe because the only stable one available is proprietary and you refuse to use it, or it doesn't work on your kernel), or might be in a limited virtual machine, or you might be printing the screen contents to PDF or to an HTML/SVG page, or perhaps you want to reserve the GPU for actual 3D games or 3D modelling software and not have the desktop hog it.

In those cases, a limited 2D API is much easier to make work efficiently and in a stable way then a generic 3D one.

For example, in software, Pixman is far more stable, much simpler, more portable and probably faster than LLVMPipe (at least if you use only a single thread).

And in the "print to PDF/PostScript/HTML/SVG" case, then you absolutely NEED a 2D API, since those formats just can't do 3D rendering (excluding WebGL and other very new innovations).

The same discussion again and again...

Posted Nov 7, 2011 1:27 UTC (Mon) by drag (guest, #31333) [Link] (3 responses)

Having OpenGL API support is a basic feature that has existed for _decades_ on any respectable desktop system.

Forcing yourself to use multiple backends of APIs that are poorly suited to what you want do seems incredibly counter productive.

Why should a developer abandon the use of perfectly good APIs just because some users choose to go out and buy hardware that requires closed source software to function, then refuses to run closed source software? It's that a bit weird?

What I find disturbing is that Linux users are still willing to go out and spend good money on hardware and put a great deal of time into setting up software that will invariably lead pathologically broken setup. Then they expect that everybody must work around the problems they inflicted on themselves.

[quote]In those cases, a limited 2D API is much easier to make work efficiently and in a stable way then a generic 3D one.
[/quote]

I think what we are trying to do is point out to you that: No, this is not a true statement at all. The work required to get decent 2D acceleration on modern hardware is not any different then what it takes to get 3D acceleration.

OpenGL is not DirectX. Hardware support is entirely optional. It's just that Mesa OpenGL software reference, sucked. Now it does not suck nearly as much.

[quote]And in the "print to PDF/PostScript/HTML/SVG" case,[/quote]

A) I don't know why this case matters at all. Can you point me to a single example of people rendering their entire desktop output to PDF?

B) The choice of Gnome-shell developer's API does not impact the decision of the application developer. They can use whatever APIs they please. If they want to use something that has the ability to render their output to SVG or PDF or whatever then the Window manager does not enter into it at all.

The same discussion again and again...

Posted Nov 7, 2011 16:33 UTC (Mon) by raven667 (subscriber, #5198) [Link]

[quote]And in the "print to PDF/PostScript/HTML/SVG" case,[/quote] A) I don't know why this case matters at all. Can you point me to a single example of people rendering their entire desktop output to PDF?

AFAIK MacOSX uses PDF as its display language but it is accelerated by the GPU hardware so PDF is not mutually exclusive with using the GPU. In fact I don't understand this segue at all, what possible reason would rendering a PDF or HTML document cause you to not be able to use hardware accelerated graphics.

The same discussion again and again...

Posted Nov 9, 2011 19:31 UTC (Wed) by wookey (guest, #5501) [Link] (1 responses)

Why should a developer abandon the use of perfectly good APIs just because some users choose to go out and buy hardware that requires closed source software to function, then refuses to run closed source software?

There isn't necessarily much choice about it:

This here desktop was provided by IT in a large corp. The built-in graphics is disabled and an nvidia card shoved in. I can't change that - it's locked-down by IT.

More seriously, on ARM there are 6 different GPUs available and _all_ of them require binary drivers. I can't buy _any_ hardware that has accelerated graphics without getting into the above sticky situation. We both agree it's a crappy situation to be in, but short of telling me I should choose an entirely different CPU architecture, I can't really be blamed for _choosing_ this situation.

You can change OS, but not the hardware?

Posted Nov 9, 2011 20:41 UTC (Wed) by khim (subscriber, #9252) [Link]

This here desktop was provided by IT in a large corp. The built-in graphics is disabled and an nvidia card shoved in. I can't change that - it's locked-down by IT.

And yet you can change the OS? Hard to believe. If everything is indeed nailed down then "user" here is not you, it's your IT department. You are merely a commuter. You have my condolences but I don't see why I should jump hoops to accommodate your needs.

We both agree it's a crappy situation to be in, but short of telling me I should choose an entirely different CPU architecture, I can't really be blamed for _choosing_ this situation.

Then we should all agree that today ARM is crappy architecture - not because of CPU (which is also not all that rosy: try to find a free documentation on ARM architecture, for example), but because of GPU - and side with the Intel in it's fight with ARM. Or use binary blob if you prefer that.

You can not pay money to people who push GPUs which require binary blobs and then expressly refuse to use said blobs: you are not helping anyone because producers of said GPU don't really care all that much what you do with them if you paid good money anyway. Either start cracking blob (if you live in a country which allows such activity) or use it "as is" or stop buying this stuff. In this case you don't even have my condolences.

As I've noted before: if you've bought piece of hardware which requires binary blob - you've accepted it. Period. End of story. Live with the consequences. If you throw away said blob after the sale you'll just make your life more miserable - it does not affect the manufacturer at all.

P.S. Of course there are possibility that you've bought ARM chip for some highly specialized project and you don't actually need a GPU at all... but then the question arises: what this have to do with GNOME? GNOME is desktop environment intended for desktops, laptops - how is it relevant to your highly specialized ARM endeavour?