Ya.. That was the original idea behind the now-oldschool XGL and whatnot.
Rewrite everything to run on OpenGL and then use OpenGL drivers to drive everything.
That effort is what lead to Kernel-Mode-Setting (OpenGL has no API for display control) and proper in-Kernel memory management. (having lots of applications open will use more video RAM then there is video RAM at one point or another).
Of course driving 2D displays on OpenGL is a huge pain in the ass. So that is why we have Gallium were you have 1 driver to drive multiple APIs.
API State Tracker --> Gallium Winsys --> DRI2 --> Linux-DRM --> Hardware
Anyways this is a better design since the world has moved on from OpenGL acceleration to much more general purpose GPUs with programmable shading languages and other things. This is leading to a full-featured GPGPUs which we will see in a year or two. This is were the GPU is just a co-processor of sorts and 'video acceleration' is just software that is compiled/designed/optimized to use the coprocessor as well as the GPU.