An ancient kernel hole is closed

Well as long as your using decent open source video drivers then your X Server can run as just a regular application _right_now_, rather then some monster that needs to fiddle with bits on your PCI bus like is traditionally needed.

It's certainly and absolutely possible.

But it probably breaks most closed source drivers so development is going to continue to be painfully slow.

There are probably a lots of problems with it, like was mentioned above with input devices, but it's absolutely possible that at some time we can have a non-root-privileged X. But people do have it working in a more-or-less fashion.