indirect rendering or direct rendering as little to do with the DRM infrastructure. Indirect rendering is when rendering goes through the X server, it's was glx extension offer. Direct rendering allow you to directly speak with the GL driver and thus stuff doesn't go through the X server.
In this picture the DRM infrastructure in kernel as no role to play, what DRM provide is use by lower level ie the GL driver or the X driver. Last you don't want X to handle memory what about GPGPU or others use of GPU ? GPU offer resource, you need a central place to allow proper sharing of resources among consumer and given the capabilities of GPU (capable of overwritting otherwise protected system memory like kernel structure) you likely don't want to have too much in userspace.