Trinity keeps KDE 3 on life support

I guess the ideal would be to send the GPU commands over the wire to be executed on the remote end. But you'd have a protocol strongly tied to the particular hardware in use, unless you wanted to restrict yourself to OpenGL or Vulkan or whatever. And even then the software isn't written with the assumption of high latency -- it will be much too "chatty" and too likely to make synchronous calls expecting a result before proceeding. You'd have to batch up the operations somehow.

Maybe the compressed video stream you mention could actually be a list of graphics primitives the GPU has recently executed. If both sides of the remote desktop have identical hardware this "compression" could be done by the video hardware itself transparently to the software. It wouldn't be in any way a standardized protocol, and could no longer be decoded if the hardware on the other side changes, but that hardly matters for something ephemeral like a remote desktop stream. This kind of exists for GPU-intensive applications ("NVIDIA Quadro GPUs support an RDP bypass functionality allowing OpenGL applications to be fully accelerated with remote use.") but I was thinking of your ordinary desktop applications where you want a pixel-perfect, low-latency remote display.