SPICE can certainly work (in theory) without QEMU and other stuff. It should be fairly easy to start Wayland on an offscreen video buffer and use it for SPICE remoting.
That way you'll even get hardware graphics acceleration on your host. The only thing you'll need is a working EGL library.
What's more, you can in theory later just attach Wayland surface to a real device using the hotplug mechanism. Just imagine - you work remotely on your computer using a tablet while your wife is watching YouTube on it. Then you come to your computer and with a flick of hand transfer your workspace from offscreen buffer to the real console.