It seems like this solution is a little to close to the metal. I would be more inclined to deal with the actual event and not the keystroke. The data would be my concern and leave control of mouse/keyboards in the hands of the client itself. Perhaps I am missing a use case that would require that level.
Even in real time gaming I have graphics cached on the client and I only send information like position and direction type information to the server. The individual keystrokes and mouse events are handled completely on the client side. The results of their events are sent along the wire.