No, that's not why. SSH doesn't expose the local X server to remote systems by default because it's more secure to have this feature disabled unless the user has specific need for it. Not because X is inherently insecure, if anything an SSH client that did not do this would be insecure.