Ow. Yeah, that's plausible, but unfortunately it would apply to all other shared-secret mechanisms too :/ basically if people can steal your key, you've lost. (But if they can steal your key they can presumably steal anything else they care to, as well.)