On second thoughts I realize that even if you knew the secret value you wouldn't be able to reverse the hash function to get the original pointers back. So if you wanted to hide the pointer values you'd need to encrypt them, with a much longer secret key than a mere 64 bits.