Not sure what you mean by racey. If you mean the value could change after you read it, sure, but that's not important. It could be that someone other than yourself tries to access the segment after you've checked you're the only one, but there are ways to ensure it's at least not another copy of yourself.
Given that is this situation attachments are created by fork() only (other than the initial one) if you have nattach == 1, you know there won't be another attachment other than by starting a complete new process. (The 1 is ofcourse yourself).
As for /proc/*/fd/*, that's hardly portable and more importantly, you're not required to have a file descriptor for a shared memory segment which means you need /proc/*/maps which is even less portable. Besides the fact that processes owned by other users are not examinable.