Actually I'm fairly sure the "ROM" term comes from the modding community. At least for older consoles (NES/SNES/N64/etc.), the original memory images were extracted from literal ROM chips for modification or emulation; the term "ROM" thus identifies the source of the data. Modern consoles (and Android) use optical media or SDDs / hard drives rather than actual ROMs, but the term stuck.
Of course, Android is nominally open source, so it shouldn't be necessary to mess with binary-level modifications. However, the fact that one can't actually build the unabridged equivalent of the Android system image shipping on any real hardware from just the published source code means that we are still at least partly at the "mod" stage rather than "distributions". Android has quite a bit of maturing to do to catch up with Linux on that front.