Missing the reason

True in theory, false in practice.

Things like image/video decoding, encryption/decryption, ... are still dependent on the data width. You can write a portable version of those but it won't be as performant as one coded with the data width in mind.
Similarly, 64-bits platforms also come with newer instructions and more registers, which can make the 64-bit implementation even more different than a 32-bit one.

I don't know if that affects Firefox (it quite possible it does for things like sandboxing/security), but a 64-bit pointer has some unused bits than can be used for other things (like tagging what a specific memory block is for). 32-bit pointers don't have that luxury and so require other workarounds, which are likely less performant and/or less secure. So Firefox would either need two different implementations, or it would use only one and make the 64-bit binary worse than what it could be.

And 32-bit applications are limited to <4G of RAM. This might force Firefox to implement tricks to reduce the memory usage of today's unoptimized ad-laden HTML pages.

And of course, as kanru mentioned, even if the code was 100% identical, there is still be a cost in having extra binaries to test and maintain.