Hmm true... By modules you mean shared objects? What if the program does need pointers larger than 32 bits? Is there some gcc flag I must pull to generate narrow code, or a flag to use for wide code?
I confess my main horror with 64-bit code came from seeing a java program that took 3x the memory when it was run in 64-bit jvm compared to the 32-bit one. (Everything else being equal, just uninstalled the 64-bit one and replaced it by 32-bit jvm.) Might be it is constrained in the tricks it can do, differently from your average small app.