So you cripple your other two machines to satisfy one?
I would suggest setting up distcc or icecream (you have 3 machines) and build a x86_64 kernel as well for the x86_64 machines. It makes the build go much faster. And you wont even need to do anything special as the userspace is all i386 anyway. But if the boxes are not all the same distro, you can use my guide to build your own gcc's for distcc kernel compiling: