A big chunk of that server BIOS time is ECC init. It might be nice if there was some sort of BIOS flag that could tell the OS, "Hey, I have done ECC init on the first megabyte. Have fun: do the rest yourself."
Then Linux could boot and hot plug each new piece of memory as ECC init completed.
Your system would be a little odd: during the first minute it'd have less than a gigabyte, growing toward the full 64 GB as time went on.