Try RHEL or CentOS. Even better with Xen 3.3.1 from gitco.de. Google my username and xen for details.
I am using this a production rack across a few 1st gen Opterons and it is fantastic. As an aside, VMWare wont do 64bit guests on these CPUs but Xen will.