I've only gotten multi-second pauses like that when I had a hard drive going bad, or I was swapping rather furiously.
With that much RAM, I wonder if VM housekeeping itself could cause such lags. 12Gb would have 3 million 4K pages. If you are running at 3GHz and did something that averaged 3000 cycles/page across all 3 million pages, that's 3 seconds.