When playing just music on N900, the dynamic frequency scaling scales the CPU speed down to 250MHz (easiest to see from /proc/cpuinfo Bogomips value changes). Pulseaudio needs to do on N900 more complex/heavier stuff than normally on desktop (sound volume increase, speaker protection...).
Oprofile tells that about third of CPU goes to pulseaudio internal workings, rest is sound data manipulation which is accelerated with NEON SIMD instructions (as you can see by objdump'ing the related libraries code).
N900 uses TI Omap3 (ARM v7) i.e. it has HW floating point support. Sound HW is AFAIK 48kHz natively.