I did a pile of benchmarking of font scrolling performance on the latest X server (now released as 1.6.0).
With the EXA glyph cache patch applied, AA and non-AA client-side fonts are nearly equivalent in performance, but there *is* a little CPU consumption from the X server which wasn't visible before, and peak scrolling speed (on my 1650x1050 widescreen, 1250MHz Athlon, Radeon 9250 AGP) is about half that of core fonts (where the fonts spray past in an utterly unreadable blur with no CPU consumption to speak of). Antialiased font rendering is actually about 5% *faster* than non-AA rendering, so I suspect the overhead of the actual antialiasing is drowned by other overheads (or perhaps it's just that antialiased rendering, as the common case, has been optimized more).
However, the difference between an utterly unreadable blur and a nearly unreadable blur isn't very significant to me: it still scrolls text off the top of the screen in a tiny fraction of a second.
(Without the EXA glyph cache, non-AA text in some cases can be hugely *slower* to render than AA text, and in both cases I was seeing >20s to scroll the screen up by a single line. But that's not AA overhead: that's transfers from VRAM, which are always going to be slow as treacle.)