If you are using Emacs 24, this is probably the new bidirectional text support. You definitely want to use the latest bzr trunk for that: it has sped up a *lot* in the last few months, to the point where I at least can no longer see the CPU load of scrolling, and it is scrolling smoothly again.
(If it's anything else, please come to emacs-devel and discuss it there!)