I've had experience with ARM in the past (Cirrus Logic EP9315) , It's an ARMv4T instruction set and unfortunately the support for them was pretty poor. If it'd been an ARMv5 chip support was much easier. Debian supported that as well as build root and open embedded from memory.
We ended up going x86 (currently 500mhz Geode). We weren't impressed with cirrus logics embedded linux offering (or their support and porting efforts). Whilst the x86 is an ugly instruction set , the support out there x86 for is beautiful from a developers standpoint. When it comes to linux, most of the code is developed and tested on x86 and it gets far more visibility than other architectures. I can develop on my laptop , test on it and be pretty confident I've got most of the basic bugs out , and then test on the embedded device and find the remainder. If the architectures were different then I'd feel much less confident.