I totally agree that attaining the freedom to run what we want on our phone is important. And, as tajyrink pointed out, Android and the ADP1 have some "freedom issues" (for lack of a better way of putting it) I think it is still a good starting point.
I myself use an OpenMoko Neo Freerunner running QTextended. This is working reasonable well as a daily phone (since my last one is being used by my girlfriend after her phone broke) but once the Openmoko FSO based images are stable then I will give them a try. Also, the Openmoko Wiki is a great resource.
I also ran Android on my Neo but I could not answer calls since Android assumed the presence of a physical button - which was not there. However, I will try Android again once the Neo version has matured a bit more.
All in all the fact that we can SSH into our phones is already a major milestone.