The freedom to modify a copy of your phone's software is clearly insufficient, you also need to be able to run your modified version on the phone. You can argue that this is not about software but hardware freedom, and I would agree, but it doesn't make it any less important.