You are right that code which doesn't use the VFPv3 isn't specifically impacted by the VFP passing convention in section 6. But the way the ABI change was done (this isn't Fedora's issue), there was no clear set of standardized magical flags just magically inserted into ELF files to tell us whether an application actually uses FP (as opposed to just targeting the/an ABI that handles it). There is the "eabi" section, e.g.:
This will tell us what we build against but it will not tell us whether any floating point instructions were actually used (whether generated or inline assembly). Fedora is not alone in taking the action of treating this as an incompatible ABI break, and in taking an approach similar to what we have done. There were some other reasons for treating this as an entire bootstrap that will be spelled out explicitly in part 2.