but shrinking should never fail in a way that leaves the filesystem invalid.
block numbers will need to change, but I don't see why inode numbers would have to change (and if you don't change those, then lots of other problems vanish), they are already independent of where on the disk the data lives.
this seems fairly obvious to me, what am I missing that makes the simple approach of
identify something to move
copy the data blocks
change the block pointers to the new blocks
free the old blocks
repeat until you have moved everything
not work? (at least for the file data)
if you try to do this on a live filesystem, then you need to do a lot of locking and other changes to make sure the data doesn't change under you (and that new data doesn't go into the space you are trying to free), but if the filesystem is offline for the shrink this shouldn't be an issue.
moving metadata will be more complex, but the worst case should be that something can't be moved, and so you can't shrink the filesystem beyond that point, but there should still be no risk.