Fedora and fallback DNS servers

If you control the network, you can reroute all 53/udp traffic to whatever you want, regardless of the IP address in the headers. If the traffic uses DoT or DoH, then you can't reroute it or otherwise tamper with it, again regardless of the IP address in the headers.

My conclusion is that the IP address in the headers is not relevant to the attack vector which you describe (hostile network/router, active MitM attacks, etc.), except perhaps for cases where an attacker can reroute by IP address but not by port. This should be rare, but given how frequently we see ridiculous BGP leaking/hijacking, I wouldn't put it past them...