Fedora and fallback DNS servers

Posted Feb 25, 2021 17:09 UTC (Thu) by madscientist (subscriber, #16861)
Parent article: Fedora and fallback DNS servers

Absolutely agree that default DNS servers are a bad idea. Yes, they will allow you to get access to the internet, but now your set of problems is actually much harder to understand! You can get to google.com but now your printer doesn't work, or you can't see your spouse's shared Windows drive, or whatever. The chances that you will realize that the problem is DNS, even for technical users, is somewhere between slim and none.

What's needed, instead, is better support for handling DNS problems!!! At the moment DNS is so much in the "background" that people don't even realize that it's there. This is great when it works, but DNS is also one of the more complicated things we have especially these days where people are using VPN regularly; maybe even multiple VPNs simultaneously!

We need to make it obvious what is wrong and provide easy-to-understand ways to fix the problem. And clearly, that has to be part of the base installation since as Lennart rightly says, without DNS you can't get help to fix DNS. Today there's no help available locally: THAT is the problem.

We could do better at the command line, for sure, but more importantly we need to do better at the desktop. I don't know if this falls into NetworkManager, or some separate GUI utility, but DNS troubleshooting and problem resolution must be made straightforward, and it must be installed by default. When the network is first set up, or when a user first logs in, it should be standard to check DNS connectivity and provide some kind of troubleshooting tool if it doesn't work.

Maybe we should also have the tool invoked automatically if a DNS issue is detected (obviously exactly what "is detected" means needs to be carefully considered, since people fat-finger hostnames all the time).

It cannot be that hard to create a troubleshooting tool that clearly shows which DNS servers you have configured, whether they are responding or not, and asks the user to choose one of a few different options to resolve it. It should be possible to easily explain the DNS server info: if the server IP is on the local LAN you know that's a DNS server being provided by your local router for example. If you have multiple routes (for VPN split tunneling) you can map the DNS server to one of them, and show which ones are associated with which VPN. One of the options for a solution surely would be "use default public DNS servers". And when that option is chosen the ramifications of that MUST be made clear in simple English, so people understand that when they choose this option they won't be able to see their local hosts, or if they're using VPN to get to work they won't be able to see their internal corporate hosts.

There's no question that DNS issues are some of the most frustrating, opaque, and obscure kinds of network problems we have today. I work with a group of amazingly smart software developers and many times they have no idea what is going on or how to fix it when DNS is problematic.

But, making DNS even more magic and inscrutable than it already is is NOT the way forward. Instead we need to be raising DNS issues up to the user and giving them the information and tools they need to fix it.

Fedora and fallback DNS servers

Posted Feb 25, 2021 21:49 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (9 responses)

> You can get to google.com but now your printer doesn't work, or you can't see your spouse's shared Windows drive, or whatever. The chances that you will realize that the problem is DNS, even for technical users, is somewhere between slim and none.

To the best of my understanding, most of those things use mDNS, or mDNS-like-technologies, which is completely independent of "real" DNS.

(Basically, devices regularly send broadcast packets that say "Hi, my name is..." and everything that cares listens for those packets. It's kind of horrible, but it has the advantage of nearly always working, no matter how incompetently the network is administered. It does not involve the use of real DNS on port 53 at all, and can't be broken by an invalid DNS configuration or other such nonsense.)

> It cannot be that hard to create a troubleshooting tool that clearly shows which DNS servers you have configured, whether they are responding or not, and asks the user to choose one of a few different options to resolve it. It should be possible to easily explain the DNS server info: if the server IP is on the local LAN you know that's a DNS server being provided by your local router for example. If you have multiple routes (for VPN split tunneling) you can map the DNS server to one of them, and show which ones are associated with which VPN. One of the options for a solution surely would be "use default public DNS servers". And when that option is chosen the ramifications of that MUST be made clear in simple English, so people understand that when they choose this option they won't be able to see their local hosts, or if they're using VPN to get to work they won't be able to see their internal corporate hosts.

The thing I think you are missing is that non-technical users don't want to understand the problem at all. They want to go on Facebook. That's all they ever wanted to do. They don't want to learn something, they don't want to figure out why it's broken, they just want to go on Facebook.

They will dismiss the wizard as soon as it comes up, and then they will find that they still can't go on Facebook. And then they will find my number in their phone, and call me, and I will have to spend 2+ hours first figuring out what is broken (they won't even mention the wizard, so I'm debugging from first principles here) and then explaining to them that they need to relaunch the wizard and here's how to do that.

And at the end of the whole charade, I will probably tell them to click the "use default public DNS servers" button anyway, because in practice, it will probably get them onto Facebook (and out of my hair) faster.

Fedora and fallback DNS servers

Posted Feb 25, 2021 22:25 UTC (Thu) by pizza (subscriber, #46) [Link]

> The thing I think you are missing is that non-technical users don't want to understand the problem at all. They want to go on Facebook. That's all they ever wanted to do. They don't want to learn something, they don't want to figure out why it's broken, they just want to go on Facebook.

Exactly. And for those non-technical users, the "$bigtech will know what web sites you're looking at" argument is rather ludicrous when $bigtech is the entire point of them "getting online"

Fedora and fallback DNS servers

Posted Feb 25, 2021 23:27 UTC (Thu) by nivedita76 (subscriber, #121790) [Link] (1 responses)

The thing I think you are missing is that non-technical users don't want to understand the problem at all. They want to go on Facebook. That's all they ever wanted to do. They don't want to learn something, they don't want to figure out why it's broken, they just want to go on Facebook.

But is there any evidence that the fallback ever helps non-technical users? The bug report was for a cloud instance, and the fallback was previously papering over a bug that cloud-init didn't configure DNS correctly.

Fedora and fallback DNS servers

Posted Feb 26, 2021 0:13 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

I was responding to the suggestion that we have a wizard to help users fix their DNS on the desktop. Cloud deployments obviously have different requirements.

Fedora and fallback DNS servers

Posted Feb 26, 2021 7:55 UTC (Fri) by madscientist (subscriber, #16861) [Link] (5 responses)

> To the best of my understanding, most of those things use mDNS, or mDNS-like-technologies, which is completely independent of "real" DNS.

I'm quite familiar with zeroconf / mDNS / Rendezvous. I worked writing software for routers and switches for the first half of my career. However they are not used everywhere for everything. In fact I just replace my home router last month and had this exact issue, where systems on my home network were not being seen/not available due to DNS problems.

You're missing my point. I surely understand that people want things to just work without having to mess with it. _I_ want things to just work. The question is what the best thing is to do when they DON'T work.

If it were the case that there was a simple solution with no downsides in behavior then selecting that behavior by default without checking with the user would be a good option. The problem is that choosing 8.8.8.8 makes some things work but not other things, and by using it automatically you make it even harder to figure out what is really wrong.

Please note I am NOT arguing about privacy here. I'm talking about correct behavior.

If the system magically chooses a partly-successful attempt to fix things without the user even knowing there's an issue, now you have a system that mostly works, but not completely, and figuring out why it doesn't completely work will be much much harder than it was otherwise. If someone comes to you and says "whatever host I try to reach I get an error 'hostname not found'", you pretty much know what's going on. If someone says "I can't reach the remote storage drive attached to my router from Linux but it works from my Mac", or "I can't talk to my printer from my Linux system, but it works fine from my Windows system", etc., well, it's going to take you quite a while to figure out that the reason for that is DNS related.

Automatically choosing a default is better than what we have today, which is that you're totally dead in the water and you're basically having to resort to browsing FAQs on your phone. But a much better solution than choosing a default would be to provide a simple troubleshooting facility.

Fedora and fallback DNS servers

Posted Feb 28, 2021 7:19 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (4 responses)

> In fact I just replace my home router last month and had this exact issue, where systems on my home network were not being seen/not available due to DNS problems.

Really? You have devices on your home network that are, completely automatically and with no human intervention, registering .home.arpa addresses, and the router has a DNS server which is letting them do that? That's amazing. I thought[1] this sort of thing was still just an IETF pipe dream. I hadn't realized people were actually going around implementing it.

[1]: I formed this impression by skimming RFC 7368, until I eventually realized that it is not a spec at all, but just a vague listing of the IETF's general aspirations for home networking on IPv6.

Fedora and fallback DNS servers

Posted Feb 28, 2021 13:33 UTC (Sun) by madscientist (subscriber, #16861) [Link] (3 responses)

No. I don't see where I said anything like that.

I said that local systems could not see other systems on the local network because DNS was misconfigured to not use the local server, exactly as this proposal suggests should become the fallback behavior.

Fedora and fallback DNS servers

Posted Feb 28, 2021 18:25 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (2 responses)

OK. Well, people who are technical enough to run their own custom BIND server (or whatever it is that you're doing) can also figure out how to configure DHCP correctly. I believe this proposal is "use the DHCP DNS if one is provided, and only fall back to the public servers if DHCP gives us nothing usable."

My assumption is that we start from the premise of "make it easy for non-technical users, and possible to configure for technical users." But perhaps you have a different set of priorities and if so, I don't think we have any common ground to debate.

Fedora and fallback DNS servers

Posted Mar 1, 2021 15:16 UTC (Mon) by madscientist (subscriber, #16861) [Link] (1 responses)

I am definitely not running BIND. Every home router available these days has its own DNS service available. That doesn't mean everything always works.

> I believe this proposal is "use the DHCP DNS if one is provided, and only fall back to the public servers if DHCP gives us nothing usable."

That's not clear to me: it would be interesting to know exactly WHAT the removed behavior is. The article uses terms like "fallback mechanism" and "last resort", but without actually defining what these mean. Does that mean that if there's no DNS server _configured_ then the fallback is used, so if you have configured servers but they are wrong or don't work you're back to no DNS? Or does it mean if there's no DNS server _available_ (either no configured servers OR none of the configured servers respond to DNS requests) the fallback is used? If the latter, when is this checked?

Either way, things can still go wrong.

> My assumption is that we start from the premise of "make it easy for non-technical users, and possible to configure for technical users."

The premise we start from is "DNS is not working". If DNS does not work, because we don't get the right configuration via DHCP or for some other reason, what is the best thing to do?

Fedora and fallback DNS servers

Posted Mar 5, 2021 10:12 UTC (Fri) by cortana (subscriber, #24596) [Link]

Does that mean that if there's no DNS server _configured_ then the fallback is used, so if you have configured servers but they are wrong or don't work you're back to no DNS

Yes, see resolved.conf(5):

FallbackDNS=

A space-separated list of IPv4 and IPv6 addresses to use as the fallback DNS servers. Please see DNS= for acceptable format of addresses. Any per-link DNS servers obtained from systemd-networkd.service(8) take precedence over this setting, as do any servers set via DNS= above or /etc/resolv.conf. This setting is hence only used if no other DNS server information is known. If this option is not given, a compiled-in list of DNS servers is used instead.

See also systemd-resolved(8) for a general description of how it resolves names via unicast DNS:

The following query routing logic applies for unicast DNS traffic:
If a name to look up matches (that is: is equal to or has as suffix) any of the configured routing domains (search or route-only) of any link, or the globally configured DNS settings, "best matching" routing domain is determined: the matching one with the most labels. The query is then sent to all DNS servers of any links or the globally configured DNS servers associated with this "best matching" routing domain. (Note that more than one link might have this same "best matching" routing domain configured, in which case the query is sent to all of them in parallel).
In case of single-label names, when search domains are defined, the same logic applies, except that the name is first suffixed by each of the search domains in turn. Note that this search logic doesn't apply to any names with at least one dot. Also see the discussion about compatibility with the traditional glibc resolver below.
If a query does not match any configured routing domain (either per-link or global), it is sent to all DNS servers that are configured on links with the DefaultRoute= option set, as well as the globally configured DNS server.
If there is no link configured as DefaultRoute= and no global DNS server configured, one of the compiled-in fallback DNS servers is used.
Otherwise the unicast DNS query fails, as no suitable DNS servers can be determined.