DNS over HTTPS in Firefox

Oops, meant "is by default decentralised" for DNS in its current form.

Mozilla has both volunteers who provide automatic data about their production version of Firefox which is aggregated and then presented in public so that anybody can look for patterns, AND volunteers running pre-production versions with deliberate test features ("Shield Studies"), in this case one of those features runs your existing resolution and DoH in parallel, throws away the answers from DoH and reports back metrics to Mozilla.

So the answer is they're collecting that data. I'm sure if the data they get says "This is slower and often doesn't work" they will accept that and re-evaluate their approach.

It's important to actually collect data, that's how we got Natural Philosophy (and so Science) as a huge upgrade over previous approaches to philosophy that proceed as you've demonstrated by arguing from premises that seem to support the philosopher's position. Among the things philosophers felt comfortable they'd figured out by your method are that the Earth is fixed at the centre of the universe, flies are spontaneously generated by the decay of living matter, and all numbers are rational.

Chrome too has programmes collecting data (although it seems Google is more comfortable just "volunteering" its own users without their knowledge), it has found all sorts of interesting things through such experiments, and since it often shares the results that lets everybody avoid getting blind-sided. For example the reason TLS 1.3 on the wire looks so silly today is that the earlier drafts (before Draft 23) were randomly enabled for Google Chrome users, and Google's metrics showed that a frighteningly high proportion failed -- even though your approach (write down axioms, scratch your beard, come to a conclusion) said that these drafts were definitely back-compatible with TLS 1.2, reality insisted otherwise. So from there experimentation found a way to sneak past most of the garbage middleboxes that weren't compatible and the draft with those changes (plus other language that doesn't change anything on the wire) is what we have today.

[ For TLS 1.2 nobody did that before publication, they relied on your approach. The standard sat on paper for _years_ largely unused, because if you tried to enable it for ordinary users a huge proportion of them got a non-working browser, and so they'd just switch to a competitor who wasn't trying to do TLS 1.2. ]

https://www.theregister.co.uk/2017/12/14/protecting_dns_p... says DNS performance is often surprisingly slow - "the 75th percentile for lookups is 181 milliseconds". (I don't know where that data originally comes from, but I don't doubt it exists). It sounds like many ISPs just implement DNS badly, and they don't care enough to improve it, and the people who do care (like end users and Mozilla) didn't have the power to fix anything. But now Mozilla can fix it by moving users onto more efficient DNS implementations.

The latency of a single DNS request isn't hugely important in a web browser - the problem is when you need to sequentially make one request for the HTML page, then another to a different domain for the JS, then to a third domain for some JSON data, then to a fourth for an image, etc. Doing the handshake per request would be silly, but you can fix that by just keeping the DoH connection alive for a few seconds - you don't need an endless flood of keep-alive packets. If it times out and you need to restart the connection when the user clicks a link an hour later, that's fine.

https://bitsup.blogspot.com/2018/05/the-benefits-of-https... notes that DoH will be able to easily transition from HTTP/2-over-TCP to HTTP/2-over-QUIC-over-UDP, for more efficient transport and better handling of packet loss (which I assume is really bad in DNS-over-UDP), and there's the possibility of providing a speculative response before the request has even been sent (if they can work out how to preserve security/privacy/etc). Since browsers and CDNs obviously want to improve HTTP performance anyway, so the protocols and implementations will already exist, and DoH only requires coordination between a single browser and a single DNS provider (not every single ISP in the world) to provide the full benefit to the browser's users, it seems a reasonably straightforward approach.

I'm not doing this work. I don't even work for Mozilla. So I don't have the data myself. But I know the people who are doing it, and I trust them and their claims.

https://bitsup.blogspot.com/2018/05/the-benefits-of-https...
This is also relevant: https://developers.google.com/speed/public-dns/docs/perfo...
The average ISP just isn't that good at managing this kind of service.

Having said that: you're right. I overstated the case to say it's "actually faster for almost all users". Sorry. It probably is for many users, but we don't have that data yet, and improvements like QUIC and push are probably needed to make it really fast.

> Cloudflare cackles as they get all the data because Mozilla buried the option to change the DoH "service" somewhere in about:config, and any alternative DoH service wouldn't have as good peerings as Cloudflare does.

Google probably does, and they're running a public DoH service: https://github.com/curl/curl/wiki/DNS-over-HTTPS#publicly...

DNS can use UDP. But if you're using DNSSEC then the replies are most likely larger than your MTU, prompting a fallback to TCP. So now you have UDP request followed by a TCP request.

Simply keeping a pool of open TLS connections is probably going to be a speed win.

Cloudflare may have a lower latency connection to the authorative nameserver (very often they are the authorative nameserver) so in practice the lookup can be a lot faster.

Hi - these are interesting thoughts that we have considered. They will need more attention as we get more experienced doing DNS with an authenticated endpoint.

In the soft fail (expected) mode, the resolver never waits for the DoH connection to be ready - if the http2 session isn't up, then it uses the os resolver. Hard fail mode will block if that's what you want but that won't be the default config. Honestly, the connection is pretty much there unless you haven't been browsing at all lately - the browser looks up a surprising number of names. (nothing about this project changes that)

the cache is interesting. Firefox has a cache of its own - meant to cover the lack of an OS cache. The cloud cache might be farther away than an ISP cache (but sometimes not), but its probably actually better populated than most ISPs but not all. It will be interesting.

Lots of ISPs just forward silently to quad 8 anyhow as they sit in the middle. (note they don't assign the end user e2e quad 8 - wonder why that is.) Especially in asia.

and tcp absolutely can outperform 1 packet per message in udp some of the time because DNS senders have absolutely terrible strategies for loss recovery and congestion control and tcp has been working on those problems for a long time. Indeed I've seen DNS stacks that actually induce their own losses (and then crudely repair them with 1second timers) through too much parallelism.. so there are some definite non obvious tradeoffs here.

I don't think you grasp just how really, really, really terrible many ISP DNS implementations are.

My ISP is one of the largest in the US, and the latency to the nameservers they put in the DHCP replies is worse than the latency to 8.8.8.8... They also hijack NXDOMAIN, and they don't play particularly nicely with DNSSEC (would probably break the revenue stream from NXDOMAIN hijacking.)