LWN: Comments on "Making WiFi fast"

Making WiFi fast

mtaht — Mon, 30 Mar 2020 19:11:27 +0000

I just wanted to note that AQL finally landed for the ath10k in openwrt head yesterday, and the results are *wonderful*.

Making WiFi fast

mtaht — Fri, 06 Sep 2019 21:58:23 +0000

That paper was ultimately published as "Ending the anomaly" - the capstone to solving a 16+ year old problem in wifi that nobody, until us - had figured out how to solve.

https://www.usenix.org/system/files/conference/atc17/atc1...

Making WiFi fast

gdt — Sat, 09 Mar 2019 01:29:02 +0000

A reminder that wired connections also have downsides, mostly its inconvenience and cost.

A good RJ-45 jack is rated for 2,500 cycles. So wireless is a much better fit for high-traffic areas such as cafes and libraries. Patch leads are a small but ongoing expense, and staff and students don't like "BYO patch lead".

A wired port costs around $200 per wallport to cable. But this can blow out when a custom solution is required. Wiring a cafe table will cost more than than table.

Wireless networks work without any further action by the user. Once set up (which is far too hard) Eduroam connects your laptop or phone to the campus network the moment you go to use the device. No searching for a jack and patch lead. Wireless is so convenient that it's common to see a person sitting next to a wall port but using wireless.

Wired from modern devices is difficult. Using wired ethernet from a phone or tablet requires special cabling (a OTG cable) to the ethernet dongle. The dongle itself is a optional purchase. Cheaper dongles meant for laptops might not have driver support in a phone. Using wired ethernet from a recent laptop requires a USB-C/ethernet dongle, which means the laptop can't be powered whilst using the wired network. To have both power and wired networking requires a bulky and expensive "docking station".

We should be telling people who need network performance to use wired. But that may not end up being the bulk of the connections on a campus network.

Making WiFi fast

mtaht — Wed, 28 Dec 2016 17:48:14 +0000

This is overlong, but does do a "typical family" evaluation, as you suggest.

http://caia.swin.edu.au/reports/161107A/CAIA-TR-161107A.pdf

Making WiFi fast

Wol — Fri, 18 Nov 2016 20:49:19 +0000

So, you have "Taht == Taht".

Okay, it won't pick up a mis-spelt "that" at the start of a sentence, but when was that ever "proper" English". :-)

Iirc the complete rule, WordPerfect ignored case if the dictionary entry was all lower case. As soon as you had mixed or upper case, it had to be a perfect match.

Cheers,
Wol

Making WiFi fast

zlynx — Fri, 18 Nov 2016 08:34:44 +0000

The 5GHz fade is also a great thing in apartment buildings. All of your neighbors have a WiFi router that they picked up somewhere and they managed to make it though the configuration wizard. So every single apartment is at 100% transmission power, all of the time. Having the walls cut that down (and having all of the extra channels) really helps.

Making WiFi fast

Sertorius — Fri, 18 Nov 2016 07:11:55 +0000

If you're going to make comments like that, please make sure you actually *know* the physics. The relevant equation in this case, the Friis path loss equation, has a lambda squared on the top, or if you prefer f squared on the bottom. So yes, path loss is significantly lower at lower frequencies; this is the reason that satellites use the lower of a pair of frequencies to transmit (because they are power-constrained); likewise frequency-division duplex phones will use the lower frequency channel for the uplink (again, power-constrained). This is also the best-case scenario; usually the path loss exponent is higher than 2 due to multipath fading (due to reflections).

Here's a graph if you aren't convinced.

5 GHz is severely attenuated by relatively mild obstructions (such as gyprock/drywall or timber) that 2.4 penetrates very easily. If you have concrete or brick walls, you'll want an AP in every room.

The main benefits of 5 GHz are that you have a lot more non-overlapping channels, so it is easier to avoid interference - it is also good if you have a lot of users to support and want to have a LOT of short-range APs.

Making WiFi fast

neilbrown — Fri, 18 Nov 2016 04:47:42 +0000

> In the vast majority of cases you want the spellchecker to automagically correct taht to that.

Speak for yourself :-)

In every single imaginable case I do *not* want any spellchecker to automagically correct anything.
I'm very happy for possible errors to be highlighted and for probable corrections to be only a gesture away. But if I ever make an error (whch I do), I want it to be *my* error, not a machine's.

Making WiFi fast

zlynx — Thu, 17 Nov 2016 23:44:02 +0000

Yes but the vast majority of words which are capitalized and not at the beginning of the sentence are proper nouns which are often spelled differently from regular words.

Examples: Kaycee, Maryanne, Marianne, Marian, Britney, Britnee, Destynie.

Making WiFi fast

mtaht — Thu, 17 Nov 2016 21:19:34 +0000

In the vast majority of cases you want the spellchecker to automagically correct taht to that.

Making WiFi fast

Wol — Thu, 17 Nov 2016 10:06:02 +0000

> I filed bugs with every spell checker maker (in the 80s and 90s) with an algorithm to fix this, none deployed it (Anything without a period in front of it and a capital Taht, don't respell). The bastards!

WordPerfect had an incredibly simple fix for this, in response to all sorts of problems.

1) Let the user dictionary correct a word to itself, eg "MSc == MSc".

2) The user dictionary is the first one consulted.

3) As soon as any dictionary or rule fired, terminate all further checking.

Fixes ANY and ALL spellcheck problems :-) (Oh, and this was back in WP5.1 for DOS, if not earlier.)

Cheers,
Wol

Making WiFi fast

Wol — Thu, 17 Nov 2016 09:55:35 +0000

To try and keep Ham happy ... how about a "contended bandwidth benchmark"?

Pick a standard number of devices typical in the home - let's say 5. And let's say our wifi router has a max bandwidth of 100Mb/s.

Now fire up 5 devices, clobber the access point, and see the total throughput. If each device manages 10Mb/s, that means the router is achieving 50% of theoretical maximum throughput.

I know it's pretty much the same benchmark as they currently use, except they use one device so they achieve 100% of theoretical max, and they quote the throughput.

If we can go to Ham and say "in a typical home, with 5 devices, half your throughput is unusable", that's a benchmark he won't like becoming public ... :-)

Cheers,
Wol

Fixed-function hardware

mtaht — Wed, 16 Nov 2016 20:09:58 +0000

As a counter-example of how better onboard firmware could cut observed latencies down below what we can achieve by moving more ops into the kernel, see:

http://blog.cerowrt.org/post/a_look_back_at_cerowrt_wifi/

Some chipsets already expose a per-station concept, in particular.

Making WiFi fast

mtaht — Wed, 16 Nov 2016 20:06:50 +0000

Let me point out a link to a talk that describes the enormous technical debt that needs to be paid down in just one (other) wifi driver set in order to make forward progress:

https://www.linuxplumbersconf.org/2016/ocw//system/presen...

These sorts of issues are almost universal in wifi, in every chipset and OS.

Fixed-function hardware

mtaht — Wed, 16 Nov 2016 19:47:43 +0000

The core need for offloaded into the hardware firmware is that wifi has the need to do certain things under very hard realtime constraints that the Linux kernel cannot meet. In other words, it's latency, once again, driving the need for intelligence "down there". From a signal processing perspective, we care about nanoseconds - and there are like 400+ DSPs on a modern 802.11ac chip. Up from there, in the core wifi standards are need for sub 10us response times for many operations.

What we showed was that at the higher levels of the wifi stack - at the txop level - linux is more than responsive enough to fare well at the 500+us latency range, and we can put a lot more intelligence there, that can make a huge difference in actual network behavior.

I have outlined on my blog multiple ways for even smarter firmware can do even better than we do today shifting more stuff back into the core processor, instead onto the onboard firmware

As well as multiple ways to do more smart things in the core linux networking layer, building on top of this work. If made more universal, we can also make a dent in several other nagging problems in wifi, like better routing metrics.

Many of the trials, travails, missteps, and other bugs we've had
to fix along the way are in my blog and/or discussed on the make-wifi-fast list.

http://blog.cerowrt.org/post/

There is so much more that can be done to improve wifi! The best document we have on all that, is here:

https://docs.google.com/document/d/1Se36svYE1Uzpppe1HWnEy...

Making WiFi fast

mb — Wed, 16 Nov 2016 18:50:45 +0000

>They really don't. In empty space, they fade _exactly_ the same way (it's physics).

Except that my house consists of a little bit more matter than empty space and that the 5GHz signal certainly is a lot weaker than the 2.4 GHz signal after it passed a few walls.

Making WiFi fast

eduard.munteanu — Mon, 14 Nov 2016 06:13:49 +0000

In practice, though, WiFi is often added on to the premises as an afterthought. Whoever set up the space didn't plan properly for networking, so WiFi gets used as a stop-gap measure. As with all last resort measures, it kinda sucks, not necessarily because there's something inherently wrong with WiFi.

Making WiFi fast

mtaht — Fri, 11 Nov 2016 16:44:42 +0000

Of possible interest is the pathological piece I wrote while wrestling with who I am today. This epiphany is why I'm now "dave", and not "mike" taht - and it's where Ham the analogy came from: http://the-edge.blogspot.com/2003_06_08_the-edge_archive....

I've cultivated out of the box thinking ever since, in myself, and everyone.

Making WiFi fast

osma — Fri, 11 Nov 2016 12:20:45 +0000

I'm glad you asked it, since I was wondering the same!

Making WiFi fast

lkraav — Fri, 11 Nov 2016 06:55:51 +0000

ACKACKACKA

(because min. 10 char ACK needed here)

Making WiFi fast

mtaht — Fri, 11 Nov 2016 01:13:29 +0000

Heh. I have varied explanations for my last name.

1) My family's spaceship crashed in estonia in the late 1880s. Since the tradition there was serfs' last names were their job (e.g. "Joe Plumber"), thus, we named ourselves after a "star or planet", and have been trying to get off earth ever since.

My hobby is asteroid exploration, as it happens.

2) Ever type in just "Taht" into nearly any word processor, and watch it respell it to "That"? Yep, well over 30% of the documents I have to sign have screwed up my name, and I've watched people really struggle to get past the autocorrect to get it right. I filed bugs with every spell checker maker (in the 80s and 90s) with an algorithm to fix this, none deployed it (Anything without a period in front of it and a capital Taht, don't respell). The bastards!

Thus the umlaut.

3) It's a good test of i18n capability and interop with various protocols and tools.

4) I used to be a death metal fan.

I am glad I could clear this mystery up. Back to WiFi!

Making WiFi fast

cesarb — Fri, 11 Nov 2016 00:13:06 +0000

Or your wire is broken. Wireless is much harder to cut.

(A coworker just found out that the Ethernet wire to one of the WiFi APs at work was broken, which explains network issues they were having.)

Making WiFi fast

cesarb — Fri, 11 Nov 2016 00:08:04 +0000

> Looking for 802.11n and Linux compatability, I failed to notice that a 500€ machine does not do 5GHz nowadays.

There's a solution for that now: look for 802.11ac. Since 802.11ac is 5 GHz only, its presence means that the WiFi adapter can do 5 GHz.

Once 802.11ac becomes more popular, it should reduce the annoying tendency of offering professional-grade laptops with only 2.4 GHz WiFi.

Making WiFi fast

tohojo — Fri, 11 Nov 2016 00:01:35 +0000

A driver can opt in to using minstrel, or it can do it's own rate control. Iwl does the latter (not sure if it's in the driver or in firmware). Not sure what it would take to get the Intel drivers to use these changes, but I don't think it's trivial, unfortunately (my laptop also has an Intel card in it).

Making WiFi fast

tohojo — Thu, 10 Nov 2016 23:58:15 +0000

Not explicitly. However, while we are theoretically doing more work per packet, we have not seen anything that even registers on the CPU usage monitor. Crypto is still offloaded on most hardware, and most of the power save features of WiFi work by letting idle stations sleep for a while to conserve power. That is unchanged; these changes only really kick in when a device is busy.

Making WiFi fast

tohojo — Thu, 10 Nov 2016 23:54:33 +0000

Actually, we have been doing most of the testing of this on APs. And the companion airtime fairness work is explicitly AP-centric. But it benefits clients as well; basically, it needs to go where the queues build, and that becomes the WiFi link as soon as the upstream internet speed increases above the effective WiFi rate...

Making WiFi fast

lkraav — Thu, 10 Nov 2016 23:54:33 +0000

You guys are all missing the most important question: why does Dave have an Estonian word Täht ("star") for his last name?

Making WiFi fast

kamil — Thu, 10 Nov 2016 17:23:34 +0000

Check if the WiFi in your laptop is upgradable. It's often on a separate mini-PCIe card that is trivial to replace with little more than a screwdriver, and the cards can often be found for under $/€20 on eBay and such.

Making WiFi fast

fredex — Thu, 10 Nov 2016 16:31:54 +0000

This article seems to be speaking from the point of view of the Linux workstation/laptop/whatever. But what about the wifi router/AP itself? wouldn't there be similar benefits there? (I'm aware of the firmware that implements codel for certain Netgear routers, but this article sounds as if it has more tweaks in mind than just the ones there.)

Fixed-function hardware

farnz — Thu, 10 Nov 2016 10:04:02 +0000

It's worth noting that the original Larrabee design was intended to be a pure software system on a massively parallel chip; by the time they got as far as cancelling Larrabee the GPU in favour of Knights Ferry, they'd had to add traditional fixed-function samplers to ensure that the GPU design would be competitive. Similar applies to CPUs - in some senses, the Cell Broadband Engine SPUs are what you get if you replace a fixed-function L2 cache controller with a software-controlled L2 cache, while weak memory models are what you get if you make software responsible for cache coherency only.

In general, it looks like there's a (movable) happy medium between hardware and software; where the hardware's function is well-understood, and unlikely to change in the next decade (texture samplers, cache controllers, Ethernet checksum handling etc), then it's best as fixed-function hardware. Where there's still debate about what the function should be (not just how fast you can make it), then it's best as programmable hardware (TCP offloads, graphics shaders etc) under the control of software.

Making WiFi fast

sourcejedi — Thu, 10 Nov 2016 08:58:24 +0000

I could have quoted more

> The goal is to have one aggregated frame in the hardware for transmission

- that's all I was really thinking about. If you tell this to original hardware designers, one would expect them to be somewhat surprised. (Or maybe I'm wrong: they'd tell you how ill-suited the network stacks they targetted were for wireless, and it's about time).

Making WiFi fast

Sesse — Thu, 10 Nov 2016 08:38:35 +0000

While I agree with most of your point, there really is something as 3D acceleration. Even the most modern of GPUs will have a triangle rasterizer, a texture mapper and a framebuffer blend unit, all of them large fixed-function blocks (well, instantiated lots of times). This is _not_ done by the more CPU-like units (the shader cores), even though they certainly are flexible these days.

You can imagine moving all of these functions up into software, but it doesn't seem to work all that well in practice (witness e.g. Larrabee).

Making WiFi fast

Sesse — Thu, 10 Nov 2016 08:33:54 +0000

They really don't. In empty space, they fade _exactly_ the same way (it's physics). And like I said in another comment, for most obstructions, they fade very similarly, too. The effects of “the band is seven times as wide” (really!) and “ambient noise tends to be about 3 dB lower on 5 GHz” drowns out these considerations in practice.

Making WiFi fast

Otus — Thu, 10 Nov 2016 03:23:05 +0000

Has power use been a consideration and been measured?

I can imagine that pushing more logic to software and handling packets in smaller batches could be detrimental to power consumption.

Making WiFi fast

drag — Thu, 10 Nov 2016 03:15:27 +0000

The trend has always been towards sucking as much functionality out of the computer and into the processor die as possible and out of hardware logic and into the software as much as possible.

It's a cost performance thing.. as in cost and performance and reliability improves the dumber the hardware gets and the faster the cpu gets. This is generally speaking, of course. The deal here is software is much more flexible, much cheaper, and is much easier to patch to correct bugs.

That's one of the really big take-home points about Moore's law.

It's true for everything in computing, not just networking. Phone modems had their guts ripped out and became winmodems. I hated winmodems until I learned how to chance the software drivers Linux to get different algorithms and bump up my connection speeds. Then it moved to sound cards and into network and into harddrive controllers, and now things like software raid is superior for most purposes over hardware raid. Even now there isn't really any such thing as '3D acceleration' anymore, instead you just have different types of processor cores that are optimized to graphics workloads with most of the logic in the 'drivers'.

The problem with networking is that we deal with such small MTU sizes that _sometimes_ you can get better performance by offloading some of it. But for most server purposes turning off all the 'offload features' on network cards isn't a bad idea.

Making WiFi fast

mtaht — Thu, 10 Nov 2016 03:11:45 +0000

The current 20ms target in the mainline merged wifi-fq_codel code is an artifact of a number of other performance problems we were having at the time - notably powersave was broken by the patch set for a long while.

So... we backed off from the more aggressive 5ms default. Most of our recent testing has been against the original 5ms target with pretty good results. We also reverted back to the quantum 1514 default, rather than 300, as that hurt us on cpu on small platforms.

So, after more stuff lands it is my hope that we will revert these two changes back to the theoretically more correct 5% of 100ms that the target represents. Also, codel's taking place 2-10ms behind the actual packet delivery, presently.

In the talk I suggested shrinking txops more explicitly under contention. There is also the idea of turning codel's drop scheduler off at either a "good sized" aggregate, or an aggregate close to the ideal size for a given station, to move the knee of the curve closer to full utilization, where currently it turns off at a single large packet outstanding. We've also discussed dynamically modifying the target via ewma based on the workload and other common delays in the system (contention and interference).

Testing this stuff is HARD! Nobody's ever applied AQM technology to wifi or aggregating macs before, so far as I know. We will never get a perfect result, the goal is merely to get one that is reasonably good across most rates, and across most common numbers of stations.

I struggle to rewrite the description - one thing that is not obvious is that the fq_codel implementation for wifi is one very large set of queues for the entire device, with the per-station pointer within it disambiguating things. This was michal kazior's innovation - prior to that I'd been stuck on the idea of a full fq_codel instance created per station, with perhaps 64 queues each (and possibly derived from cake's set associative version of fq_codel). Now there's tons of queues and if there is a station collision on a given queue, it gets sorted out. Much better than what I'd had in mind!

It was my first talk on the work, mae culpa!

Still working on rephrasing the troublesome bit in the article, give me a few hours.

Making WiFi fast

drag — Thu, 10 Nov 2016 03:04:02 +0000

2.4ghz is nice if you need longer distance. Different frequencies have different strengths.

Making WiFi fast

drag — Thu, 10 Nov 2016 03:01:02 +0000

maybe its a hardware or firmware limitation.

Making WiFi fast

mtaht — Thu, 10 Nov 2016 02:53:29 +0000

I'd sent a few nits regarding this section of the article to jon earlier, as the description is unclear. Let me get to that in another post.

If I said "there is no need for intelligence in the network hardware" I did not mean that. There is plenty! of room for more intelligence there, just no need (for non-mu-mimo) for more than 2 txops of queuing in the onboard memory, or firmware, or driver.

It is kind of my hope actually that by reducing the max queuing in the wifi chip that we can fit more code into the firmware, like keeping better statistics or putting in better rate control information, or doing saner things with interrupts, or presenting a better API, etc.

Making WiFi fast

mtaht — Thu, 10 Nov 2016 02:46:37 +0000

while we are working on various modifications to fq_codel to make it more robust across a wide range of rates and numbers of stations, the airtime fairness patches we have currently do indeed behave better than what we call the fq-mac version without an explicit modification to codel.

I certainly welcome more testers (see the links to patches I posted earlier), ideas, and data. In addition to the data on the slides there, we have a large paper on the airtime fairness stuff pending academic review, which I can provide privately if you would like to see it.