Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 19:13 UTC (Mon) by joey (guest, #328)
Parent article: Meltdown/Spectre mitigation for 4.15 and beyond

Worth noting that kernel mitigations to Spectre don't protect user-space processes from dumping memory belonging to other user's processes. We're only at the very beginning of an exceedingly long road to getting user-space Linux fixed.

Perhaps this won't be a big deal in practice, after all once an attacker has gotten into the machine as one user, they generally only need to wait to exploit an future security hole to fully own the machine.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 20:09 UTC (Mon) by jcm (subscriber, #18262) [Link] (18 responses)

This is why a complete fix requires either turning off the indirect predictor in all of userspace (we support this in RHEL, for that reason - we thought about this) or repolining the world, which requires a mass rebuild. In our case, this will happen in Fedora, though the exact timing of that is subject to upstream gcc work.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 20:29 UTC (Mon) by Otus (subscriber, #67685) [Link] (5 responses)

> turning off the indirect predictor in all of userspace

Is there something in current processors that allows turning off the branch predictor or what does this mean?

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 20:41 UTC (Mon) by jcm (subscriber, #18262) [Link] (4 responses)

This (disabling the indirect predictor logic) is what IBRS achieves (restricts indirect branch prediction). But the current patches only do this for entry into the kernel by default. The other half is IBPB, which you can think of as a flush of the predictor state so that when you do a world switch you can flush the state. But there are possible gaps. I'll leave it to the reader to figure them out.

Those IBRS/IBPB interfaces are special MSRs because they're not real MSRs. They're hacks in the microcode to make what look like new MSRs (hence the always-write logic requirement). On other arches, we have more direct control over CPU chicken bits that control indirect predictors and we can whack those directly. But we still have to account for userspace-to-userspace attacks in general. The example from the spectre papers focuses on the latter by way of example anyway.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 20:49 UTC (Mon) by Otus (subscriber, #67685) [Link]

Ah, so prediction is not disabled, just prevented from interacting between processes?

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 17, 2018 5:29 UTC (Wed) by paulj (subscriber, #341) [Link] (2 responses)

So what does IBRS do exactly (as was also the question in the linux-kernel thread)?

Is it /disabling/ IBP CPU logic? In which case, Andrea Arcangeli's belief that setting it once is sufficient surely must be true?

Or have Intel, with the microcode update, managed to add some bits of context (privilege level, address space?) to the branch-prediction table, and setting this IBRS pseudo-MSR is needed to get the CPU to update its view of the context in some way, so that IBRS must be set on every security relevant context change? Which would be more in-line with David and Arjan's views in that thread?

The lack of documentation and explanation is less than ideal. The security issues are now public. It doesn't make sense to try 'manage' what information is made public about any mitigation features - it can only hamper the speed at which any flaws/issues with those mitigations are uncovered.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 17, 2018 11:08 UTC (Wed) by dwmw2 (subscriber, #2063) [Link] (1 responses)

Intel documentation is here. I haven't seen public AMD documentation yet (they have IBPB but not IBRS).

No, it isn't just disabling branch prediction completely. I think that what they could achieve in the microcode hacks was fairly limited. So in some ways setting IBRS is a partial barrier, and flushes certain predictions from the store. But leaving IBRS set also makes things go slow, which implies that it's doing some checking at all times. The details are opaque and will vary from generation to generation.

Thankfully we don't really need IBRS except on Skylake (where it doesn't suck quite so much anyway).

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 19, 2018 12:43 UTC (Fri) by anton (subscriber, #25547) [Link]

Unfortunately, the Intel documentation is quite abstract. It does not tell us what these things actually do (probably because that's different for different generations); instead it tries to specify how to use them and/or what guarantees these things give (but even that is not very clear).

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 21:20 UTC (Mon) by epa (subscriber, #39769) [Link] (10 responses)

Can't it be fixed in the same way the attack on the kernel was fixed: by unmapping the memory? So each user space process would get its own memory mappings.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 15, 2018 23:47 UTC (Mon) by andresfreund (subscriber, #69562) [Link] (9 responses)

That's the fix for meltdown, not spectre, which is the danger talked about in the subthread.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 13:31 UTC (Tue) by epa (subscriber, #39769) [Link] (6 responses)

I was thinking of the "Attacks using native code" section of the Spectre paper. As I understand it, that variant of Spectre relies on the victim process being in the same address space as the attacker. You can speculatively execute arbitrary instruction sequences from the victim process and so read its memory space. This attack is prevented if different userspace programs don't share an address space.

There are other Spectre variants but I didn't think they were quite as powerful; they could not usually "dumping memory belonging to other user's processes" as the first post in this thread talks about.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 14:52 UTC (Tue) by excors (subscriber, #95769) [Link] (5 responses)

As I understand it, the Spectre paper's "Example Implementation on Windows" has the victim process and attacker process *not* sharing an address space. The attacker process puts its own code at virtual addresses that coincide with interesting code in the victim, and uses that to train the branch predictor. The branch predictor only cares about virtual addresses (and maybe not even all their bits), and doesn't care about address spaces or physical addresses. The CPU switches to the victim's address space when running the victim process, but it keeps the attacker's branch predictor state, so the attacker can control what code is run (speculatively) by the victim.

Then the attacker chooses to make the victim run code that transmits the victim's memory through some covert channel - mainly timing of reads to cache lines that are shared between the two processes, but there are other variations of covert channels that don't rely on shared cache lines at all (e.g. a read will evict unrelated data at different addresses that happen to map onto the same cache set).

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 14:59 UTC (Tue) by epa (subscriber, #39769) [Link] (4 responses)

Ah, I see. So the only way to prevent that by changing the address space would be to put each user process at its own set of virtual addresses that do not overlap at all with virtual addresses used by other processes. Even on a 64-bit system this would limit the number of user processes.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 15:26 UTC (Tue) by matthias (subscriber, #94967) [Link]

Would it be enough to put the text segments into non overlapping regions? Or is it possible to train the branch predictor to jump to virtual addresses that are marked non executable in the attackers context? After all such jumps can never really succeed.

If the attacker can only train the branch prediction to jump to addresses that are not mapped by the victim, there should be no information leak.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 15:44 UTC (Tue) by excors (subscriber, #95769) [Link]

Haswell reportedly only uses the lower ~31 bits of addresses for branch prediction, so you'd have to make sure processes' address spaces don't overlap modulo 2^31, which is impractical. (And other CPUs could use even fewer bits.)

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 20:14 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

> Ah, I see. So the only way to prevent that by changing the address space would be to put each user process at its own set of virtual addresses that do not overlap at all with virtual addresses used by other processes.

Or could one empty the branch predictor between process switches, possibly using a similar training mechanism?

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 20:18 UTC (Tue) by mjthayer (guest, #39183) [Link]

And yes, I know that is roughly what IBPB does where it is available.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 16, 2018 20:23 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

Regarding the fix for Meltdown, I was wondering whether emptying the TLB on context switches and rewriting the page tables in memory so that the kernel parts were not accessible to user processes would also have worked. Seems potentially slightly less invasive than the duplicate page table thing, assuming the page table structure makes it doable.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 17, 2018 7:12 UTC (Wed) by mjthayer (guest, #39183) [Link]

Though it would probably break PCIDs on systems which have them. And on second thought, I don't see much benefit over the duplicate page tables.

Meltdown/Spectre mitigation for 4.15 and beyond

Posted Jan 17, 2018 5:36 UTC (Wed) by foom (subscriber, #14868) [Link]

Given the discussion about retpoline being somewhat ineffective on Skylake, it seems like it may not actually even be sufficient to mass-rebuild the entire userspace with retpoline?

Reading https://software.intel.com/sites/default/files/managed/c5... it looks like there's not really a "turn off the indirect predictor in all of userspace" option, either. (Although I could imagine IBRS might have that effect on some CPUs.)

And even though IBRS ("Indirect Branch Restricted Speculation") is initially described like a mode switch, the intel doc also says it actually needs to be used as a command at every privilege transition for some of the CPUs (those without the "Enhanced IBRS" feature), and even then it doesn't protect code running at a given privilege level from other code on the same core at the same privilege level but in a different process. So you gotta also use IBPB.

And I have to imagine that IBRS and IBPB must be pretty slow given all the work being put into retpoline.

But if we need retpoline on old processors, and can't depend on retpoline on new processors...does userspace code running on skylake need to take the hit of IBRS-always, and IBPB, *and* retpoline (because presumably there won't be multiple variants of all the distros, built with and without retpoline)? How slow is that gonna be?

(Too bad user code can't use the kernel boot-time alternatives patching trick to eliminate the retpoline manipulation where it's not useful. Well, I guess it could via the vsyscall stuff, but you'd need to do an indirect jump through PLT to get there...which kinda ruins the point.)