LWN: Comments on "VMX virtualization runs afoul of split-lock detection"

VMX virtualization runs afoul of split-lock detection

robbe — Sun, 19 Apr 2020 21:09:11 +0000

How it has worked before with VMware Workstation: the product ships a ton of pre-built kernel modules, but these are invariably not for my distribution, and even if they were, they get old pretty fast, if you are tracking Linus releases. You can fall back on compiling from the provided module sources, but if your kernel is 6-12 months newer than your release of Workstation, this will usually fail and needs manual fixing.

So… the module breaking regularly is par for the course, at least when considering VMware.

VMX virtualization runs afoul of split-lock detection

hmh — Sun, 12 Apr 2020 21:48:07 +0000

Not to mention the emulators... Yes, the kernel can, and does, have to emulate (a few instructions of) the running processor sometimes...

VMX virtualization runs afoul of split-lock detection

nix — Sat, 11 Apr 2020 22:10:55 +0000

> Given how many problems they have had with TSX (which includes some so bad that they had to completely disable TSX on the affected processors in a microcode update), it's a good thing they didn't do it that way.

After a huge amount of effort to get TSX working for locks in glibc, actual benchmarking was done. TSX was only slightly faster than no-TSX, and algorithmic changes made no-TSX massively faster than TSX. Whoops. (TSX remains dead useful for attackers trying to trigger speculative attacks that then erase the evidence of their execution.)

VMX virtualization runs afoul of split-lock detection

cesarb — Fri, 10 Apr 2020 15:44:52 +0000

> This ought to be have been implemented using a TSX-like concept

Given how many problems they have had with TSX (which includes some so bad that they had to completely disable TSX on the affected processors in a microcode update), it's a good thing they didn't do it that way.

It makes sense that they chose the simplest possible implementation: other than legacy MS-DOS era software, no software should do atomic operations on values which are not naturally aligned in memory. Their mistake was to take so long to implement a trap on these misaligned atomic operations.

VMX virtualization runs afoul of split-lock detection

nix — Fri, 10 Apr 2020 13:06:54 +0000

I have a question: does this essentially mean putting a disassembler in the kernel?

There's been a disassembler in the kernel for ages (or actually many, for many architectures: used e.g. for kprobes and uprobes etc to analyze function prologues to drop probe points there). This is just one of many users.

VMX virtualization runs afoul of split-lock detection

jcm — Fri, 10 Apr 2020 05:40:24 +0000

Intel. This ought to be have been implemented using a TSX-like concept by putting both cache lines into effectively a write set consisting of lines exclusively held by a single core at a time, not by locking the uncore of the chip.

VMX virtualization runs afoul of split-lock detection

kmweber — Thu, 09 Apr 2020 19:23:02 +0000

As someone who knows nothing about this, I have a question: does this essentially mean putting a disassembler in the kernel? Because presumably that particular sequence of bytes that constitutes the opcode could appear in other contexts, right? E.g. as initialized data, as the end of one instruction opcode followed by the beginning of another, etc. Since x86 instructions are variable in length, is there a way to reliably determine this short of disassembling the entire thing from the beginning?

Like I said, I know nothing about this, so this isn't meant as a commentary on whether it is a good solution or not, whether the benefits outweigh the costs, and so forth--I'm just teying to better understand what's going on.

VMX virtualization runs afoul of split-lock detection

jkingweb — Thu, 09 Apr 2020 13:42:28 +0000

For clarity, who is "they"?

VMX virtualization runs afoul of split-lock detection

jcm — Thu, 09 Apr 2020 04:08:08 +0000

What isn’t spelled out clearly anywhere is just how badly they screwed up the split cache line lock implementation. There isn’t one “memory bus” in a modern system, there are many cache coherent agents talking to one another and via home agents (contained within home nodes in Intel parlance) to external memory. There are so many nasty ways that you could implement “locking the bus”.

VMX virtualization runs afoul of split-lock detection

imMute — Wed, 08 Apr 2020 18:01:04 +0000

It'll also completely screw over people who use out-of-tree modules and *up-to-date* kernels too.

VMX virtualization runs afoul of split-lock detection

Tov — Wed, 08 Apr 2020 14:18:27 +0000

Which is essentially status quo...

VMX virtualization runs afoul of split-lock detection

dottedmag — Wed, 08 Apr 2020 13:35:50 +0000

Well, it will cause these appliances to be stuck on ancient kernels with out-of-tree modules.

VMX virtualization runs afoul of split-lock detection

amacater — Wed, 08 Apr 2020 12:37:59 +0000

Or just kill out of tree modules as unfixable and blacklist them from linking ... that would kill off the unfixable cheap appliances that are always out of tree :)

VMX virtualization runs afoul of split-lock detection

cesarb — Wed, 08 Apr 2020 11:47:27 +0000

> but developers like Peter Zijlstra also see an opportunity to prevent the loading of modules that engage in other sorts of unwelcome behavior, such as directly manipulating the CPU's control registers.

I wonder if the end game won't be something like requiring all out-of-tree modules to be compiled to WASM or similar.