LWN: Comments on "Emulating Windows system calls in Linux"

Emulating Windows system calls in Linux

Hi-Angel — Sat, 11 Jul 2020 13:30:48 +0000

> No, you would:
>
> * patch WINE libraries (the only ones that should issue Linux system calls) to
> go through a trampoline page

You can't achieve anything here by patching WINE libs because as the prev. author
said, there's no problem with apps that go through them. The problem being
discussed is that some apps make system calls without going through WinAPI/WINE
libs. Let me quote the original mail:

> Modern Windows applications are executing system call instructions directly
> from the application's code without going through the WinAPI. This breaks Wine
> emulation, because it doesn't have a chance to intercept and emulate these
> syscalls before they are submitted to Linux.

Emulating Windows system calls in Linux

raoni — Fri, 03 Jul 2020 13:28:37 +0000

IIRC from when I read the thread, they are OK with overhead on syscalls from windows code, they are not the concern for performance, they are concerned that applying some sort of overhead for all syscalls is a bigger performance hit because the linux libraries and the winAPI emulation code.

Emulating Windows system calls in Linux

khim — Thu, 02 Jul 2020 10:13:55 +0000

Windows 7 uses SYSENTER and doesn't work on CPUs without it.

When applications dropped WINXP support then got the chance to use it, too.

Emulating Windows system calls in Linux

rwmj — Thu, 02 Jul 2020 09:31:10 +0000

Don't Windows and Linux system calls use entirely different mechanisms? Does Windows still use an IDT to enter the kernel?

Emulating Windows system calls in Linux

mirabilos — Tue, 30 Jun 2020 12:31:17 +0000

Except that the code needed to handle it almost certainly isn’t signal handler-safe or can be made to…

Emulating Windows system calls in Linux

NYKevin — Sun, 28 Jun 2020 19:04:16 +0000

> as long as you can tolerate those syscalls being slow.

I imagine this will depend on the game. If it's isolated into a bunch of small levels with loading screens between them, well, the loading screens will suck, but the rest of the game should basically work most of the time, assuming the game engine isn't trying to do something weird (like constantly telling the OS which pages to evict first).

But if it's an open world game that dynamically loads stuff in and out of memory all the time, then you're in trouble.

Emulating Windows system calls in Linux

roc — Fri, 26 Jun 2020 21:09:14 +0000

Probably better to ask "when will it be safe to assume syscalls are fast again?"

Emulating Windows system calls in Linux

roc — Fri, 26 Jun 2020 21:08:35 +0000

When will it be safe to start assuming no-PTI when designing APIs?

Emulating Windows system calls in Linux

roc — Fri, 26 Jun 2020 21:05:17 +0000

Wine uses glibc and a bunch of other system libraries which do need to be patched. Those libraries aren't trying to stop us patching them, but they're not providing any hooks to avoid the need for patching, either.

Again: you don't need to patch the tricky game code with this approach ... as long as you can tolerate those syscalls being slow.

Emulating Windows system calls in Linux

pbonzini — Fri, 26 Jun 2020 15:03:58 +0000

Yes, the core idea though is the same, distinguishing trapped and pass-through system calls from the address.

Emulating Windows system calls in Linux

pm215 — Fri, 26 Jun 2020 14:09:28 +0000

That would work, but it's not the approach suggested at the top of this comment thread, which includes "The ptracer looks at the code around the syscall and if it matches certain common patterns, we patch it with a jump to a stub"... (You don't need to runtime-patch the wine libraries -- wine controls that code so it can just be built to do whatever.)

Emulating Windows system calls in Linux

pbonzini — Fri, 26 Jun 2020 12:40:54 +0000

No, you would:

* patch WINE libraries (the only ones that should issue Linux system calls) to go through a trampoline page

* use seccomp-bpf to raise SIGSYS for almost all code except that single trampoline page

* now if you get SIGSYS you now it's a Windows syscall, and you handle it from the SIGSYS handler

Emulating Windows system calls in Linux

pm215 — Fri, 26 Jun 2020 11:49:31 +0000

The patchset says it's addressing the way that "Modern Windows applications are executing system call instructions directly from the application's code without going through the WinAPI" -- so I think your approach would imply patching game code. It sounds like they already have workable approaches for apps that are traditional "call the winapi library which makes the syscall" style.

Emulating Windows system calls in Linux

luto — Fri, 26 Jun 2020 05:12:08 +0000

Pre-Meltdown, trivial syscalls like this were very fast. A trampoline oughtn’t hurt that much.

Emulating Windows system calls in Linux

roc — Fri, 26 Jun 2020 01:39:07 +0000

The rr approach applied to Wine would not require patching the Windows game code, only the Wine/Linux libraries which *are* somewhat cooperative.

Discussing it on LKML, the problem with our approach for Wine is probably the issues with multiple threads potentially racing with system-call patching.

Emulating Windows system calls in Linux

smcv — Thu, 25 Jun 2020 23:24:13 +0000

As mentioned in the article, patching Windows game code is unlikely to work well, because in some cases it tries to detect external modifications to itself as an anti-cheating mechanism, and it's deliberately obfuscated to make modification and tracing harder. I'm aware rr isn't usually tracing actively cooperating processes, but it isn't usually tracing a process that is actively uncooperative either.

Emulating Windows system calls in Linux

roc — Thu, 25 Jun 2020 22:56:00 +0000

rr grapples with a similar problem. We need to intercept commonly-executed system calls and wrap them with our own processing, with minimal overhead. I think Wine could probably use our approach.

We use a seccomp filter to trap on all syscalls except for those called from a single specific trampoline page. When a library makes a syscall, the filter triggers a ptrace trap. The ptracer looks at the code around the syscall and if it matches certain common patterns, we patch it with a jump to a stub that does the extra work we need and then issues a real syscall via the trampoline. Thus, a library syscall is slow the first time and fast the rest of the time. (Another possible variant of this approach, probably faster when applicable, would be to avoid using a ptracer and have the filter trigger a SIGSYS whose handler does the patching.)

Maybe I should post this to the list...

Emulating Windows system calls in Linux

chris_se — Thu, 25 Jun 2020 21:00:03 +0000

I think the user-space trampoline is the most promising solution here: it requires extremely minimal code changes to the kernel and offers full flexibility to the user-space programs, because they can use any code they want for their trampoline, and that code can be extremely hand-optimized assembly for that specific use case.

And if somebody much more clever than me when it comes to assembly can figure out a way to do that without requiring a context switch for the instruction that triggers the trampoline (don't know if that's possible) then this would have the same performance as eBPF, because whether you do the comparison in ring 3 or ring 0 shouldn't actually change anything.

On the other hand, if you do need two additional context switches (one for entering the kernel and one for jumping back to the trampoline) then one would have to measure the performance impact here. (Though in that case one optimization could be for the kernel to postpone the speculative execution mitigations until after it has determined the syscall comes from the trampoline and should hence be used directly, and don't do any of the mitigations both for entering and exiting ring 0 if it wants to jump immediately back to the trampoline, that should reduce the cost of the context switches quite a bit.)

Emulating Windows system calls in Linux

rvolgers — Thu, 25 Jun 2020 17:57:31 +0000

To clarify the kind of design I'm thinking of, you could perhaps get something more flexible than a BPF syscall filter by making all executable pages PROT_NOSYSCALL, requiring all syscalls to go via a single non-PROT_NOSYSCALL page which performs the filtering in userspace and then performs the syscall. Assuming CET is turned on, as it's needed to ensure the filtering is done before the syscall instruction.

Obviously there are still a lot of potential problems when you have security code running in the same address space as the code it's defending against, so there might be something I'm missing. Certainly the trusted code that does the filtering would have to be very carefully written to defend against the program altering the filter code, and there are lots of potential race conditions if you'd want to check anything stored in memory.

Still, it seems like a pretty flexible approach, especially considering syscall filtering via bpf is also pretty limited, and worth thinking about?

Emulating Windows system calls in Linux

rvolgers — Thu, 25 Jun 2020 17:36:37 +0000

I was all set to dislike this proposal, but I actually think the original idea is great, and is kind of underselling itself.

The PROT_NOSYSCALL could be turned into a remarkably elegant and effective security barrier when combined with Intel's Indirect Branch Tracking (part of CET) to remove the possibility of jumping directly to a SYSCALL instruction.