Emulating Windows system calls, take 2

> > I'm clearly afraid of security implications of such a mechanism.
>
> Can you forward any specific security concerns that are not already mitigated to the original
> thread? We definitely want to have a good look at any security implications of this feature. I
> expect that most issues would be mitigated by making it unable to cross fork/exec boundaries.

Of course not. This is just a /personnal fear/. I have a profound distrust in anything that runs
under Windows. Not that I trust blindingly softwares that runs on Linux, but I've seen so many
malware hidden in documents, images and vulnerabilities even in windows softwares made by
"security" teams (last time I had been affected was with Cisco Webex) that I can imagine that
some ways will be explored to gain access to a Linux system through faulty Windows
programs.

> The goal is to provide an infrastructure for emulation in userspace. This means exactly that we
> don't need to go adding support for whateverOS in the kernel. :)

It's, indeed, better out of the kernel. So a fuse-like API for personalities :)

> But, saying we are hiding it is not fair. In fact, I called it a personality mechanism in my first
> submission, but we dropped that name to avoid confusion.

I'm sorry, I didn't mean to be unfair. I missed that point (I do not follow LKML anymore).

Thank you for clearing my mind :)

> Can you forward any specific security concerns that are not already mitigated to the original thread? We definitely want to have a good look at any security implications of this feature. I expect that most issues would be mitigated by making it unable to cross fork/exec boundaries.

Quite. This has no more security implications than a signal handler (to which it is very similar), so as long as it takes the same approach (reset signal handlers on address space reset at exec() time) we should be fine. Sure, if you use this mechanism *wrong* all sorts of things can happen, but the same is true of signal handlers and indeed of general bugs in code. The implications are no worse, except that making a mistake here is more likely to be obvious because it will probably break the program's use of lots of syscalls at once :)

It did -- but this wasn't done the way (say) Solaris did it, by piling the entire threading abstraction into the kernel and complicating the bejeezus out of everything. Instead we got away with a few relatively simple and noninvasive things: thread groups, a bit of tweaking to signal handling (which was painful, sure, but much less painful than almost everything else around signal handling already was and is), futexes, and some nice souping up to PID allocation so you could have silly numbers of them.

(And a good few of those abstractions were, I vaguely recall, there *already*, and used by LinuxThreads: tgids, for instance. I'm sure futexes were new for NPTL though.)

Thanks for letting me know!

Yes absolutely agreed. Being able to do "deep" stuff would basically require "composition over configuration", so an exokernel, to reign in the complexity. Great idea, but Linux absolutely is not that.

This is a very late comment I guess, buts that's mostly because I still don't know what is being proposed.

> The problem with classic personalities is that they are very shallow. They can be used to adapt syscalls, but little else.

I presume that means the design of personality API forces you to place the code in the kernel. "The code" is something using Linux to emulate the foreign syscall. I can tell you with reasonable authority[0] that's true.

Surprisingly the code doesn't care where it lives. In fact it mostly doesn't change. Even if the code was a kernel module moving to user space. I know that because I've done it [1].

I can also tell you from the maintainer of the said code's perspective, user space is the nicer place to be. Far, far, far nicer. To paint the picture, the kernel code I maintained has to do syscall's in some way. Why syscall's? Because they are the only interface in the kernel that's stable, and life's far too short spend it migrating out of tree code to each kernel version. But getting to syscall table was mission bloody impossible. The mechanism I inherited was unwinding the kernel stack, disassembling return addresses until you found the code that dispatched to you, then extract the pointer to the syscall table from the previous instruction. (Sorry if I wrecked your meal.) This is of course not easy to port between kernel versions either, but at least you are porting just one obnoxious thing.

Turns out there is only one real challenge to moving it to user space. That is getting the kernel to deliver the syscall to some user space trampoline address. That ends up in being one of two challenges. The first kind is when the syscall mechanism is what the Linux kernel uses. In that case is becomes 'redirect any syscall outside of the trampoline range to the trampoline'. This case is covered in the article.

The 2nd kind is when the syscall mechanism is not what the kernel uses. A software interrupt is the obvious way, but the x86 / amd64 designers in particular have displayed commendable imaginative flair in this area. However, in every case I've seen the mechanism is illegal to do in user space, so either the kernel handles it, or it's SIGSEGV. So all you have to do is arrange for that mechanism (whatever it is) to boomerang to an address in the trampoline, with minimal overhead.

Handling the second case doesn't seem hard: a standardised API like syscall_trap(SYSCALL_TRAP_INT80, trampoline_trap, trampoline_begin, trampoline_end, flags), a kernel module for each SYSCALL_TRAP_??? mechanism. Job done.

How the proposed mechanism do this is a bit of a mystery. I guess I should look at the code.

[0] http://ibcs64.sourceforge.net/
[1] http://ibcs-us.sourceforge.net/