New AT_ flags for restricting pathname lookup [LWN.net]

New AT_ flags for restricting pathname lookup

Posted Oct 4, 2018 21:36 UTC (Thu) by wahern (subscriber, #37304)
Parent article: New AT_ flags for restricting pathname lookup

I don't understand why the Go team is so resistant to adding the ability to explicitly pin a goroutine to a machine thread. Goroutines are an amazing, almost ideal construct. But there's a very obvious and unresolvable impedance mismatch between how a goroutine implement threading (linear flow of logical execution) and how traditional operating systems do. A similar mismatch exists with FFI ABIs (i.e. stack details) and with the blocking semantics of some syscalls. In those cases a goroutine *is* pinned to a machine thread; indeed, the very architecture of the Go runtime (the [G]oroutine, OS [M]achine thread, and [P]rocessor scheduling abstractions) is built around this mismatch. It's inexplicable to me why they refuse to expose the scheduling levers that must necessarily exist.

New AT_ flags for restricting pathname lookup

Posted Oct 5, 2018 7:13 UTC (Fri) by kostix (guest, #119803) [Link] (1 responses)

That wouldn't have helped anyway: the problem with not being able to do the classic fork+exec in Go programs is that the code executing in each of them heavily relies on the live Go runtime (which is linked with/into any compiled Go executable and actually manages the whole lifecycle of the program), and that runtime exploits multiple OS threads — both to run the program's goroutines and do its own chores.

Since fork() clones the state of just a single thread — the one which happened to execute that syscall, — as soon as the control resumes in the child process, there is literally no Go runtime anymore around the goroutine "awoken" in the cloned thread, and as soon as it happens to call anything which would normally reach for the runtime, it is hosed. And normally such a call would happen pretty soon.

So basically the only sensible thing one might safely do after forking a process running a Go program is to do a controlled set of preparations and exec().
And actually that's what the syscall.ForkExec does — with some added complexity stemming from Go having an execution model other than C ;-)

You can look at ForkExec in https://golang.org/src/syscall/exec_unix.go and then at forkAndExecInChild in https://golang.org/src/syscall/exec_linux.go — the code is very easy to follow for any programmer with a C background, and it is extensively commented.

New AT_ flags for restricting pathname lookup

Posted Oct 6, 2018 1:37 UTC (Sat) by wahern (subscriber, #37304) [Link]

Shouldn't it be possible to quiesce the runtime (pause GC, park all other goroutines, and join all kernel threads)? All the machinery in the scheduler must already be there, more or less. Maybe some component is currently running in a dedicated thread in an infinite loop, but conceptually it could be refactored to be able to enter and exit its core loop.

It might not be particularly efficient and come with a ton of gotchas, but it would at least make some currently impossible things possible, such as using geteuid and forking helper processes. Those things tend to happen early on, anyhow, so performance and other limitations wouldn't matter much.