LWN: Comments on "Go filesystems and file embedding"

Go filesystems and file embedding

HelloWorld — Fri, 14 Aug 2020 17:43:38 +0000

Passing a context around is trivial in purely functional programming, it's called the Reader Monad. It's built right into some modern effect systems like ZIO...

Go filesystems and file embedding

Cyberax — Fri, 31 Jul 2020 19:56:35 +0000

Inheritable thread locals work just fine in Java. The problem is that Java is a bit old-fashioned regarding threading, so a lot of parallel stuff is done in shared global thread pools because thread creation is expensive. This kinda makes inheritable thread-locals a moot point.

On the other hand, in Go pretty much nobody uses long-lived goroutine pools so GLS variables can work just fine. Moreover, pprof labels already work this way (there's no way to read them, you can only set them).

Go filesystems and file embedding

ibukanov — Fri, 31 Jul 2020 14:28:04 +0000

By cancelable in my initial response I meant ability to interrupt a given blocked call so it returns immediately. Timeouts are a subset of that.

What would be ideal if FileHandle can be used as a pseudo-channel in the select statements in Go. But that requires very non-trivial implementation especially on Linux.

Go filesystems and file embedding

gray_-_wolf — Fri, 31 Jul 2020 12:35:46 +0000

Context is not used just for cancellation. For example, we are passing logger in it. So for those cases it would still be useful even if you cannot actually cancel the operation.

Go filesystems and file embedding

benhoyt — Fri, 31 Jul 2020 09:14:38 +0000

By cancellation in this context, do you mean timeouts/deadlines? Or actually being able to cancel a read/write at any given time? If it's timeouts, does it work to specify a read/write timeout when you open the filesystem client (or Open() a file), rather than for example having a ctx/timeout on every Read call? Similar to what Russ Cox asks here: https://www.reddit.com/r/golang/comments/hv976o/qa_iofs_d...

Go filesystems and file embedding

smurf — Fri, 31 Jul 2020 08:35:17 +0000

Define "does not work". My experience (async Python code with contextvars) says otherwise.

Go filesystems and file embedding

ibukanov — Fri, 31 Jul 2020 07:54:54 +0000

Automatic inheritance for thread locals does not work. Java, where new threads are not created as often as in Go, provides interface to manually deal with the inheritance but even with that subtle bugs are possible.

And any thread-local with a mutable state that is shared between threads pretty much implies that it uses some form of locking internally. At which point using a global map with a mutex looks like a reasonable replacement for a thread-local especially for such heavy thing as Context.

Note I agree that it sucks that a language with GC does not provide a simple way to associate a piece of data with its threads, but at least one can see where the resistance is coming from.

Go filesystems and file embedding

Cyberax — Fri, 31 Jul 2020 07:38:25 +0000

Not all filesystems are local. Cancellation is very much possible for NFS (for which I'm actually already using a Go-based client).

Go filesystems and file embedding

ibukanov — Fri, 31 Jul 2020 07:23:05 +0000

On the other hand Linux still lacks a good story with cancelable local file reads. Adding context to file-related API would be a lie on a most popular server system.

Go filesystems and file embedding

Cyberax — Fri, 31 Jul 2020 04:31:51 +0000

There's no other choice than to carry explicit context object in absence of GLS. I don't mind the absence at all, actually. It leads to cleaner code, because developers stop doing things like storing database transactions in thread locals.

And quite a few companies actually have code style that enforces context as the last (or first) parameter to every method in the exported interfaces.

Go filesystems and file embedding

benhoyt — Fri, 31 Jul 2020 00:02:51 +0000

I'm in two minds about context. It's definitely useful, but I agree with Michal Štrba's 2017 article that it's like a virus that starts polluting all your APIs with "ctx context.Context" (which apart from being something you have to add everywhere, the naming is stutter-y). I personally wouldn't mind "goroutine local storage" for values, though I think that can be solved in other ways too (eg: in an HTTP context, a map of *http.Request to data). And of course there's the very hack-ish libraries that add GLS, like jtolio/gls. As for timeouts, those can usually be done more explicitly, or with an overall timeout per client. For cancellation, I have much less experience and don't know if there's a good solution there.

Go filesystems and file embedding

Cyberax — Thu, 30 Jul 2020 19:10:24 +0000

Each goroutine would have its own TLS context (GLS?), with potential automatic inheritance for new goroutines.

In other words, exactly like pprof labels work right now: https://rakyll.org/profiler-labels/

Go filesystems and file embedding

ehiggs — Thu, 30 Jul 2020 19:08:12 +0000

How would TLS even work in the face of goroutines that can be scheduled in different threads?

Go filesystems and file embedding

mgk — Thu, 30 Jul 2020 17:48:56 +0000

While there are places contexts don't make sense, I'm 100% behind you that there's a lot coming out without context support, that is a head scratcher: Like filesystems. I had hoped we stopped assuming filesystems were always available, always fast, never hung, etc. back in the 90's...

Go filesystems and file embedding

Cyberax — Thu, 30 Jul 2020 17:40:14 +0000

The inconsistency from Go team is frustrating. On one hand they are stoically opposed to any kind of thread-local storage (which I don't mind at all), on the other hand they are introducing new APIs that don't accept contexts.

And without contexts it's not possible to do context-based logging at all.