One and a half colors

Posted Dec 2, 2025 17:00 UTC (Tue) by smurf (subscriber, #17840)
Parent article: Zig's new plan for asynchronous programs

So instead of function-marked-async-or-not you shlep an additional "io" parameter all around your call chain whenever something might or might not need to do anything async?

Doesn't seem much different from tagging everything you want to async-ize with "async" and "await" …

Also I'm interested in how zig plans to manage an event loop this way. I mean you need to save and restore the call stack somehow, and stacks may not exactly be small in a production setup.

to post comments

One and a half colors

Posted Dec 2, 2025 18:53 UTC (Tue) by daroc (editor, #160859) [Link] (1 responses)

It's a bit different because the syntax of the functions involved doesn't change. And you can store an Io in a data structure or global variable if you want to. But I agree it's separately onerous to have another piece of context to port around everywhere.

As for managing the event loop: I believe the plan is for there to be built-in functions that can start executing a function on a user-provided stack, so the event loop can allocate separate stacks and then give them to the running functions. But Zig has also had a long-term goal to eventually be able to statically determine the needed stack size of any given function, at which point it should be possible to write comptime code that does better than that.

One and a half colors

Posted Dec 2, 2025 21:20 UTC (Tue) by softball (subscriber, #160655) [Link]

> piece of context to port around everywhere

Similar to Go, which also has "one and a half colors". Functions taking a context.Context as their first argument are most likely performing I/O. It's not required though: a number-crunching function performing no I/O might still check the context for cancellation every so often. Likewise, I/O without taking a context.Context is possible.

Similarly, in async languages (Rust, ...), a function marked async might end up performing no I/O at all (no await points). A function marked sync might spawn its own runtime (or grab the global one) and start spawning futures (= called async functions) on it.

Lots of grey areas.

One thing I wonder about is mixing threading for CPU-bound work and eventing for I/O-bound work. In Rust, one solution is having the application be fundamentally async (tokio, ...) and hand around a dedicated threadpool (rayon, ...). If there's enough available parallelism, both can coexist without interference and communicate via channels. Rust makes this explicit and relatively ergonomic at compile time (Send + Sync, ...). I wonder how equivalent Zig code would look like (I suppose Io would be the evented variant, and for the CPU-bound work just spawn OS threads normally).

One and a half colors

Posted Dec 2, 2025 20:09 UTC (Tue) by quotemstr (subscriber, #45331) [Link]

I'd actually people carry an IO object around. The ambient nature of IO makes doing things like shimming the outside world for testing annoying. Programs are better overall when you give a component an explicit pass from above capability for anything you'd like it to do.

One and a half colors

Posted Dec 2, 2025 21:18 UTC (Tue) by excors (subscriber, #95769) [Link] (11 responses)

Sounds like there's still two colours, but instead of the colours being "uses async IO" and "doesn't use async IO", they're now "uses IO" and "doesn't use IO". You still have the problem of propagating a colour (i.e. the presence of an `Io` parameter, either direct or carried in some other struct) all the way up the call graph, until you reach a high enough level that can decide the application's policy on concurrency.

Arguably that's a good case of colouring, because the presence of IO _should_ be part of your API: users ought to be aware of the security and performance and portability and testability implications of an API that accesses the filesystem/network, and should have some control over it. But users shouldn't have to care about serial IO vs concurrent IO - that's just an implementation detail and a performance optimisation - and in this model they don't have to care, because the API is IO-coloured either way, unlike the async/await model where migrating to concurrent IO changes your API colouring.

That's similar to how the use of dynamic memory allocation should be part of your API (and in Zig it is); it's too important to hide. And error handling (in most languages that don't use exceptions). And the general concept of dependency injection.

I suppose the main downside is that once you start making everything an explicit part of every API and avoiding implicit global state, it gets annoyingly verbose, and it's hard to read code when the important logic is obscured by boilerplate. But I think it's interesting to see Zig try a different tradeoff here.

One and a half colors

Posted Dec 2, 2025 22:23 UTC (Tue) by khim (subscriber, #9252) [Link] (8 responses)

That's much better solution that what Rust did. Of course Zig has the benefits of hindsight.

In practice there are more than two colors — except in Rust it's not obvious from the source and almost unmanageable in practice.

That's because you couldn't simply pass any random async function into any random executor… the functions that do actual work have to match the executor or else the whole things falls apart — and these functions are invisible in Rust's async fn signature.

In Zig one may simply have more than two implementations of the interface.

People are talking about “two colors” because in practice that's something that actually works, but try to mix two executors in one program in Rust… and the whole thing falls apart, you couldn't do that. It's not “2 colors” problem, but “2+ colors problem”.

One and a half colors

Posted Dec 3, 2025 9:17 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (2 responses)

> That's much better solution that what Rust did. Of course Zig has the benefits of hindsight.

I would like to understand how IO functions are compiled. If they are stackful coroutines, Zig's solution is very clever but that's a very different design space than the stackless coroutines you have in Rust or, for that matter, in Zig's previous attempt at asynchronous I/O. Stackless coroutines need compiler support but are more efficient (a couple years back I had a (IMO) really nice design for C stackless coroutines, but no time to implement it...).

Or does the compiler effectively treat the IO argument as a request to turn the function into a state machine and pass it to the threaded or evented run-time?

One and a half colors

Posted Dec 3, 2025 10:17 UTC (Wed) by spiffyk (subscriber, #173891) [Link]

As I understand it, the way the function will ultimately be compiled is entirely up to the chosen implementation of the Io interface. May be stackful, may be stackless – the point is that to the function taking the Io parameter, this is just a transparent implementation detail.

What is important to note is that the language provides no special treatment to the Io parameter, it is entirely an API convention to pass it. Io is an extension of the standard library, which in and of itself does not change how the language works. The only point at which the compiler itself will need to provide special functionality is when an implementation of the Io interface uses the (as of yet not finalized, afaik) constructs for stackless coroutines. But those are planned to be a separate feature of the language, and, from the perspective of the Io, will only be used by specific Io implementations.

One and a half colors

Posted Dec 3, 2025 12:40 UTC (Wed) by lukasl (guest, #180745) [Link]

There is no compiler support for this. It's either threads or stackful coroutines, depending on the implementation. The current dev versions of Zig only support threaded version of this. There are work-in-progress implementations of event loops in the zig std. I also created one alternative stackful coroutine runtime, outside of the zig std, that also implements the interface.

One and a half colors

Posted Dec 3, 2025 9:58 UTC (Wed) by muase (subscriber, #178466) [Link] (2 responses)

> That's because you couldn't simply pass any random async function into any random executor… the functions that do actual work have to match the executor or else the whole things falls apart — and these functions are invisible in Rust's async fn signature.

Could you elaborate what you mean with that? Because I have heavily used async within the embedded world, and I struggle to understand your point.

Async has two relevant API components: futures and wakers. As long as I implement a correct future, and use the provided opaque waker to wake the executor, I don’t see how my implementation has to match the executor?

And in my experience that works pretty well IRL; I have quite a few projects where I switched from my own executor to embassy, and two projects where I did vice versa, and I’ve never encountered any problems so far…

One and a half colors

Posted Dec 3, 2025 11:22 UTC (Wed) by khim (subscriber, #9252) [Link] (1 responses)

> Async has two relevant API components: futures and wakers.

Nope. It has three components, or, more precisely, two and half: futures+wakers and executors. These have to match or the whole thing goes down in flames.

> As long as I implement a correct future, and use the provided opaque waker to wake the executor, I don’t see how my implementation has to match the executor?

Because your waker has to match the executor. And said waker, as you have said, is opaque and is not present in the function signature so compiler couldn't verify that they match.

> I have quite a few projects where I switched from my own executor to embassy, and two projects where I did vice versa, and I’ve never encountered any problems so far…

And how many of these have both of these executors running, simultaneously? Note that it's not even a purely theoretical interest. There are tokio and tokio_uring — and they are incompatible (certain things in tokio make it impossible to use in one project). You may want, e.g., use tokio for network-related tasks and tokio_uring for disk access (where tokio is lacking). But if you try to do that you would quickly find out that mixing different futures (and thus different wakers) can easily lead to trouble.

One and a half colors

Posted Dec 4, 2025 21:40 UTC (Thu) by ofranja (subscriber, #11084) [Link]

> And how many of these have both of these executors running, simultaneously?

You can have multiple runtimes side-by-side in Rust, unless there is a specific limitation on the implementation of your runtime that prevents it. Nevertheless, it would be an implementation issue and not a language issue.

If you want an example, send this request to the LLM of your preference:

"Please write a simple example of a Rust program using async-std and tokio simultaneously in the same program."

> But if you try to do that you would quickly find out that mixing different futures (and thus different wakers) can easily lead to trouble.

You might not be able to use certain functions from a runtime's library in another runtime if they are implemented by the runtime itself - ie: what you call is a function that signals an action to the runtime and the function itself does not "execute" anything, and just waits for the action to complete.

In any case, this is easily solvable by communicating between different runtime contexts using channels and matching the library function calls to the particular runtime.

One and a half colors

Posted Dec 3, 2025 13:56 UTC (Wed) by smurf (subscriber, #17840) [Link] (1 responses)

> People are talking about “two colors” because in practice that's something that actually works

Two colors because one has "async" and one has not, or one requires a somewhat-special "Io" and one does not.

Variants (shades of a color, if you'd like) are of course a related problem. Rust isn't the only language that has more than one mostly-incompatible async variant. Python has two (asyncio and trio (*)), and a third (anyio) which provides an API that runs on top of both of them. Also you can run trio on top of asyncio (in guest mode) and vice versa (trio-asyncio). All in all that makes the situation annoying but somewhat manageable.

(*) OK so there's also curio and Twisted and whatnot. Details.

One and a half colors

Posted Dec 3, 2025 15:09 UTC (Wed) by khim (subscriber, #9252) [Link]

> Two colors because one has "async" and one has not, or one requires a somewhat-special "Io" and one does not.

You can easily predict that “sync” is just a special executor that does everything in place. That's trivial transformation and easily done if you may support more than one executor.

> All in all that makes the situation annoying but somewhat manageable.

Yes, but it's kludges on top of kludges. If you can, somehow, adopt the model that may have more than one executor then all the difference fall by wayside automatically because “sync” becomes just an async with trivial executor. But if you don't do that the every time you add new executor you make the mess worse: two executors (certain flavor of async and “sync”) are the typical norm today, three are a mess, try to converge dozen of executors in one app and the whole thing would collapse under its weight.

To untangle that mess we need to, somehow, parametrise our async functions… and Zig does precise that.

One and a half colors

Posted Dec 2, 2025 23:02 UTC (Tue) by sionescu (subscriber, #59410) [Link] (1 responses)

Correct. It still has two colours, and still suffers from a lack of preemption, so a CPU-bound code block can still starve an async worker thread. Sucks as much as the other approaches (Rust, Go), just in a slightly different flavour.

One and a half colors

Posted Dec 3, 2025 2:55 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Go can actually pre-empt computation-heavy goroutines. It's implemented using signals: https://github.com/golang/proposal/blob/master/design/245...