Zig 2024 roadmap

By Daroc Alden
February 2, 2024

The Zig language 2024 roadmap was presented in a talk last week on Zig Showtime (a show covering Zig news). Andrew Kelley, the benevolent dictator for life of the Zig project, presented his goals for the language, largely focusing on compiler performance and continuing progress toward stabilization for the language. He discussed details of his plan for incremental compilation, and addressed the sustainability of the project in terms of both code contributions and financial support.

Priorities for Zig

Kelley opened by discussing the upcoming 0.12 release, confirming that the GitHub issues tagged with that milestone constitute an accurate list of what he expects to land in the release, and that there are no large disruptive features planned for 0.12. Kelley said that he expected 0.12 to be fairly stable, and that many users would prefer it because it mostly contains bug fixes to 0.11, only a select handful of which are going to be backported to 0.11.1. He then talked about the process for deciding on the most important problems to fix as part of aiming for stability, and invited anyone who had a big, real-world Zig project to add their problems with the language to the tracking document.

Kelley then segued into discussing the number of open issues with the compiler, pointing out that if he fixed one bug a day, it would take years to resolve all of them. So, naturally, the Zig project needs some way to solve open issues faster. Kelley's solution to this problem is to work on making the compiler more performant — if contributors need to spend less time compiling, they can spend more time actually working on contributions. Currently, the Zig compiler takes ten minutes to build from scratch on my laptop, but the tests take more than 600 CPU-minutes. A large portion of that time is spent on code generation using LLVM, which is often slow (despite recent improvements by the LLVM project).

LWN previously reported on the Zig project's attempts to make LLVM an optional dependency. Kelley said that great progress has been made on that goal, and that the native x86 backend is complete enough that "you can start playing with it and seeing if it works for your use case", although it is not passing all of the behavior tests quite yet. He expects those changes to land in 0.12, although the LLVM backend will remain the default for at least one more release. Kelley hopes that the native backend will be useful for debug builds, where producing code quickly is more important that producing optimal code.

He was also excited about progress on incremental builds, saying: "This feature is unique to Zig because you just can't do it on accident. You need to design for it from the beginning". While other languages support incremental compilation, the Zig project has plans to go further than other languages do, including directly patching changed functions into on-disk binaries. In support of incremental compilation, Kelley has been working on re-designing the compiler's internal data structures to make incremental compilation as smooth as possible.

He has converted the compiler's internal representation to avoid the use of pointers, instead storing information as flat arrays accessed by index. This representation permits the compiler's internal knowledge of the program to be saved directly to disk and then loaded again without parsing or fixups, so that incremental compilation can just mmap() the previous analysis and access it directly. Kelley went on to say that incremental compilation "is actually Zig's killer feature, and we haven't even launched it yet". Eventually, he wants to make incremental rebuilds of a project so fast as to appear instantaneous.

Later, Kelley claimed that the four main priorities for Zig before tagging version 1.0 were performance, improvements to the language itself, bringing the standard library up to a consistent level of quality, and writing a formal language specification. He presented each of these as being a necessary prerequisite of the later steps, with work on performance informing what language changes are necessary, which themselves inform what goes into the specification. He addressed concerns about how long it would take for Zig to reach 1.0 by remarking that "things are pretty unstable right now, but maybe 1.0 is not really what you're waiting for. Maybe you're just waiting for, you know, the draft of the language specification to come out. I think that's going to be a stability turning point".

During the Q&A after the main talk, one person asked how they could pitch the use of Zig to their company, given that the language isn't stable yet. Kelley's response was that people should "ask for forgiveness, not permission". Loris Cro, the host of Zig Showtime, elaborated by explaining that focusing too much on whether Zig has reached version 1.0 is missing the point. Cro said that it's more important to have a good mental model of where Zig stands compared to the alternatives, and whether using the language would actually be better for your use case. Kelley agreed with that assessment, saying that one should introduce Zig at one's company only if it would actually make some piece of software better, not just for the sake of introducing a new language.

Another question concerned whether the Zig project intends to focus on external tooling, such as a better language server. Kelley responded that he intends to get incremental compilation working first, before integrating an officially supported language server, because it could reuse many of the same internal components. He did assert that anyone can influence the path and timeline of Zig by submitting a pull request, and encouraged anyone who was particularly interested in a specific tool or feature to try contributing to the project.

A different audience member asked whether there had been any fix for the issue highlighted in Martin Wickham's 2023 talk at the Software You Can Love conference where a function argument that is implicitly passed by reference can cause problems when the same memory is also aliased through a mutable pointer. Kelley responded that this had not yet been addressed, and remarked that while this problem has drawn a lot of attention in tech news circles, he does not consider it very important. He noted that TigerBeetle, a financial database written in Zig that cares a lot about correctness, listed several other problems with the language as being more pressing. Despite that, he assured the audience that the problem would be fixed eventually, it just hasn't been a priority yet.

One potential contributor said that they were excited to write some compiler optimization passes for Zig, and wondered when the compiler will have settled down enough that they could look at contributing those. Kelley responded that there was still some additional research work to be done around efficient internal representations, and that he had been looking in particular at the Sea of Nodes representation. Kelley continued to say that high-level optimizations would need to wait until that had been explored.

Another question had to do with when support for async-await would return to the language. Kelley responded that the asynchronous I/O support in previous versions of Zig had been experimental, and that they had learned a lot about what problems come up when trying to do something like that in a systems language. He said that he really liked single-threaded asynchronous event loops, because of how they let you be very explicit about what order I/O operations ought to occur in, and he wanted to bring it back to the language, but the prerequisites weren't in place. He spoke about the difficulties with debugging asynchronous programs (including lack of support in DWARF for asynchronous functions), and the problems with LLVM's coroutine code generation. Ultimately, he said that asynchronous programming would need to wait for the native backend, which can exercise greater control over how the code for asynchronous functions is generated.

Finances

Kelley also spent a portion of the talk discussing the Zig Software Foundation's finances. The foundation recently published a report as part of its 2024 fundraiser explaining how it used donations in 2023. While many volunteers contribute time to the Zig project directly, nearly two thirds of the foundation's expenditures went to paying contractors to work on the core language. The remaining third was split between paying Kelley for his time working on the language and paying for necessary infrastructure and legal expenses.

Donations diminished in the last portion of 2023, but Zig is more popular than ever. There were a record number of GitHub issues filed and pull requests merged in the past year. In 2024, the Zig Software Foundation is hoping that a large portion of its revenue can once again come from individual donations, which accounted for a third of its revenue in 2023. It is also experimenting with new funding sources, including "Donor bounties". The foundation has previously expressed discomfort with feature bounties, saying "bounties are a poor form of sponsorship when it comes to software development", because they discourage cooperation and maintainable code. Donor bounties are a proposed alternative where the bounty is paid to the foundation, instead of directly to contributors. In this model, bounties can still contribute to the development of Zig, without setting contributors at odds with each other or encouraging the development of shoddy code. Kelley was quick to emphasize that donor bounties are an experiment, and "are not a new thing we want everyone to do". No-strings-attached donations remain the foundation's preferred method of contribution. But the few donors who have specific, urgent requests for particular features can reach out to the Zig Software Foundation to fund a bounty.

Conclusion

While the Zig project is continuing to make progress on its ambitious goals for the language, there are numerous difficult technical challenges ahead. Kelley's vision for the language in 2024 focuses mainly on performance, as a way to increase development velocity and unblock other work. There is lots of additional work needed in the long term on the standard library, documentation, and language specification. Despite that, the Zig project is healthy, with 354 contributors in the past year and a growing pool of serious real-world users. Kelley noted that he doesn't know how long it will take to tag 1.0; like any free software project, Zig's progress depends on people's willingness to help. Despite this reality, Kelley asserted that the project would get there with time.

to post comments

Zig 2024 roadmap

Posted Feb 3, 2024 1:55 UTC (Sat) by quotemstr (subscriber, #45331) [Link] (73 responses)

Zig is a non-memory-safe language. Why are we, as an industry, pouring resources into non-memory-safe languages? The advantages of memory safety are compelling.

Zig 2024 roadmap

Posted Feb 3, 2024 2:15 UTC (Sat) by b7j0c (guest, #27559) [Link]

Who is "we"? The Zig folks are scratching an itch and you don't have to care if you don't want to.

Zig 2024 roadmap

Posted Feb 3, 2024 6:13 UTC (Sat) by IntrusionCM (subscriber, #140256) [Link] (5 responses)

Der industry,

please eliminate JavaScript first.

Being non-memory safe is the least of its many problems.

Thanks.

Zig 2024 roadmap

Posted Feb 4, 2024 15:55 UTC (Sun) by kentonv (subscriber, #92073) [Link] (4 responses)

What do you mean? JavaScript is memory-safe.

Zig 2024 roadmap

Posted Feb 4, 2024 17:18 UTC (Sun) by IntrusionCM (subscriber, #140256) [Link] (3 responses)

In theory, thanks to Garbage collection, yes.

In practice, especially given its (ab-)use in backends, no.

Pure JavaScript as a browserspecific, client side only language has become niche.

Most complex frameworks have the need to bypass refcounting and thus make the GC collector useless.

Same goes for example for Java.

Memory safety isn't perfect in Rust, either - just far better as it was integrated into the core design of the language.

Zig 2024 roadmap

Posted Feb 4, 2024 17:56 UTC (Sun) by kentonv (subscriber, #92073) [Link] (2 responses)

What do you mean by "bypass refcounting"? JavaScript doesn't use refcounting.

Also "memory safety" isn't really about collecting garbage / leaks, it's about preventing memory access errors like out-of-bounds access or use-after-free. A garbage collector which never actually collects anything would technically be "memory safe".

Are you saying that e.g. JavaScript in Node.js doesn't sufficiently prevent such memory errors? Can you give a specific example?

Zig 2024 roadmap

Posted Feb 6, 2024 8:50 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (1 responses)

Changing the definition to have the definition apply… classic!

Zig 2024 roadmap

Posted Feb 6, 2024 14:30 UTC (Tue) by kentonv (subscriber, #92073) [Link]

From: https://en.wikipedia.org/wiki/Memory_safety

> Memory safety is the state of being protected from various software bugs and security vulnerabilities when dealing with memory access, such as buffer overflows and dangling pointers.[1] For example, Java is said to be memory-safe because its runtime error detection checks array bounds and pointer dereferences.[1] In contrast, C and C++ allow arbitrary pointer arithmetic with pointers implemented as direct memory addresses with no provision for bounds checking,[2] and thus are potentially memory-unsafe.[3]

Zig 2024 roadmap

Posted Feb 3, 2024 7:44 UTC (Sat) by rrolls (subscriber, #151126) [Link] (65 responses)

Rust is strict about memory safety to the exclusion of everything else.

When you're coming from C, Zig gets you 99% of the memory safety that Rust does.

The fact that it doesn't get you the remaining 1% is then a blessing, not a curse.

Rust is hard, and involves a lot of fighting with the compiler before you can get anything done. Zig is pragmatic.

Someone wrote a detailed explanation far better than I could: https://zackoverflow.dev/writing/unsafe-rust-vs-zig/

Zig 2024 roadmap

Posted Feb 3, 2024 10:01 UTC (Sat) by b7j0c (guest, #27559) [Link] (24 responses)

I wish more Rust advocates committed to writing something non-trivial with it (10k lines or more) before bashing us over the heads with advocacy. Please be sure to introduce async into the mix too. I don't dispute that some Rust advocates have actually passed through the crucible, but most have only trivial experience with it.

I recall (and was sadly part of) the Haskell hype wave about ten years ago...screaming that shared mutable state was a grievous sin and the only path to redemption was through Haskell's purity. Then I actually tried writing a large project in Haskell and was instantly deprogrammed. Rust in 2024 feels like a do-over of the Church of Haskell.

Zig 2024 roadmap

Posted Feb 3, 2024 10:24 UTC (Sat) by mb (subscriber, #50428) [Link] (13 responses)

>Please be sure to introduce async into the mix too.

Could you please show us what Zig does better than Rust when it comes to async programming?

>that shared mutable state was a grievous sin

Well, shared mutable state is one of the biggest sources of memory safety bugs.

Zig 2024 roadmap

Posted Feb 3, 2024 14:39 UTC (Sat) by b7j0c (guest, #27559) [Link] (12 responses)

Zig rolled back async which is outlined in the article above.

Zig is years away from being stable so they have years to get it right

Rust on the other hand is stuck with its async approach forever

Zig 2024 roadmap

Posted Feb 3, 2024 21:17 UTC (Sat) by khim (subscriber, #9252) [Link] (10 responses)

Does it mean that we are comparing something that actually exist and works (however poorly) to something that doesn't even exist?

And if you think Rust async story is awful I recommend you to try C++ one. Then come back and talk.

> Rust in 2024 feels like a do-over of the Church of Haskell.

It's only half-truth. Haskell asked you to redesign all your code and write everything from scratch.

It's literally impossible to translate many idioms from other languages into Haskell, not even unsafe Haskell.

Rust offers unsafe that actually makes it possible to adapt practically any designs from any language. It may not be a good idea to do that in many cases, but it's mere availability separates Rust craze from Haskell one.

As for Zig… I actually hope it'll do well: we really need a target to send all these “we code for the hardware” guys and Zig looks like nice one.

It would even, probably, work, for some time, at least.

Zig 2024 roadmap

Posted Feb 4, 2024 2:20 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (9 responses)

My thinking for those "Portable assembler" people is a revision of C which eliminates all the Undefined Behaviour and in the process also removes any semblance of modern convenience or decent performance.

This hypothetical language would have wrapping overflow for all its integer types, it would define all bounds misses to result in a zero value, and its memory model would be an idealized vast array of raw bytes, thus enabling Provenance Via Integers. Know your allocator will have put this new Goose at an address in memory sixteen more than this string you're about to free up? Just make a pointer to it by adding sixteen to the string pointer, no problem, it's not important when you do this, before you free it, after, either works. To deliver Sequential Consistency it would deliberately lack multi-threading. To deliver type safety it would eliminate new-fangled types like booleans, while underscoring that enums are just integers, strings are just arrays of bytes, etc. instead of "Make illegal states unrepresentable" instead this language would strive to "Make all representations legal states".

I think such a language could be made to deliver software that's maybe a thousand times slower than Python, while remaining no easier to learn than C today, and with no better tooling, this would be entirely safe (though it's probably important not to tell the beneficiaries that you've made their language safe, say instead that it's now "authentic" or something). They wouldn't necessarily _write_ software in this revised C, but they could spend their days arguing with each other about the definitions, which keeps them off the streets.

Zig 2024 roadmap

Posted Feb 4, 2024 18:10 UTC (Sun) by khim (subscriber, #9252) [Link] (8 responses)

> My thinking for those "Portable assembler" people is a revision of C which eliminates all the Undefined Behaviour and in the process also removes any semblance of modern convenience or decent performance.

It wouldn't work. Because they are after efficiency, just feel that compiler have to provide it if possible and leave their low-level tricks alone if it doesn't understand something.

Usually they even realize that some unlimited undefined behaviors are needed (look on their semi-serious proposals… Loosening the basic model part is precisely about that), they just couldn't accept the fact that it's either fully defined behavior, or fully undefined behavior, their dreams of, somehow, pulling ~~the rabbit out of the hat~~ what-the-hardware-does behavior into high-level languages just wouldn't work.

> This hypothetical language would have wrapping overflow for all its integer types, it would define all bounds misses to result in a zero value, and its memory model would be an idealized vast array of raw bytes, thus enabling Provenance Via Integers.

And even if you would invent a way to define show right shift is supposed to work… they would immediately turn around and ask why compiler doesn't do the sensible thing and doesn't put local variables in registers (which is impossible in such a language).

> I think such a language could be made to deliver software that's maybe a thousand times slower than Python

No, it wouldn't be even that much slower if you would use it as “portable assembler” and would write code like you are writing assembler and compiler (that may usually do some optimizations, but not if you “code for the hardware”) just doesn't do anything. The problem is that they wouldn't be satisfied, anyway.

> They wouldn't necessarily _write_ software in this revised C, but they could spend their days arguing with each other about the definitions, which keeps them off the streets.

They don't really spend that much time arguing about things. They actually produce code and their rants are usually limited to complains about how this or that compiler misunderstands their genious ideas and instead of producing optimized code breaks their valuable programs.

When asked about what compiler should do instead they usually either provide random incompatible answers or just say that “compiler should preserve hardware semantics” without elaborating what that phrase even means.

Zig 2024 roadmap

Posted Feb 5, 2024 14:53 UTC (Mon) by pbonzini (subscriber, #60935) [Link] (7 responses)

I never quite understood why shifts of negative values are still an issue now that C23 mandates twos complement representation. They should be changed to be equal to x*2^n for left shift, and x/2^n rounded towards negative infinity for right shift.

Zig 2024 roadmap

Posted Feb 5, 2024 15:01 UTC (Mon) by farnz (subscriber, #17727) [Link] (1 responses)

That then makes x >> y and x << y a multiple instruction sequence, not a single instruction sequence. On AArch64, for example, LSL x1, x2, x3 is defined as "take the bottom 6 bits of x3, shift x2 left by that amount"; this ignores the sign bit completely, and to implement the behaviour you're suggesting, I'd have to check the sign bit of x3, then choose whether to do an LSL, LSR, or ASR instruction based on signedness of x2's current contents and sign bit of x3.

Zig 2024 roadmap

Posted Feb 5, 2024 15:17 UTC (Mon) by khim (subscriber, #9252) [Link]

You are talking about right argument, pbonzini talks about left one.

Zig 2024 roadmap

Posted Feb 5, 2024 15:18 UTC (Mon) by khim (subscriber, #9252) [Link] (4 responses)

P1236R1 changed definition to precisely what you say in C++20.

Are you sure C23 haven't picked up that change?

Zig 2024 roadmap

Posted Feb 5, 2024 15:24 UTC (Mon) by pbonzini (subscriber, #60935) [Link]

Oh, I would be happy to be wrong!

Zig 2024 roadmap

Posted Feb 6, 2024 9:02 UTC (Tue) by pbonzini (subscriber, #60935) [Link] (2 responses)

Nope:

> 6.5.7 Bitwise shift operators
>
> [...] 4 The result of E1 << E2 is E1 left-shifted E2 bit positions [...] if E1 has a signed type and nonnegative value, and E1 * 2^E2 is representable in the result type, then that is the resulting value; otherwise, the behavior is undefined
>
> 5 The result of E1 >> E2 is E1 right-shifted E2 bit positions [...] If E1 has a signed type and a negative value, the resulting value is implementation-defined

Zig 2024 roadmap

Posted Feb 6, 2024 10:13 UTC (Tue) by khim (subscriber, #9252) [Link] (1 responses)

> Nope:

Ugh. I think someone should propose to fix it, then. C++ have finally removed “undefined behavior” from there (crazy values still trigger undefined behavior when E2, the right operand is “strange”, but anything is accepted as E1, the left operand — starting from C++20), thus in practice compilers already have code to handle everything properly

Zig 2024 roadmap

Posted Feb 6, 2024 15:25 UTC (Tue) by pbonzini (subscriber, #60935) [Link]

GCC and clang have promised for over 10 years to not use that latitude (not sure if left shift overflow is undefined, but anyway negative numbers can be shifted left and right with no other worries). So yeah it should be a no brainer.

Zig 2024 roadmap

Posted Feb 4, 2024 7:59 UTC (Sun) by pbonzini (subscriber, #60935) [Link]

If it's either async or portability (because they are not going to add LLVM support for it), it's dead on arrival. There are some good things about Zig but it has serious NIH problems.

Zig 2024 roadmap

Posted Feb 3, 2024 11:44 UTC (Sat) by kleptog (subscriber, #1183) [Link] (5 responses)

I'm not sure what your point is. There are several large Rust projects (firefox 2.3MLOC, rust-lang 1.8MLOC / redox 1.3MLOC / diem, tikv, servo 300KLOC / verloren 200KLOC). Are you trying to suggest that Rust is unsuitable for large projects? Because that's clearly not the case.

In my experience, I found Rust coding simpler than C++ for two main reasons: the error messages from the Rust compiler are more helpful, and you don't have the mental load of remembering what the lifetimes are are of all the variables, because it's all stated in the code.

What is true is that it's much harder building certain algorithms. That article tries to build a mark-sweep garbage collector, because it's something that a borrow-checker would have trouble with. And comes to the obvious conclusion it's much harder in Rust. But that's not representative of the majority of programs.

That it's harder to write such programs in Rust is not a downside in my eyes. Most applications do not need a mark/sweep garbage collector. A Rust program requires more planning up front to make sure the data structures are well defined, but that more than pays for itself later on as your application gets larger. In my experience, after junior developers get used to writing Rust, they become better C++ programmers because they understand that the concept of object lifetimes is very important for a correct program, even if the compiler can't check them.

Zig 2024 roadmap

Posted Feb 3, 2024 14:50 UTC (Sat) by b7j0c (guest, #27559) [Link] (1 responses)

Where did I say Rust was unsuitable for lage projects?

What I said was most advocacy for Rust is coming from people who likely aren't using it as such, but should.

Zig 2024 roadmap

Posted Feb 6, 2024 8:54 UTC (Tue) by LtWorf (subscriber, #124958) [Link]

Most people believe that their project (however simple it might be) is the apex of difficulty.

Zig 2024 roadmap

Posted Feb 23, 2024 18:05 UTC (Fri) by pawel44 (guest, #162008) [Link] (2 responses)

> There are several large Rust projects (firefox 2.3MLOC[...]).

I didn't know Firefox is written in Rust. I thought it's mainly C++.

Zig 2024 roadmap

Posted Feb 26, 2024 10:31 UTC (Mon) by kelvin (guest, #6694) [Link] (1 responses)

> I didn't know Firefox is written in Rust. I thought it's mainly C++.

Firefox originates from the Mozilla organization, and while C++ is still the most used language in Firefox as measured by lines of code, significant parts of Firefox are now written in Rust.

Here's a project which tracks the language stats of Firefox: https://4e6.github.io/firefox-lang-stats/

C++ ~9.9 MLOC
JavaScript ~9.5 MLOC
C ~5.3 MLOC
Rust ~4.3 MLOC

Zig 2024 roadmap

Posted Feb 26, 2024 10:33 UTC (Mon) by kelvin (guest, #6694) [Link]

> Firefox originates from the Mozilla organization

that was supposed to be "Rust originates from the Mozilla organization"

Zig 2024 roadmap

Posted Feb 3, 2024 13:06 UTC (Sat) by wtarreau (subscriber, #51152) [Link] (2 responses)

I'm sure there are plenty of good developers with good intents in the rust church, but I also think that a good part of the most vocal ones who constantly repeat the same buzzwords in the face of anyone have no clue about what a computer is, how memory works, how a CPU works, what an instruction is, and so on, and they're causing more harm than good to the group they claim to represent.

Zig 2024 roadmap

Posted Feb 3, 2024 13:19 UTC (Sat) by mb (subscriber, #50428) [Link]

Don't worry. Nobody is taking C or even Zig away. You can use them forever. I promise.
And you can ignore Rust or anything else. No problem at all.
Deal?

Zig 2024 roadmap

Posted Feb 4, 2024 1:37 UTC (Sun) by roc (subscriber, #30627) [Link]

What evidence do you have for this?

Almost all the Rust developers and advocates *I* know (mostly from Mozilla and Google) are like myself: veteran developers of large C and C++ projects who have been burned over and over again by severe bugs that Rust prevents, and want the pain to stop for ourselves and our users. People who know very well what a computer is and what an instruction is.

Zig 2024 roadmap

Posted Feb 4, 2024 19:51 UTC (Sun) by roc (subscriber, #30627) [Link]

FWIW I wrote about half of a 250K-ish line system (Pernosco) in Rust, using async for parts of it. It was a great experience and I'll use Rust for new production systems whenever I possibly can. It's not just about reliability and safety but also easy parallelism and maintainability. And FWIW I've been coding in C++ for over 30 years including stints at Mozilla and Google.

*My* wish is that more advocates of less safe languages such as C, C++ and Zig had experience working on multi-million line multithreaded programs that are constantly scrutinized by malicious experts. Then we'd hear less of "have you tried just not writing memory safety bugs?"

Zig 2024 roadmap

Posted Feb 3, 2024 10:14 UTC (Sat) by mb (subscriber, #50428) [Link]

>Someone wrote a detailed explanation far better than I could

I didn't read the whole text, but it seems to compare Unsafe Rust to Zig.
Which doesn't make a whole lot of sense to me.

Of course, writing Unsafe Rust is hard.

But the application, driver or embedded developer hardly writes any Unsafe Rust code. It's pretty common that whole applications are implemented in Rust and the Unsafe Rust parts are left to the generic abstraction layers. That makes the Unsafe Rust code be centralized and not spread around into each and every application.

>Rust is hard, and involves a lot of fighting with the compiler before you can get anything done

I would rather "fight" with the compiler than with the debugger to find my memory safety bugs.

Zig 2024 roadmap

Posted Feb 3, 2024 12:06 UTC (Sat) by ojeda (subscriber, #143370) [Link] (20 responses)

When you're coming from C, Zig gets you 99% of the memory safety that Rust does.

Could you please explain what do you mean by "99%"? To my understanding, 99% means catching the vast majority of bugs, but from their own documentation, it seems like memory safety is essentially like C's:

It is the Zig programmer's responsibility to ensure that a pointer is not accessed when the memory pointed to is no longer available. Note that a slice is a form of pointer, in that it references other memory.

In order to prevent bugs, there are some helpful conventions to follow when dealing with pointers. In general, when a function returns a pointer, the documentation for the function should explain who "owns" the pointer. This concept helps the programmer decide when it is appropriate, if ever, to free the pointer.

And from a quick try, I see that indeed trivial use-after-frees are not caught in 0.11/master, with -OReleaseSafe/-ODebug): https://godbolt.org/z/EG1dKz5MK, https://godbolt.org/z/zaW9e13aG.

In Rust, the equivalent program does not even compile.

Zig 2024 roadmap

Posted Feb 3, 2024 14:36 UTC (Sat) by rrolls (subscriber, #151126) [Link] (19 responses)

I suppose it depends on your viewpoint.

In my background I'm used to manually thinking about and managing lifetimes anyway. For example, using array indices rather than pointers, minimising the total number of allocations, documenting exactly how data is stored and in what order things should be "set up", "used" and "torn down" (in whatever means is appropriate for the language), passing responsibility for allocation towards the caller as much as is practical, and so on. I just don't write code in the first place that does "lots of little allocations and deallocations here there and everywhere"*; I think carefully about every single allocation; so, effectively by necessity, "lifetimes" isn't a very complex topic in my code, with the consequence that Rust's advantages don't really apply to it. (A function that returns a pointer to part of its own stack frame, like your example, is something I learned never to write a long time ago.)

*unless I'm using a high-level language like Python, where allocations happen constantly, but the garbage collector prevents any memory-unsafety troubles at the cost of performance. However, even in Python, I'm still thinking about the "semantics" of setup and teardown in the same way, because although memory management isn't involved in it, you can still run into the same traps with higher-level resources such as files, sockets, threads, processes, async tasks, or the state of a database.

On the other hand, I constantly find myself running into "completely obvious and stupid once you find it" buffer overflows, off-by-one errors, and so on. This is a class of problems that Zig solves immediately with slices. In one particular example, I had a known bug (as in, the effect was known, but not the explanation or the solution) in some C code that lasted for years because, each time the edge case that caused it arose, it would set a byte one beyond the end of a buffer to zero when that byte is usually zero anyway, but then just occasionally that byte isn't supposed to be zero, so something completely unrelated fails long after the code that wrongly sets that byte is done, making it impossible to figure out why. If that was Zig code, using a slice for that buffer would have given me a panic with a relevant stack trace immediately the very first time that write was detected, and I would have been able to fix it straight away. (Even valgrind didn't detect this bug: although it was out of bounds of the array, it was still part of a single memory allocation, hence why instead of causing an immediate crash it corrupted some unrelated behavior later.)

Of course it's not just slices. Zig also brings a bunch of other useful features that C doesn't have: tagged unions, comptime, RLS and its unique approach to error handling, to name a few. The first and last of these are also places where memory safety bugs completely unrelated to lifetimes show up in C over and over again. Rust has some of these features too, but that's irrelevant to the fact that Zig is still far better than C.

Basically my argument is that I feel Zig would solve the vast majority (if not all) of the "extremely hard" bugs I actually run into in practice, and while yes, Rust would not let code with those bugs compile, it would also make it a lot harder to write anything in the first place, but wouldn't bring me any further benefits beyond what I'd get from Zig. This is what I meant by "Zig is pragmatic." (The caveat of course being that I've only experimented a little so far with Zig, and almost not at all with Rust, so I can't exactly speak from experience with either language.)

So, yes, I was wrong to make such a sweeping generalisation with my original comment. But I hope I have at least demonstrated that the dismissive, rhetorical question of "Why are we, as an industry, pouring resources into non-memory-safe languages [like Zig]?" is equally misplaced :)

Zig 2024 roadmap

Posted Feb 4, 2024 1:47 UTC (Sun) by roc (subscriber, #30627) [Link] (11 responses)

Use-after-free (including dangling pointers into the stack) is a huge problem in large C and C++ applications and the source of a lot of CVEs. We've been trying "think carefully about every single allocation" for decades and it does not work at scale. ASAN, fuzzing, etc help but not enough; rare race and error conditions are too hard to hit.

Zig 2024 roadmap

Posted Feb 4, 2024 10:42 UTC (Sun) by b7j0c (guest, #27559) [Link] (7 responses)

It's not like Zig is absent on this

The testing allocator tells you about mishandled/unmanaged memory

Zig 2024 roadmap

Posted Feb 4, 2024 11:15 UTC (Sun) by mb (subscriber, #50428) [Link]

Does this also help, if the code path is not actually hit?

Maybe you can catch use-after-free with tests, if the use-after-free often happens in normal program flow, too.

But many security problems come from code paths being entered with incorrect input data that would never be executed alike in normal program flow.
In normal program flow with normal input data there often is no actual invalid memory access.

Can Zig find these problems?
Can Zig find multi thread data races?

Zig 2024 roadmap

Posted Feb 4, 2024 11:37 UTC (Sun) by atnot (guest, #124910) [Link] (5 responses)

As said above:
> ASAN, fuzzing, etc help but not enough; rare race and error conditions are too hard to hit.

The testing allocator is just basically built-in ASAN.

I should say Rust has this issue a bit too, thanks to the regrettable decision to disable overflow checks in release mode by default. I sort of get why they did it, it avoids the "why is my math microbenchmark 2% slower than the C version" from people first trying out the language, which would inevitably create a public impression that Rust is slower than C. But it's usually barely measurable in real world programs[1]. Granted, it's not as bad as in C because bounds checks are still applied post-overflow, but it's still annoying to find. And since the overflow checks won't show up in the profilers by default, people never optimize for that configuration. But I digress.

[1] Safe custom allocators, ime, tend to be very measurable, although there's a lot of work by e.g. Google on making it better. Still, it means Zig is arguably slower than Rust at iso safety, if you wanted to look at it that way.

Zig 2024 roadmap

Posted Feb 5, 2024 13:49 UTC (Mon) by khim (subscriber, #9252) [Link] (4 responses)

> But it's usually barely measurable in real world programs.

10x slowdown is “barely measurable”? What world do you live in?

> I should say Rust has this issue a bit too, thanks to the regrettable decision to disable overflow checks in release mode by default.

If you are talking about arithmetic (which is wrapping in Rust when not in debug mode) then they really had no choice: while trying to vectorize code with these checks is not impossible infrastructure is just not there. And they really had no resources to add custom passes to LLVM which would make the whole thing usable from Rust with non-wrapping arithmetic.

10x or more slowdown is not that uncommon if you enable these checks and then process large arrays of integers, which is definitely easily measurable in real world programs.

And having different rules for integers in arrays and standalone integers would be just too weird (although it may be interesting optional mode of compilation, now that I think about it).

Zig 2024 roadmap

Posted Feb 5, 2024 15:09 UTC (Mon) by atnot (guest, #124910) [Link] (2 responses)

> 10x slowdown is “barely measurable”? What world do you live in?

This is just based on just testing my own programs. Which are generally bottlenecked on memory not integer math, as most programs are. Even then, 10x is a ridiculous number, I can't find anyone reporting anything even close to 2x in real world programs.

But here, let's look at some actual data, someone running specint:

> On the other hand, signed integer overflow checking slows down SPEC CINT 2006 by 11.8% overall, with slowdown ranging from negligible (GCC, Perl, OMNeT++) to about 20% (Sjeng, H264Ref) to about 40% (HMMER). [...]
> Looking at HMMER, for example, we see that it spends >95% of its execution time in a function called P7Viterbi(). This function can be partially vectorized, but the version with integer overflow checks doesn’t get vectorized at all. [...]
> Sanjoy Das has a couple of patches that, together, solve [some missed optimizations]. Their overall effect on SPEC is to reduce the overhead of signed integer overflow checking to 8.7%.
https://blog.regehr.org/archives/1384

Specint is a bit biased towards HPC but we see, even there most normal business logic style code doesn't lose out at all. The losses are dominated by a few, very hot functions that are presumably heavily optimized already.

As you note, overflow checks interfere with vectorization, but so do millions of other things, it's notoriously finicky. Rust regularly misses autovectorization because of bounds checks too. It's very hard to write non-trivial code that vectorizes perfectly and reliably across platforms by accident.

Which gets me to the actual point I was getting at you ignored: In a hypothetical world where overflow checking was enabled by default, here's how this would have gone in a profiling session:

"why is hmmer::P7Viterbi() so slow now?"
"oh, it's not vectorizing because of overflow"
"let me replace it with a wrapping add, or trap outside of the loop body, or use iterators since I'm using Rust"
"that's better"

And millions of mysterious production bugs and a hundred CVEs would have been avoided, at barely any cost to most programmers and one extra profiling iteration of a thousand for a few people writing heavily integer math code.

Zig 2024 roadmap

Posted Feb 5, 2024 15:36 UTC (Mon) by khim (subscriber, #9252) [Link] (1 responses)

> In a hypothetical world where overflow checking was enabled by default

…Rust would have played the role of “new Haskell”: something which people talk about but don't use, except for a few eggheads and then rarely.

> And millions of mysterious production bugs and a hundred CVEs would have been avoided, at barely any cost to most programmers and one extra profiling iteration of a thousand for a few people writing heavily integer math code.

Nothing would have been avoided because Rust would have been just ignored. Rust, quite consciously, used up it's weirdness budget for other, more important, things.

Perhaps Rust with slow-integers-by-default would have saved someone from themselves, but chances are high that it would have hindered adoption of Rust too much: people are notoriously finicky about simple things and seeing these dozens of checks in the program which should be, by their understanding, two or three machine instructions long would have gave Rust a bad reputation for sure.

> This is just based on just testing my own programs.

If you are happy with that mode then why couldn't you just enable it in your code? -Z force-overflow-checks exists precisely because some people like these overflow checks.

I'm not big fun of them because in my experience for hundreds of bugs where some kind of buffer is too small and range checks are catching the issue there exist maybe one or two cases where simple integer overflow check is capable of catching the issue which is not also caught by these range checks. Certainly not enough to warrant these tales about millions of mysterious production bugs (why not trillions if you go for imaginary unjustified numbers, BTW?)

Zig 2024 roadmap

Posted Feb 5, 2024 16:26 UTC (Mon) by atnot (guest, #124910) [Link]

> Nothing would have been avoided because Rust would have been just ignored.

You're just making my own arguments back at me now, but snarkier. I said this two messages ago.

Look, I like Rust too, it's my personal language of choice. There's no need for this level of aggressive defensiveness over someone on the internet thinking it would be useful to make some pretty marginal tradeoff differently. With the data we have, I'm convinced it makes sense today and I've explained why. You're welcome to disagree.

Zig 2024 roadmap

Posted Feb 5, 2024 16:14 UTC (Mon) by mb (subscriber, #50428) [Link]

>10x or more slowdown is not that uncommon if you enable these checks and then process
>large arrays of integers, which is definitely easily measurable in real world programs.

You do not have to decide globally, if you want overflow checks or not.

You can enable overflow checks in release builds and for the performance critical code you can use https://doc.rust-lang.org/std/num/struct.Wrapping.html
With that you get fast code and safe code where it matters.

Zig 2024 roadmap

Posted Feb 5, 2024 7:02 UTC (Mon) by rghetta (subscriber, #39444) [Link] (2 responses)

I don't know for C, but I work on a 5MLOC C++ very active project, and in my experience memory errors are not so frequent, especially after C++/11. Each year we have *some* memory bugs (and none in production) but hundreds of logic errors. RAII, smart pointers, references, even templates can make a huge difference in preventing resource bugs, imho.

Zig 2024 roadmap

Posted Feb 5, 2024 19:00 UTC (Mon) by roc (subscriber, #30627) [Link] (1 responses)

Is it multithreaded and being fuzzed by experts?

Zig 2024 roadmap

Posted Feb 7, 2024 12:59 UTC (Wed) by rghetta (subscriber, #39444) [Link]

Multithreaded yes. Fuzzed only on import interfaces, not on the complete codebase.

Zig 2024 roadmap

Posted Feb 4, 2024 19:39 UTC (Sun) by ojeda (subscriber, #143370) [Link] (6 responses)

So, you are saying we can avoid temporal memory safety mistakes by "thinking carefully". But somehow, the same argument would not apply to spatial memory safety mistakes, and in fact, we would "constantly" make them.

Well, according to the Chromium project (and other projects), some of those issues you say we would not "run into in practice" are, in fact, quite prevalent sources of vulnerabilities: "Around 70% of our high severity security bugs are memory unsafety problems (that is, mistakes with C/C++ pointers). Half of those are use-after-free bugs."

> If that was Zig code, using a slice for that buffer would have given me a panic

...unless the safety checks are disabled, e.g. via -OReleaseFast.

> Rust would not let code with those bugs compile, it would also make it a lot harder to write anything in the first place

I mean, you state this as a fact, but you also recognize you have almost no experience with Rust. If you already think in terms of lifetimes anyway as you said, then writing safe Rust should be a very nice experience.

> I hope I have at least demonstrated that the dismissive, rhetorical question of "Why are we, as an industry, pouring resources into non-memory-safe languages [like Zig]?" is equally misplaced :)

It is not misplaced. The point is that nowadays we know how to do better. The industry (and other entities) is interested in getting away from memory unsafety as much as possible. Thus introducing a new language that essentially works like C (especially if you consider existing tooling for C) is not a good proposition.

Zig 2024 roadmap

Posted Feb 4, 2024 20:28 UTC (Sun) by roc (subscriber, #30627) [Link] (5 responses)

I think the question is misplaced because we're not actually pouring resources into Zig. Even advocates agree that it's years away from stabilization, and it's not going to get used much until after that happens. In the meantime, there are some interesting ideas like comptime that will get some testing.

We might even discover at some point in the future that "Rust, but comptime instead of generics and macros" would be an improvement on Rust.

Zig 2024 roadmap

Posted Feb 5, 2024 10:22 UTC (Mon) by farnz (subscriber, #17727) [Link] (4 responses)

comptime instead of generics is a non-starter, I suspect, since generics in Rust lay out data differently, not just change the code. But comptime instead of macros and some uses of traits would be extremely interesting to see; I suspect that there's a lot of cases where people currently have to write procmacros in Rust where comptime would be a good fit.

Zig 2024 roadmap

Posted Feb 5, 2024 13:44 UTC (Mon) by atnot (guest, #124910) [Link] (3 responses)

> comptime instead of generics is a non-starter, I suspect, since generics in Rust lay out data differently, not just change the code

You totally can do that! Zig and other languages with dependent-ish types generally have a unified type system for all types, including types themselves. In practical Zig terms, this means that you can't only return an integer from a comptime function, but the integer type itself, or a different type _depending_ on the arguments (that's where the term comes from), or an arbitrary struct type you just made, or a function, or anything really.

So in mathematical terms it's actually far more powerful than anything Rust has. Rust can do a few of these things as bespoke features but it's not a generalized system in the same way it is in dependently typed languages. This has its advantages and disadvantages, with dedicated syntax generally being more compact, readable and debuggable but also leaving weird incongruences between various parts of the language that are hard to solve, as Rust is experiencing.

It's been a pretty hip thing to experiment with somewhat recently (at least before effect systems really hit the scene) so I'm looking forward to seeing how Zig fares with it in a non-academic setting.

Zig 2024 roadmap

Posted Feb 5, 2024 19:02 UTC (Mon) by roc (subscriber, #30627) [Link] (2 responses)

What Zig does is not "dependent types" as in academia.

Zig 2024 roadmap

Posted Feb 5, 2024 20:16 UTC (Mon) by atnot (guest, #124910) [Link] (1 responses)

Yes, for anyone curious, one reason is that comptime isn't statically typed, it's dynamically typed but at compile time. For example, there is no way to write things like "comptime function that returns a function that returns either int or float", you can only write "comptime function that returns a function that returns some mystery surprise type". Like C++ templates, there's no type checking going on until after things have already been evaluated.

That said, you can do a lot of similar constructions and I think it faces a lot of similar issues regarding ergonomics when used in a non-fp language. Plus I think it does also demonstrate some of the benefits of having a single unified type system nicely without having to teach someone to read haskell-like syntax.

Zig 2024 roadmap

Posted Feb 5, 2024 20:35 UTC (Mon) by atnot (guest, #124910) [Link]

C++ templates are probably actually a good example in multiple ways here. Anyone really into their types would sneer at someone calling templates generics, they aren't really generics for similar reasons as comptime isn't dependent typing. But they're still extraordinarily popular and helpful, and they make it very easy to sell people on proper generics by pointing at them and saying "they're like that, but better".

Zig 2024 roadmap

Posted Feb 3, 2024 16:02 UTC (Sat) by quotemstr (subscriber, #45331) [Link] (17 responses)

99% memory safety is still memory unsafe. What proponents of memory unsafe languages (like the apologists on the C++ standards committee) don't understand is that there's a ton of value in proving a program can't have certain classes of defect. Zig can't do that. Languages that don't support memory safety proofs are all obsolete.

Zig 2024 roadmap

Posted Feb 3, 2024 17:00 UTC (Sat) by ballombe (subscriber, #9523) [Link] (8 responses)

Sound Rust does not provide 100% memory safety, only the illusion of 100% memory safety.

I work on a C software that use a custom memory manager. It works this way:
Use mmap to allocate a large ( 1GB to 1TB) array of virtual memory.
The custom memory allocator will give you a range of indices that you can use in this array.
From the point of view of rust (and valgrind!), all the memory is owned by main(), there is no pointers, all accesses are checked to be within the bound of the allocation, all memory is initialized etc.

But really, there is nothing that prevent the code to write outside the allocated range of indices as long as it is inside the array. So buffer overflow are still possible, even though the memory manager is safer than malloc.

Zig 2024 roadmap

Posted Feb 3, 2024 17:32 UTC (Sat) by mb (subscriber, #50428) [Link]

You are right. C software can have memory safety bugs.

Zig 2024 roadmap

Posted Feb 3, 2024 17:35 UTC (Sat) by mpr22 (subscriber, #60784) [Link]

What's that? If you disconnect the sensor wire from your sawstop table saw, it cuts your finger off when you touch the blade instead of triggering the pyrotechnics to drop the saw?

Zig 2024 roadmap

Posted Feb 3, 2024 17:41 UTC (Sat) by quotemstr (subscriber, #45331) [Link] (2 responses)

BTW: there are annotations you can use to tell valgrind about how your custom heap works and how to detect unsafe memory accesses within it

Zig 2024 roadmap

Posted Feb 3, 2024 20:42 UTC (Sat) by ballombe (subscriber, #9523) [Link] (1 responses)

Yup, we use those.
But my point is that we could port the software to rust and it would not be anymore memory safe than in C.

Zig 2024 roadmap

Posted Feb 3, 2024 21:07 UTC (Sat) by mb (subscriber, #50428) [Link]

It just shows that you don't know what Safe Rust is.
That is fine, though.
I invite you to learn new things.

Zig 2024 roadmap

Posted Feb 4, 2024 16:39 UTC (Sun) by matthias (subscriber, #94967) [Link] (2 responses)

> Sound Rust does not provide 100% memory safety, only the illusion of 100% memory safety.

Rust is quite precise in what memory safety means. First and foremost it means that it is impossible to invoke undefined behavior, e.g. data races. It certainly does not mean that the owner of a portion of memory cannot write data to the memory.

> I work on a C software that use a custom memory manager. It works this way:
> Use mmap to allocate a large ( 1GB to 1TB) array of virtual memory.
> The custom memory allocator will give you a range of indices that you can use in this array.

The question is: Who owns the array? If it is owned by the allocator, then all reads and writes must go through the allocator. Only the function/struct that owns this array is allowed to access it. Of course you can give away mutable access to the array, but again, there can be only a single owner of the mutable reference.

> From the point of view of rust (and valgrind!), all the memory is owned by main(), there is no pointers, all accesses are checked to be within the bound of the allocation, all memory is initialized etc.

Why should the memory be owned by main()? It should be owned by your custom allocator (which is transitively owned by main()).

> But really, there is nothing that prevent the code to write outside the allocated range of indices as long as it is inside the array. So buffer overflow are still possible, even though the memory manager is safer than malloc.

It depends how you give access to the memory. If you really only give indices and have a functions in the allocator for reads and writes, then yes, you have a function that can write to arbitrary locations in the array. It is perfectly safe to do so and this will not invoke undefined behavior. The program might not do what you want, but this is not what memory safety means.

If you you give slices of the array, then each receiver of a slice can only write to that slice and you have bounds checking etc. This is the only way different parts of the program can modify parts of the array independent of each other. However, rust will make sure that they only write to the parts of the array, the custom allocator has reserved for them. Writing such an allocator certainly requires some unsafe code to split up the one array into several smaller pieces that have independent lifetimes. There is at least split_at_mut() as a safe abstraction, so you could build this type of allocator using only safe code, but you certainly will need a bit more fine grained control if you really write an allocator yourself.

Valgrind will only see individual allocations, but rusts ownership tracking can be much more fine grained than a single allocation from the system allocator. And clearly it will ensure that you cannot overwrite the stack and that there are no data races and much more.

Zig 2024 roadmap

Posted Feb 4, 2024 17:18 UTC (Sun) by mb (subscriber, #50428) [Link]

>you could build this type of allocator using only safe code

Certainly.

But note that the allocator traits are unsafe. That means if an allocator implements these traits, it promises to not violate memory safety rules. Therefore, an allocator can't reduce Rust's 100%-safe guarantee, unless it is unsound. An unsound allocator is a bug.

And if you don't register the allocator to Rust by implementing the unsafe trait and your allocator is 100% safe Rust, then it's just a normal piece of code that is also 100% memory safe. Doesn't reduce Rust's guarantees either.

Rust's safety guarantees assume and depend on all unsafe code and the operating environment (all linked libraries, the operating system and the hardware) to be sound and not violate Rust's memory model.

Of course that means in practice one will probably find bugs in unsafe or foreign language code that breaks Rust's safety guarantees. A CVE in libc, for example (https://lwn.net/Articles/960289/). But that is an argument for pushing Rust code even further down the chain of dependencies, into the operating system.

Zig 2024 roadmap

Posted Feb 5, 2024 10:13 UTC (Mon) by ballombe (subscriber, #9523) [Link]

> The program might not do what you want, but this is not what memory safety means.

This is my point. rust defines memory safety, not the other way round.

Zig 2024 roadmap

Posted Feb 3, 2024 21:56 UTC (Sat) by pebolle (guest, #35204) [Link] (6 responses)

> 99% memory safety is still memory unsafe.

100% safe $THING does next to nothing, at prohibitive costs. I'm not an engineer, but I think that's engineering 101.

What would an 100% safe hammer look like?

Zig 2024 roadmap

Posted Feb 3, 2024 22:18 UTC (Sat) by mpr22 (subscriber, #60784) [Link] (2 responses)

Terrible analogy.

Hammers, being physical artifacts designed for applying a concentrated mechanical impulse to other physical artifacts, are inherently dangerous.

Most interesting code does not have an inherent need to be allowed to violate (or even honour in a hard-to-automatically-validate way) memory safety.

Zig 2024 roadmap

Posted Feb 3, 2024 23:00 UTC (Sat) by pebolle (guest, #35204) [Link] (1 responses)

Knifes, "being physical artifacts designed for applying a concentrated mechanical impulse to other physical artifacts, are inherently dangerous." I'm sure your household includes knifes. You get my point.

Is there an example of a 100% memory safe language that allows one to do interesting things at an acceptable cost?

Zig 2024 roadmap

Posted Feb 4, 2024 3:25 UTC (Sun) by tialaramex (subscriber, #21167) [Link]

I'm going to guess that you aren't English.

In English law it's a criminal offence to have any "bladed article" in public unless either you have a specific lawful excuse for why you needed that object here and now (and no "Self defence" is specifically never a lawful excuse) or it meets some very narrow criteria which protect small folding pocket tools such as a "Swiss army" knife.

It doesn't even matter whether the blade is sharp. I used to carry a butter knife to open the front door of a friend's flat when I was a student (saves going down to the building entrance to let her know I'm outside, and this is before everybody has a phone), but it was important not to take it off site, because that's a bladed article, even though you'd be lucky to cause a graze never mind serious injury a police officer knows that's enough to arrest you.

And yes, WUFFS is an entirely safe language. The price you pay is Generality, WUFFS is a special purpose language for Wrangling Untrusted File Formats Safely (hence the name) and so you cannot write general purpose programs in it. In exchange for this high price you get entirely safe programs (for example bounds checking isn't added by the compiler, it will just inherently reject any programs with potential bounds misses, they won't compile, so either your code just has no misses or you wrote the bounds checking correctly yourself and averted such misses) and of course you get much better performance than a human programmer would achieve in languages like C.

Zig 2024 roadmap

Posted Feb 4, 2024 9:55 UTC (Sun) by jmalcolm (subscriber, #8876) [Link] (2 responses)

There is no such thing as "compile time checking" for a hammer. It has unlimited flexibility at "run time". So no, there is no such thing as a 100% safe hammer.

Zig 2024 roadmap

Posted Feb 4, 2024 10:50 UTC (Sun) by mb (subscriber, #50428) [Link]

Absolutely correct.

But we can actually write the Rust->hammer analogy:

A hammer is intrinsically unsafe.
Rust recognizes that a hammer can't be safe. It's intrinsically unsafe, just like a pointer. By itself it can't be used safely.
Rust builds safe abstractions around intrinsically unsafe hammers. Let's call them trained professionals, or "carpenters". In this analogy the carpenter is trained extremely well and can never make mistakes. The abstraction is carefully reviewed for mistakes.
These abstractions can be used safely. It is 100% safe to call the carpenter to use the hammer for you safely.
Yes, it is a little bit more complicated to call a carpenter instead of using the hammer yourself.
Yes, you will get compile time errors (carpenter saying: It is not safe to hammer on this screw).
But it's safe.

Instead of training 1000 people about safe usage of a hammer, we train 1 carpenter extremely well so that the 1000 people don't have to worry about tears and broken thumbs.

Zig 2024 roadmap

Posted Feb 4, 2024 16:40 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

There kind of is "compile-time checking" for a hammer; the manufacturer calls it "quality control", and it's there to make sure that the carpenter, if they do their job properly, won't get injured in a way that attracts an unacceptable financial liability to the manufacturer or tool supplier.

Zig 2024 roadmap

Posted Feb 3, 2024 23:21 UTC (Sat) by ju3Ceemi (subscriber, #102464) [Link]

Obsoletes
Hahaha
You are a funny guy

Zig 2024 roadmap

Posted Feb 3, 2024 7:34 UTC (Sat) by rrolls (subscriber, #151126) [Link]

Ah, I watched this one! Excellent writeup.

I've been keeping an eye on Zig since about version 0.8 or 0.9, and I've messed around with it a little. Of all the new languages I've seen pop up over the last couple of decades, this is the one that's excited me the most. As well as the language itself, I like what I see from their ethics and financial model, too.

I do expect and hope it to become my go-to language at some point, though currently I'm in that "waiting for it to become stable" camp.

Zig 2024 roadmap

Posted Feb 3, 2024 14:50 UTC (Sat) by amodm (subscriber, #33731) [Link] (5 responses)

I'm actually excited that there's a systems language camp that's considering speed of compilation as an ambitious enough goal to actually want to replace LLVM. It pushes for much needed competition in the compiler space.

Zig 2024 roadmap

Posted Feb 4, 2024 1:49 UTC (Sun) by roc (subscriber, #30627) [Link] (4 responses)

Rust's cranelift project has a similar motivation.

It's ironic, because one of the original marketing points for LLVM over gcc was faster compilation.

Zig 2024 roadmap

Posted Feb 4, 2024 11:57 UTC (Sun) by atnot (guest, #124910) [Link] (3 responses)

I feel like there is this inevitable curve every compiler backend goes through where, unless given some external constraint, people will always find one more optimization that's just a tiny bit more expensive but makes huge gains in output quality in some scenario. And if you could just tweak this data structure a little bit they would be so much easier to write. Then you repeat that 100 times and you end up just where GCC and LLVM are with huge, slow data structures built to get every last bit of performance out at the cost of baseline compile times.

I'm a bit more optimistic about cranelift there because it always has to remain usable for interactive use with WASM. Go achieves that constraint by refusing to make a release if the compile time is slower than before. I hope Zig does something similar.

Zig 2024 roadmap

Posted Feb 4, 2024 14:53 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (2 responses)

My optimism for Cranelift is that they seem more enthusiastic about delivering on clear semantics for their IR. A recurring problem with LLVM is that the IR was built by C++ people, who have a very lax attitude to tricky semantic correctness problems because C++ itself just punts all those problems to the programmer, knowing the programmer will just ignore them and so too bad the end users eat the consequences, however bemusing or outright lethal those might be.

With an already unsafe language like C++ the resulting miscompilations barely register, but for Rust it "optimizes" enough routine provably safe (even if sometimes daft) code into nonsense that it's a significant blight.

Zig 2024 roadmap

Posted Feb 4, 2024 19:19 UTC (Sun) by roc (subscriber, #30627) [Link] (1 responses)

I don't think LLVM is a lost cause there --- John Regehr and others are doing heroic work in this area --- but it sure is a problem.

Zig 2024 roadmap

Posted Feb 4, 2024 20:18 UTC (Sun) by Wol (subscriber, #4433) [Link]

Yup. I got the impression the LLVM people were quite happy to fix the problems, but if it's pervasive right through the IT and implementation, I can understand it being a long and painful process.

Certainly I've seen a fair few complaints that "C/C++ isn't strict, so the LLVM isn't strict. Rust is strict and it breaks".

Cheers,
Wol