LWN: Comments on "Five years of Rust"

Five years of Rust

mathstuf — Tue, 26 May 2020 17:43:00 +0000

They also expect `.pc` files for crate artifacts (which cargo doesn't bother with by doing builds of external dependencies directly) or a stable Rust ABI so that `.rlib` files could be installed and shared. The former doesn't make sense because the latter doesn't exist. I also want it, but it isn't there yet and it's one of those things that needs a decent amount of foresight before freezing it.

Five years of Rust

Cyberax — Mon, 25 May 2020 05:13:31 +0000

So to distill it:
1. Meson developers first wanted to do package and dependency resolution themselves. Which is impractical, because Cargo is complicated.
2. They can also work with Cargo developers to provide missing introspection API, but I couldn't get the sense of what exactly is missing from libcargo.
3. Meson doesn't support external build systems at all. There's work to do it: https://github.com/mesonbuild/meson/pull/6920

So I'm kinda confused. I think it's mostly Meson's problem, it should be more flexible about the way it treats external dependencies.

Five years of Rust

marcH — Mon, 25 May 2020 00:56:32 +0000

Long version: https://github.com/mesonbuild/meson/pull/2617
Shorter version: https://github.com/mesonbuild/meson/issues/2173

Five years of Rust

rillian — Thu, 21 May 2020 17:49:11 +0000

The mach front-end generally calls cargo to build rust source. For code linked directly into the core rendering engine, there's a top-level crate which acts as the head of the rust dependency and generates a static library where all the C/C++ ABI symbols are available. This lets cargo do its thing while providing a central point of inclusion, which avoids duplicates, and the whole thing integrates as a sub-target within the larger build system.

There are other uses of rust of course: ancillary tools, tests, binding generators in both directions, wasm targets for inclusion in the greater application and so on. Mozilla uses a monolithic repo, so all rust source is vendored into the source tree to allow offline builds. Cargo helps maintain that as well.

The rustc command-line compiler works perfectly well when called from other build systems. I think one of the reasons people say you need to use cargo is that cargo is a package manager as well as a build tool. That makes it very easy to publish and use external libraries. That library ecosystem is another advantage rust has over traditional C/C++ code. A small commandline tool like ripgrep is made of 10 internal modules and includes 9 external libraries which themselves pull in 15 more libraries.

To build all that you not only must implement reading (or manually rewrite) the build description intended for cargo, but you need to implement the package retrieval functions as well. And at that point, you might as well have used cargo.

Five years of Rust

mgedmin — Thu, 21 May 2020 14:14:01 +0000

They have a very nice build tool called 'mach': https://developer.mozilla.org/en-US/docs/Mozilla/Develope...

Technically it's not a build system but a frontend to a build system. I've no idea how it all works under the hood.

Five years of Rust

farnz — Thu, 21 May 2020 09:13:45 +0000

Building Rust with Buck is something the Buck developers support; the hard part is that you then need tooling to do the dependency analysis and importing that Cargo does for published crates to make building Rust code with Buck useful.

Realistically, you can only do this by using cargo metadata to get what Cargo thinks the world should be, and then rewriting the resulting metadata into your own build rules.

Five years of Rust

Cyberax — Thu, 21 May 2020 03:26:47 +0000

> We really, really, really would like to support building rust code with meson, and we're trying as hard as we can. But I have to agree, rust has basically told us "cargo is the only way to build rust code."
I'm curious, what is the problem with Rust? You surely can invoke rustc directly from Meson and you can fairly easily use Cargo to vendor the dependencies locally. I've done this for a Makefile-based project that has weird Rust-C++-Rust dependency loops (don't ask).

I guess it might be tricky if you want to do dependency resolution entirely in Meson.

Five years of Rust

ofranja — Thu, 21 May 2020 01:54:52 +0000

Do you have any link/reference for this statement about cargo being the only way to build Rust code? Was it done by someone on the language team?

I know of a couple of projects using rustc without any issues, so I'm curious about the reasons.

Five years of Rust

mathstuf — Wed, 20 May 2020 12:38:51 +0000

If you're not using other crates, calling `rustc` directly is probably fine (it can just dump out a static lib for you to link to or objects if you want to link yourself). It's the inter-crate dependency resolution and such you really want `cargo` for. So incremental replacement with stdlib-only Rust code should be fine.

Five years of Rust

epa — Wed, 20 May 2020 09:39:03 +0000

What are the Mozilla developers doing to build their mixed C++ and Rust codebase?

Five years of Rust

tbleher — Tue, 19 May 2020 20:27:43 +0000

As a C++ developer developing realtime software, this is sad to hear. I consider Rust a very interesting language (though I haven't really used it yet), but being tied to a single build system is a big no-go from my point of view.

Five years of Rust

dbaker — Mon, 18 May 2020 22:16:53 +0000

Meson developer here.

We really, really, really would like to support building rust code with meson, and we're trying as hard as we can. But I have to agree, rust has basically told us "cargo is the only way to build rust code." And that stinks for a whole lot of reasons. As a user, maybe I want to incrementally switch a large project to rust. Maybe I only want to use rust for a small part of my project, maybe I have other real reasons I don't want to use cargo. From meson's point of view we don't want to call another build system, we want to call c++ or rustc or dmd or javac etc, and run all of the build from a single ninja invocation, where all of the build is ordered as optimally as possible and ninja knows what needs to be rebuilt at any given time.

Five years of Rust

zdzichu — Mon, 18 May 2020 12:23:10 +0000

In theory, you do not download "whatever" but specific sha1 commit. You can omit sha1 when defining dependencies, but you give up reproducibility.

Five years of Rust

LtWorf — Mon, 18 May 2020 10:41:47 +0000

How is it reproducible when you connect to github and start downloading whatever?

Five years of Rust

HelloWorld — Sun, 17 May 2020 13:12:15 +0000

> TL;DR I really like Rust as a lang, but needing 15 minutes to compile a text editor on a 3950X gives me a bit mixed feelings.

I think the right solution for this is better tooling. There is no reason for your computer to compile stuff that has been compiled hundreds and thousands of times before on other people's machines. Bazel gets this right:
https://docs.bazel.build/versions/master/remote-caching.html

Five years of Rust

farnz — Sun, 17 May 2020 10:37:20 +0000

Cargo does have support for local registries, in two different forms.

The one that's less useful for distros is separate registries, where you can have a secondary source of private packages (useful for in-house development with Cargo, for example).

The more useful form is source replacement, where you tell Cargo that when it's looking for packages from an upstream registry, it should instead look at the local subset mirror. This enables you to set up a registry of just packages that you've packaged, and have Cargo pull from there. If you patch a package in this registry (e.g. libgit2-sys), then all the packages built against it will pick up the patched form.

Five years of Rust

mathstuf — Sun, 17 May 2020 00:15:55 +0000

I feel your pain. I did some Haskell Fedora maint and haven't really done much the past few years. Stackage really helped though. I suspect Rust will get something like it some day. https://lib.rs might be the start of that.

> It's super annoying to patch Rust crates (e.g. when we bump libgit2 we usually have to patch each rust crate individually because we need to add a patch-section to the Cargo.toml of each package...).

I think it'd be better if Cargo supported local registries. Then you could install the `-devel` package for a Rust library (likely the only thing to exist for them) which adds it to some distro-maintained registry store that uses trigger scripts to sync up. Custom index, crate storage, etc. Then you shouldn't need to patch anything since everything finds what's in the local index.

> It also seems like the API is getting broken more often in Rust land because it's so easy to pull in old versions, so backwards compat isn't as much of a concern.

I haven't seen crates breaking backwards compat all that often. It's pretty good with its semver adherence (MSRV not withstanding, but distros usually can ignore that problem since if the compiler's not getting updated, the tools probably aren't either). I think what tends to happen is that you add a dep at time T. Later, after some `cargo update` calls, you end up using an API that has introduced in some point or minor release and never go back to update your dependency version to be accurate. On that front, I've not been idle there:

https://github.com/pulls?q=is%3Apr+author%3Amathstuf+arch...
https://github.com/pulls?q=is%3Apr+author%3Amathstuf+arch...

But it seems some maintainers don't care. If you want to put some weight behind any of the open ones, I'll not complain :) . Getting this supported and used in CI widely is the long-term solution for this problem:

https://github.com/rust-lang/cargo/issues/5657

I do it for my crates (at least as far as I can; these deps not supporting it blocks me about halfway through my stack).

> (and it's hard to cache anything when every crate uses a slightly different version of a dep)

sccache is the tool of choice here, but it is sensitive to build directory location (though I think that's fine for at least Koji builds). And distros don't want caches because of reproducibility then any verification is more expensive anyways, so caching doesn't actually help. Or packages could ship their sccache stores in another package (at least what they add to the store over the packages it depends on)? I think getting support for an ad hoc registry is likely the way to go. That said, even if it did become widespread, I'd still just use crates.io for development (CI could verify any packaging subsets for my dependencies).

Five years of Rust

Cogitri — Sat, 16 May 2020 22:48:02 +0000

At least from a distro perspective I'd take meson/CMake (or even Autotools) over cargo any day of the week. I understand that cargo _is_ very convient for devs - it sure is nice when adding one line to your cargo.toml is all you have to do to get things running. But it's a lot of trouble for distros. It's super annoying to patch Rust crates (e.g. when we bump libgit2 we usually have to patch each rust crate individually because we need to add a patch-section to the Cargo.toml of each package...). It also seems like the API is getting broken more often in Rust land because it's so easy to pull in old versions, so backwards compat isn't as much of a concern. Compile times are also pretty meh because all the deps are getting compiled for each crate (and it's hard to cache anything when every crate uses a slightly different version of a dep).

TL;DR I really like Rust as a lang, but needing 15 minutes to compile a text editor on a 3950X gives me a bit mixed feelings. The amount of deps everything has also makes me a bit uncomfortable, it's uncommon to see binary crates with less than 200 deps for me.

Five years of Rust

ibukanov — Sat, 16 May 2020 20:42:46 +0000

Not everything is re-written in Go. Sqlight still implies cgo, for example.

Five years of Rust

josh — Sat, 16 May 2020 17:29:43 +0000

That's exactly how I would write that, as well. Trying to find combinator functions to make a one-liner doesn't always make the code more readable.

Also, for any argument parsing beyond the simplest test program, I'd likely pull in the "clap" crate, which handles command-line arguments exceptionally well.

Five years of Rust

josh — Sat, 16 May 2020 17:28:13 +0000

Exactly. If you don't have an argument you want to use the default, but if you have an argument that doesn't parse as a port, you should produce an error.

Five years of Rust

oever — Sat, 16 May 2020 17:17:32 +0000

let port: u16 = args.get(2).and_then(|n| n.parse().ok()).unwrap_or(7878);

This way is not one line but perfectly readable and gives an error message when the port argument cannot be parsed as in integer.

let port: u16 = if let Some(n) = args.get(2) {
    n.parse()?
} else {
    7878
};

Five years of Rust

Cyberax — Sat, 16 May 2020 17:11:39 +0000

This actually should work just fine, each plugin would essentially have its own private runtime.

Five years of Rust

Tobu — Sat, 16 May 2020 17:06:16 +0000

Then again, your example code conflates no port being provided (picking a default seems fine), and the port not parsing correctly. I would want to fail fast in the later case, instead of doing something the caller would not expect. The difference was evident when I tried to refactor that method chain to use "if let", so for that snippet, the language is driving me towards more correct code.

Five years of Rust

bluss — Sat, 16 May 2020 16:36:56 +0000

And cargo integrates with make jobserver, so it should do the right thing w.r.t parallelism and jobs, if it is invoked from make.

Five years of Rust

mathstuf — Sat, 16 May 2020 13:28:56 +0000

> You can call Go code just fine from C

If I write a PAM plugin in C, who is responsible for starting the Go runtime? What if I have two such plugins? How do they coordinate their Go runtime usage?

Five years of Rust

mathstuf — Sat, 16 May 2020 13:26:28 +0000

> Unless you're dong something unusual, you almost never want to call .ok();

And that's true here. There's no indication that:

./prog foo bar

is going to use 7878 as its port (assuming that's what this is for) rather than the "bar" service's port (which might also be a viable behavior. But silently ignoring it is not good. Even `unwrap()` might be better here because at least that lets the caller know they done goofed.

Five years of Rust

rvolgers — Sat, 16 May 2020 12:57:20 +0000

Could you give some more details about what you don't like about Cargo? It just feels strange to me since one of Rust's major advantages is easy integration with other languages, and I don't really see a specific complaint in your comment.

You can drive external build steps from Cargo's build using build.rs (https://doc.rust-lang.org/cargo/reference/build-script-ex...), and Cargo can produce statically linked object files with all Cargo dependencies for consumption by other builds (https://github.com/alexcrichton/rust-ffi-examples/tree/ma...).

If it's just a matter of "I want my build to just be makefiles", then I guess the reason nobody is doing that is because it's a bad idea and you lose much, much more than you gain.

Five years of Rust

dottedmag — Sat, 16 May 2020 10:17:35 +0000

That's because the build story for multiple programming languages is a dumpster fire.

Go clearly and unequivocally displayed that it is definitely possible to do one-command reliable reproducible builds, so doing anything less than that is just not defensible.

Five years of Rust

cxtuttle — Sat, 16 May 2020 01:05:53 +0000

Yeah, cgo is still used quite a bit. Things like database drivers and other interfaces tend to use it quote a bit. There is a performance cost to calling outside of Go, so it's used sparingly. Things which require more performance tend to use asm https://golang.org/doc/asm, as Go has an assembler and is actually an easy way to learn assembly. Some of the standard Math functions are in assembly.

Also the point about not exporting a C ABI. You can call Go code just fine from C, and also compile a Go library as a shared lib and call from any language that can call it.

For debugging Go, you'll want to use Delve: https://github.com/go-delve/delve

Five years of Rust

josh — Fri, 15 May 2020 23:57:28 +0000

> let port: u16 = args.get(2).and_then(|n| n.parse().ok()).unwrap_or(7878);

Unless you're dong something unusual, you almost never want to call .ok(); that's throwing away the error. And .and_then() exists for a chaining style that's occasionally useful but not the common case. Rather than trying to combine Option or Result values, I normally try to turn them back into what's inside them as soon as possible (unless I'm holding onto an Option because it represents something truly optional that I may or may not use later). Given a result, I *immediately* use ? on it, and then I don't have a Result anymore.

Five years of Rust

mathstuf — Fri, 15 May 2020 23:16:40 +0000

Oh, I'm well aware of CMake's faults :) . But if you're in its realm, there's nothing else that's viable IME. And if you're Unix-bound, I can see not having any dependencies that use CMake, but that's not really a thing on Windows once you start venturing much of anywhere.

Five years of Rust

Jonno — Fri, 15 May 2020 21:29:05 +0000

> I'm maybe less sold on Result, at least the fact that I can't easily just treat it as an Option

Of course you can. If you have a Result<T, E> named foo and want an Option<T>, you just write foo.ok(). Though most of the time you don't have to, as the ? operator and most methods works the same on both, and "if let Ok(x) = foo { ... }" is equivalent to "if let Some(x) = foo.ok() { ... }".

Five years of Rust

Sesse — Fri, 15 May 2020 21:20:25 +0000

I have enough hatred for CMake that I won't spill it in this thread (we use it at work), but rest assured, none of the Meson build files I've ever written needed CMake to find anything. :-) (When it finds something, it prints out where it came from.)

Five years of Rust

mathstuf — Fri, 15 May 2020 21:02:45 +0000

Ok, going back and reading this again, I think I see better now. I think the main thing is that HKT is missing. It's allow for some nicer combinators between Option and Result automatically. But without that, every combo would need a new method. But someone could make an OptionExt or ResultExt trait to fill them out.

FWIW, I work on CMake and you certainly don't have to bundle your deps (some projects do to ease Windows). You do need find modules, but they're generally pretty easy for simple packages and CMake ships the common things. IIRC, Meson gets around this by basically using CMake to find the richer packages it can represent, so if be careful giving it *too* much credit on that front. It's still stuck with pkg-config anemia otherwise (AFAIK).

The problem with leaving it to the systems or OS is that you get the wild west of C and C++ today where building is generally a nightmare. I mean Nix has ways of just making its packages from various language package management systems, but even it needs *something* to start from.

The biggest problem with pkg-config is the lack of support for mixing toolchains. That realm basically assumes GCC-comoatible flags which is...not the case in the real world.

Five years of Rust

mathstuf — Fri, 15 May 2020 20:51:45 +0000

You can drop the error of a Result<T, E> and make an Option<T> with the .ok() method.

Five years of Rust

joib — Fri, 15 May 2020 20:37:34 +0000

Oh, I forgot to answer your chained behavior question; isn't that exactly what the '?' operator does: https://doc.rust-lang.org/edition-guide/rust-2018/error-h...

And yes, I agree with your desire for the one true build tool to rule them all. But while we're waiting for that, cargo is IMHO a very fine single-language build tool, by far the best of all the ones I have tried.

Five years of Rust

Sesse — Fri, 15 May 2020 20:37:08 +0000

Sure, Option is an std::optional, and I think it's fine to have a forced error check (it's much better than exceptions). I'm maybe less sold on Result, at least the fact that I can't easily just treat it as an Option (either things failed or they didn't, I don't always care about the result code).

Five years of Rust

joib — Fri, 15 May 2020 20:30:21 +0000

I you're used to C style programming, it might help to think of Option as something which a C programmer in a similar situation would use a C pointer for, which can either point to a (hopefully!) valid value or be NULL. Except that Rust semantics forces you to test the value instead of dereferencing the NULL pointer, and yes, an Option isn't a pointer but a strongly typed enum, but in this case they are used similarly.

A result, OTOH, is a combination of a result and an error code. And yes, again, it's a strongly typed enum which doesn't have any direct equivalent in C.

tl;dr: Option - A value which is either present or not, vs Result - A result of an operation, or an error if the operation failed.

HTH, HAND.

Five years of Rust

Sesse — Fri, 15 May 2020 19:55:42 +0000

The idea about having something like std::optional is just fine, but… why are there two? Why are they so generally incompatible? And given that they explicitly have looked at Swift, why is it so hard to do chained behavior (if an error, keep it an error)?

Here's an example from my own program (albeit very small!). I want to use argv[2], or use 7878 as the default:

let port: u16 = args.get(2).and_then(|n| n.parse().ok()).unwrap_or(7878);

When you have this kind of system, why shouldn't I be able to do (I'm just inventing syntax here)

let port: u16 = args.get(2).then(parse).unwrap_or(7878);

I really do agree with the concept, it's just that the execution feels so unneccessarily clumsy. Maybe I've just been missing something, but I did ask more Rust-experienced people, and nobody could really suggest anything better.

As for package systems, I genuinely believe this is an OS / development environment task. As a user, I don't want to care whether my software (and likewise libraries) is built in C, Rust or something else, and I _certainly_ don't want to have to engulf every dependency I have into my own build system. (CMake in particular is a bad case of this. meson _can_ do it, but it's thoroughly optional, which is great. If you look to the scripting world, you have cases where some Ruby gems need to repack npm libraries because they depend on building JavaScript code for the client side—just another example that packaging systems should not be about which language the software is written in.) pkg-config doesn't install anything, so it doesn't solve all issues, but it is a fairly standardized interface to discover them.