Rustaceans at the border [LWN.net]

Rustaceans at the border

Posted Apr 14, 2022 19:59 UTC (Thu) by mb (subscriber, #50428)
In reply to: Rustaceans at the border by Gaelan
Parent article: Rustaceans at the border

I think the question is more like:
Should we depend our kernel build on external sources such as crates.io?

Yes, we should allow no_std crates to be used from kernel code.
(after careful review for things like floating point, etc..)

But how should we integrate the crate?
Should we only declare the version in the kernel's Cargo.toml and fetch the latest compatible SemVer at kernel build time? That would result in kernels with the same version having different content.
Should we pin a specific version via Cargo.lock or similar?
Should we copy the crate into the kernel tree to be independent of crates.io?

to post comments

Rustaceans at the border

Posted Apr 14, 2022 20:29 UTC (Thu) by djc (subscriber, #56880) [Link] (1 responses)

Keeping the Cargo.lock file in the Git repository is pretty common for Rust applications, we do that at work.

I don't see the point in Rust for Linux forbidding the use of crates.io crates. Yes, you'd probably want to enforce that only no_std crates are pulled in (I don't know if that's currently supported), but otherwise it seems like a great way of sharing functionality with other parts of the ecosystem.

I can see how pulling in an async runtime would be a bigger deal, but at the same time being able to write async code instead of callback hell seems like it might be important for making code easier to understand the same way it does in userspace.

Rustaceans at the border

Posted Apr 14, 2022 22:48 UTC (Thu) by ssokolow (guest, #94568) [Link]

Keeping the Cargo.lock file in the Git repository is pretty common for Rust applications, we do that at work.

I think it'd be more likely they'd want to use cargo vendor (which automatically vendors the specified dependencies) so the kernel repository can use third-party crates but rely on crates.io only as a means of standardizing/automating the rote steps in their existing workflow for manually OKing updates to vendored dependencies.

I can see how pulling in an async runtime would be a bigger deal, but at the same time being able to write async code instead of callback hell seems like it might be important for making code easier to understand the same way it does in userspace.

They'd almost certainly want to write their own executor which is just an API adapter for the relevant kernel machinery.

That's the whole point of Rust's async system being designed as it is. To avoid tying everyone to a single executor.

(Yeah, most people use tokio because the APIs provided beyond the executor itself are still working toward standardization, but those are generally std things like userspace I/O anyway, so that's not relevant to kernel or embedded use-cases.)

Rustaceans at the border

Posted Apr 14, 2022 22:42 UTC (Thu) by josh (subscriber, #17465) [Link] (28 responses)

I absolutely *don't* think we should pull in third-party crates over the network. I do think we should have a means of "vendoring" unmodified upstream crates that work in the kernel, without having to rewrite them.

Rustaceans at the border

Posted Apr 15, 2022 0:32 UTC (Fri) by IanKelling (subscriber, #89418) [Link] (21 responses)

> I absolutely *don't* think we should pull in third-party crates over the network.

I strongly agree.

For multiple reasons, including the fact that when I go to https://crates.io I get a blank page which says I should enable javascript. Please don't make kernel development require remote code execution on my machine.

Rustaceans at the border

Posted Apr 15, 2022 0:55 UTC (Fri) by mjg59 (subscriber, #23239) [Link] (18 responses)

I just want to be clear on this - you'll download kernel source code from the internet and type "make", resulting in a great deal of code running on your system, but you're not ok with javascript running inside a strongly sandboxed environment?

Rustaceans at the border

Posted Apr 15, 2022 8:39 UTC (Fri) by tux3 (subscriber, #101245) [Link] (3 responses)

Presumably a possible defense for the more general version of this position is that the review process is very different, between web pages in general and the kernel.

While I personally run most JavaSscript without a second though, some websites that are not crates.io sometimes run undesirable JS - cryptominers, supercookies, what have you. Sometimes there is third-party JS code that has clever ways of escaping the sandboxes.
I am told Firefox's content process lost connection to the X11 server just a month ago. Although the sandbox is improving as well, hardness may sometimes be a relative thing.

In contrast, I will trust Debian to update many millions of lines of code straight from the internet into a running system, much of it which could enter my $HOME.
If the kernel were to pull in more external libraries, like it did with zstd, there's more than a couple git remotes that I'd feel I could pull from without being gifted any crypto miner or other code that might try to poke at a sandbox.

Rustaceans at the border

Posted Apr 15, 2022 8:49 UTC (Fri) by mjg59 (subscriber, #23239) [Link] (2 responses)

I have no reason to believe that kernel code gets significantly more review than crates.io does - I agree that the web as a whole doesn't meet that bar, but frankly most software I pull from Debian has also had much less review than the kernel does (https://lists.debian.org/debian-devel/2003/02/msg00771.html is an example of this not going well) so it feels like what's missing is a way to express what level of trust I place in any provider of code I end up executing rather than just to assert that websites that use Javascript are unacceptable .

Rustaceans at the border

Posted Apr 15, 2022 9:08 UTC (Fri) by tux3 (subscriber, #101245) [Link] (1 responses)

I agree with that. As for crates.io, I have respect for the work they do and I'm happy to run their code (though I have not read it).
This may be getting off-topic, but now I'm curious if you have anything specific in mind when you write about expressing levels of trust — would that look like further sandboxing?

Rustaceans at the border

Posted Apr 15, 2022 9:27 UTC (Fri) by mjg59 (subscriber, #23239) [Link]

Great question! I spent a while looking into whether it was feasible to apply different LSM profiles (SELinux/Apparmor/whatever) to dpkg depending on where the package was downloaded from, and unfortunately the architecture doesn't make that terribly possible. From the web perspective, I think that probably comes down to extension-level handling at the moment? In an ideal universe we'd have infrastructure to tie any given piece of javascript back to an upstream repo and make a trust decision based on things like licensing and review assurances, but that feels like kind of a lot of work.

Rustaceans at the border

Posted Apr 15, 2022 13:51 UTC (Fri) by IanKelling (subscriber, #89418) [Link]

Hah good point. I do generally try to also run make within a sandbox too though. Websites should not requiring people to download and run a program to simply read a page of text. Crates.io is almost entirely used for text that could be simple html.

Rustaceans at the border

Posted Apr 19, 2022 8:28 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (12 responses)

> but you're not ok with javascript running inside a strongly sandboxed environment?

The fact that there is a CVE a day or so for chromium and firefox contraddicts your statement about "strongly sandboxed".

Rustaceans at the border

Posted Apr 19, 2022 8:50 UTC (Tue) by mjg59 (subscriber, #23239) [Link] (11 responses)

Very few of those CVEs involve sandbox escapes.

Rustaceans at the border

Posted Apr 19, 2022 9:29 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (10 responses)

Only one is needed though.

Rustaceans at the border

Posted Apr 19, 2022 9:45 UTC (Tue) by mjg59 (subscriber, #23239) [Link] (9 responses)

Sure, but only one local privilege escalation bug in Linux is needed for any non-Javascript attack vector and there's more of those than there are Chrome sandbox escape chains.

Rustaceans at the border

Posted Apr 19, 2022 13:43 UTC (Tue) by LtWorf (subscriber, #124958) [Link] (8 responses)

But I trust anything coming from apt much more than anything coming from some website.

Rustaceans at the border

Posted Apr 19, 2022 13:59 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (7 responses)

> But I trust anything coming from apt much more than anything coming from some website.

The conversation isn't really about what you personally trust but what ought to be considered trusted by people and that should be based on empirical data. For the most part, distributions don't vet code from upstream for security issues and sometimes distribution specific patching can even introduce security holes (A notorious example is https://www.debian.org/security/2008/dsa-157).

Rustaceans at the border

Posted Apr 19, 2022 14:01 UTC (Tue) by farnz (subscriber, #17727) [Link] (6 responses)

Your link is truncated - I'm guessing you mean this OpenSSL predictable RNG security advisory?

Rustaceans at the border

Posted Apr 19, 2022 14:30 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (5 responses)

> Your link is truncated - I'm guessing you mean this OpenSSL predictable RNG security advisory?

Correct. Thanks

Rustaceans at the border

Posted Apr 19, 2022 16:58 UTC (Tue) by amacater (subscriber, #790) [Link] (4 responses)

This is a favourite stick to drag up to beat Debian with: what's more interesting is to actually go back and realise what happened when it came to light. The work by Luciano Bello was first class: Debian's response was clear and relatively immediate - explain the problems with OpenSSL and thereby OpenSSH fully, create a tool to deny-list problem keys, explain what needed regenerating and why.

The mistake came as a result of actually querying with the upstream maintainers what should be done and doing it inappropriately once too often. I haven't seen other incidents handled as well by other teams and other distributions - let alone by commercial software. It's worth looking at as something being handled well at the time and in retrospect, not necessarily trotted out at every opportunity 14 years later

Rustaceans at the border

Posted Apr 19, 2022 17:36 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

> This is a favourite stick to drag up to beat Debian with

As I specifically noted already, it was just an example of a distribution specific vulnerability that is more well known (and it has some significance due to OP's reference to apt but mostly incidental). Your defensive reaction to that specific example does nothing to address the broader point. Replace that example with another distribution example if it helps you, here you go:

https://nvd.nist.gov/vuln/detail/CVE-2007-5962

We can certainly find more across distributions since everytime backporting or distribution specific patching happens (even as simple as a permissions change in the filesystem), there is deviations from upstream that introduces some potential risk of bugs including security vulnerabilities that don't exist upstream. So the broader point is that humans are infallible and occasions make mistakes (not even counting malicious attackers) and we shouldn't automatically rely on distributions to provide us a better security compared to well vetted upstream projects. Atleast in some cases, they do worse.

Rustaceans at the border

Posted Apr 19, 2022 17:39 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

Correction: humans are not infallible.

Rustaceans at the border

Posted Apr 19, 2022 18:11 UTC (Tue) by amacater (subscriber, #790) [Link] (1 responses)

Well vetted upstream projects is the difficulty. "With enough eyeballs, all bugs are shallow" - not necessarily if the eyelids (or codebases) happen to be effectively closed ... and see the discussions round the security of Chrome/Chromium regularly, for example.

The case of Rust in the kernel is similar to the problems of vendored code necessary to keep some ecosystems running: Node is the obvious one that causes problems to Debian and others.

It's possible that there's just an instinctive awareness of potential problems to come.
There's _so_ much code there that's intertwined and impenetrable and we've all been subject on occasion to being force-marched forwards by large scale change in someone else's ecosystem that we can't control - see, for example init system flamewars or Python 2 -> Python 3 (or a.out to ELF if your memory is long enough).

Rustaceans at the border

Posted Apr 19, 2022 18:28 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

> Well vetted upstream projects is the difficulty

To be clear, the original context here was visiting crates.io website compared to installing a distro package.

> The case of Rust in the kernel is similar to the problems of vendored code necessary to keep some ecosystems running

Vendored code already exists in the Linux kernel. The Rust mechanisms for tracking vendoring is strictly better than the ones we have for vendored C based projects in the kernel.

Rustaceans at the border

Posted Apr 15, 2022 6:41 UTC (Fri) by qyliss (subscriber, #131684) [Link]

It's not a solution to any of the broader points being discussed, but I recommend https://lib.rs/, an alternative frontend to crates.io, which does not require JavaScript (and also has a much nicer interface).

Rustaceans at the border

Posted Apr 18, 2022 3:40 UTC (Mon) by ssokolow (guest, #94568) [Link]

Bear in mind that crates.io, the package index, and crates.io, the web-based browser frontend for it, are separate systems.

Think of it like GitHub or GitLab or BitBucket or Gitea. When using git with one of those sites or cargo with crates.io, you only need to enable JavaScript if you want to publish to the site and, once you've set things up and retrieved your API key(s), you don't need to log in.

Heck, comparing cargo to git is an understatement. With subcommands like search, publish, and owner, it's more like hub. (A wrapper for git which adds subcommands like git issue create for GitHub-proprietary APIs so you probably never need to use the web UI once you've set up your credentials.)

Beyond that, they do have plans to enable server-side rendering for the browser frontend... it's just that the team who handle the frontend is a bit understaffed so it's taking a while to rework it.

Rustaceans at the border

Posted Apr 15, 2022 8:23 UTC (Fri) by marcH (subscriber, #57642) [Link] (4 responses)

> "vendoring" unmodified upstream

Problem: everyone knows "copy/paste/diverge" sounds bad.

Solution: keep inventing new synonyms to disguise it. Like when $BIGCORP changes name after some scandal.

The real solution is of course some form of branching / forking == copy/paste under (version) control. Probably what the author of a git tool written in rust had in mind :-)

Rustaceans at the border

Posted Apr 15, 2022 16:14 UTC (Fri) by pj (subscriber, #4506) [Link] (2 responses)

I think it's pretty common these days to 'vendor' code via git submodules, which seems to be what you're advocating for.

Rustaceans at the border

Posted Apr 15, 2022 17:35 UTC (Fri) by smurf (subscriber, #17840) [Link] (1 responses)

"git submodule" doesn't vendor anything. The code is still pulled from a remote repository, it's just pinned to a specific version. Which is exactly what you want to avoid spurious external changes that introduce more-or-less-subtle security problems.

"Real" vendoring, aka copy/paste/ignore, disconnects the copy entirely from its source. (Thanks but no thanks.)

Rustaceans at the border

Posted Apr 16, 2022 7:08 UTC (Sat) by riking (subscriber, #95706) [Link]

The 'cargo vendor' subcommand does copy-paste-track by saving the copied version to the Cargo.lock file.

Rustaceans at the border

Posted Apr 15, 2022 17:12 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

What we do is we have a separate git history for the import that is subset from upstream (remove test suites, test data, docs, sources we don't care about, etc.) and commit that on the respective tracking branch. One can even transform the import to do things like sqlite's amalgamate to make a smaller set of sources. This then gets `-Xsubtree` merged into the main tree and is checked that:

- changes to this directory only come from such a merge
- the merge came from the "right" branch (tracked by its root commit)
- the merge did not modify the subtree in the merge commit (yay)

We also check that *all* such changes come in through this mechanism and we track our patches in a fork of upstream that we tag for each import for posterity.

Rustaceans at the border

Posted Apr 16, 2022 8:21 UTC (Sat) by jsakkine (subscriber, #80603) [Link]

This is more like a constraint set by the kernel process rather than an opinion :-) It's in some ways analogous to OOT modules ("sort of" same different). We cannot import unreviewed code to the kernel. It's a brickwall.

Rustaceans at the border

Posted Apr 15, 2022 15:01 UTC (Fri) by cesarb (subscriber, #6266) [Link]

> Should we copy the crate into the kernel tree to be independent of crates.io?

As mentioned in a comment above, there's precedent with things like zstd, which is also fully copied into the kernel tree instead of being kept as an external package. So I'd expect every crate included in the kernel, with the exception of the ones which come with the compiler (i.e. the "core" crate), to be vendored directly into the kernel tree.

Rustaceans at the border

Posted Apr 16, 2022 9:39 UTC (Sat) by jsakkine (subscriber, #80603) [Link]

I think this whole thing is in the scope of kernel maintainers to decide because they have to take the responsibility of using external crates. You can have an opinion but unless you are one who that has to deal with its consequences, it unfortunately does not count that much at all.

Rustaceans at the border

Posted Apr 16, 2022 14:55 UTC (Sat) by jsakkine (subscriber, #80603) [Link]

As long as pulling them does not require Internet connection :-) One must be able to build kernel without downloading external code. Importing in some cases is of course considerable but that must be done case by case.