"fringe platforms which can't even run a Rust compiler" [LWN.net]

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 20:48 UTC (Sat) by Cyberax (✭ supporter ✭, #52523)
In reply to: "fringe platforms which can't even run a Rust compiler" by sthibaul
Parent article: Python cryptography, Rust, and Gentoo

These days it's more like "you need LLVM for your platform". This will automatically give you a C/C++ compiler, Rust and soon Go. If anything, it's actually easier to port non-trivial userspace to new platforms today.

to post comments

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 20:50 UTC (Sat) by sthibaul (✭ supporter ✭, #54477) [Link] (14 responses)

No, LLVM won't immediately give you Rust, see other comments about the workload to port libc.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 21:51 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (13 responses)

Rust doesn't need libc.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 21:58 UTC (Sat) by sthibaul (✭ supporter ✭, #54477) [Link] (12 responses)

But cargo does, and building rust needs cargo?

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 22:12 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (11 responses)

It depends on what you WANT to do. If you're bringing up a small embedded platform then you don't need to run cargo on it. You just do cross-compilation, which only requires LLVM for the target.

If you want to bring up a full large-scale OS (think Fuchsia or Linux) then you'll need a libc for Python and other tools anyway. You'll need to port the libc crate to that OS as well, but this will be a pretty small task overall compared to the amount of work you have to do.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 22:20 UTC (Sat) by sthibaul (✭ supporter ✭, #54477) [Link] (10 responses)

> You'll need to port the libc crate to that OS as well, but this will be a pretty small task overall compared to the amount of work you have to do.

As I mentioned in other comments, porting the libc create is currently *not* a small task, you have to go over all system-provided libc headers to check for precise bits and bytes. This step deserves some automation, like perl/python/etc. have.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 22:29 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (9 responses)

Here's an example of libc for VxWorks (a non-Unix OS): https://github.com/rust-lang/libc/blob/master/src/vxworks...

It's 2000 lines and most of them can be trivially generated by cut&pasting headers and doing string replaces. You can write it in within a day.

Automating it would be nice, but different OSes can have wildly different header conventions.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 22:37 UTC (Sat) by sthibaul (✭ supporter ✭, #54477) [Link] (6 responses)

> most of them can be trivially generated by cut&pasting headers and doing string replaces.

cut&pasting+string replace from the C .h headers??

> You can write it in within a day.

For somebody that doesn't even know Rust from the start?

> Automating it would be nice, but different OSes can have wildly different header conventions.

I'm not saying to automate string replacements, but simply do like all other languages implementations I have seen do: interpret the C headers in C, i.e. bindgen.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 23:13 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

> cut&pasting+string replace from the C .h headers??
Yup. Get your platform's headers, copy the definitions and then massage them until they are valid Rust.

If you want something minimalistic, you can even skip most of them (I'd estimate that around 500 lines are truly needed).

> For somebody that doesn't even know Rust from the start?
Add one more day for cargo-culting from an existing file. If you ever had to debug autohell, you're qualified enough to do it.

> I'm not saying to automate string replacements, but simply do like all other languages implementations I have seen do: interpret the C headers in C, i.e. bindgen.
That is possible, but you'd spend quite a bit of time debugging the generated code for your custom platform anyway. And given that these files are pretty much static once they are written, they don't need a lot of maintenance.

In reality, bringing up a non-Unix platform is such a huge undertaking that you'd likely be better off by writing the libc in _Rust_.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 23:20 UTC (Sat) by sthibaul (✭ supporter ✭, #54477) [Link] (4 responses)

> Get your platform's headers, copy the definitions and then massage them until they are valid Rust.

glibc, for instance, has ~500 headers, the definitions are split among arch-specific and non-arch-specific files. Just to give an instance for the st_dev member of the stat structure: it is actually defined in bits/stats.h, with type __dev_t. Type __dev_t is defined in bits/types.h from __DEV_T_TYPE. That type is defined in bits/typesizes.h from __UQUAD_TYPE. That type is defined in bits/types.h from __uint64_t.

> That is possible, but you'd spend quite a bit of time debugging the generated code for your custom platform anyway.

What is left to debug, when the C interpretation is *by definition* the correct answer?

> In reality, bringing up a non-Unix platform is such a huge undertaking that you'd likely be better off by writing the libc in _Rust_.

Why Rust?

Really, that's exactly the point: it looks like Rust people want to bring the whole world into Rust, and not bother with the rest of the world that isn't Rust.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 13, 2021 23:25 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

> glibc, for instance, has ~500 headers, the definitions are split among arch-specific and non-arch-specific files.
If you have a libc from your device vendor, then just do "#include <socket.h>" and then pre-process the file. If you are making a completely new arch, then YOU have to write these 500 files.

> What is left to debug, when the C interpretation is *by definition* the correct answer?
For example, Rust has special definitions to navigate the socket headers (these are macros in C, not functions). C also doesn't specify the error returns.

> Why Rust?
Because it's safer than C and faster to write.

> Really, that's exactly the point: it looks like Rust people want to bring the whole world into Rust, and not bother with the rest of the world that isn't Rust.
Honestly, that would be a great outcome for the world.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 14, 2021 0:20 UTC (Sun) by sthibaul (✭ supporter ✭, #54477) [Link] (2 responses)

> just do "#include <socket.h>" and then pre-process the file.

Yes, but that still doesn't magically collapse all the typedefs. With just socket.h that produces 500 lines of C, that a C compiler and scripts shared by *all* ports would much better grok than doing seds by hand.

> For example, Rust has special definitions to navigate the socket headers (these are macros in C, not functions).

Ok, but that's only a very tiny part of the Posix interface.

> C also doesn't specify the error returns.

You mean the set of errors that a function may return? Yes, and a kernel never makes such a promise as the exact set of errors it might ever return in the coming decades.

> > Why Rust?
> Because it's safer than C and faster to write.

Let me rephrase: why Rust in particular? And not the myriad of other languages that have been invented by mankind?

> > Really, that's exactly the point: it looks like Rust people want to bring the whole world into Rust, and not bother with the rest of the world that isn't Rust.
> Honestly, that would be a great outcome for the world.

So that's exactly what I wrote in another comment: so much for the free software dream, if you're free to use any language, as long as it is Rust.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 14, 2021 16:05 UTC (Sun) by nix (subscriber, #2304) [Link]

> Yes, but that still doesn't magically collapse all the typedefs. With just socket.h that produces 500 lines of C, that a C compiler and scripts shared by *all* ports would much better grok than doing seds by hand.

Aha, thank you for giving me another use case for CTF! The upcoming CTF format v4 work will include an objdump option that emits a collection of C header files given a .ctf section. They won't be the same as the inputs, of course (no comments, for starters, and with the default linker options you'll get one big header for non-ambiguous types and then small headers #including the big one for all the ambiguously-defined ones), but it'll be better than nothing -- but until now I didn't have an answer for why this would ever be better than using the original .h files. Now I have one: transformations on the headers to help people doing work like this. (A transformation that collapses away all typedefs, leaving everything textually defined in terms of the base types, and leaving the typedefs around but with no actual users, should be trivial to implement. It's probably a bad idea to *use* those headers, but for human analysis for FFIs it's probably likely to be useful.)

"fringe platforms which can't even run a Rust compiler"

Posted Feb 15, 2021 0:28 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

> Ok, but that's only a very tiny part of the Posix interface.
You actually don't need much to bootstrap Rust. The now-removed CloudABI was about 500 lines of code for the libc, which is probably about the minimal amount.

> Let me rephrase: why Rust in particular? And not the myriad of other languages that have been invented by mankind?
For stuff like libc you need a language that has no runtime (no GC, no threads created behind your back, etc.), natively compiled, memory-safe and preferably with additional type-safety-enforced guarantees.

Not many languages fit this bill. Out of modern-ish ones only Rust and Swift fit the bill.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 14, 2021 20:25 UTC (Sun) by geert (subscriber, #98403) [Link] (1 responses)

IIUIC, Rust expects to call into snprintf() on VxWorks?

Last time I developed software for VxWorks (15y ago), its C library didn't have snprintf(), so we had to use our own implementation.

"fringe platforms which can't even run a Rust compiler"

Posted Feb 14, 2021 21:57 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

> IIUIC, Rust expects to call into snprintf() on VxWorks?
Nope. Rust has its own string formatting support (in stdlib) that is independent of the OS.