Native support for Rust on the GPU

Christian Legnitto is the maintainer of rust-gpu and Rust CUDA, two libraries that make it possible to program a computer's graphics processing unit (GPU) from Rust. He isn't satisfied with the current state of GPU support in Rust, however. In a talk at RustConf 2026, he explained his vision for how the GPU could become an ordinary compiler target for normal Rust code, without the need for any special libraries or new ecosystem support. That vision is not yet fully implemented, but he does have a prototype that he is preparing to release.

Today

There are lots of existing libraries that can be used to program GPUs from Rust, including the ones Legnitto maintains. The problem with them, as he sees it, is that they start by looking at the features of the GPU and trying to represent them in Rust. This does work — these projects produce good code, which runs in production — but it causes friction where there are mismatches in expectations between the GPU and Rust. Worse, it forks the Rust ecosystem, in that projects written to use one library are usually incompatible with projects written to use a different library.

This LWN.net subscription-only content has been made available to you by an LWN subscriber. To see more of this content, please take advantage of the following special offer.

Free trial subscription

Try LWN for free for 1 month: no payment or credit card required. Activate your trial subscription now and see why thousands of readers subscribe to LWN.net.

The current GPU libraries for Rust seem to implicitly assume that "GPUs are weird" and need special treatment. Legnitto doesn't think that is the case. The thing is that "CPUs are also really weird," Legnitto said, "we're just used to it." Recently, he has been exploring a different approach: start with a Rust-first design, rather than a GPU-first design. His goal is to preserve Rust's native semantics at all costs, even if that causes some inefficiency. The result, while still experimental, is a system that can run normal Rust applications, including libraries that are not intended for use on a GPU, directly on the GPU's parallel hardware.

Rust already has support for a variety of CPU architectures, each of which has its own problems and quirks. The language handles this by using zero-cost abstractions where possible, and opt-in escape hatches, target-specific compilation, and inline assembly where it isn't. The standard library is, in many ways, a facade over the various incompatible implementations of the same basic concepts.

The plan for his experiment is to do anything that the GPU can reasonably support on the GPU itself, with other operations (such as filesystem interaction, etc.) being dispatched to the host CPU. The standard library already does this on the CPU, except that operations are dispatched to the kernel, instead of a different piece of hardware. The key point is that Rust application code does not need to know or care how its requests are handled.

Tomorrow

The one part that requires more thought is the integration of concurrency primitives. The thing that GPUs do better than CPUs, and the reason to use them at all, is that the hardware supports a greater degree of parallelism. Still, Legnitto thought that the parallelism provided by GPUs mapped well to the use of processes, threads, and single instruction multiple data (SIMD) instructions in typical Rust code.

To run code on a GPU, he said, the CPU first sets any necessary configuration up, and then "launches a kernel" on the GPU. This can be considered a kind of process that is assigned some number of "blocks" and "warps" (smaller hardware units that provide concurrency) on the GPU. Multiple kernels can run at once, as long as they are assigned to different sets of warps. The difference between a GPU kernel and a CPU process is that the kernel runs on all of its assigned warps at once; it does not start on a specific CPU core. Still, there is an obvious way to remedy that: just have all but one of the running GPU kernel instances go to sleep (entering a busy loop, because they cannot sleep in the way that processes do) before handing execution off to the user's code. Legnitto acknowledged the inefficiency of that solution, but did not think it was too much of a problem.

Hardware components such as warps don't enforce the use of concurrency, Legnitto said. They describe how much concurrency is possible, but the code remains in control of how it is actually used. Anything that might normally be done with a process can be implemented by launching a kernel: "it has a lifecycle, multiple can run in parallel, there's a heap, etc.". Kernels can even launch new ones by making a request to the CPU to launch them on their behalf. So, everything that Rust code expects to be able to do with processes can be implemented in terms of kernel launches.

Similarly, GPU warps can mostly fulfill the role of threads. They have shared memory, a program counter, a stack, thread-local storage, etc. The one difference is that warps can't create new warps, since a warp is a physical piece of hardware, of which there are a fixed number. The solution is to make use of the other kernel instances that went to sleep when the kernel started executing. A warp that wants to spawn a new thread pushes a function pointer to a shared list, and then sets a flag that is checked by the busy-looping kernel instances. One of them grabs the function pointer and calls it. In that way, even though the actual number of warps is fixed, a dynamic number of Rust threads can essentially be scheduled across them.

Finally, the CPU's SIMD instructions are a perfect match for GPU "lanes" — the smallest parallel piece of hardware in the GPU, which can apply a single simple operation across multiple inputs simultaneously. On both the CPU and GPU, these represent the lowest level of concurrency, with a static hardware width and limited supported operations (mostly math). The only real difference is that GPUs tend to be wider, but CPUs vary in how wide their SIMD instructions are, so the Rust compiler already needs to handle that.

Legnitto then showed an example of what Rust code for a GPU using this approach would look like. At the time of writing, he has not shared his slides, but a simple example that spawns a thread and uses SIMD intrinsics to add four pairs of numbers at once would look like this:

    fn example() {
        let handle = thread::spawn(|| {
            let a = i32x4::from_array([10, 20, 30, 40]);
            let b = i32x4::from_array([1, 2, 3, 4]);
            let result = a + b;
            println!("SIMD Result: {:?}", result.to_array());
        });

        handle.join().unwrap();
    }

That same code would run unmodified on a CPU. Real code would typically spawn more threads and do something more interesting with them, but the fundamental operations remain the same.

This approach is not, in theory, any slower than current approaches, he said. A Rust program that spawns multiple threads which all start using SIMD should have performance equivalent to hand-written CUDA code. The real benefit is that "normal Rust stuff" just works. Existing libraries and language features are usable as-is, and can have their performance improved by incrementally adopting threading and SIMD where it makes sense, which will also benefit users running on the CPU. For truly performance-critical sections, it's still possible to drop down into inline assembly and write code for the GPU directly.

It also means that all Rust programmers become GPU programmers overnight, without having to learn anything new, Legnitto said. If a proposal like his lands, it would make Rust the largest GPU ecosystem in the world, he continued.

That's not to say that this approach is without drawbacks. For some hardware, there may be better ways to structure GPU programs that aren't compatible with Rust's model. Also, if libraries originally written for the CPU work, that might deter people from writing better GPU-specific libraries. The size of compiled binaries could also be a problem; most GPU code is short and simple compared to an entire Rust application. GPU code has become more complex over the years, he said, so he does think that all GPUs are going to need to deal with larger programs — Rust applications would just encounter the problem immediately.

Improving the language

It's also possible that adopting Rust on the GPU could motivate the design and adoption of language features that would be useful elsewhere. For example a "disjoint slice" type that allows referencing multiple non-overlapping parts of an array would be useful on both the GPU and CPU. Standard library support for tensors and matrix multiplication is another area that Rust will probably need to incorporate at some point anyway, Legnitto claimed. "Just because something is useful on GPUs doesn't mean that it's GPU-specific."

Ultimately, Legnitto believes that having first-class support for GPUs in Rust is essential to the future of the language. More software is making use of GPUs all the time, and newer CPU designs are increasingly incorporating GPU-like elements. There will always be competing software ecosystems, he said, and software that does not work well on GPUs or GPU-like CPU cores will be at a disadvantage.

There was a short time for questions. One audience member asked about whether compilers that had been optimized for CPUs really produced GPU code that worked well. Auto-vectorization for GPU code is still "pretty raw", Legnitto admitted. That is to be expected, however, given that this is the first time that LLVM is being applied in this way. There will be additional optimizations that bring performance more up to par with existing GPU toolkits, he said.

[ Thanks to the Linux Foundation, LWN.net's travel sponsor, for assistance in traveling to RustConf. ]