What influenced this addition?

Posted Jul 19, 2024 19:47 UTC (Fri) by mb (subscriber, #50428)
In reply to: What influenced this addition? by bluss
Parent article: Silva: How to use the new counted_by attribute in C (and Linux)

Thanks for explaining. I learnt something today.

What influenced this addition?

Posted Jul 20, 2024 12:13 UTC (Sat) by bluss (guest, #47454) [Link] (3 responses)

It doesn't really make me happy to see the complexity of a "language with UB" reproduced in Rust, but it's sort of like that, for the `unsafe` block enclosed part of the language.

Plain flexible length allocations can use `Box<[T]>`. For the true flexible array member use case, some non-native library solution is needed, and there seems to be crates available that implement this. I think I would look at https://docs.rs/thin-dst/ for this, but I don't know it well.

What influenced this addition?

Posted Jul 22, 2024 17:54 UTC (Mon) by ms-tg (subscriber, #89231) [Link] (2 responses)

Q: Is there a semantic difference (a difference in _intended_ meaning and use) between the C flexible array pattern here, and in Rust having simply a `Box<[T]>` boxed slice in the Struct?

If so, is there a way to return a `Box<[T]>` in a general way to use in Rust code that is handed a C flexible array, that is close to zero cost?

(these are genuine questions, I don't know the answer)

What influenced this addition?

Posted Jul 22, 2024 21:29 UTC (Mon) by atnot (guest, #124910) [Link] (1 responses)

No, but also yes.

I'll start with the no: Box is just a heap allocation, and [T] is just a contiguous set of elements. So a Box<[T]> compiles down to a pointer to a variable sized heap allocation, which is different.

The reason it is done this way is that [T] is one of the few types in Rust that are "Unsized", that is, do not have a size known at compile time. The things you can do with unsized types in safe Rust are extremely limited. They can only exist behind a pointer, you can't create them, put them on the stack, or any of the other things you can do in C. Really the only thing you can do is put them in a Box to get a nice, constant size type again, hence Box<[T]>. You will never see a raw [T] writing normal Rust because of how useless it is.

So no, it's not a direct replacement, unless your flexible array is just {size, buffer[]}. Anything else doesn't really translate to safe Rust. However, this does represent the far majority of cases where flexible array members are used in C in practice. There are a few exceptions, but it's generally not a clear performance win and they often use funky bit stashing tricks that counted_by wouldn't understand either anyway. So it's close enough in practice.

However, while Box<[T]> might only be equivalent in some cases, [T] does exist and can actually be the last member of a struct to create a dynamically sized struct, exactly like in C. In fact to my knowledge it was added for C ffi. It won't be pretty and it will be highly unsafe. But you can totally do it, if you wish to.

Why not in safe Rust? Probably could be done, I just don't think anyone cares enough.

What influenced this addition?

Posted Jul 24, 2024 22:03 UTC (Wed) by rodrigorc (guest, #89475) [Link]

There are a few new features, that hopefully will be stabilized soon, that will make the "unsized struct with unsized last field" trick quite workable.

In particular the `ptr_metadata` feature is quite handy. This sample code runs in nightly Miri without warnings:

#![feature(ptr_metadata)]
use std::ops::Index;
use std::slice;

#[repr(C)]
struct S {
    n: usize,
    array: [u32]
}

impl Index<usize> for S {
    type Output = u32;
    fn index(&self, index: usize) -> &u32 {
        unsafe {
            &slice::from_raw_parts(self.array.as_ptr(), self.n)[index]
        }
    }
}

fn main() {
    unsafe {
        // Fake C code:
        let layout = std::alloc::Layout::from_size_align(100, 4).unwrap();
        let ptr = std::alloc::alloc_zeroed(layout);
        ptr.cast::<usize>().write(10);
        
        // Use the flex array:
        // First with size=0, a distinct Sized type for the header might look nicer
        let n = (*std::ptr::from_raw_parts::<S>(ptr, 0)).n;
        let s = &*std::ptr::from_raw_parts::<S>(ptr, n);
        for i in 0 .. s.n {
            println!("{} {}", i, s[i]);
        }
        
        // More fake C code
        std::alloc::dealloc(ptr, layout);
    }
}

Playground link.