Proper Community Engagement
Proper Community Engagement
Posted Aug 31, 2024 5:05 UTC (Sat) by rc00 (guest, #164740)In reply to: Proper Community Engagement by roc
Parent article: Rust-for-Linux developer Wedson Almeida Filho drops out
> Which leaks virtual memory (to catch heap UAF) and is not suitable for production use.
The general-purpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.
https://www.openmymind.net/learning_zig/heap_memory/
> ReleaseSafe does not affect the choice of allocator and therefore does not detect heap UAF unless you opt into the general-purpose allocator, which is not suitable for production use.
Please read the previous links. This was directly addressed.
> And FWIW Zig has *nothing at all* to detect stack UAF.
The language is still not 1.0 yet. Here is the issue where use-after-free stack allocations will be addressed:
https://github.com/ziglang/zig/issues/3180
I am sharing these links in good faith. Please take the appropriate time to process them.
Posted Aug 31, 2024 9:04 UTC (Sat)
by roc (subscriber, #30627)
[Link] (1 responses)
Yes, I checked all your links (and more) before I responded, and the information in those links is consistent with what I said. Your new link is also consistent with what I said.
> The general-zurpose allocator is configured with never_unmap set to false by default. Only when set to true is there a possibility that every allocation can be leaked. Again, this allocator can be used for production release modes.
This doesn't support your claims. It doesn't mention never_unmap or describe exactly what GeneralPurposeAllocator guarantees or say what the performance impact is. In fact those things are not documented anywhere outside the source code, as far as I can tell; certainly nowhere you have linked to. So I find your attitude irksome.
But in the spirit of https://xkcd.com/386, I went ahead and looked at the source code: https://github.com/ziglang/zig/blob/master/lib/std/heap/g.... The situation is pretty complicated, but as far as I can tell, the following things are true in the default configuration with ReleaseSafe build mode:
In more detail: the source code comments say:
It's not entirely clear from the comment what "memory slots" means here, but looking at the code it's more clear: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
However, when freeing a sub-page-size object, if that means the bucket is *completely empty*, the entire bucket is freed: https://github.com/ziglang/zig/blob/37df6ba86e3f4e0f5d6a2...
Note that the comment "for larger ones it is handled in the backing allocator" is false for the default PageAllocator. There is no logic to avoid reusing virtual memory in that code. So virtual memory isn't leaked, but in exchange you get UAF bugs.
For allocations >= the page size, you enable UAF bugs immediately because PageAllocator can reuse addresses.
This Reddit thread agrees with me that "UAF will not be reliably detected": https://www.reddit.com/r/Zig/comments/1eysv2k/general_pur...
andrewrk did write: https://github.com/ziglang/zig/issues/3180#issuecomment-6...
> I am sharing these links in good faith. Please take the appropriate time to process them.
Oh, I've spent way too much time processing this.
> Here is the issue where use-after-free stack allocations will be addressed: https://github.com/ziglang/zig/issues/3180
Will it really, though? It is not easy at all to detect and prevent stack UAF bugs. For example, if you take the address of a stack value and store it in a heap object, is that allowed? It's often useful to be able to do that, so I guess Zig would want to allow it, but if you allow it, it can be very difficult to prove that the pointer is not used after the function has returned. If they do solve this, there will be some overhead and/or some existing useful Zig code that is no longer legal.
For now, safe Rust prevents UAF and all Zig has is partial, inefficient solutions and promises.
Posted Aug 31, 2024 13:34 UTC (Sat)
by corbet (editor, #1)
[Link]
Proper Community Engagement
> https://www.openmymind.net/learning_zig/heap_memory/
-- GeneralPurposeAllocator reuses addresses of allocated objects in many cases and does allow UAF bugs in those cases
-- GeneralPurposeAllocator avoids reusing addresses of allocated objects in other cases, and therefore must suffer from serious fragmentation problems leading to performance degradation
-- GeneralPurposeAllocator lacks a lot of the optimizations essential to modern high-performance allocators and cannot be expected to be competitive with those allocators
> //! ### `OptimizationMode.debug` and `OptimizationMode.release_safe`:
...
> //! * Do not re-use memory slots, so that memory safety is upheld. For small
> //! allocations, this is handled here; for larger ones it is handled in the
> //! backing allocator (by default `std.heap.page_allocator`).
GPA passes through objects >= the page size through to the underlying page-level allocator (which defaults to td.heap.page_allocator). For smaller objects, it has a list of page-sized "buckets" per size class, with each bucket carved up into "slots" of that size class. To allocate an object, it finds a bucket for the right size class with empty space, and uses the *next slot in the bucket that has never been used*. I.e., even if some objects in the bucket have been freed, those slots won't get used. This is what inevitably leads to fragmentation and poor cache usage.
With the default PageAllocator and GPA configuration, this memory is returned to the operating system. The next time GPA needs to allocate a page, PageAllocator will likely return memory at the same address. GPA will hand out references to that memory and so classic UAF bugs can then be triggered --- references through a pointer to the freed object will work but access memory in the freshly allocated object, perhaps of a different type.
> Use after free is now solved as far as safety is concerned with heap allocations, if you use the std lib page_allocator or GeneralPurposeAllocator.
And maybe you believed him, but unfortunately that's not true, or at least it's not true now.
This language-advocacy discussion has gone fairly far from the original topic and seems unlikely to resolve anything. This seems like a good time to let it go.
Proper Community Engagement
