| |||||||||||||
jemallocjemallocPosted Mar 13, 2008 7:00 UTC (Thu) by rsidd (subscriber, #2582)In reply to: jemalloc by jayavarman Parent article: A look at memory usage in Firefox 3 jemalloc is the default on FreeBSD since 7.0 (replacing the older phkmalloc, which was also widely used outside FreeBSD, but had scaling issues). Why isn't it used in glibc -- you should ask the maintainers. Could be it's too new, could be NIH.
(Log in to post comments)
jemalloc Posted Mar 13, 2008 13:09 UTC (Thu) by nix (subscriber, #2304) [Link] Also, `just rewrite it dammit' is not always the right answer. The malloc() in glibc has a lot of arcane knowledge and nifty speedups for obscure situations in it: replacing it wholesale would throw all that knowledge away. A malloc() which works slightly better in most circumstances but drastically worse in some is perhaps not an improvement (not that this is necessarily true of jemalloc: I haven't checked, but *consistent* behaviour has always been a major focus of glibc malloc()).
jemalloc Posted Mar 13, 2008 15:31 UTC (Thu) by jasone (guest, #2423) [Link] Indeed, arbitrarily replacing the system malloc implementation is dangerous, because it will cause memory layout changes which are almost certain to expose latent application bugs. An allocator has to be substantially better to warrant such a change. glibc uses ptmalloc, which is a multi-threaded derivative of dlmalloc. It works quite well for most typical uses, which is rather impressive, given the difficulty of tuning extent-based allocators. The only weak spots I've observed are its fragmentation behavior for some allocation patterns (such as lots of variation in allocation sizes, as for many C++ applications), and threading scalability above ~6 CPUs. In practice, I'm not convinced that a switch to a different allocator is warranted yet for Linux, because the existing allocator doesn't fail horribly for any common uses. When we all have laptops with 16+ CPUs, glibc's malloc may need some tender loving care, but we're not quite there yet. With regard to jemalloc in Firefox 3, the primary reason it was worthwhile was the horrible fragmentation behavior of the malloc implementations on Windows. The Linux version of Firefox also benefitted, but not to a degree that would have by itself warranted the serious effort we had to expend. Your comment about *consistent* behavior is spot-on. Early versions of jemalloc used extent-based allocation, and I found it essentially impossible to make the algorithms work consistently across a wide range of applications. This is why jemalloc now uses what people typically call zone allocation. The behavior is much easier to predict and reason about, thus avoiding extreme fragmentation due to complex interactions with allocation patterns.
|
|||||||||||||