| |||||||||||||
Short topics in memory managementShort topics in memory managementPosted Mar 7, 2007 13:35 UTC (Wed) by dion (subscriber, #2764)In reply to: Short topics in memory management by k8to Parent article: Short topics in memory management
I think you might be right.
Firefox probably thinks we are living in 1970 when there wasn't anything called virtual memory, so it allocates gobs of RAM for caches and does explicit file io.
A much better way would be to have just one cache in one huge memory mapped file, that way there is never any explicit io and the kernel is free to write stuff out to disk whenever it needs to.
It would make a persistent cache a bit tricky, you could never do anything with the cached files using normal tools and crashing would be interesting, though.
Someone working on firefox should take a peek at Varnish, which is incredibly fast because it lets the kernel do all the hard memory/io work.
(Log in to post comments)
Stuck with 1970s VM? Posted Mar 7, 2007 15:57 UTC (Wed) by jzbiciak (✭ supporter ✭, #5246) [Link] Don't forget, Firefox has to run on Windows, too. ;-)
Stuck with 1970s VM? Posted Mar 7, 2007 16:38 UTC (Wed) by ibukanov (subscriber, #3942) [Link] As far as I remember Windows got memory-mapped files starting at least with Windows-95. Surely the API is different, but it is straightforward to isolate the difference behind a tiny wrapper. The best thing about it is that since the memory mapping removes explicit reads/writes from the application, the end result is more portable code.
Stuck with 1970s VM? Posted Mar 7, 2007 16:44 UTC (Wed) by jzbiciak (✭ supporter ✭, #5246) [Link] *whoooosh*
That's the sound of a tongue-in-cheek jab at Windows going over your head. :-)
Short topics in memory management Posted Mar 7, 2007 16:50 UTC (Wed) by ibukanov (subscriber, #3942) [Link] > It would make a persistent cache a bit tricky,
The trick is to use relative offsets instead of pointers in the data that goes into the mapped file. It has an extra benefit of a more secure code since it is easier to add bound-check for the offsets then checking the pointers.
Short topics in memory management Posted Mar 7, 2007 17:47 UTC (Wed) by zlynx (subscriber, #2285) [Link] Checking offset bounds may look easier than a pointer, but it isn't really. A pointer is nothing but an offset from memory location 0, after all.
if( pointer > mmap_base && pointer < mmap_end ) ...
Using an unsigned int offset instead saves you from checking the lower bound, but you lose that savings again the instant you segment your memory into different uses, like using some for text and some for graphics.
Short topics in memory management Posted Mar 8, 2007 23:57 UTC (Thu) by jzbiciak (✭ supporter ✭, #5246) [Link] "A pointer is nothing but an offset from memory location 0, after all." Well, these days it is on most architectures...
Short topics in memory management Posted Mar 7, 2007 17:49 UTC (Wed) by zlynx (subscriber, #2285) [Link] One problem with large mmaps is fragmented virtual memory. It is very possible with 32-bit addressing to end up in a situation where you *cannot* allocate a single block of virtual. Especially if you are doing things where you grow your maps, reposition them, etc. Then you need crazy stuff like a mmap defragmenter where you flush it all to disk, unmap it and remap everything again.
All the problems could be solved of course, but who is going to rewrite all that code? Again.
Short topics in memory management Posted Mar 7, 2007 18:14 UTC (Wed) by dion (subscriber, #2764) [Link] Well, the solution is to "not do that, then".
If I were to implement a browser cache in a memory mapped file then the mapping would be the first thing to get allocated and it would never change.
|
|||||||||||||