| |||||||||||||
Still waiting for swap prefetchStill waiting for swap prefetchPosted Jul 26, 2007 1:56 UTC (Thu) by sobdk (guest, #38278)In reply to: Still waiting for swap prefetch by mgb Parent article: Still waiting for swap prefetch
Now I've been programming since magnetic drums were hip so I may be a bit confused here but it seems to me that I can remember a time less than a decade ago when a system didn't need a couple of gigs to run Linux well. I would say it still does. I'm writing this as we speak from a PIII 700 MHz with 256 MB of RAM and honest Linux runs great. I even use Firefox which tends to eat up 50% of my RAM according to top, but still no complaints. In fact even with Firefox using half my RAM I'm still not even touching swap. Interestingly enough when I read some of the posts on lkml claiming that they had 1GB of RAM and swap prefetch drastically improved their workload, all I could think is "what on earth are these people running?" Until a couple of months ago my dev-machine at work only had 512 MB of RAM and had all kinds of nasty things like Firefox, beagle, and Lotus Notes running but my swap was rarely used. Conversely on days when I was forced to boot it into Windows XP for one reason or another I couldn't leave an application minimized for more than 10 minutes before Windows decided to swap it out. Waiting 1 minute every time I switch apps on Windows is enough to make me go crazy and be thankful that Linux has a sane swapping algorithm.
(Log in to post comments)
Still waiting for swap prefetch Posted Jul 26, 2007 2:41 UTC (Thu) by Richard_J_Neill (subscriber, #23093) [Link] Hmm - I've just upgraded to a 64-bit desktop machine. The principal reason is so that firefox can address > 4GB of swap (!). Really, FF crashes on me about once a week, because although I have 2GB RAM and 6GB of swap, firefox manages to malloc() 4 GB! No idea where it is going - admittedly I tend to have about 200 tabs open, but that alone shouldn't be the problem.
Still waiting for swap prefetch Posted Jul 26, 2007 14:51 UTC (Thu) by lysse (guest, #3190) [Link] Well, 200 tabs at about 2Mb per tab would use up your 4Gb quite effectively...
Off by an order of magnitude? Posted Jul 26, 2007 20:29 UTC (Thu) by martinfick (subscriber, #4455) [Link] 200 * 2MB = 400MB != 4GB
Off by an order of magnitude? Posted Jul 27, 2007 6:04 UTC (Fri) by man_ls (subscriber, #15091) [Link] Let's make it 20 MB then per tab. If each page is three screens high, and my screen is 1680x1050, with 4 bytes per pixel (3 colors + alpha):1680 x 1050 (pixels/screen) x 4 (bytes/pixel) x 3 (screens) = 20672 MB. We do want the pages to be cached when we switch tabs, don't we?
Off by three orders of magnitude! Posted Jul 27, 2007 6:06 UTC (Fri) by man_ls (subscriber, #15091) [Link] That should read 20672 KB, sorry.
Off by an order of magnitude? Posted Jul 27, 2007 14:36 UTC (Fri) by Los__D (guest, #15263) [Link] Yeah we do, but does it really cache a bitmap of the page? That would seem a bit silly
Bitmap cache Posted Jul 27, 2007 21:57 UTC (Fri) by man_ls (subscriber, #15091) [Link] Yeah we do, but does it really cache a bitmap of the page?I don't know exactly, but I would say it does. Loading a couple of big JPEG files takes quite longer than changing tabs between them. That would seem a bit sillyWhy, exactly? If you have the RAM to spare, it seems to be as good as any other use. I seem to remember some discussion on LWN about Firefox caching even pages in the history. When people complain that "Firefox eats up 2 GB" in a 4 GB machine, it gives the wrong impression. Firefox runs fine on my 128 MB laptop, and memory seldom goes above 80 MB.
Bitmap cache Posted Jul 28, 2007 13:38 UTC (Sat) by Los__D (guest, #15263) [Link] I would never (never EVER) suggest using JPEG for computer graphics, but caching in PNG would seem smarter than a bitmap, and not really that much slower. Of course, when you only have 3-4 tabs, it wouldn't really matter, but as the count goes up, you can much better afford the CPU than the memory. Plus, the CPU is only when you change, the memory usage is constant.
I'm not completely sure how uncompressed PNG would handle the JPEG's on the page, but I couldn't imagine it getting worse (memory wise) than bitmap, just equal. (I've been known to be naive in my expectations from formats though).
The browser (toolkit?) could of course also do some kind of checking of available memory, and change algorithm from that, but maybe that is an unneeded complexity.
I don't know, maybe it's easiest/simplest and mostly correct to just keep the rendered pages as is in memory, but for some usage patterns, it does become a huge waste of memory.
Bitmap cache Posted Jul 28, 2007 13:51 UTC (Sat) by man_ls (subscriber, #15091) [Link] I would never (never EVER) suggest using JPEG for computer graphicsI wouldn't either. My little experiment with JPEG images involved decompressing a JPEG image vs. caching an uncompressed bitmap. Caching images in a lossy format would be ludicrous. But caching in a lossless format such as PNG isn't such a good idea either. An important aspect of caches is that you should store an artifact which you already have, not one you have to generate. If you have to compress a bitmap to PNG before caching it, you are wasting a lot of CPU time just to generate a cache which you might as well never use again. An example: you download a JPEG image, then uncompress it to show it, and then you compress it to PNG before caching it in memory. Messy.
Bitmap cache Posted Jul 28, 2007 14:00 UTC (Sat) by Los__D (guest, #15263) [Link] It really depends on the cost of CPU vs. the cost of memory.
For most usage patterns, I agree, but for those that likes to have 30+ tabs open (for some reason), caching the uncompressed bitmap is a _big_ waste.
The question is if the best approach wouldn't be to just forget or limit those caches for that kind of usage. At least the bitmap approach is asking for disk trashing.
Bitmap cache Posted Oct 9, 2007 18:49 UTC (Tue) by Blaisorblade (guest, #25465) [Link] I've read time ago a comment from a OLPC developer on exactly this issue. I'm writing a summary of his comments I recall.Basically, an application can store an uncompressed bitmap in the X server. And Firefox caches all images this way! That's the problem. Probably avoiding the caching would be even faster (due to no swapping).
IMHO (and this is my opinion, I don't recall his one) a reasonable solution would be to extend the X protocol to allow caching compressed images in most reasonable formats (including especially JPEG). Then the X server can keep a cache of most shown decoded images.
Since the JPEG compression ratio is huge, there would be a huge gain from this.
Off by an order of magnitude? Posted Aug 9, 2007 1:51 UTC (Thu) by lysse (guest, #3190) [Link] Quite right. Sorry. Oops.
Still waiting for swap prefetch Posted Jul 28, 2007 3:43 UTC (Sat) by roelofs (guest, #2599) [Link] Really, FF crashes on me about once a week, because although I have 2GB RAM and 6GB of swap, firefox manages to malloc() 4 GB!Geez, what are you doing to the poor thing?? No idea where it is going - admittedly I tend to have about 200 tabs open, but that alone shouldn't be the problem. OK, that might do it... I have two FF windows containing a total of 9 tabs, and it's been running since early March. Total memory usage (per ps(1)): 197M allocated, 157M resident. (Of course, there's also X11 pixmaps, as was noted in another LWN article a while back. I've forgotten how to check that, but even if FF were the sole X client--which it's not, by far--that's only another 194M allocated and 46M resident.) Greg
Still waiting for swap prefetch Posted Aug 1, 2007 2:23 UTC (Wed) by set (guest, #4788) [Link] xrestop will show you the X memory usage, such as pixmaps.
Still waiting for swap prefetch Posted Aug 1, 2007 16:43 UTC (Wed) by zlynx (subscriber, #2285) [Link] One of these days someone needs to add the amount of CPU X spent per client to xrestop.
Still waiting for swap prefetch Posted Aug 3, 2007 23:45 UTC (Fri) by roelofs (guest, #2599) [Link] xrestop will show you the X memory usage, such as pixmaps.Danke, sir! That's the command I couldn't recall. Greg
|
|||||||||||||