Weekly Edition
Return to the Front page
| |||||||||||||
In defense of Firefox
Normally, one would not expect a posting about a performance-related bug in
pre-release software to generate a great deal of interest. But when LWN pointed to a weblog entry
describing the Firefox 3 fsync() bug, the result was (at last
count) over 90 comments. Clearly something is going on here to stir up
this level of conversation. It's not clear that all participants are
taking away the right conclusion, though.
Firefox 3 uses the sqlite database, as do a number of other applications. In this case, the database is being used to store the user's browsing history; an active user can create quite a bit of database traffic in a short amount of time. As part of updating the database, sqlite will call fsync() to ensure that the data gets to the disk. All fairly normal stuff which shouldn't create trouble for anybody. Except it does create trouble because (1) these fsync() calls are made frequently, and (2) Linux filesystems do not handle fsync() as well as they should. As a result, heavy use of Firefox 3 on a Linux system can cause the system as a whole to perform poorly. It's clearly an issue that the Firefox developers need to fix, even if it's not entirely their fault. And it appears that they do, indeed, plan to fix it. One could argue that Firefox 3 should never have gotten as far as a release candidate with this sort of bug unfixed. The problem was first reported in early March, but the conversation did not get going in earnest until late April. So there was not a whole lot of time to figure out a fix for the release candidate. More justifiably, it could be said that the developers' consideration of shipping the final release with this problem unfixed was insensitive to the needs of Linux users. The notion that they would bless distributors who patched the problem themselves does not help much in this regard. That thought does raise an interesting question, though: what would have happened if Mozilla did not bless a patch, then denied use of the "Firefox" trademark to distributors who fixed the problem anyway? But all of this is moot; it appears that the Firefox developers have decided to do a second release candidate which includes a fix for this problem. The other common complaint has to do with why Firefox is using a relational database in the first place. It is, arguably, a significant bit of bloat for an already large application. The answer from the developers is that a real database is needed to provide the kind of features that Firefox users are asking for. As the discussion on LWN showed, there really are users who want quick access to significant amounts of history. Your editor has, so far, not had his socks knocked off by the "awesome bar," but other users clearly have. There is value in adding features that your users will call "awesome." There is also value, of course, in the creation of a small and fast application. The Firefox developers have had to chart a course between the addition of features and keeping the overall size reasonable, and they have taken grief for their decisions on both sides. One user's bloat is another user's indispensable tool; it's hard to keep everybody happy. But one generally keeps more users happy by including the features they want. Despite all of that, Firefox 3 clearly shows the results of some attention to performance and memory use. Your editor has not done any sort of formal benchmarking, but a few months of use of the beta releases leads to the conclusion that Firefox 3 is more responsive than its predecessor, and that many of the worst memory usage problems have been addressed. It has been a while since the days when regular restarts to combat the effects of memory leaks were required. Firefox will never be a lightweight program - or even a middleweight program - but it does appear that, for now, the monster's growth has been restrained. Firefox 3 also includes greater GTK integration, which has inspired complaints of its own. But better integration with the Linux system has been something users have been requesting for a while. It is hard to fault the developers for trying to satisfy that request. All told, it would appear that the Firefox community is trying to follow through on its promise of better support for Linux users. They seem to be doing what has been asked of them. In so doing, they have produced a new major release which, for whatever faults it may have, is a real improvement on what came before. The development process which helped to rescue the net from proprietary software and standards continues in full swing. There will, beyond doubt, be no shortage of things to criticize the Firefox developers for in the future. But, before we do that, it's worth taking a moment to back off, let them get the 3.0 release out the door, and congratulate them for a job which is truly, in many ways, well done. (Log in to post comments)
In defense of Firefox Posted May 29, 2008 3:50 UTC (Thu) by kjp (subscriber, #39639) [Link] >>The other common complaint has to do with why Firefox is using a relational database in the first place.<< The relational db part of sqlite isn't the issue, nor is it bloated. It's the (easily turned off) paranoid transaction safety that's the question. I have no fear though, that this bug will be easily fixed by turning off synchronous transaction writes (something I have done in my own sqlite app as it gives like ~50 times performance) and simply atomically copy/rename the file periodically to suit mobile or performance users.
In defense of Firefox Posted May 29, 2008 6:06 UTC (Thu) by roelofs (guest, #2599) [Link] The relational db part of sqlite isn't the issue, ...I believe it's one of the issues. Some of us like the ability to hand-edit the bookmarks file, grep it, diff it, copy/merge it between machines, etc. (Granted, there are probably better ways to handle the latter...) If I'm not mistaken (and here I [blush] have to admit that I have yet to try FF3, so I easily could be mistaken), bookmarks are one of the things now hidden within the binary blob that is the sqlite database. Greg
In defense of Firefox Posted May 29, 2008 6:45 UTC (Thu) by tetromino (subscriber, #33846) [Link] So just run sqlite3 ~/.mozilla/firefox/*/places.sqlite and hand-edit the bookmark database via simple SQL commands...
In defense of Firefox Posted May 29, 2008 11:02 UTC (Thu) by k8to (subscriber, #15413) [Link] Because knowlege of use of text editors implies knowledge of SQL commands?
In defense of Firefox Posted May 29, 2008 11:43 UTC (Thu) by hummassa (subscriber, #307) [Link] I would say knowledge of how to use a text editor to edit your bookmarks.xml files pretty much implies sufficient knowledge to learn a couple of SELECTs.
In defense of Firefox Posted Jun 5, 2008 9:52 UTC (Thu) by anandsr21 (guest, #28562) [Link] I find your faith in the cut copy paste generation, disturbing ;-).
In defense of Firefox Posted May 29, 2008 8:23 UTC (Thu) by roc (subscriber, #30627) [Link] The really good news is that *history*, which used to be in the appalling "mork" format, is now also in SQLite. So accessing your history data just got increadibly easier.
In defense of Firefox Posted May 29, 2008 12:59 UTC (Thu) by nix (subscriber, #2304) [Link] Seconded. Anything which kills mork off for good and hurls it into the Sun with a stake through its heart is a good thing in my book.
In defense of Firefox Posted May 29, 2008 9:23 UTC (Thu) by Tuxie (guest, #47191) [Link] Normally I'd agree with you but Sqlite is a pretty standard and is supported just about everywhere, and the flexibility improvements it adds to Firefox are significant and definitely worth it. If you want to manipulate the data outside of FF, run "sqlite3 path/to/db.sqlite" and manipulate whatever you want with standard SQL. You can also use just any programming language there is to automate your FF-data manipulation tasks, something that was way harder before.
In defense of Firefox Posted May 29, 2008 13:43 UTC (Thu) by iabervon (subscriber, #722) [Link] The bookmarks.html file is still there; it's just no longer authoritative. So you can search it (and use it as your home page) as before; it probably wouldn't be hard to write something to take a HTML file and update the database with it, either, but I don't know if anyone's written that. Personally, what I'd like is if they made "live bookmarks" in the HTML file have a button to use AJAX to get recent items.
In defense of Firefox Posted May 29, 2008 6:18 UTC (Thu) by roc (subscriber, #30627) [Link] Thanks for that. One thing though: > That thought does raise an interesting question, though: what would have > happened if Mozilla did not bless a patch, then denied use of the > "Firefox" trademark to distributors who fixed the problem anyway? This reads to me like "what if Mozilla just decided to torment Linux distributors for fun?" It's hard to answer because I can't imagine a plausible scenario where it would happen. At worst, e.g. the majority of Mozilla staff become possessed by demons, it would be Iceweasels all round. Yay for software freedom. On the other hand I can imagine a scenario where some distro produces a harebrained patch that corrupts user data a lot, and Mozilla people would have to think hard about whether that was acceptable. But I wouldn't classify that as "fixing the problem".
In defense of Firefox Posted May 29, 2008 8:18 UTC (Thu) by ssam (subscriber, #46587) [Link] I am not sure that i have seen it said anywhere, but firefox 3 has increased the default history lifetime from 9 to 180 days. This is vastly more data that the old history system probably would not scale to. Has anyone tried reducing the history life time, so see if that improves performance?
In defense of Firefox Posted May 29, 2008 9:17 UTC (Thu) by ekj (guest, #1524) [Link] Fine. But this is just an -example- of a -general- problem in FF. The fact that the UI is (or appears to be) single-threaded. I don't particularily -care- if some action that FF needs to perform takes 0.1 seconds, or 3 seconds. Offcourse I -prefer- if it takes 0.01s, but I can accept any if it's really nessecary. What irks me though, is that I've got 10 tabs open, and ALL of them, the ENTIRE user-interface freezes COMPLETELY because of something happening in ONE of the tabs. That is annoying and useless. There's no reason whatsoever that tab1 needs to not render or respond because tab2 is doing something or other.
In defense of Firefox Posted May 29, 2008 10:51 UTC (Thu) by Thue (subscriber, #14277) [Link] Agreed. It should really be multithreaded.
In defense of Firefox Posted May 29, 2008 15:09 UTC (Thu) by docwhat (subscriber, #40373) [Link] The threading isn't actually the whole of the problem. When fsync() is called on certain filesystems (ext3 without certain optimizations turned on is mentioned in the bug) it causes the whole desktop to freeze/slow as all the data is written to disk. fsync() in these filesystems is really just a call to sync(). :-( Ciao!
In defense of Firefox Posted May 29, 2008 15:50 UTC (Thu) by tjc (subscriber, #137) [Link] Some of the "freezes" in Firefox last 30 seconds or more, so I don't think it's the file system in all cases.
In defense of Firefox Posted May 29, 2008 18:48 UTC (Thu) by vonbrand (subscriber, #4458) [Link] For me, FF (on Fedora rawhide) freezes for good when accessing trac sites, or certain blogs (Groklaw, when trying to post a comment FF went off and didn't come back in an hour or so), and trying to comment on LWN got it frozen too.
In defense of Firefox Posted May 29, 2008 15:55 UTC (Thu) by BenHutchings (subscriber, #37955) [Link] In my experience, UIs are always required to be single-threaded. The locking needed to make multi-threaded UIs work will tend to kill any speed advantage. Now some time-consuming operations maybe ought to be done asynchronously so the UI thread doesn't have to wait for them. But that's easier said than done. I suspect that the problem you're seeing (and which I recognise) is due to running Javascript in the UI thread. I don't know whether it's possible to avoid that, without introducing potential for deadlocks.
In defense of Firefox Posted May 29, 2008 16:46 UTC (Thu) by nix (subscriber, #2304) [Link] Well, yes, but if instead of single-threading you desynchronize the (event-driven) UI from the backend processing *and rendering* work, it can get fast again. (After all, what else is an X server?)
In defense of Firefox Posted May 29, 2008 16:47 UTC (Thu) by nix (subscriber, #2304) [Link] That is, of course, instead of *multi*-threading. :)
In defense of Firefox Posted May 30, 2008 7:19 UTC (Fri) by ekj (guest, #1524) [Link] The UI may need to be single-threaded. But handling the events need not be. Control could, and should, return to the UI-thread without waiting for the event to be finished handling. Just because tab a is currently in the process of repainting, there's no reason to -not- process new events coming in. Yeah, yeah, I know -why- FF is like this. But that doesn't change the fact that this particular aspect truly sucks.
In defense of Firefox Posted May 29, 2008 16:07 UTC (Thu) by intgr (subscriber, #39733) [Link] > There's no reason whatsoever that tab1 needs to not render or > respond because tab2 is doing something or other. The reason is the ancient design decision that the UI and behavior would be customizable through JavaScript extensions. As extensions are global (i.e. they can see and manipulate all of your tabs and UI elements), they need to live in a single interpreter. JavaScript has classically been a single-threaded language; making it multi-threaded now would create a lot of confusion and unexpected race conditions; most developers cannot even begin to imagine all the ways that, for example, the renderer and concurrent changes to a document's DOM could interact. Adding coarse-grained locking does not have much benefit: any interaction between concurrent threads would still likely cause one to wait for the other to yield a lock. On the other hand, with fine-grained locking, writing code to handle race conditions would make extensions and Firefox more complex and more error-prone. Furthermore, adding concurrent multithreading support to a high-level language is no easy task and has a significant performance impact on the interpreter alone -- as Perl, Python, Ruby, Smalltalk etc developers can confirm. Now, I'm just playing the devil's advocate and pointing out all the problems with this approach; clearly it is *possible* to rewrite Firefox as a multi-threaded browser, but it's a decision that has to be made with great precaution.
In defense of Firefox Posted May 29, 2008 18:13 UTC (Thu) by oak (subscriber, #2786) [Link] What would you lose if each Firefox window would be a separate process and only tabs within same window would be in the same process? At least moving a tab from one window to another would then be much harder, maybe even impossible if all the tab state needs to be moved too. Maybe windows-as-separate-processes could be a config option?
In defense of Firefox Posted May 29, 2008 18:48 UTC (Thu) by thoffman (subscriber, #3063) [Link] How about one thread per source domain? So if I have a bunch of windows and tabs open -- say, a Google Mail Compose window, plus the main Google Mail screen in a tab, then those two windows will be rendered by the same single thread, eliminating dangers of race conditions there. Meanwhile, in another tab, I have LWN open. It will be rendered by another thread. So, if Firefox's slow Javascript machine starts to suck wind in GMail, at least LWN will be nice and snappy on my multicore machine...
In defense of Firefox Posted May 29, 2008 19:53 UTC (Thu) by tjc (subscriber, #137) [Link] The reason is the ancient design decision that the UI and behavior would be customizable through JavaScript extensions. As extensions are global (i.e. they can see and manipulate all of your tabs and UI elements), they need to live in a single interpreter.That explains a lot! A repeatable example (at least for me) is linuxjournal.com, which freezes Firefox for about 15 seconds on my system.
In defense of Firefox Posted Jun 19, 2008 17:01 UTC (Thu) by tjc (subscriber, #137) [Link] This is no longer a good example -- the Linux Journal front page loads much faster than it did a few weeks ago, at least for me.
In defense of Firefox Posted Jun 5, 2008 5:30 UTC (Thu) by renox (subscriber, #23785) [Link] I agree that this is an important issue: I switched to Opera for this reason. And I doubt that FF3 will make me switch back as the 'one tab freeze the whole browser' issue will still probably be there.
In defense of Firefox Posted Jun 7, 2008 7:04 UTC (Sat) by muwlgr (guest, #35359) [Link] Konqueror gives better multitask experience as well. Otoh, its GUI does not use JavaScript.
In defense of Firefox Posted May 29, 2008 14:12 UTC (Thu) by drfickle (guest, #1093) [Link] I'm using Fedora 9 at home and have yet to encounter this behavior. What does it take to trigger it? 20 tabs? Limited memory/swap? So far I don't have any complaints about FF3. However, like our editor, I am still waiting for the "awesome" to kick in.
In defense of Firefox Posted May 29, 2008 14:52 UTC (Thu) by felixfix (subscriber, #242) [Link] I believe you have to have some other process(es) with heavy disk I/O, so when ff3 fsync()s, it also fsync()s all disk I/O, not just its own. I have had it when surfing while a compile is running, and it is awful.
ext3 fsync is the real issue Posted May 30, 2008 7:41 UTC (Fri) by Cato (subscriber, #7643) [Link] The real issue for me is that ext3 has a broken implementation of fsync() - since fsync is a file based operation there's no excuse for syncing *all other files on the filesystem*. Having said that, I think FF3 or sqlite could equally have used O_SYNC on write, which has lower performance in the single file case but far better performance when other files need syncing with fsync().
ext3 fsync is the real issue Posted May 30, 2008 17:05 UTC (Fri) by nlucas (subscriber, #33793) [Link] The SQLite main developer has already tried to make sqlite to use O_SYNC instead of fsync. The performance dropped to half.
In defense of Firefox Posted Jun 2, 2008 20:18 UTC (Mon) by jch (guest, #51929) [Link] > I'm using Fedora 9 at home and have yet to encounter this behavior. What does it take to trigger it? The issue is that Firefox will perform large numbers of fsync system calls on files in ~/.mozilla, and fsync is implemented as a global sync on journaled filesystems. One way to reproduce the slowdown is to put ~/.mozilla on an ext3 filesystem which has heavy unrelated write traffic. Another is to put it on a busy NFS filesystem (having home on NFS is rather common in academic settings).
In defense of Firefox Posted Jun 1, 2008 17:20 UTC (Sun) by jch (guest, #51929) [Link] They are using a full-fledged relational database with strong integrity guarantees to manage by browsing history, of all things. No amount of spin can hide how incredibly brain-damaged that is.
In defense of Firefox Posted Jun 1, 2008 23:03 UTC (Sun) by njs (guest, #40338) [Link] There's a lot of satisfaction to be had in reflexively ranting about things and slamming on them. In my experience, there is ultimately more satisfaction -- just of a slower and less instantly-gratifying sort -- in learning and building. Unfortunately, the two approaches are often at odds, because one is fundamentally about protecting one's preconceptions, while the other requires with-holding judgement and seeking out ways to challenge them. In this case, you're being factually wrong, utterly ignorant of the relevant technical issues, and actively hostile to changing either -- but you do probably *seem* smart to some readers, and that's a choice you can make. Many technical people do. YMMV.
In defense of Firefox Posted Jun 3, 2008 20:51 UTC (Tue) by dmag (subscriber, #17775) [Link] @jch: Please do some research before posting. SqlLite doesn't pretend to be an RDBMS. It doesn't support (enforce) basic datatypes. The shared library is ~1MB on my system (and can go down to ~100K when embedded and features removed). Yes, it supports "strong integrity guarantees". The opposite of that is called "corruption". If you wrote a browser, would you just call "truncate(bookmarks); write(bookmarks, data); close()"? I hope not. You would need to design a fail-safe recovery mechanism that doesn't loose data. It might be fairly simple when updating bookmarks, but for storing browsing history and doing indexing, you'd basically be re-writing most of SqlLite anyway.
In defense of Firefox Posted Jun 5, 2008 16:58 UTC (Thu) by pointwood (guest, #2814) [Link] I've been running FF3 since beta 1 on 2 machines and I've been quite satisfied with it. I would go as far as to say that it's been more stable than FF2. I have experienced my profile "go bad" which meant that it wouldn't start. That's quite some time ago though, I think it was while beta 2 was the latest. I like the awesome bar. Once you get used to how it works it is a very nice improvement. What I like the most though is all the work they have done on fixing the memory leaks and performance. It was very much needed and they have certainly delivered.
In defense of Firefox Posted Jun 6, 2008 4:31 UTC (Fri) by cybernytrix (guest, #5727) [Link] If you guys go check out the upcoming HTML5 spec, it talks about persistent client side storage and SQL queries to access that data! sqlite is tiny. Personally I don't care about the awesome bar, but I think including sqlite is good.
|
|||||||||||||