LWN: Comments on "LinuxCon: A tale of two bootcharts"

LinuxCon: A tale of two bootcharts

Aissen — Tue, 07 Sep 2010 15:34:11 +0000

if the phone booted more rapidly, I'd turn it off during movies instead of just turning it to vibrate.

That's what "Airplane Mode" is for :-)

LinuxCon: A tale of two bootcharts

roelofs — Fri, 03 Sep 2010 21:13:02 +0000

But I really wish my Garmin GPS unit would boot up quicker. It's kind of annoying that by the time it's started up and gotten a satellite lock, I can be halfway to my destination. :)

Ditto, but at least it has an excuse: the world has changed every time it boots. (AFAIK, the long lock time is due to it trying different antenna configurations in order to optimize overall signal strength from the current spread of satellites. Changing its orientation while it's doing that can seriously slow it down, btw...)

Greg

LinuxCon: A tale of two bootcharts

oak — Fri, 03 Sep 2010 20:19:03 +0000

The Android fsck_msdos execution was interesting, was it for an empty FAT file system or a full one? If latter, how many files & dirs it had? (thousands, tens of thousands?)

LinuxCon: A tale of two bootcharts

jschrod — Thu, 02 Sep 2010 11:20:02 +0000

For LaTeX it is still of interest in special circumstances where one preloads classes and packages as well and then dumps the fmt file.

In a current project of mine, I was able to improve the format time of a document from 4 sec to 0.4 secs with usage of such a format file. It's not much for one document, but since (a) we're producing ca. 15 million documents a year and (b) this is also used in an interactive environment, such performance improvements were very well received by our users.

sreadahead

giraffedata — Thu, 02 Sep 2010 06:09:50 +0000

That's the whole point. The HD is doing a lot of seeking when it could just be moving the head down the platter across all of the tracks linearly, which means zero seek time.

That's the whole point of what? It wasn't the point of the article or any previous comment, since they talked about SSDs (no seek time) and about CPU overlapping disk time. (Simply overlapping CPU and disk time, on mechanical disk systems I know would be barely noticeable, because the disk time is so great compared to the CPU time).

On the SSD-based system, though, the overlap is significant because the disk time is so much shorter. If overlapping buys you 7 seconds toward a 10 second boot, the CPU time and disk time must be of the same order.

sreadahead

blitzkrieg3 — Thu, 02 Sep 2010 03:18:22 +0000

> But just based on what I've seen and heard my disk drive do during painfully slow boots, I'll bet the CPU time is dwarfed by the seek time, so overlapping CPU and disk time wouldn't buy a whole lot.

That's the whole point. The HD is doing a lot of seeking when it could just be moving the head down the platter across all of the tracks linearly, which means zero seek time.

Also look at the charts, even if you discount the HD seeking, it buys about 7 seconds. Otherwise these seconds would be spread about the boot while a process was blocking instead of actually running.

Linux on Grandma's phone

dmarti — Tue, 31 Aug 2010 17:04:25 +0000

If the cell carriers can eke out enough additional revenue per low-end subscriber using "crapware", then the Android phone becomes the free-with-service phone, and the fast-booting phone vanishes from the market. So it's reasonable to start making Android usable for Grandma's emergency glove compartment phone now.

LinuxCon: A tale of two bootcharts

dlang — Tue, 31 Aug 2010 04:26:03 +0000

I have mixed feelings on priorities, to a large extent speeding up boot time involves finding a way to not need to do things at boot, and that ends up paying dividends overall, not just at boot time.

LinuxCon: A tale of two bootcharts

jcm — Tue, 31 Aug 2010 04:20:17 +0000

I think you answered the critical-use only situation quite well - don't get a smartphone (computer). It's not that boot time doesn't matter to anyone, but I made my comment in that way to exaggerate the absurdity of caring about boot time before everything else is fixed. There are so many other actual issues out there that matter more before that does.

LinuxCon: A tale of two bootcharts

dlang — Mon, 30 Aug 2010 22:57:16 +0000

one situation where phone boot time could be critical.

My grandmother (92) carries a cell phone for emergancy use only, it's turned off unless she needs to make a call.

In an emergancy, I could very easily see a 1-minute boot time being critical.

now the phone she uses isn't a smartphone, but the point is that just because you leave your phone on all the time to receive incoming calls, doesn't mean that everyone will, and for those that don't, the boot time is significant.

on my laptop I don't care much about boot time, but that's because I so seldom turn it off, it goes from external power at home to external power in my turck to external power at the destination...

I don't use suspend, but I don't boot it more than once every couple of months.

LinuxCon: A tale of two bootcharts

foom — Mon, 30 Aug 2010 22:31:24 +0000

> Why does it matter how long it takes for a cellphone to boot?

Probably not a huge deal.

But I really wish my Garmin GPS unit would boot up quicker. It's kind of annoying that by the time it's started up and gotten a satellite lock, I can be halfway to my destination. :)

LinuxCon: A tale of two bootcharts

dlang — Mon, 30 Aug 2010 22:25:41 +0000

I find it annoying, but then again it may be usual for people to use their phones continuously enough to drain the battery to the point where they shut off, then scramble for power and have to sit and wait for the phone to turn back on.

if the phone booted more rapidly, I'd turn it off during movies instead of just turning it to vibrate.

if you can boot fast, then you can power off and reboot instead of doing a suspend (with all the overhead of saving your entire memory to durable media)

LinuxCon: A tale of two bootcharts

jcm — Mon, 30 Aug 2010 22:13:27 +0000

Same goes for laptops that don't dual-boot. If you're only rebooting your laptop or netbook every few days/weeks, it's going to be tough to convince me that it really really matters.

LinuxCon: A tale of two bootcharts

jcm — Mon, 30 Aug 2010 22:12:06 +0000

Why does it matter how long it takes for a cellphone to boot?

I say that to be intentionally provocative, but let's be honest, I reboot my Android phone maybe once every few weeks, and the less-than-a-minute it takes to boot isn't a huge deal over that kind of timeframe. Yea, it's annoying for developers of Android core bits, but it's not the end of the world. We probably need to be a bit more honest about why we like low boot times, given that it is likely more for pretty graphs than because the user really boots their phone enough times to find it a selling feature.

Jon.

sreadahead

giraffedata — Mon, 30 Aug 2010 21:59:42 +0000

The main advantage is to never have to do iowait but always have the data you need at hand. Eg doing 10 seconds of disk reads and 15 seconds of CPU time in 15 seconds rather than in 25 seconds.

OK, I forgot about that. So I take it sreadahead runs in parallel with other boot processes?

But just based on what I've seen and heard my disk drive do during painfully slow boots, I'll bet the CPU time is dwarfed by the seek time, so overlapping CPU and disk time wouldn't buy a whole lot.

sreadahead

Jonno — Mon, 30 Aug 2010 20:07:35 +0000

The only advantage I know to reading stuff into cache ahead of time is that you can do it in optimal order.

The main advantage is to never have to do iowait but always have the data you need at hand. Eg doing 10 seconds of disk reads and 15 seconds of CPU time in 15 seconds rather than in 25 seconds.

sreadahead

giraffedata — Mon, 30 Aug 2010 16:01:19 +0000

On a solid-state block device, where order doesn't matter, what is the point of sreadahead? The only advantage I know to reading stuff into cache ahead of time is that you can do it in optimal order.

With disk drives, I don't think user space should be ordering the reads -- that expertise belongs in the block layer and below. All you have to do is pile all the read requests into the block layer and it will order them properly (of course, you don't have to get everything in the queue at once -- eliminating 99% of the seek time is fine).

And though I don't know what the state of implementation is, in theory you shouldn't be piling anything explicitly into the block layer either; you should just be advising the block layer of your plans to read that stuff later (madvise).

LinuxCon: A tale of two bootcharts

i3839 — Sat, 28 Aug 2010 17:36:21 +0000

Or in other words: Pigs don't fly. ;-)

LinuxCon: A tale of two bootcharts

Trelane — Fri, 27 Aug 2010 20:21:01 +0000

I think you're confused about how hardware probing is done.

Oh, that's a virtual certainty. ;)

[excellent lesson here; thanks!]

The 'probing' I refer to involves drivers checking which variant is present and then initialising it appropriately; drivers that you don't need will never be invoked.

Is this part done via ACPI, as outlined above? Or is this done 90's style? What's the slow part of this, precisely? Is it the initialization? If so, why is it significantly faster to have the module in-kernel?

That might be worth a try.

wish I had time to. It sounds like perhaps a rather extensive project, as it touches on fundamentals of how things work. If static linking is truly a significant portion of the boot process, though, it might well be worth it. Only testing would tell.

LinuxCon: A tale of two bootcharts

BenHutchings — Fri, 27 Aug 2010 20:13:51 +0000

It strikes me, however, that the basic problem *here* is finding out what hardware is present and not present.

I think you're confused about how hardware probing is done. Back in the 90s PCs would have a load of ISA devices that each had to be probed in their own way. Today almost every device is described by ACPI tables or the standard PCI or USB configuration mechanisms. The kernel uses these to find which devices are present and only invokes the drivers associated with them. It also sends events out to udev, which loads modules if needed. The 'probing' I refer to involves drivers checking which variant is present and then initialising it appropriately; drivers that you don't need will never be invoked. (There are exceptions, e.g. 8139cp and 8139too handle different devices with overlapping sets of device IDs. Normally they will both be loaded and may both probe the same device.)

This could be mitigated by means of a table of "known to be present" hardware information supplied to a bulk-loaded module set (something along the lines of putting all the .ko's into a single .ko with tables specifying that it's a bulk module and where everybody's at in it, as outlined above.)

That might be worth a try.

LinuxCon: A tale of two bootcharts

Trelane — Fri, 27 Aug 2010 18:04:13 +0000

deciding if the module should be loaded or not, and if you load a module that isn't needed, waiting for the module to decide that the hardware it's trying to initialize really isn't there.

I think the remedy for this could be implemented as I replied back to the other subthread. I don't know, however, if this is at all feasible, but the core problem is that upon first boot the kernel doesn't know what hardware is present and what isn't. Supplying information pertaining to what is likely to be found again (a simple list with information on what modules were most recently used would be a simpler algorithm, and since it could clearly rely on userspace tools to put together the information, this could be a policy thing) would avoid this. The problem would then be making sure we fall back gracefully (simple idea would be to just fall back to scanning as is currently done).

LinuxCon: A tale of two bootcharts

Trelane — Fri, 27 Aug 2010 18:00:02 +0000

I am painfully aware that I'm hitting the envelope of my knowledge, pretty hard.

"The annoying thing about driver modules is that module initialisation, which includes binding to and probing all the devices they can handle, is serialised."

It strikes me, however, that the basic problem *here* is finding out what hardware is present and not present. In the vast majority of cases, I'd wager that this really only needs done once (rather like the crypto hardware tests): upon installation. Beyond that, what's needed is graceful fallback for the rarer cases the hardware isn't present anymore, and what we already have: new hardware detection.

This could be mitigated by means of a table of "known to be present" hardware information supplied to a bulk-loaded module set (something along the lines of putting all the .ko's into a single .ko with tables specifying that it's a bulk module and where everybody's at in it, as outlined above.) If intialization of the known hardware fails, then it could perhaps fall back to probing.

LinuxCon: A tale of two bootcharts

dlang — Fri, 27 Aug 2010 17:45:06 +0000

the slowest part is deciding if the module should be loaded or not, and if you load a module that isn't needed, waiting for the module to decide that the hardware it's trying to initialize really isn't there.

a monolithic kernel that is tailored for your hardware (includes all the drivers you need and few or none that you don't) will boot up _much_ faster as a result.

LinuxCon: A tale of two bootcharts

BenHutchings — Fri, 27 Aug 2010 14:57:52 +0000

"The problem is that vmlinuz and initramfs must be loaded prior to accessing the driver for your sata/pata/scsi/etc controller. There are special bios calls that let you do this, but those are crappy. Each call only fetches a single block (512 bytes) and requires a pass through the kernel/bios boundary as well as a call to the harddrive."

No, it's not that bad; you can read multiple sectors at a time. And the kernel is not involved at this point. There's just a boot loader which is probably running in real mode, at least while it's looping over the block list.

The annoying thing about driver modules is that module initialisation, which includes binding to and probing all the devices they can handle, is serialised. Initialisation of built-in drivers can be parallelised. I assume this is hard to change, otherwise someone would have done it already.

LinuxCon: A tale of two bootcharts

Jonno — Fri, 27 Aug 2010 00:35:52 +0000

I must be missing something. The Intel guys were loading Gnome, and got to 5 seconds, right?

No, they used a slimmed down XFCE on a feature stripped X...

LinuxCon: A tale of two bootcharts

nix — Thu, 26 Aug 2010 22:47:20 +0000

Emacs did that for a long time, then both Emacs and XEmacs shifted to a more explicit serialization mechanism (the resulting files are mmap()ed in, so still shared between instances IIRC), mostly because keeping unexec() working was a pain.

LaTeX never did the coredump trick because it wasn't originally written on a Unix system at all (hell, it's written in Pascal). It's always written out a serialized representation of its state ('format files') then loaded them later. I'm not sure how valuable this is for LaTeX proper anymore (who cares about a fraction of a second?) but it's very valuable for big formats like ConTeXt, which takes ages to load even on modern systems.

LinuxCon: A tale of two bootcharts

Jonno — Thu, 26 Aug 2010 22:26:52 +0000

What specifically is the slow part of modules? Having to suck them in individually?

Actually, modules loaded after rootfs is mounted isn't the problem. The problem is the initramfs, ormore acurately vmlinuz+initramfs . So simply building all modules normaly included in the initramfs into the kernel won't help either (well, you do loose the early userspace bits, which helps somewhat, but are neglible in the big picture). What you need is fewer modules to load prior to rootfs is mounted, and that can only be done by knowing the hardware in advance.

The problem is that vmlinuz and initramfs must be loaded prior to accessing the driver for your sata/pata/scsi/etc controller. There are special bios calls that let you do this, but those are crappy. Each call only fetches a single block (512 bytes) and requires a pass through the kernel/bios boundary as well as a call to the harddrive.

A kernel/bios boundary is essentially the same as a userspace/kernel boundrary, but bioses are generally crappy, and will usually trash all register, most or all L1 cache, and up to 1MB of the L3 cache. Doing that thirty-five-thousands-and-thirty-seven (35,037) times (my Ubuntu netbook) can take a few seconds...

Touchpad from Dell hell (OT)

sflintham — Thu, 26 Aug 2010 22:15:53 +0000

There is some information in this bug report. I think I installed some package from a PPA with Ubuntu 9.10 to largely fix the issue. I must admit I haven't used my Mini 10 much since I upgraded to 10.04, but it appears to be fixed by default there. (FWIW, I do recall the Dell-supplied Ubuntu 8.04 (?) didn't have any problems either.)

LinuxCon: A tale of two bootcharts

PhracturedBlue — Thu, 26 Aug 2010 21:58:23 +0000

I must be missing something. The Intel guys were loading Gnome, and got to 5 seconds, right?

I assume they didn't use compiz (and while I use the Ubuntu netbook edition on mine, I dunno that I really need it either), but even that seems to be only 4 seconds additional. So what did the Intel guys do to gnome to make gnome so much faster than the Ubuntu folks can manage?

I'd love my netbook to boot in 5 seconds, but after trying MeeGo, I'm not willing to move to it to achieve that goal.

I like the interface Ubuntu provides on my netbook, but I'd be willing to try a different environment if it brought a significant speedup.

LinuxCon: A tale of two bootcharts

Trelane — Thu, 26 Aug 2010 20:36:58 +0000

"Though Intel claimed a speed boost from statically compiling modules into the kernel, the Canonical team has to be able to support more than just Intel netbooks."

What specifically is the slow part of modules? Having to suck them in individually?

It seems like it ought to be possible to build some sort of metamodule to avoid one-by-one; put them all in one glob and suck them in en masse.

If that's not it, it seems like it ought to be possible to create a hybrid kernel with quasi-modules for the hardware you installed on already in and ready to go.

LinuxCon: A tale of two bootcharts

Trelane — Thu, 26 Aug 2010 20:30:46 +0000

"have to buy a whole bunch of "standard" computers."

This would be the nonstandard standard?

Grammmar error

jake — Thu, 26 Aug 2010 20:28:34 +0000

> "can be run both environments" is incorrect.

yup, it is ... seemed like 'in' fit in well there, so that's what i did.

thanks for the report, just a friendly reminder that typos should go to 'lwn@lwn.net'.

jake

Grammmar error

tmassey — Thu, 26 Aug 2010 20:25:13 +0000

"can be run both environments" is incorrect. I can't come up with a simple word to insert there and correct (and complete) the thought. I guess it would have to be something like "can be used to improve both environments"?

Touchpad from Dell hell (OT)

boog — Thu, 26 Aug 2010 19:15:49 +0000

'They chose the Dell Inspiron mini 10 netbook, dubbed the "touchpad from hell" by Scott because it was hard to use without the pointer jumping around.'

Interesting. Although I don't have that precise Dell notepad, I think my Latitude D430 has a touchpad from the same place. The pointer will jump suddenly (possibly under load). Though some progress has been made in that it will often return now after its jump, it still causes plenty of problems and is *very* ugly.

Do any of the assembled geniuses happen to know where the problem resides and if it is fixable? I hardly know where to start - kernel, X, ...

Best regards

LinuxCon: A tale of two bootcharts

iabervon — Thu, 26 Aug 2010 18:01:25 +0000

I thought emacs switched at some point from actually dumping core to more explicitly byte-code compiling the elisp and writing it to files, on account of needing a substantial chunk of optional elisp from a very large library when starting up; dumping only the fundamental stuff was still too slow, and dumping everything would require reloading too much, so they needed a quick way to load individual modules, and this obsoleted the core dump mechanism. But I could be wrong; it's been ages since I've actually watched emacs build.

LinuxCon: A tale of two bootcharts

Fowl — Thu, 26 Aug 2010 14:30:16 +0000

<OT>
I've always found hibernate on Windows to be significantly faster than a fresh boot. Perhaps I'm stuck with hardware that is too low end though - 1GB of IO is faster than 4GB of IO shock horror!

Sleep (and "Hybrid Sleep", aka suspend to ram + hibernate) _are_ marvellous, barring any firmware SNAFU's.
</OT>
re: old is new
A good idea is a good idea. ;)

sreadahead

blitzkrieg3 — Thu, 26 Aug 2010 13:07:53 +0000

Apologies. I based that on what Scott said at the discussion as well as this forum post. Can those ioctls be written for ext4? In the LPC article it was mentioned that Ted Tso suggested this for ext4, but I don't know if it has been done since.

sreadahead

liljencrantz — Thu, 26 Aug 2010 09:53:21 +0000

So what, then is the difference between sreadahead and über-readahead?

LinuxCon: A tale of two bootcharts

liljencrantz — Thu, 26 Aug 2010 09:52:13 +0000

The difference is that back in the 80s, a lot fewer people used prepackaged emacs.

LinuxCon: A tale of two bootcharts

thedevil — Thu, 26 Aug 2010 05:39:15 +0000

Back in the 80s? It still does precisely that AFAIK. Other programs which provide both scriptability and a built-in core written in the scripting language do something similar, or else they just ship the core blob in the source (hello ocaml!)