| |||||||||||||
A report from OSCON 2007A report from OSCON 2007Posted Aug 3, 2007 0:03 UTC (Fri) by jwb (guest, #15467)Parent article: A report from OSCON 2007
Looking for files or directories that don't exist? This is 99% or more of what glibc does behind your back. Look at this strace of cat(1):
open("/usr/lib/locale/en_US.UTF-8/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT
and on and on and on. Executing a subprocess is a veritable ENOENT bonanza as the linker/loader looks in all sorts of improbable places for bits of your program. If looking in the wrong directory for files that don't exist is a power consumption problem, we have a ton of work ahead of us.
(Log in to post comments)
A report from OSCON 2007 Posted Aug 3, 2007 1:15 UTC (Fri) by dberkholz (subscriber, #23346) [Link] I don't think it would be as problematic upon application startup as during runtime (for example, looking to see whether a file's appeared every 100ms).
Importance of application startup Posted Aug 3, 2007 3:34 UTC (Fri) by jreiser (subscriber, #11027) [Link] One dose of 'powertop' or other profiler will find the "sore thumb" applications or usages that poll; but these are rare. Over half of all processes run for 0.1 CPU seconds or less, and for these the "startup" consumes the vast majority of resources. A shell with builtin busybox (mv, ln, cp, rm, date, ls, ...) can be a significant savings. The general case would be a learning shell that could identify and remember the state after startup of a large class of frequently-run applications.
A report from OSCON 2007 Posted Aug 3, 2007 3:05 UTC (Fri) by joey (subscriber, #328) [Link] I think the question is, why is your cat looking for locales at all, if it's just catting a file. Mine only looks for locales if I run it with --help or something.
A report from OSCON 2007 Posted Aug 3, 2007 4:48 UTC (Fri) by carenas (subscriber, #46541) [Link] LANG=C
would get rid of all those extra "locale" file lookups
A report from OSCON 2007 Posted Aug 3, 2007 7:36 UTC (Fri) by tzafrir (subscriber, #11501) [Link] Actually the help message is surely somethng that needs translation. Hence there is a point in looking at LC_MESSAGES for the translation just for running cat --help .
Error messages cat spits out in case it can't find your file can also be translated.
A report from OSCON 2007 Posted Aug 3, 2007 17:56 UTC (Fri) by ajross (subscriber, #4563) [Link] strace -eopen cat /dev/null
It reads the (non-existent) locale files even when no output is generated.
A report from OSCON 2007 Posted Aug 4, 2007 12:10 UTC (Sat) by tzafrir (subscriber, #11501) [Link] right, so you want to complicate the code of every program:
if(open failed) {
print_help_message(){
A report from OSCON 2007 Posted Aug 4, 2007 20:36 UTC (Sat) by madscientist (subscriber, #16861) [Link] That's not required. Every time a program needs to translate a message, it calls a function to do the translation. It should not be too difficult to modify the code for "lazy evaluation", where the locale catalog is not set up until the first time a message needs to be translated, and hide this inside the i18n libraries.
A report from OSCON 2007 Posted Aug 3, 2007 4:15 UTC (Fri) by Nick (guest, #15060) [Link] Trying to open file names that don't exist should not always go to disk.The kernel has negative dentry caching, so only the first such access would hit the disk (no different from opening a file that does exist).
A report from OSCON 2007 Posted Aug 3, 2007 4:56 UTC (Fri) by arjan (subscriber, #36785) [Link] that's indeed true in theory. In practice it seems that the lifespan of negative dentries is so short that you still go to disk ;(
A report from OSCON 2007 Posted Aug 3, 2007 14:49 UTC (Fri) by drepper (subscriber, #5153) [Link] The open calls only happen if you or your distribution provider (or both) make mistakes or you are using truly ancient code. On Fedora and RHEL there is exactly one single open call
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
In fact, the individual files don't even exist anymore.
So, stop complaining if you cannot even keep your machine running correctly.
A report from OSCON 2007 Posted Aug 3, 2007 18:33 UTC (Fri) by jwb (guest, #15467) [Link] I can see that your reputation for being a flaming jerk is well deserved. Thanks for your practically useless reply and commercial endorsement of your employer's product.
A report from OSCON 2007 Posted Aug 9, 2007 2:25 UTC (Thu) by lysse (guest, #3190) [Link] Maybe drepper was out of line, but I have to say that from my perspective, this response was completely out of proportion.
A report from OSCON 2007 Posted Aug 9, 2007 8:47 UTC (Thu) by nix (subscriber, #2304) [Link] Well, it is notable that Ulrich gave no information at all about how to create it (although a one line response would have given all that was needed) and that there is essentially no documentation anywhere that describes it, either.
`Flaming jerk' is over the top. `Unhelpful' or `uninformative' would be more accurate.
A report from OSCON 2007 Posted Aug 10, 2007 11:37 UTC (Fri) by liljencrantz (subscriber, #28458) [Link] Pot. Kettle. Black.
A report from OSCON 2007 Posted Aug 3, 2007 21:30 UTC (Fri) by tetromino (subscriber, #33846) [Link] Ulrich, that's the first time I've ever heard of /usr/lib/locale/locale-archive. None of the distributions I use (Ubuntu and Gentoo) have it. No manpage on any of my systems mentions it. Is there some documentation somewhere for what this file is, what format is it in, what it is used for, how is it written or generated, and how distros (the ones that do not have the main glibc maintainer on their staff) can use it?
A report from OSCON 2007 Posted Aug 3, 2007 23:03 UTC (Fri) by johnkarp (subscriber, #39285) [Link] I fixed the issue on my gentoo box by modifying /usr/sbin/locale-gen (itsa script). One of the strings inside has "--no-archive", which I removed.
I reran locale-gen, and the archive was created. The number
I filed a gentoo bug:
A report from OSCON 2007 Posted Aug 4, 2007 0:14 UTC (Sat) by jwb (guest, #15467) [Link] On Ubuntu, this is the rational given for using --no-archive:
"The rationale is that with an archive it is not possible to mix system and user defined locales by setting LOCPATH."
Presumably this complaint was raised by some user who wanted to so exactly that thing. On Ubuntu, this can be changed in /etc/belocs/locale-gen.conf
A report from OSCON 2007 Posted Aug 5, 2007 16:51 UTC (Sun) by jbailey (subscriber, #16890) [Link] Right. In Ubuntu we don't use the locales from the upstream glibc tree on the grounds that they're quite frequently wrong and (justifiably) hard to get updated. Since we already have communities of folks through Launchpad who are doing the translations for the software, we trust them to tell us the locale-specific needs that they have.
There are a few ways that we could fix this better and work on having multiple locale archives or some such, but we just haven't gotten there yet.
Ideally this could be resolved Posted Aug 9, 2007 2:21 UTC (Thu) by JoeBuck (subscriber, #2330) [Link] If the translations can be pushed upstream, everyone can benefit and you can use the cache.
A report from OSCON 2007 Posted Aug 9, 2007 8:34 UTC (Thu) by nhippi (subscriber, #34640) [Link] > In Ubuntu we don't use the locales from the upstream glibc tree on the grounds that they're quite frequently wrong and (justifiably) hard to get updated. Since we already have communities of folks through Launchpad who are doing the translations for the software, we trust them to tell us the locale-specific needs that they have.
This sounds a lot like you are saying you are trading "high quality and working with upstream" with "whatever we get from users without verifying".
Or maybe let me put it the other way:
If the upstream really is "frequently wrong", despite of being "(justifiably) hard to get updated", how come this "(justification)" does not apply to updates via launchpad? What magic makes launchpad updates so much more high quality that it's worth to fork locales?
> There are a few ways that we could fix this better
Please do. Ubuntu is getting a reputation of doing quick hacks to hide the problem instead of actually fixing the issue.
re: bitten by: http://lkml.org/lkml/2007/1/4/232
A report from OSCON 2007 Posted Aug 9, 2007 3:37 UTC (Thu) by jdub (subscriber, #27) [Link] I updated an existing Ubuntu bug with information from this thread, and some strace logs of 'cat /dev/null' with and without a locale-archive.
https://bugs.launchpad.net/bugs/55906
Thanks for raising this!
A report from OSCON 2007 Posted Aug 4, 2007 13:40 UTC (Sat) by csamuel (✭ supporter ✭, #2624) [Link] You can create the archive yourself by doing (as root):
A report from OSCON 2007 Posted Aug 15, 2007 7:04 UTC (Wed) by set (guest, #4788) [Link] hmmm... strace of cat for me produces less than 25 lines of outputand only 3 files opened, none of them locale. Probably because I run gentoo and hate on nls....
|
|||||||||||||