LWN: Comments on "Quotes of the week"

Quotes of the week

BenHutchings — Wed, 20 Jun 2012 23:34:04 +0000

You can even choose it at installation time, in expert mode.

Quotes of the week

drago01 — Sun, 17 Jun 2012 08:33:51 +0000

> Proprietary programs that are only available in i386 versions.

That does not mean that you need a i386 userspace for them. Multilib works just fine in that case.

> That's not my experience. Quite often, proper distribution-supported packages (i.e., not roll-your-own installations) for i386 shared libraries in x86_64 base installations are missing. I.e., one has a 32bit program, and no vendor-supported package is available with its needed libraries that can be installed.

Not sure what distribution you are referring to but at least on fedora every lib shipped does have a installable i686 counterpart in the repos. So for every libfoo.x86_64 there is libfoo.i686 ...

> And no luck at all on RHEL.

What does that mean? Just install libfoo.i686 (you can do that for every library shipped assuming they are doing the same as fedora which I have no reason to doubt).

Quotes of the week

drago01 — Sun, 17 Jun 2012 08:30:20 +0000

No he means *only* i386 userspace binaries and libs.
I am saying you can just have an x86_64 userspace and still being able to install i686 libs where needed to run the i386/i386 binaries.

Quotes of the week

drago01 — Sun, 17 Jun 2012 08:29:08 +0000

> On an x86_64 system, you can install i686 packages. But then you have already base x86_64 userland packages installed, especially those with shared libs. Additional package installations will cause conflicts with those base-installed packages.

That's just not true.
You can install those just fine ... they won't conflict because the shared libraries go into different directories (lib vs lib64). Which is the whole point of multilib. Binaries in both packages won't conflict either because rpm has a concept of "coloring" which means if you install foo.i686 and foo.x86_64 and both ship /usr/bin/foo it *wont* conflict. /usr/bin/foo will just be the x86_64 one.

Name examples of what you mean. You might just have hit some kind of packing bug for a specific library but they are designed to not conflict and it works fine for me here.

32 bit kernel on 64 bit CPU

giraffedata — Sat, 16 Jun 2012 23:53:51 +0000

So you cripple your other two machines to satisfy one?

Sure, that's reasonable. It's like having a 10G disk drive and a 15G disk drive and not using 5G of the larger drive so you can use them as a mirrored pair.

I too maintain a single kernel binary that I run on lots of systems. Many of them even access the same actual file, over the network. Some are capable of running 32 or 64 bit and others only 32 bit. The extra speed I'd get out of the 64 bit machines, in my case, would not justify the extra maintenance effort of having two kernels.

And note that there's no hypocrisy here. I never intended to have full x86_64 speed when I acquired the x86_64 machines; That feature is just an unwanted by-product.

Quotes of the week

faramir — Fri, 15 Jun 2012 23:18:05 +0000

If you are running an x86 user space...

1. syscalls probably won't be faster.
2. there certainly won't be more registers for user space apps.
3. multiprecision arithmetic is rare no matter what.

Even with those issues, I would still be happy to stop using a PAE kernel if I could retain my x86 user space. Unfortunately, I don't run Debian and it appears that is the only distribution that supports this out of the box.

Quotes of the week

faramir — Fri, 15 Jun 2012 23:06:16 +0000

You are talking (probably) about execution time/memory bandwidth. He is talking about total memory footprint. Which matters more is going to depend on the context.

Quotes of the week

dlang — Fri, 15 Jun 2012 19:53:50 +0000

the problem isn't in the mechanism, it's in how far it's been implemented.

apparently libaspell isn't multiarch (or at least it's not being installed appropriately if it is), so when you install the 32 bit version it uninstalls the 64 bit version that was already there.

blocking multiarch doesn't solve this problem, the same problem existed before multiarch, it's just that everyone knew not to try to do that (you only used 'special' 'prepared' libraries that were part of the ia32 package)

multiarch reduces the problem, but by making it work more of the time, you do run into the cases where it doesn't work.

If you try to block it until it's perfect, you will block it forever, because whenever you actually use something new, you find the corner cases.

multiarch works perfectly if you stick to doing only the things that you could do without it.

Quotes of the week

rgmoore — Fri, 15 Jun 2012 19:45:20 +0000

That sounds like exactly the kind of problem that convinced Guillem Jover to block Ubuntu's version of multiarch and not accept it into Debian until all the kinks were worked out. Maybe he was onto something.

Quotes of the week

jengelh — Fri, 15 Jun 2012 10:29:07 +0000

>Also, i386 does have the added benefit that these tools have smaller pointers and do not take up as much memory to begin with.

i386 is hardly a benefit - it has much less registers and you don't have the guaranteed SSE2 availability.

The cake seems to reside near x32 [64-bit instruction set, but with ILP32] instead. (But hey, that's what people already do (mostly - save for a gcc issue) on sparc64 and ppc64 for years, have the x86 folks been sleeping?)

Quotes of the week

fb — Fri, 15 Jun 2012 08:33:15 +0000

> causing the entire KDE desktop environment to be removed,

Sounds like a feature to me!
*ducks*

Quotes of the week

rganesan — Fri, 15 Jun 2012 05:40:30 +0000

Debian does. I am really pissed debian derivatives like Ubuntu/Mint etc don't.

lftp ftp.debian.org:/debian/pool/main/l/linux-2.6> ls linux-image*amd64*i386.deb 
-rw-rw-r--    1 1176     1176     28704056 Nov 03  2011 linux-image-2.6.32-5-amd64_2.6.32-39_i386.deb
-rw-rw-r--    1 1176     1176     28769240 May 06 09:17 linux-image-2.6.32-5-amd64_2.6.32-45_i386.deb
-rw-rw-r--    1 1176     1176     23154160 Jun 01 22:48 linux-image-3.2.0-2-amd64_3.2.19-1_i386.deb
-rw-rw-r--    1 1176     1176     23208626 May 18 17:32 linux-image-3.3.0-trunk-amd64_3.3.6-1~experimental.1_i386.deb
-rw-rw-r--    1 1176     1176     23391324 Jun 06 18:18 linux-image-3.4-trunk-amd64_3.4.1-1~experimental.1_i386.deb

Quotes of the week

jschrod — Thu, 14 Jun 2012 23:50:59 +0000

On an x86_64 system, you can install i686 packages. But then you have already base x86_64 userland packages installed, especially those with shared libs. Additional package installations will cause conflicts with those base-installed packages.

On an i686 installation, no x86_64 kernel package is available, like it is in Debian. "yum list 'kernel*'" tells so.

Of course, you can always create your own completely-self-made-up package list and use Anaconda or AutoYast or such. But this is what I called "roll-your-own installation" above, then you create a spin. It is not an installation mode that is supported by a distribution out-of-the box -- which is what I was asking for.

I don't deny that it ain't possible; after all, one can also always roll one's own kernel and install that. I hinted to the fact that such an installation mode is not widespread because it is *not a mode commonly offered by distributions*.

Quotes of the week

mathstuf — Thu, 14 Jun 2012 22:27:29 +0000

> And no luck at all on RHEL.

Hmm? I have i686 and x86_64 packages installed at the same time on Fedora and I remember RHEL 5 being the same. I haven't played with RHEL 6 enough to remember.

Quotes of the week

ballombe — Thu, 14 Jun 2012 21:44:07 +0000

and much faster multiprecision arithmetic too!

Quotes of the week

dlang — Thu, 14 Jun 2012 19:07:55 +0000

> It has worked flawlessly ever since.

you are a little lucky.

there have been a handful of low-level interfaces that have had problems. I remember tripping over a iptables bug, and there was the recent automounter related issue.

in general it works, but it has not worked perfectly all along.

Quotes of the week

dlang — Thu, 14 Jun 2012 19:06:18 +0000

keep in mind that multi-arch is not perfect.

I've run into problems with some libraries that don't work properly with multiarch (specifically installing the 32 bit aspell library removed the default library on ubuntu 11.04, causing the entire KDE desktop environment to be removed, not fun. Yes I did submit a ticket, no action in two months.)

Quotes of the week

PaXTeam — Thu, 14 Jun 2012 18:11:36 +0000

sure, an amd64 kernel manages more RAM in a better way but you had a (rhetorical?) question as to why would someone still stick to i386, i just gave you one possible reason (better kernel self-defense), that's all :).

Quotes of the week

nevets — Thu, 14 Jun 2012 16:56:28 +0000

Which gives more reason to add that message.

It will give the major distros more incentive to make their i386 userspace include a x86_64 kernel. Otherwise the kernel will keep calling their users idiots ;-)

Quotes of the week

jschrod — Thu, 14 Jun 2012 16:45:26 +0000

You are right. I didn't know about the linux-image-amd64 package.
So, Debian has the ability to install a 64bit kernel on a 32bit system.

According to http://packages.ubuntu.com/search?suite=precise&arch=..., Ubuntu has no such package.

I know that RHEL 6 and its derivates have no such package, and neither does SLE nor openSUSE. I assume that Fedora has none either.

I'd say, even with this correction, my original statement stands: If one looks at the major distributions, one sees the reason why not more people use such an installation.

Quotes of the week

nevets — Thu, 14 Jun 2012 16:38:00 +0000

> yes (if 'happen' = 'are exploitable'). UDEREF on i386 uses segmentation to prevent exactly this class of bugs from becoming exploitable beyond DoS.

Which goes back to my original point. Why run vanilla Linux i386 on x86_64. It's better to just run x86_64 kernel with a i386 userspace, then an i386 kernel if you have extended RAM.

Quotes of the week

PaXTeam — Thu, 14 Jun 2012 16:23:48 +0000

> Modifying config options does modify the kernel. Do you need to recompile?

i think for most people (esp. kernel devs) modify = patch, and not 'reconfigure'. but for all i care, i got the point across :).

> but I was thinking that userspace had the added protection of segments to protect against accessing the kernel.

even if the default segments were set up this way, there's modify_ldt and TLS to get around them.

> Are you saying that since Linux uses a flat segment space that the bugs happen for both i386 and x86_64?

yes (if 'happen' = 'are exploitable'). UDEREF on i386 uses segmentation to prevent exactly this class of bugs from becoming exploitable beyond DoS.

Quotes of the week

nevets — Thu, 14 Jun 2012 16:22:02 +0000

I was waiting for someone to reply about ulimit. I'm just too lazy to implement a limit on my x86_64 boxes, that I just wait till the system slows down and restart instead.

Also, i386 does have the added benefit that these tools have smaller pointers and do not take up as much memory to begin with.

Quotes of the week

nevets — Thu, 14 Jun 2012 16:11:34 +0000

> it's an existing kernel config option, nothing needs to be modified.

Modifying config options does modify the kernel. Do you need to recompile?

> pre 2.0 (iirc) task switching? set_fs()? TLS? ;)

I started working with 2.0, so I should never had said 'never' ;-)

Yeah, I knew about TLS, that's why I said 'much of segmentation', instead of saying 'any of segmentation'.

> nope, (vanilla) linux uses flat segments, there's no separation at the segment level.

Ah sorry, was thinking that it did. It's been a while since I've actually worked with segments. I know the kernel was not limited against accessing userspace, but I was thinking that userspace had the added protection of segments to protect against accessing the kernel.

> when the declared policy from high up is to actively suppress them is... too funny if it wasn't so sad at the same time

Are you saying that since Linux uses a flat segment space that the bugs happen for both i386 and x86_64?

Quotes of the week

PaXTeam — Thu, 14 Jun 2012 15:39:43 +0000

> If you're going to modify the kernel, might as well just install x86_64 instead.

it's an existing kernel config option, nothing needs to be modified.

> Linux never bother with much of the segmentation crap that i386 uses to begin with.

pre 2.0 (iirc) task switching? set_fs()? TLS? ;)

> Sure, it separates userspace from kernel space,

nope, (vanilla) linux uses flat segments, there's no separation at the segment level.

> but it does nothing to protect one task from another. Page tables are used for that purpose.

true but what's that matter here? ;)

> Show me one security errata that was the result of removing segmentation from x86. And I mean for Linux.

every single kernel-dereferences-unintended-userland-pointer bug (something that UDEREF in PaX protects against if you want to see how it's done properly). and asking for actual security errata when the declared policy from high up is to actively suppress them is... too funny if it wasn't so sad at the same time :P. in any case, http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=linux+ker... should get you started.

Quotes of the week

nybble41 — Thu, 14 Jun 2012 15:39:40 +0000

> I like it because it keeps evolution, firefox and chrome all limited to 3Gigs of RAM.

You can limit the virtual address space of specific programs or sessions with ulimit -v, while still benefiting from x86_64 everywhere else. Even better, with ulimit you can choose _any_ limit, not just 3 GiB.

Quotes of the week

nevets — Thu, 14 Jun 2012 15:16:25 +0000

So you cripple your other two machines to satisfy one?

I would suggest setting up distcc or icecream (you have 3 machines) and build a x86_64 kernel as well for the x86_64 machines. It makes the build go much faster. And you wont even need to do anything special as the userspace is all i386 anyway. But if the boxes are not all the same distro, you can use my guide to build your own gcc's for distcc kernel compiling:

http://rostedt.homelinux.com/distcc/

It's a little out of date, but should still be useful to help you set up.

Quotes of the week

nevets — Thu, 14 Jun 2012 14:19:23 +0000

> on i386 the userland/kernel address space split is configurable

If you're going to modify the kernel, might as well just install x86_64 instead.

> an area where amd64 is worse is security due to the lack of segmentation, the baby went with the bathwater unfortunately

Bah! Linux never bother with much of the segmentation crap that i386 uses to begin with. Sure, it separates userspace from kernel space, but it does nothing to protect one task from another. Page tables are used for that purpose.

Show me one security errata that was the result of removing segmentation from x86. And I mean for Linux.

Quotes of the week

PaXTeam — Thu, 14 Jun 2012 13:14:23 +0000

on i386 the userland/kernel address space split is configurable, so if one can live with less virtual address space for userland, the kernel can be more efficient with 2-3GB RAM as well, but beyond that HIGHMEM is inevitable. an area where amd64 is worse is security due to the lack of segmentation, the baby went with the bathwater unfortunately...

Quotes of the week

nevets — Thu, 14 Jun 2012 13:14:11 +0000

You missed my point. I never said that the system should be a x86_64 base, only the kernel. And *that* works just fine on i386 installations.

I've been running an i386 userspace system on an x86_64 kernel since 2005. Never had any issues with it.

Debian today now even supplies a 64bit kernel with its i386 distribution. Although, I think you still need to install the i386 kernel first, and then upgrade to the 64bit kernel after install. The package is:

linux-image-amd64

Quotes of the week

k3ninho — Thu, 14 Jun 2012 13:08:15 +0000

And you get twice the CPU register count.

Quotes of the week

nevets — Thu, 14 Jun 2012 13:02:41 +0000

> This page table must always be present and takes up that 885 megs

I meant to say "takes up part of that 885 megs", the way I wrote it, it looks like it takes it all up. But with a big enough amount of RAM, it could.

Quotes of the week

nevets — Thu, 14 Jun 2012 12:57:02 +0000

> And the reason for wanting a i386 userspace is ... ?

I like it because it keeps evolution, firefox and chrome all limited to 3Gigs of RAM. I have two boxes I use (one is very work specific, the other is personal, although I use it for most my upstream work as well). My work box is a true x86_64 install base. My personal box is i386 userspace running a x86_64 kernel. Both have 8Gigs of RAM.

My i386 box never has issues with fire-fox, chrome or evolution taking up all my memory. When they get too big, the worse that happens is that they crash and I just restart them.

On my x86_64 box, these apps take over all RAM and it's not till I notice the slow down before I restart them.

Both these boxes run 24/7 so a daily shutdown is not there to clean things up.

The reason:

przemoc — Thu, 14 Jun 2012 12:45:59 +0000

You're right. I tried <pre> at the beginning, but there is this ridiculous requirement of dealing by hand with & and other stuff in HTML that I went with other tags before carefully reading the note. There is also an old <xmp> tag, treating everything in it as-is, but it's not universally supported.

Having Markdown as a third option for format would be really nice in such cases. The only problem is indentation that is tiring unless you have a vi/xclip+sed/etc. close. The solution is a fenced code blocks (~~~) extension, available in many implementations.

But it's OT here, sorry.

Quotes of the week

nevets — Thu, 14 Jun 2012 12:45:35 +0000

In 2005 when AMD came out with an affordable x86_64 processor, I bought one and built my main workstation with it. At the time, there was no x86_64 distros that I knew of, so I installed Debian testing i386 and did a cross compile to add a x86_64 kernel on it. It has worked flawlessly ever since.

I still have that system (I'm typing this on it), even though I've changed everything from the motherboard, RAM, CPU, Hard disks, and power-supply. But I've never reinstalled the system. (If you are wondering about how I did that and changed the HDs, I have a RAID 1 setup, and changing HDs is just a matter of swapping one at a time and doing a resync).

Today, I no longer do a cross compile to build my kernels for it. It handles building a x86_64 kernel just fine, even though my userspace is still i386. And yes, it comes with a x86_64 kernel:

Package: linux-image-3.2.0-2-amd64

I believe the 'Multi-arch' is to run both x86_64 and x86_32 userspace. But has nothing to do with the kernel (although you need an x86_64 kernel to run x86_64 userspace apps). But if you only have x86_32 userspace, then all you need is that linux-image and you're good to go. Debian has supported this for a while.

Quotes of the week

nevets — Thu, 14 Jun 2012 12:32:13 +0000

The kernel only has access to 885 megs at a time. Userspace has 3G address range and 1G for the kernel. But the kernel uses over 100 megs for things like virtual memory (where modules are loaded) and accessing the hardware (iomem mapping), and even mapping pages for the rest of RAM.

Also the kernel needs to create a page descriptor for every page of RAM. This page table must always be present and takes up that 885 megs. When you have a lot of RAM, this table grows, but the total access does not. It's like raising taxes on people on social security or other fixed income.

All kernel related operations take place in that 885 megs. Even if you have 2Gigs or 4Gigs of RAM, the kernel does not use any more. If the kernel needs to do any operations on the memory outside this range, (besides copy_from_user) it must map that page into its own address space (where the extra 100 megs are used for). This slows down the kernel and the system in general.

For systems with 2, 4, or possibly even 8Gigs, the kernel is still remarkably efficient in doing this, but it still has its cost. When you go above 8gigs, now you are just asking for trouble. I may have been a bit extreme when saying anything over 1Gig should produce that message. But it definitely should say it for over 8G (and possible just over 4).

Read Documentation/vm/highmem.txt for more information.

The reason:

cortana — Thu, 14 Jun 2012 12:25:53 +0000

AFAIK packages in non-free are not auto-built by default. So a lack of an officially-build amd64 instance of this package probably just means that the maintainer didn't go through the steps to request that it by autobuilt.

On the other hand, now that multiarch is present in testing/unstable:

$ dpkg-architecture -qDEB_HOST_ARCH
amd64

# apt-get install lgrind
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Recommended packages:
  tetex-bin:i386
The following NEW packages will be installed:
  lgrind:i386
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 81.8 kB of archives.
After this operation, 250 kB of additional disk space will be used.
Get:1 http://ftp.uk.debian.org/debian/ wheezy/non-free lgrind i386 3.67-3 [81.8 kB]
Fetched 81.8 kB in 0s (303 kB/s)
Selecting previously unselected package lgrind.
(Reading database ... 447986 files and directories currently installed.)
Unpacking lgrind (from .../lgrind_3.67-3_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for doc-base ...
Processing 1 added doc-base file...
Registering documents with scrollkeeper...
Setting up lgrind (3.67-3) ...
texhash: Updating /usr/local/share/texmf/ls-R... 
texhash: Updating /var/lib/texmf/ls-R-TEXLIVEMAIN... 
texhash: Updating /var/lib/texmf/ls-R-TEXLIVEDIST... 
texhash: Updating /var/lib/texmf/ls-R-TEXMFMAIN... 
texhash: Updating /var/lib/texmf/ls-R... 
texhash: Done.
W: Operation was interrupted before it could finish

$ ldd /usr/bin/lgrind
	linux-gate.so.1 =>  (0xf77d4000)
	libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf764d000)
	/lib/ld-linux.so.2 (0xf77d5000)

Not too impressive since I already had libc6:i386 installed, and lgrind has no other dependencies, but you can see how multiarch nicely makes obsolete the old biarch problems that jschrod lists in a post above.

The reason:

nix — Thu, 14 Jun 2012 12:19:58 +0000

You want <pre>, I think.

Quotes of the week

richmoore — Thu, 14 Jun 2012 12:12:17 +0000

IIRC the syscall convention for x86-64 is much faster.

The reason:

przemoc — Thu, 14 Jun 2012 10:18:23 +0000

This is strange. Quick check (possibly inaccurate) shows that it's not a x86-specific package.

$ cat /etc/debian_version
wheezy/sid
$ uname -m
x86_64
$ apt-get source lgrind
...
$ ( cd lgrind-3.67/ && dpkg-buildpackage -b -us -uc )
...
$ dpkg -I lgrind_3.67-3_amd64.deb
...
Package: lgrind
Version: 3.67-3
Architecture: amd64
...
$ sudo dpkg -i lgrind_3.67-3_amd64.deb
...
$ lgrind
lgrind -- general purpose "pretty printer" for use with LaTeX.
usage: lgrind [options] <name> ...
...

BTW what was the tag for using fixed-size font in HTML comments at LWN.net? <code>, <samp> and <tt> seem unsupported...