64-bit? [LWN.net]

64-bit?

Posted Apr 17, 2004 3:40 UTC (Sat) by RichardRudell (subscriber, #7747)
Parent article: The Grumpy Editor goes 64-bit

Wow. Lots of feedback on what should now be an old topic (-:

Redhat Enterprise Linux 3 was my choice and it works flawlessly. Pricey, yes, but all of the features needed for compute-intensive applications which need more than 32-bits. Robust, solid, and supported by the third-party software I need.

I notice that LWN has never done a piece on how RHEL3, which is a 2.4 kernel, has all of the most useful features of the 2.6 kernel backported to a solid and stable platform. If you need me to list what they are, perhaps a detailed piece analyzing the RHEL3 kernel is more in order ...

[I have no relationship to RedHat other than being a satisfied customer].

BTW: it's not 64-bit addressing. Hypertransport has a physical limitation of 40-bits. Yeah, I couldn't believe it either until I read the specs. 1 TB is a nice improvement over 4 GB, but it's not the 2^64 talked about in an earlier post. Forget virtual address space (currently 48-bits on Opteron), I'm talking about physical address space. There is a hardwired limit which is going to hurt sooner than later.

Worse, because "int" is still 32-bit, we should really call this the "36-bit" machine, i.e., many applications will break because they count the "number of things" using an "int"; if the "thing" is 32-bytes, the "int" overflows at 2^36 bytes. That's only 16 GB which is too near current reality for my liking.

Also, 64-bit mode beats 32-bit mode on Opteron only for "small memory" applications. "Big memory" applications, i.e., why you want 64-bits in the first place, still lose by 5-20%. See, the problem is pointers being 64-bits reduces the effective size of the L1/L2 cache and cause increased traffic to DDR. Typical applications show a 1.6x increase in memory space (each struct being a mix of pointers (64-bit) and ints (32-bit). The extra 8 registers and the new ABI (pass arguments in registers, no frame pointer) help, but you just can't overcome a 60% increase in memory traffic when you really need to use all that memory.

Sincerely,
Richard Rudell

(Log in to post comments)

64-bit?

Posted Apr 17, 2004 4:02 UTC (Sat) by RichardRudell (subscriber, #7747) [Link]

Oops. 2^36 is 64 GB. But you can tweak the numbers however you please, make the "thing" 128-bytes instead of 32, and the result is the same. Still too close for comfort.

I would like to add that my field is Computer-Aided Design for Integrated Circuit Design which has been fighting the 4 GB memory barrier for the last 5 years. First came 3 GB on Linux, then 3.5 GB, then 4 GB with the "HUGEMEM" Linux kernel patches (flush TLB on every syscall). The only other solutions have been Sparc-64 and IA64, which have been, hmm, less than price/performance competitive.

64-bit?

Posted Apr 18, 2004 5:39 UTC (Sun) by evgeny (guest, #774) [Link]

> Worse, because "int" is still 32-bit, we should really call this the
> "36-bit" machine, i.e., many applications will break because they count
> the "number of things" using an "int"; if the "thing" is 32-bytes, the
> "int" overflows at 2^36 bytes. That's only 16 GB which is too near current
> reality for my liking.

I didn't get it at all. Nor the 32->36 step neither (more importantly) the connection between sizeof(int) and addressable memory - unless you're talking about braindamaged apps casting pointers to ints of some kind.

> Also, 64-bit mode beats 32-bit mode on Opteron only for "small memory"
> applications. "Big memory" applications, i.e., why you want 64-bits in
> the first place, still lose by 5-20%. See, the problem is pointers being
> 64-bits reduces the effective size of the L1/L2 cache and cause increased
> traffic to DDR.

By "big memory" you probably assume not big enough to hit the 2/3/4GB (depending on kernel patch) native limit of x86. Anything above should clearly favor 64bit.

> Typical applications show a 1.6x increase in memory space (each struct
> being a mix of pointers (64-bit) and ints (32-bit). The extra 8 registers
> and the new ABI (pass arguments in registers, no frame pointer) help, but
> you just can't overcome a 60% increase in memory traffic when you really
> need to use all that memory.

Uh? A typical app spends half of its allocation in pointers?! There are such apps certainly, but they are 1) not memory-hungry AND 2) don't need a high CPU<->RAM bandwidth. Anything really memory-intensive should use a contigous memory areas (arrays or some custom mappings) in the first place, in which case the overhead is negligible. The only exception is probably memory debuggers (BTW, one thing I miss badly for x86_64 is valgrind).

64-bit?

Posted Apr 18, 2004 15:21 UTC (Sun) by RichardRudell (subscriber, #7747) [Link]

Sorry, I should have been more clear.

I'm not talking about "brain-damaged" apps which mix int and pointer. I'm talking about a different type of brain-damage -- using an "int" to count, for example, the number of transistors on a chip. This "int" overflows at 2^31 meaning that, regardless of the fact that I can now address 1 TB, I'm going to be hit by a different type of problem before I reach that limit.

This is a different problem from the "make applications 64-bit clean". It is to "upgrade" applications so they use 64-bit ints when necessary.

Re: "uh"; I'm speaking from experience as a user of IC design tools which are (1) memory hungry (4GB-12GB for a single process is not atypical); (2) need high CPU/RAM bandwidth (all of that memory is aggresively used); and (3) show a 60% increase in virtual memory when running in 64-bit mode. I am not attempting to present these as "typical" applications; just that they are important to me ...

I anxiously await Intel's entry into the field with IA32e. The ensuing competition will be great for everyone, both in increasing performance and lowering price.

64-bit?

Posted Apr 27, 2004 21:05 UTC (Tue) by slamb (guest, #1070) [Link]

That's not so bad. They left room in the instructions to address the full 64 bits. They'll be able to replace the hardware with something that supports the full 64 without rewriting the compilers and recompiling all the software.

IIRC, the first "32-bit" Intel machines couldn't actually address 4GB (or even 2GB) of memory, either. But it didn't really cause a problem; by the time people actually wanted to do that, they'd bought new processors. I expect this will be the same situation.