And they are happy with 32bit Flash too [LWN.net]

And they are happy with 32bit Flash too

Posted Oct 18, 2008 7:35 UTC (Sat) by k8to (subscriber, #15413) [Link]

You overstate your case in several places.

x86_64 is pretty space efficient. The increase in binary size is small enough that programs compiled this way run faster despite the memory hit.

x86_64 binaries do have benefits vs x86 binaries on an x86_64 platform. Each duplicated library is wasted resources, and these can add up to a lot depending upon how many they are (try loading all the gnomelibs for both arches, it's not tiny).

x86_64 is also faster than x86, although not so much as to make it super obvious.

So there's basically no upside to x86 binaries.

But the problem isn't bitness, the problem is it's buggy and crashy and crappy. The lack of a 64bit compile is just a symptom of the lousy state of their codebase.

And they are happy with 32bit Flash too

Posted Oct 18, 2008 8:47 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]

one additional readon for 64 bit, there are twice as many registers available to a 64 bit program than for a 32 bit program (on x86)

x86_64 is also faster ? Not so fast...

Posted Oct 18, 2008 10:18 UTC (Sat) by khim (subscriber, #9252) [Link]

x86_64 is also faster than x86, although not so much as to make it super obvious.

If you actually check the facts you'll find that x86-64 is only faster for fp, for integer calculations results are quite unclear - from 30% gain to 40% (yes, 40%) loss. Average win of 1.6% to 3.8% does not look very compelling.

So there's basically no upside to x86 binaries.

Oh, there are is: huge population of 32bit systems. That's HUGE upside. If you want to support just one binary (and that's significantly easier then to support two) it's huge win. And as there are few downsides...

Each duplicated library is wasted resources, and these can add up to a lot depending upon how many they are (try loading all the gnomelibs for both arches, it's not tiny).

If you compare sizes of all such libraries with 8GB of RAM (and there are no need to use 64bit binaries if you have less - kernel may be, userspace - no)... that's still tiny potatoes. There are significant pressure upon CPU cache, true, but as shown above x86-64 can be problem by itself there, so no clear win from "pure 64-bit system"... "Pure 64-bit system" is more about bragging rights than about real benefits...

x86_64 is also faster ? Not so fast...

Posted Oct 18, 2008 13:08 UTC (Sat) by tialaramex (subscriber, #21167) [Link]

I feel you've badly oversimplified about userspace. The fact is that what we really ran out of wasn't RAM but address space. And we ran out of address space years ago.

The address space consumed by applications includes allocated RAM, file backed memory maps, device backed mappings (e.g. from graphics cards for visualisation tools) as well as all the code and data associated with any libraries linked in, and any libraries those libraries link in and so on.

When the absolute hard limit of address space is 4GB (2GB on Windows and many other operating systems) it's very easy to reach this limit without needing 4GB of DIMMs, let alone 8GB. That was the point when 64-bit userspace development should have taken off.

But when it became apparent that nobody except Intel was interested in cheap low-to-mid range hardware, and that Intel had no clue how to build a 64-bit CPU, a lot of us were forced to hack around these problems, to write programs that would use huge complicated wrappers around pread() and pwrite() rather than just memory map a random access file, to allocate and de-allocate buffers only when needed for fear that we might ask for too much RAM at once if we used a faster algorithm that avoided thrashing the allocator, and so on.

And the result was software which sucks disproportionately to today's hardware capabilities. Nobody should be encouraging the development of more software like that now that AMD have shown Intel that they could just glue wider registers to their existing design and have a 64-bit CPU that performed well and was affordable (Yes, I understand this is a gross over-simplification).

Arguments based on SPEC numbers are a good antidote to the "64-bit will make everything twice as fast" weenies, but in practical everyday terms they don't mean a lot in most applications. They deliberately don't let you go through the code and say "Well, this approach is hopeless with wider pointers, let's do something different here instead". If my application runs 40% slower in 64-bit mode I'm going to go find the problem (I declared that huge array as 'long' integer? Whoops, that was stupid) and fix it.

My day job is developing and maintaining a 64-bit program, admittedly ours is a lot bigger than most, but we've been doing this for years, and its been reinforced over time that refusing to compromise for 32-bit was the right decision. The effect trickles down, and with new _laptops_ having 4GB of RAM these days it's crazy to be writing libraries (which is what the Flash player mostly is) that are explicitly written to be trapped in that 4GB address space.

Easy? I think not.

Posted Oct 18, 2008 17:49 UTC (Sat) by khim (subscriber, #9252) [Link]

But when it became apparent that nobody except Intel was interested in cheap low-to-mid range hardware, and that Intel had no clue how to build a 64-bit CPU.

Intel built one. The problem was: nobody wanted to spend time rewriting existing programs. Situation is more-or-less the same today: transtion to 64bit is underway, but there are no rush.

If my application runs 40% slower in 64-bit mode I'm going to go find the problem (I declared that huge array as 'long' integer? Whoops, that was stupid) and fix it.

What do you do if your program basically works with bunch of pointers (like Flash does)?

The effect trickles down, and with new _laptops_ having 4GB of RAM these days it's crazy to be writing libraries (which is what the Flash player mostly is) that are explicitly written to be trapped in that 4GB address space.

Flash was not written today and hardest part to port is not library of geometry objects but JIT. And JITs are hard to port (Chrome's JIT does not have support for 64bit too - despite being brand-new unlike Flash).

My day job is developing and maintaining a 64-bit program, admittedly ours is a lot bigger than most, but we've been doing this for years, and its been reinforced over time that refusing to compromise for 32-bit was the right decision.

May be it was right direction for you but why you are so sure it's right direction for everyone else? Most desktop programs don't need a lot of RAM and don't need a lot of address space - and will be Ok for years to come. Will the clock applet need terabytes of address space? Why? What for? Even browser: if you render just one page in a single process (like Chrome does) then there are basically no way you'll need even 100MiB of address space in the near future, let alone 4GiB...

Transition to "full 32-bit system" took 15 years (from 1986 when i386 was created to 2001 when last 16-bit based OS was declared obsoleted), I think transition to "full 64-bit systems" will be slower, not faster - because pressure is not so great. And Athlon64 was only introduced five years ago...

Easy? I think not.

Posted Oct 18, 2008 23:39 UTC (Sat) by tialaramex (subscriber, #21167) [Link]

"Itanic" indeed, that's exactly my point about "no clue". You correctly identify that there's a transition process, and the Itanium made no allowance for such a process, it seemed as though Intel hadn't even considered that the software then running on their x86 line would be expected to somehow transition onto this weird new architecture.

The transition process doesn't have to take decades, I think you've got that part wrong. The degree of abstraction from hardware is much greater this time, meaning you don't have all or nothing transitions. On OS X apparently even drivers can opt out of knowing about 64-bit for a while. With RAM being so cheap, most general purpose computing devices will quickly be 64-bit capable for practical reasons, and once the CPU is 64-bit you will have some applications that make use of that. You no longer have "32-bit only" as an option, so if you want the minimum number of platforms to support, "64-bit only" becomes the sensible choice. I think this creates a situation in which a "siphon effect" occurs, taking people from "Well, there is one 64-bit program I want to run" to a 64-bit only system in one upgrade cycle. I think the limiting factor will be ISVs, particularly on Windows, who loathe having to change /anything/ they were still shipping 16-bit installer programs that hadn't been maintained in almost a decade, right up until Microsoft banned such nonsense from their co-branding programme.

But then maybe I'm just counting very differently to you. I would not have said that transition to 32-bit ended in 2001. I'd say that happened in the mid 1990s, there was a lot of mess inside Windows 95, but new development of 16-bit software had ceased, PC video games (which you might think of as 16-bit because they ran under DOS for a few years still after Win 95) were actually using DOS extenders to escape 16-bit DOS and run 32-bit code. When the Pentium Pro flopped it was just barely mis-timed, its 16-bit performance was poor and the last few major applications with performance critical 16-bit code were just dying off. The Pentium II, also with fairly bad 16-bit performance, went into the market just fine.

Now, how about I show you how to burn lots of RAM on a web page without trying. Let's write a blog entry about a TV show we saw last night. Firstly, we inline the show itself, compressed of course, but still could easily be 2GB of data. Now, we add our commentary track, synchronised to the original, and then a series of images illustrating particular points which people can click on to zoom to their full HD resolution. Today it would be considered anti-social to embed a 2GB video on my hypothetical blog, but then ten years ago it would have been anti-social to put a 2MB image on there. In 2013 a 2GB video may just be the obvious way to illustrate what I'm talking about.

Of course very little of this strictly has to be in RAM at one moment, it's just that the difference between the user experience with the program desperately trying to keep all this fitted into 100MiB versus not worrying about it and letting the OS set cache policy is very noticeable. When you seek back a few seconds and a little beach ball appears and the disk goes crazy for 15 seconds, that makes you desperately want to avoid ever seeking again. Whereas if it happens instantly, you now have a useful feature. If scrolling to a different image causes the whole machine to have a heart attack while it goes off to load the image back into RAM, again you find yourself not wanting to scroll any more. This is the same as Linus' observation about git - faster isn't just a quantitative thing.

Now, as to the more general case, with the clock. You're right - in principle there are programs running on this laptop in front of me that could be 16-bit, or even 8-bit programs, They don't do very much, they don't manipulate huge files, or anything. But in practice it's much easier to just have one platform, a platform that's big enough for all the programs you run, and I argue that today and certainly tomorrow that platform is 64-bit.

Finally the pointers question. This is a design issue. There's a common Unix design where pointers are used as opaque handles, so an "image handle" or a "message handle" or whatever is just a pointer. Fast, but not necessarily space efficient. If your program typically has a few individual handles being passed around, that's a good pragmatic decision, but if you keep huge structures filled with handles then you may need to re-evaluate when porting to 64-bit, could the handles be indexes on an array or vector instead? The "indexing" style handles are more common in Windows, but each kind is found on both systems. With a 32-bit index instead of an address sized pointer you can have the same data throughput (and thus more or less performance) regardless of 64-bit vs 32-bit code. That's just one illustration, but hopefully it's helpful. Programs that absolutely /must have/ large numbers of real pointers or other address-sized things are fairly rare.

Easy? I think not.

Posted Oct 18, 2008 23:47 UTC (Sat) by tialaramex (subscriber, #21167) [Link]

Hmm, Wikipedia says the PII significantly improved the PPro's 16-bit performance. I guess I didn't notice because I personally wasn't running any 16-bit code. So, in that case I withdraw that part of my argument, perhaps Intel found 16-bit code performance made a difference to their customers as late as 1998.

Yup: pretty big difference

Posted Oct 19, 2008 9:09 UTC (Sun) by khim (subscriber, #9252) [Link]

16-bit code performance made a difference between win and loss in 1998...

Itanium lost for the same reason Pentium Pro lost

Posted Oct 19, 2008 9:08 UTC (Sun) by khim (subscriber, #9252) [Link]

"Itanic" indeed, that's exactly my point about "no clue". You correctly identify that there's a transition process, and the Itanium made no allowance for such a process, it seemed as though Intel hadn't even considered that the software then running on their x86 line would be expected to somehow transition onto this weird new architecture.

Itanium had support for such a process. But it's 32-bit performance was abysmal. Intel's engineers said that this time the transition process doesn't have to take decades and concentrated solely on 64-bit side - and of course they were wrong.

The degree of abstraction from hardware is much greater this time, meaning you don't have all or nothing transitions.

Yahoo! Yes, the abstraction is much greater, "transition pressure" is lower and that means transition will be slower, not faster.

With RAM being so cheap, most general purpose computing devices will quickly be 64-bit capable for practical reasons, and once the CPU is 64-bit you will have some applications that make use of that.

Yes, some applications. The kind of applications which need huge memory. Like Photoshop - which has 64-bit version now. For the whopping three days (CS4 was released October 15, 2008) - and only under Windows Vista (Windows XP and MacOS X are 32-bit only).

You no longer have "32-bit only" as an option, so if you want the minimum number of platforms to support, "64-bit only" becomes the sensible choice.

Why not? As I recall Photoshop is very resource-hungry application and MacOS is "platform of choice" for designers, yet Photoshop remains 32-bit application. It'll win from huge disk caches on 8GB-16GB systems, but it does not need to be 64-bit to get advantage from this memory... The pressure to make applications 64-bit, not an OS will only begin to mount 5-10 years down the road.

I think this creates a situation in which a "siphon effect" occurs, taking people from "Well, there is one 64-bit program I want to run" to a 64-bit only system in one upgrade cycle.

Sorry, but I can not see how it creates such illusions. We've switched from "32bit system with some 64bit programs" (this is what we used for last three years) to "64bit system with a lot of 32bit programs" on my work just few months ago (and there were voices to wait year or two) - and a lot of stuff is still 32bit-only with no plans to switch to 64bit any time soon. Sparc servers worked with such a model (64bit kernel, 32bit userspace) for years, why x86-64 must be different.

I would not have said that transition to 32-bit ended in 2001. I'd say that happened in the mid 1990s, there was a lot of mess inside Windows 95, but new development of 16-bit software had ceased, PC video games (which you might think of as 16-bit because they ran under DOS for a few years still after Win 95) were actually using DOS extenders to escape 16-bit DOS and run 32-bit code.

Windows95/98/ME had a lot of 16-bit code and programs for that OS often included 16bit helpers (not just installers). And games actually switched to pure 32bit only when 3D became widespread - that's few years after Quake. Till Windows 9X was alive it was to early to say "16bit era is history". Even if you count mid 1990s as the end of "16bit era" it's still 10 years after introduction of i386 architecture...

When the Pentium Pro flopped it was just barely mis-timed, its 16-bit performance was poor and the last few major applications with performance critical 16-bit code were just dying off. The Pentium II, also with fairly bad 16-bit performance, went into the market just fine.

Actually they fixed this performance loss and that saved Pentium II. Because a lot of programs depended on 16-bit - most of all Windows9X itself.

Of course very little of this strictly has to be in RAM at one moment.

This is not a problem: when you are talking about such a huge data sizes often it does not matter if it's addressable at once or not. You can always use mmap over /dev/zero to work with huge data regions like EMS back in a day. EMS worked just fine when random access is not needed (random access to video file is really random access to few thousand positions in that file).

You're right - in principle there are programs running on this laptop in front of me that could be 16-bit, or even 8-bit programs, They don't do very much, they don't manipulate huge files, or anything.

They are using GLibC - that's enough. Statically linked "Hello world" is over 128KiB today. More then 16-bit can handle without a lot of tricks. And when we are talking about code - that's different cattle of fish. EMS worked fine with a lot of data, but when program included 1-2MB of code... it was different game altogether. It was still possible but speed loss was 10-15%, sometimes more.

But in practice it's much easier to just have one platform, a platform that's big enough for all the programs you run, and I argue that today and certainly tomorrow that platform is 64-bit.

It's certainly easier if you start from scratch. But if you have 32bit legacy it's different.

Finally the pointers question. This is a design issue. There's a common Unix design where pointers are used as opaque handles, so an "image handle" or a "message handle" or whatever is just a pointer. Fast, but not necessarily space efficient. If your program typically has a few individual handles being passed around, that's a good pragmatic decision, but if you keep huge structures filled with handles then you may need to re-evaluate when porting to 64-bit, could the handles be indexes on an array or vector instead?

Ah. That's the answer I've waited for: so you suggest to switch from 32bit to 64bit, throw away EMS-style tricks used for video and add new, equally troublesome, tricks to work with data structures. Essentially zero gain for a lot of work (you still can not use a lot of data objects because your 32bit indexes will overflow). Is this wise decision? Not IMO.

That's what I've wanted to say: for a lot of programs switch to 64bit is just a replacement of one set of tricks with another. Today. When they'll start to work with billions of these handles (and that means many gibibytes of RAM, not 4GiB or 8GiB) it'll make sense to use 64bit despite speed loss (otherwise it'll just not work) and probably switch should happen somewhat earlier. But to do it today... it's too early.

Programs that absolutely /must have/ large numbers of real pointers or other address-sized things are fairly rare.

Rare but important. That's web-browsers (DOM and JavaScript are exactly such structures), Flash (the same: a lot of small objects and ActionScript) and OpenOffice.org (a lot of small objects for each spreadsheet or word document). Firefox is significantly slower in 64bit mode (OpenOffice.org is slow as hell in any more so don't count) and Adobe just decided that they don't want to spend money to produce inferior product...

The only place where 64-bit switch happened fast is FOSS because a lot of people just took it as a challenge: let's make everything pure 64-bit. Where people are counting money - process is much, much slower. Unless you are talking about few guys which needed and used gibibytes of RAM for many years (like EDA).

Back to Adobe's excuse making

Posted Oct 19, 2008 14:00 UTC (Sun) by tialaramex (subscriber, #21167) [Link]

I knew you'd use Photoshop as an example, does that tell you anything? If you Google around for people arguing that 32-bit userspace is fine, and no-one needs to worry about running out of address space and so on, the first things you run into are Adobe Photoshop developer blogs.

They've been making this argument ever since 64-bit Windows became available to ordinary mortals. You can find plenty of made up numbers about how it would actually be slower for most purposes, and certainly wouldn't get any faster until you were loading absolutely huge images with vast numbers of layers...

... and then Adobe announced the 64-bit Photoshop and suddenly the story changed overnight. 64-bit was going to be faster for even moderately sized images, a lot of existing features would see a speed-up, and so on.

This is called /spin/ - Adobe used to tell Photoshop users they didn't want unlimited Undo History, it was a crutch for people who didn't use their image editing software the right way, and then one day Photoshop implemented it and suddenly Adobe couldn't say enough about how great it was. They're trying to avoid the Obsourne effect, I get that, but the net result is that they spent a lot of time misleading customers. Why believe anything they say?

Adobe ended up in the mess with Photoshop, partly, because of a lack of abstraction. Many Photoshop users have 3rd party plug-ins. The plug-ins run "in memory" so if you upgrade to 64-bit you need some tremendous kludge or else you can't run any old plug-ins. A bit of design foresight would have been good right there.

You're mistaken about PC video games, it's nothing to do with 3D or Windows. It's all about DOS extenders. The DOS extender was supplied with your tool chain, effectively part of the run time. Despite DOS being 16-bit the "extender" would push the CPU into 32-bit flat mode, so all your actual development would be in 32-bit. DOS extenders were fairly rare when Doom had one, but by the time Quake arrived in 1996 everything in the PC section of your games store would be 32-bit code run by a DOS extender. Even simple point-and-click adventure games, which could have been coded for 16-bit DOS with overlays or similar tricks weren't bothering, 32-bit development meant less headaches for developers, which meant shorter time to market.

I'm not really sure what to make of your claims about DOM and so on. If web browsers actually need billions of DOM objects, URIs, images etc. then isn't it a surprise that they've ever worked on 32-bit machines? Or on the other hand, if as seems more likely they have a few million DOM objects, a few million URIs, and a few thousand images, then 32-bit handles are fine, and will be long after 32-bit memory isn't nearly enough. When I said most programs don't need lots of pointers, I meant that explicitly, and you don't seem to have understood that. /Choosing/ to use pointers everywhere you can is a design decision, and not automatically a good one.

Finally, I must say that although Firefox feels sluggish everywhere, it doesn't seem especially more so on my 64-bit machines. "Significance" is a tricky term, medical researchers speak of both "statistical significance" and "clinical significance" e.g. maybe you can show that your new drug has a statistically significant effect on lower back pain, but then when doctors look at the size of the effect it's equivalent to one person saying they feel a little better for every million people treated - well, that's not clinically significant, your drug is useless. Negative effects from increased native word size need to be characterised in terms of what users would actually notice. If the user won't notice, we shouldn't care.

Spin is spin, but...

Posted Oct 19, 2008 15:04 UTC (Sun) by khim (subscriber, #9252) [Link]

Adobe used to tell Photoshop users they didn't want unlimited Undo History, it was a crutch for people who didn't use their image editing software the right way, and then one day Photoshop implemented it and suddenly Adobe couldn't say enough about how great it was.

If that's really the most they can offer to 64bit users then previous position was correct: 64bit version offer only tiny improvements, not something truly spectacular.

You're mistaken about PC video games, it's nothing to do with 3D or Windows. It's all about DOS extenders. The DOS extender was supplied with your tool chain, effectively part of the run time. Despite DOS being 16-bit the "extender" would push the CPU into 32-bit flat mode, so all your actual development would be in 32-bit. DOS extenders were fairly rare when Doom had one, but by the time Quake arrived in 1996 everything in the PC section of your games store would be 32-bit code run by a DOS extender. Even simple point-and-click adventure games, which could have been coded for 16-bit DOS with overlays or similar tricks weren't bothering, 32-bit development meant less headaches for developers, which meant shorter time to market.

A lot of games were 16bit back then. I still remember that Larry7 (1996) and Enemy Nations (1997) worked fine with Windows 3.1 (I had no hardware for Windows95 back then) and thus were quite happy with 16-bit system. Only by the 1997 games started dropping 16bit mode support (and Quake was released June 22, 1996, don't forget).

I'm not really sure what to make of your claims about DOM and so on. If web browsers actually need billions of DOM objects, URIs, images etc. then isn't it a surprise that they've ever worked on 32-bit machines?

I wanted to say that browsers work with a lot of small objects and so it makes perfect sense to use 32bit mode. Only if/when web pages will start to include billions of said objects 64bit mode will make sense. Basically either they work with small pages of today (and then 32bit mode is the best choice) or they handle ubercomplex pages of tomorrow (and then all these schemes with indexes are useless). Transition phase is short - and it's some years in the future...

Or on the other hand, if as seems more likely they have a few million DOM objects, a few million URIs, and a few thousand images, then 32-bit handles are fine, and will be long after 32-bit memory isn't nearly enough.

Actually this time is short. Memory requirements double every year and half on average. That means if you have one pointer per ten words or useful data (not uncommon for many different types of applications) then time where your "32bit indexes but 64bit pointers" scheme works is five or six years! And it's just stupid to write program which can only be useful for five years - much better to just refuse to switch to 64bit till your data and indexes work in 32bit mode (that's fastest mode) - and then switch to 64mode. With some tricks (move images and video handling to separate process, etc) you can manage to work with 16GiB of RAM (may be even 32GiB or RAM) while in 32bit mode and then this "window of opportunity" shrinks even further (this tricks will slow down your program, of course, so at some point 64bit mode will be just faster with no strings attached - and THAT is the time when you need to switch).

/Choosing/ to use pointers everywhere you can is a design decision, and not automatically a good one.

Sorry, but I don't buy it. Indexes are kludges: at some point you'll have so many objects in your program that 32bit indexes will just not work (Windows 98/ME was hit by this problem pretty hard: GDI objects used indexes is 16bit tables and so it was not uncommon so see system with many mibirytes of RAM crash or behave erratically because it had no room to create yet another window or yet another brush). Wrappers around pread() and pwrite() are kludges too. But there are a difference: wrappers around pread() and pwrite() allow you to use old libraries and programs for a while in the "brave new word or multigibibyte RAM" while indexes make life easier for a while but then require redesign of the whole system (something like Windows 9X => Windows XP transition). Thus IMNSHO indexes are more evil then wrappers.

Negative effects from increased native word size need to be characterised in terms of what users would actually notice. If the user won't notice, we shouldn't care.

Sorry, but it's the other way around: positive effects must be measured and if the user won't notice there are no need to switch to 64bit yet. Why? Easy: we already have working 32bit stuff and 64bit stuff must prove itself before it'll be accepted. XP 64bit was total flop and Vista 64bit is not so hot either - and that's prerequisites! While there are no 64bit OS there are no 64bit programs... You need at least ONE program which shows that 64bit OS is worthwhile and even after that switch from 32bit to 64bit will be on case-by-case basis...

That actually the policy around here: if application can work in 32bit mode - it must be used in 32bit mode in production (most our programs are memory-bound, not CPU-bound), but all tests must pass in 64bit mode too because switch is unavoidable. At some point.

Spin is spin, but...

Posted Oct 19, 2008 17:15 UTC (Sun) by tialaramex (subscriber, #21167) [Link]

If that's really the most they can offer to 64bit users

Ah, I guess that again you have the wrong end of the stick. This was not an example of something enabled by the 64-bit version, it's just an example of how Adobe changes its story between releases. Illustrating that you can't believe what Adobe's developers tell you about the value of features their product doesn't have (well, this ought to be obvious I'd have thought). GIMP developers have been known to say that various missing features aren't important in the past and I didn't believe them either.

I still remember that Larry7 (1996) and Enemy Nations (1997) worked fine with Windows 3.1

Yes...

and thus were quite happy with 16-bit system

... No.

I don't have Larry7 code to look at, but Enemy Nations is a Free Download these days. It's easy to verify that Enemy Nations isn't a Win16 game at all, it's a Win32 game carefully tailored to use the subset of Win32 that was available through Win32s, a 32-bit flat memory runtime for Windows 3.x that Microsoft made available to encourage transition. Microsoft's demo for that technology is actually rather famous -- Freecell. It's rather a shame they couldn't come up with anything similarly compelling for Win64... On the upside this means you can still play Enemy Nations on 64-bit Vista (in theory at least) unlike 16-bit games.

And well, OK, you don't buy my design advice about handles. I'm certainly not going to write a whole web browser to try to prove you wrong. I will say that I think if you look at how memory usage grows you'll see that there's a difference between the trend for small things to get bigger, versus for there to be more small things. This hard disk is more than a thousand times bigger than the first one I owned, yet the number of /files/ on my current disk is only 400 000. Did I really only keep 400 files on my first hard disk? No, I don't think so. I expect the web browsers in five years time to have only 2-3 times more 'objects' but the total RAM used will increase more like ten fold as the objects get richer.

Spin is spin, but...

Posted Oct 19, 2008 20:59 UTC (Sun) by khim (subscriber, #9252) [Link]

It's easy to verify that Enemy Nations isn't a Win16 game at all, it's a Win32 game carefully tailored to use the subset of Win32 that was available through Win32s, a 32-bit flat memory runtime for Windows 3.x that Microsoft made available to encourage transition.

Ok, you are right. By the 1997 games dropped support for 16bit systems. I've tried to recall what I've played back then and found that most games were already 32bit or in few cases used Win32s... 1995 was the time when I've switched from 286 to 486 so I can not be too sure if games in 1996 and 1997 used Win32s or not. Before than they were 16bit as rule - only few notable exceptions (like Doom in 1993). To effectively cope with 2-4MiB of RAM was too much for 16bit mode. It means we can be pretty sure transition to 64bit systems will happen when typical RAM in desktop will pass 64GiB or 128GiB. That's few years yet... We don't have many server systems with 128GiB RAM today let alone desktops/laptops...

This hard disk is more than a thousand times bigger than the first one I owned, yet the number of /files/ on my current disk is only 400 000. Did I really only keep 400 files on my first hard disk?

I'm pretty sure you have many times more than 400'000 files on your disk if you'll count files stored in archives (like .ods or .tbz2) separately. It makes sense for a filesystem (disk is very slow and CPU is fast) but not for memory (CPU is not that fast... yet). And even that changes nothing: even if we'll assume that objects are growing in size two times per five-six years (average size of file 20 years ago was 50-60KiB, today it's more like 1MiB) it buys your technique one doubling time - 1.5-2 years. The facts is: it's the trick of the same style as pread/pwrite wrappers. Why exchange one trick for another if it buys you pretty limited time? When you can safely drop BOTH tricks it'll make sanse - but this time didn't come yet.

Stories like this one will not promore 64bit software at all. You need more then 4GB of RAM for the 64bit more to shine, perhaps as much as 32GiB or may be even 128GiB. Till then... a lot of software writers will not care, most users will not care either.

Spin is spin, but...

Posted Oct 26, 2008 23:16 UTC (Sun) by vonbrand (subscriber, #4458) [Link]

Sorry, but it's the other way around: positive effects must be measured and if the user won't notice there are no need to switch to 64bit yet. Why? Easy: we already have working 32bit stuff and 64bit stuff must prove itself before it'll be accepted. XP 64bit was total flop and Vista 64bit is not so hot either - and that's prerequisites! While there are no 64bit OS there are no 64bit programs... You need at least ONE program which shows that 64bit OS is worthwhile and even after that switch from 32bit to 64bit will be on case-by-case basis...

That MSFT still isn't able to ship a 64-bit OS while others had stuff running before the chips shipped is not exactly the architecture's fault...

Multi-gigabyte memories are commonplace today (heck, this laptop has 4GiB RAM), and anything with more than 4GiB can't be handled sanely with 32 bit architectures anyway.

SPARC64?

Posted Oct 26, 2008 22:52 UTC (Sun) by vonbrand (subscriber, #4458) [Link]

Sparc servers worked with such a model (64bit kernel, 32bit userspace) for years, why x86-64 must be different.

The situation is completely different between SPARC and SPARC64 than x86 and x86_64: There everything is twice as big, and only the need for more than 4GiB of memory space can force 64 bits. AMD learned from this mistake (and Itanium, and also from Pentium Pro's abysmal performance on then-current 16 bit code) and x86_64 programs are almost as dense as x86.

[Yes, I'm still sunning a SPARCstation Ultra 1]

x86_64 is also faster ? Not so fast...

Posted Oct 18, 2008 15:53 UTC (Sat) by gmaxwell (subscriber, #30048) [Link]

"Average win of 1.6% to 3.8% does not look very compelling."

Perhaps but the average win on typical GNU/Linux systems is greater than that: GCC does not do so well on register starved platforms. Find a comparison on GCC and you the difference is greater because x86 is really quite register starved.

Additionally, x86_64 makes mandatory a number of micro-architectural improvements which were previously optional. For example, consider SSE2 which is not universally available on x86. So generally distributed binaries must either offer alternative functions and detect at run time, or, more commonly, avoid using those instructions. x86_64 reset the baseline and you can just assume that many of these features will be there. This difference won't show up in benchmarks like the one you cited, since it used compiles directed at the particular CPUs in use, but it does show up in the real world.

There are also a couple of other advantages of x86_64 mode which were not mentioned up-thread: Much faster sysctls and improved security (no-execute pages are always available, bigger address space means that addresses can always contain null bytes, etc).

And I'm not sure why you say that ~3% by itself isn't all that compelling when the price difference between a 2.83GHz core2 quad and a 3.0GHz core 2 quad cpu is $230 US. Based on that simplistic analysis we might expect 3% to be worth $115 US while if you use a modern x86_64-painless distro and don't use non-free software the cost of 64bit mode is pretty close $0.

Whopping 5% for GCC...

Posted Oct 18, 2008 18:18 UTC (Sat) by khim (subscriber, #9252) [Link]

Perhaps but the average win on typical GNU/Linux systems is greater than that: GCC does not do so well on register starved platforms. Find a comparison on GCC and you the difference is greater because x86 is really quite register starved.

GCC was really, really bad on register starved architectures, but since IA32 was the most important architectures for many years it got many tricks which helped. This difference is not so great with gcc too: 5.2%.

Additionally, x86_64 makes mandatory a number of micro- architectural improvements which were previously optional. For example, consider SSE2 which is not universally available on x86.

Very few systems today don't have SSE2. Athlon XP was discontinued three years ago and even VIA C7 and Intel Atom support SSE2 today - but they don't support x86-64!

So generally distributed binaries must either offer alternative functions and detect at run time, or, more commonly, avoid using those instructions.

1. There are yet another alternative: barf and refuse to run. Few systems will be affected.
2. Flash was written long ago - so MMX/SSE/SSE2 versions of all routines already exist.

And I'm not sure why you say that ~3% by itself isn't all that compelling when the price difference between a 2.83GHz core2 quad and a 3.0GHz core 2 quad cpu is $230 US. Based on that simplistic analysis we might expect 3% to be worth $115 US while if you use a modern x86_64- painless distro and don't use non-free software the cost of 64bit mode is pretty close $0.

Because average difference of just 3% with difference from -40% to 30% in case by case basis means even bigger win will be to carefully select which programs must be 32bit and which - 64bit. And win from profile based optimization and whole program optimizations can dwarf that 3-5% in many cases - but nobody does that. It looks kinda hypocritical to press Adobe to rewrite the Flash for mere 3-5% win when Linux community does not do it's own homework.

Whopping 5% for GCC...

Posted Oct 18, 2008 20:32 UTC (Sat) by gmaxwell (subscriber, #30048) [Link]

Don't make the mistake of assuming that I think Adobe should do a 64-bit Linux port of flash. I don't. I don't use flash nor do I care what they do. I was responding specifically to your argument that x86_64 is not worthwhile.

You're still arguing from a non-realistic position: It's not reasonable for distributions to do profile driven compilation because of the load it imposes on the build infrastructure. Likewise they can't reasonably do whole program optimization because of the enormous memory requirements. Today in the real world x86_64 does give real measurable performance improvements. The floating point improvement alone justifies it.

Likewise, being unable to support non-P4+ systems isn't reasonable. There are still plenty of x86 CPUs sold today without full SSE2 support (Geode, for example). It makes a lot of sense to batch together all of the little non-backwards-compatible micro-architecture improvements and break compatibility once, offering x86_64 for new systems, and i486 compatible code for everything else. Which is exactly what is being done.

It seems to me that you have missed the point I was making about 3%. People are paying $240/cpu for what is *at best* a 6% improvement. If you have any floating point workloads x86_64 is a big win, and even if you have purely integer workloads x86_64 is still a worthwhile improvement in typical cases.

Your cited benchmark here is antiquated: Would you really expect GCC from 2004 to take full advantage of a CPU architecture which had just become available in 2003? I do not have a copy of the speccpu benchmarks, but I'm familiar with the mincost optimization code (mcf) that performed so poorly on x86_64 in your cited benchmark. My own quick test here shows mcf on x86_64 performing only 3% slower than when compiled for x86 with gcc 4.3.0 on core2. My test problem[1] may not be comparable to whatever is in specint, but this suggests to me that the 40% hit documented in your linked benchmark is probably no longer representative of x86_64 performance with GCC, even on one-off micro-benchmarks.

[1] http://myrandomnode.dyndns.org:8080/prob.gz

And they are happy with 32bit Flash too

Posted Oct 18, 2008 13:42 UTC (Sat) by tialaramex (subscriber, #21167) [Link]

Well, last time I looked there were still a LOT of stores selling machines with 4GB of RAM and 32-bit Vista. Big brand names too, not little outfits. And not just 32-bit Vista, a lot of those machines, particularly laptops, have their 64-bit CPU wedded to a 32-bit mainboard chipset, physically capable of only 2^32 different addresses, which means they can't use the entire 4GB of RAM no matter what you run, Vista, Linux, or OS/2, because there'd be no physical addresses left for PCI devices. But they know users aren't technically capable enough to ask what's going on, so the advert says "4GB of RAM*" and the small print for the asterisk just says that you can't use all that RAM. You're still really buying 4GB of RAM, but to actually use it you'll need to throw the rest of the machine away. In practice such machines leave about 3.5GB of RAM accessible.

But also there is also still a LOT of software out there, particularly drivers, but also all manner of other things, that just doesn't work on 64-bit Vista. Not "it has to run in 32-bit mode", nearly everything is like that, but stuff that just won't work at all, if you're lucky you get an error, otherwise it just crashes. So I don't buy this idea that there are tons of people running 64-bit Vista and not noticing. Maybe it's happening on the dual-architecture Linux distros like Fedora, but not on Windows.

Because this is prefessional software, of course :-)

Posted Oct 18, 2008 18:29 UTC (Sat) by khim (subscriber, #9252) [Link]

I don't quite get why a company (that has a long list of things linux is suposedly lacking in) is not able to port their "professional" software to any 64bit (or other) environment.

The reason is the same there are no Fedora for Alpha and Ubuntu only supports Sparc on servers: cost-benefits analisys does not look very compelling. As you can see the main complaint of Adobe (just like any commercial company) is "too many platforms". It's not just "IA32 vs x86- 84", it's also "V4L vs V4L2" and "ALSA vs PulseAudio vs OSS vs ...". All these things raise costs and make the platform less compelling to port to.

Diversity is greatest strength of Linux - and it's greatest weakness too: LSB will not help you if you want to write plugin which is usable with default browser...

Because this is prefessional software, of course :-)

Posted Oct 19, 2008 12:44 UTC (Sun) by macc (subscriber, #510) [Link]

He, he.

64bit platforms are not an obscure niche market.
Not today and certainly not tomorrow.

So Adobe seems to be mired in being incapable to jump.
( probably should shove the "white" boys out the backdoor.)

Reading that referenced blog ( even if the main
article is outdated the follow ups from this
guy read rather similar ) gcc is completely unusable
( at least for the hacks historically used by adobe/flash )
X is a pit full of poisonous snakes, linux as a platform
an unfixable mess.

How come so may OSS projects are completely cross platform
/ architecure / byteorder and compile effortlessly with gcc
to well behaved and performing binaries ?

I am regularly astonished how former windows programmers
can break a perfectly good unix system in completely
unexpected ways.
( Just tried to fix teamspeak2 on my sons box. No go )

MACC

64bit platforms? Huge market, yes. Pure 64bit platforms without 32bit support? Don't make me laugh...

Posted Oct 19, 2008 13:55 UTC (Sun) by khim (subscriber, #9252) [Link]

64bit platforms are not an obscure niche market.

Certainly not! They are widespread and Adobe Flash works just fine there! Oh, you mean "pure 64bit platforms without 32bit support"... That's not "obscure niche market", true. That's uber-obscure super-niche market.

How come so may OSS projects are completely cross platform / architecure / byteorder and compile effortlessly with gcc to well behaved and performing binaries ?

Effortlessly? Easily? Have you tried to compile VirtualDub with gcc? Or GNOME with MSVC? Few OSS projects go from Windows world to Linux world - because it's hard. It's much easier to go the other way (with help of the CygWin and MingW). OSS projects either live in Windows and don't even try to port to Linux because it's API is horrible mess or they live in Linux and then they are forced to deal with all these idiosyncrasies. Even so they often choose to care only about few platforms (TraceMonkey support is limited to x86, x86-64 and ARM right now, for example). Plus there are yet-another-problem: most "portability solutions" are designed with source availability in mind (have you tried to use rpm from RHEL on Debian or something like that? It's quite a challenge)

I am regularly astonished how former windows programmers can break a perfectly good unix system in completely unexpected ways.

Actually that's just different mindset: Linux programmers work in community and expect that other will fix their own bugs while they can be asked to change the program if there are some change in other place. Windows programmers goal is to make program compatible with given set of libraries/programs/etc but they are not ready to accept the fact that program which worked yesterday can require fix today.

64bit platforms? Huge market, yes. Pure 64bit platforms without 32bit support? Don't make me laugh...

Posted Oct 20, 2008 7:25 UTC (Mon) by quotemstr (subscriber, #45331) [Link]

What about Linux is horrible mess? You'd do well to support your accusations. I'd argue that Win32 is a horrible mess, actually, filled with compatibility kludges, automagical common control versioning, silently redirected writes to parts of the filesystem namespace, and so on. Not the worst I've ever seen, but I'd rather work on a nice POSIX system any day.

I can't speak for VirtualDub, but GTK+ programs actually run fine under Windows. Take Pidgin as a great example. If a program is hard to port, it's because that program's design is tied to the platform, not because platform X or platform Y's APIs are inherently bad.

Where is the mess? Everywhere: sound, video, embedding, etc.

Posted Oct 20, 2008 8:36 UTC (Mon) by khim (subscriber, #9252) [Link]

I'd argue that Win32 is a horrible mess, actually, filled with compatibility kludges, automagical common control versioning, silently redirected writes to parts of the filesystem namespace, and so on.

You are looking on the system from the POV of system developer. From POV of application developer lack of said automagical versioning is a mess.

Why? Some examples.

Sound. How well OSS emulation is working today? Oh, right, OSS API was a mess and so it can not be adequately supported. Now everyone must rewrite everything for ALSA... or may be GStreamer... or may be PulseAudio...

Video. There are just two interfaces: V4L and V4L2. So if I don't need the features I can safely use V4L, right? Oh, no backward compatibility... Ok, I'll use V4L2... What? No support for some devices? So I must support both? What a mess... (Note: AFAIK this particular mess was cleaned up and now most devices support V4L2 - but for years the situation was as described).

Widgets. Can I embed Qt widget in a GTK program? No? Why No? Ok. At least GTK widgets are usable in Qt (not so long ago this was impossible too).

Remember the ire Windows users directed on Microsoft for discontinuing VB6? It was supported for ten years before Visual Fred replaced it! In Linux such tectonic shifts (when you can not use old APIs anymore) happen once per year or two. And rarely developers offer support for "obsolete" technologies. The only one lib which actually supports old APIs is GLibC - yes, there are occasional glitches, but you can still take old program from the era where files were limited to 2GiB and it'll work. Oh and linux kernel itself is pretty good at keeping things backward compatible... unless you'll count aforementioned "big switches" like OSS->ALSA transition or V4L->V4L2 transitions.

Transitions in Windows are handles in the sane way (from the POV of everyone who's not system developer):
1. APIv1 is introduced.
2. APIv2 is introduces some years later. APIv1 is still usable and for a few years you can safely ignore APIv2.
3. APIv1 is declared deprecated and developers are encouraged to switch to APIv2.
4. Few years later APIv1 is... removed? no! frozen: you can still use APIv1, but some features are not available.
5. Finally, 10-15 years down the road APIv1 is removed.

If there are APIv3 or APIv4 introduced along the way - it means at some point you'll have concurrect support for three or four versions of API...

In linux it's done "in clean way":
1. APIv1 is introduced.
2. APIv2 is introduced some years later. Now you must support both APIv1 and APIv2. Because they don't play well on the same system and you never know when particular distributions will switch from APIv1 to APIv2.
3. APIv1 is removed.

This is the best way to scare away ISVs and "normal" users. Who just want to use the system and not think about versions of kernel, GLibC and GTK. Who want to select API to support all currect systems (say: 2-3years old) and future systems (again: 2-3years in the future). Why this limits? It's easy: PCs average lifespan is 2-3 years. And you want your application be available for everyone who's currently using PC. So Any API must be supported for at least five years (ten is better) and for at least three years after replacement is introduced (five is better).

Situation is slowly changing - but it's still far from being equal to Windows. Actually friedliness to ISV is almost directly proportional to market share: Linux is 1% (because of the aforementioned problems), Mac OS is 5-10% (it's better then Windows, but it still likes to do things like drop support for Carbon when 100% incompatible replacement was only available for eight years), and the rest is Windows (which is horrible in many ways but has excellent backward compatibility).

Remember: Joe Average does not care as much for "cleannes of the architecture" - but he does care about his ability ro use old software. And ISVs like to produce software which till PC lasts. This is not an option in current Linux ecosystem - unless you'll ignore it and go with separate abstraction layer (JDK, for example).

Where is the mess? Everywhere: sound, video, embedding, etc.

Posted Oct 27, 2008 0:22 UTC (Mon) by vonbrand (subscriber, #4458) [Link]

No, Linux (the kernel) still supports the very first filesystem it had, and runs its very first binaries.

OTOH, I once bought a MSFT game, and it did not run on the next version of Windows. Just the splash screen showed, then complete hang.

This "Linux is a mess" is complete FUD. Yes, I do want my programs to run on the next (and then some) Fedora. Usually that means that they got recompiled (by my distribution or somebody else, even myself) on the way. I very rarely have any use for year-old binaries (except for some self-written programs that I just copied over and am still running as 32 bits on this 64 bits, mostly out of lazyness). And where I do need to run old binaries, I use a "enterprise" distribution; there I have a very much longer guarantee than on any Windows I've seen.

Hmm... Check your facts first, them talk

Posted Oct 27, 2008 7:57 UTC (Mon) by khim (subscriber, #9252) [Link]

No, Linux (the kernel) still supports the very first filesystem it had, and runs its very first binaries.

The very firstsystem? Did you mean very first filesystemS? There were two: ext1 and xiafs. Both are unsupported today. As for binaries... yes, "Hello world" runs, but real binaries fail routinely. Programs which iteract with hardware (floppy, CD, sound) are expecially fragile.

OTOH, I once bought a MSFT game, and it did not run on the next version of Windows. Just the splash screen showed, then complete hang.

What was the name of the game and what did you do to make it work? I've found that most games can be persuaded to work in the next version of Windows if you'll just remove what copy protection software they are using. That's not a Windows problem: copy protection creators go far beyond spec.

Look, I'm not saying Windows compatibility is 100% sure thing. But Microsoft thinks about compatibility! Compare with Linux: yes, there exist the way to properly support sound in old program. But it's still unmerged - six years after ALSA introduction! You can be 100% sure Microsoft will never tolerate such a huge breach of continuity. You can be 100% sure such transtion layer will be developed and introduced alongside the new interface, not some years after the fact when poor sound support will start to really hurt Linux reputation.

Yes, I do want my programs to run on the next (and then some) Fedora. Usually that means that they got recompiled (by my distribution or somebody else, even myself) on the way. I very rarely have any use for year-old binaries (except for some self-written programs that I just copied over and am still running as 32 bits on this 64 bits, mostly out of lazyness).

Yes, that typical Linux user attitude. But Windows user are accustomed to much longer support times. And for binaries, not for source. Some companies are still using 10-15 years old programs written in Visual Basic 2 for Windows 3.1 (that's one reason as to why Windows for x86-64 is not such a big hit: you can not use Windows 3.x programs there).

And where I do need to run old binaries, I use a "enterprise" distribution; there I have a very much longer guarantee than on any Windows I've seen.

Have you actually used Enterprise Windows? Let's compare, shell we:
Windows2000: introduced in 2000 (duh), EOL July 13, 2010
RHEL2.1: introduced in 2002, EOL May 31, 2009
Windows2003: introduced in 2003 (duh), EOL July 14, 2015
RHEL4: introduced in 2005, EOL February 29, 2012
RHEL5: introduced in 2007, EOL March 31, 2014
Windows2008: introduced in 2008 (duh), EOL October 07, 2018
Sorry, but if you'll compare enterprise Linux distributions with what Microsoft is offering... you'll find Linux sorely lacking. RHEL and Ubuntu LTS fall far short of what the Microsoft is offering where SLES is barely matching it (but if I wanted an Microsoft-controlled OS I'd used Windows). Don't try to rebuff FUD with FUD of your own - it backfires.

P.S. Note: I'm not saying Linux is hopeless. On the contrary: Linux is slowly moving in right direction. But to fix the problem you first must admit yet. Sadly Linux developers tend to ignore it and blow up every time points it out. That's not the way to win ISVs and users.

Where is the mess? Everywhere: sound, video, embedding, etc.

Posted Jan 26, 2012 1:25 UTC (Thu) by HelloWorld (guest, #56129) [Link]

> Sound. How well OSS emulation is working today? Oh, right, OSS API was a mess and so it can not be adequately supported. Now everyone must rewrite everything for ALSA... or may be GStreamer... or may be PulseAudio...
This is downright bullshit. There's OSS emulation in ALSA and it works most of the time. PulseAudio has padsp for OSS-only applications and an ALSA plugin for ALSA-only applications.

> Video. There are just two interfaces: V4L and V4L2. So if I don't need the features I can safely use V4L, right? Oh, no backward compatibility...
Again, this is just bullshit. V4L2 drivers do include a compatibility mode for V4L applications.

> Widgets. Can I embed Qt widget in a GTK program? No? Why No? Ok. At least GTK widgets are usable in Qt (not so long ago this was impossible too).
I agree that having two GUI toolkits is unfortunate, but it has _nothing_at_all_ to do with backward compatibility.

> 1. APIv1 is introduced.
> 2. APIv2 is introduced some years later. Now you must support both APIv1 and APIv2. Because they don't play well on the same system and you never know when particular distributions will switch from APIv1 to APIv2.
Right, except that you failed to provide even *one* example. I have gtk2 and gtk3 applications running side-by-side right now. I've run a Qt3 application (Xilinx ISE) just a few months ago.

So please, just stop making up random bullshit here.