The x32 system call ABI [LWN.net]

The x32 system call ABI

Posted Sep 9, 2011 12:23 UTC (Fri) by NikLi (guest, #66938)
In reply to: The x32 system call ABI by gmaxwell
Parent article: The x32 system call ABI

It is not "pointer memory bandwidth bound test cases".

A vm like python uses a *lot* of pointers:

- a list of 'n' items is a buffer of 'n' pointers. Same for tuples.
- a dictionary of 'n' items is a buffer of ~6*n pointers
- every string item carries a pointer
- every instance is a dictionary plus a couple of pointers

C programmers think with memory buffers but for dynamic languages where objects work by reference are mostly based on tons of pointers; this is what makes them dynamic. And yes, making all those pointers half their size is very important. Because imagine that when you want to look up something in a list, this list is fetched to the cache and all the pointers are traversed while looking for the item. Fetching a 2k buffer is better than fetching a 4k buffer. In fact, x86 might be more suitable than x86-64 for such vms!

(It would be very interesting to see some python benchmarks for x32 vs x86, nontheless)

Now, one may say that "if you want speed, do it in C". However making a dynamic language faster will benefit thousands of programs written in that language, which is important for some people..

Using pointer offsets suffers from one extra indirection and will kill a big part of the cache. On the other hand, pointing to more than 4G of things is an overkill.

to post comments

The x32 system call ABI

Posted Sep 9, 2011 14:12 UTC (Fri) by gmaxwell (guest, #30048) [Link] (6 responses)

> Using pointer offsets suffers from one extra indirection and will kill a big part of the cache. On the other hand, pointing to more than 4G of things is an overkill.

You use a single offset (after all, we're assuming you're willing to take a 4G limit in these applications) and keep it in a register.

Alternatively, how about an ABI that promises you that you can get memory under the 4G mark and you use 32 bits internally, and covert at the boundaries to external libraries. This way single applications can be 32 bit without overhead but it doesn't drag the whole system with it?

The x32 system call ABI

Posted Sep 9, 2011 15:45 UTC (Fri) by dlang (guest, #313) [Link] (5 responses)

what you are trying to describe is basically what the x32 architecture is doing.

however you missed that libraries can allocate memory as well, and so the libraries must be compiled to only request memory under 4G as well.

The x32 system call ABI

Posted Sep 9, 2011 18:12 UTC (Fri) by gmaxwell (guest, #30048) [Link] (1 responses)

It would only take a single syscall to the kernel to tell it to never give this process access to _any_ address space outside of the first 4gb (not via sbrk, mmap, etc).

It would have ~all the performance benefits without doubling the libraries in memory. It wouldn't, however, retain the benefit of reduced porting benefit of existing 32bit crapware since pointers in library owned structures would be the other size. ::shrugs::

The x32 system call ABI

Posted Sep 11, 2011 3:23 UTC (Sun) by butlerm (guest, #13312) [Link]

What you describe could be done, but it would be difficult to implement, require special compiler support to do well, and would break source compatibility even with special compiler support.

It would be essentially the same as adding support for 80286 style near and far pointers across the code base. In C, every structure, every header file, every shared pointer declaration would potentially have to be marked whether it was using large or small pointers. The compiler certainly wouldn't know that an arbitrary function or structure declaration was referring to something from a library, and some libraries would have to come in a non-standard flavor in any case.

Now as you say, there are certain advantages to that, in terms of memory and cache footprint. They did it back in the x286 era for a reason. But it is much more impractical to implement that sort of thing across the source code for practically everything then simply to compile under a new ABI, especially if the new ABI performs well enough to be the system default.

A reasonable distribution policy could be to replace x86 with x32, and not ship x86_64 libraries in x32 distributions. It could simply say that if you want have a 64 bit user space, you should use a full 64 bit version. 64 bit addressing could be reserved for the kernel. If I were to guess, half of the people currently planning to use x32 (e.g. in embedded applications) have that sort of thing in mind in any case.

The x32 system call ABI

Posted Apr 9, 2012 21:28 UTC (Mon) by snadrus (guest, #60224) [Link] (2 responses)

What about building x32 off ia32 compatibility? There would be no kernel changes, but just compiler changes to use the additional registers. You may even be able to use ia32 or x32 libraries interchangeably if you're not passing by register.

The x32 system call ABI

Posted Apr 10, 2012 9:08 UTC (Tue) by khim (subscriber, #9252) [Link] (1 responses)

1. Open wikipedia. Read.
2. Try to pretend you never asked this question.

Perhaps then you'll be considered seriously in some future architecture dispute.

Your worst yet, and for me your last

Posted Apr 13, 2012 6:37 UTC (Fri) by biged (guest, #50106) [Link]

Khim, you have exceeded your usual levels of hostility and brashness with this comment, and so I have added you to my filter. (I mention this as a reminder to others: My Account -> Comment Filtering.)

Your response here is beyond rude: it is poisonous. You should realise that with more time and attention someone might be able to explain the misconception, help others and avoid insulting anyone.

Please stop treating LWN as your inbox: post less often, and more thoughtfully. For me, you have become a spammer.