A way out for a.out
The a.out format dates back to the first edition of Unix. When MINIX came along, it naturally used that format for its executable files; that, in turn, led to a.out being used in Linux as well. It is a simple format, and its implementation on Linux was even simpler; among other things, every Linux shared library had to be centrally assigned its own portion of the address space, since libraries could not be relocated at run time. Still, Linux used a.out for some time, until support for the newfangled ELF format was first added to the 0.99.13 development kernel in 1993.
There was a time when the crazier people among us manually converted our Slackware systems from a.out to ELF in order to be able to try it out and gain the benefits before distributions were updated. They still bear the scars from that time. Not that your editor would ever admit to knowing anybody who would have engaged in any such activity.
ELF has been the standard executable format for Linux on most architectures since 1995. One might think that would have provided enough time for any users of a.out binaries to grudgingly move on to ELF; its adoption can probably be judged to not be a passing fad at this point. But, in the real world, surprises lurk.
The initial conversation on removing a.out faded out fairly quickly, but was restarted when Eric Biederman posted a patch disabling the building of a.out support on the two architectures (Alpha and m68k) that still enable it by default. This patch did not actually remove the support, it just turned it off to see if anybody would scream. If protests were raised, support could be turned back on quickly and easily; otherwise a full removal could be done.
Linus Torvalds quickly responded
that he was "pretty sure we can't do this
". He pointed out
that the native executable format on Alpha systems running Digital's Unix
was essentially a.out, even if it went by the newer name of ECOFF. The
Linux a.out loader can run ECOFF programs by virtue of ignoring some of the
newer ECOFF features; taking it out would break any systems still using
that support. One might think that the number of users who are still using
Alpha CPUs, running ECOFF binaries on those CPUs, and tracking current
kernel releases might be a fairly small number, but one never knows.
Kees Cook did do
a little research on, seemingly, the only distribution that still
supports Alpha (Gentoo) and found that the only ECOFF files present
contained firmware, which does not run on the CPU anyway. He concluded
that there would be no harm from removing a.out support on this platform:
"Let's do it
".
It seems to be a universal rule that somebody always has to come along to ruin the party. In this case, just as it seemed like there were no further obstacles to the removal of a.out, James Jones showed up to let it be known that he was still using a.out:
The use case is running an old set of tools to build programs for the Atari Jaguar. Namely, Atari's assembler (mac) and linker (aln). The alternative is running windows versions in dosbox, or using some replacements that have been developed based on an even older, less-featureful version of the source code for mac and aln, but which still haven't managed to add back in all the features needed to build some programs or use the Atari debugging tools (Also available in a.out only).
He gave some more details on why he still needed to use these tools in this followup message
Moving a program to a new executable format is normally just a matter of rebuilding the program. But rebuilding is a rather more difficult task in the absence of the source code. Finding oneself stuck trying to run a decades-old binary on modern systems is just one of the many joys reserved for users of proprietary software — but many people have found themselves in that position at one time or another. They are legitimate Linux users, and there is no desire to break their systems. So Petkov duly gave up and asked only that Jones document his usage of a.out for the benefit of those who might try to remove it in the future.
Cook, however, was not so quick to throw in the towel; he looked at the programs in question and concluded that it might be possible to write an ELF wrapper program that could load and run old a.out binaries. One day later, he posted such a program, noting that it could run the aln program at least far enough for it to complain about its command-line arguments. Jones tried it out and was pleased with the results:
Yes, works perfectly, thank you. I like the idea of using this much better than getting emailed every time someone wants to delete the a.out code again. Consider my use case withdrawn. I've already pushed an update to my jaguar-sdk project to use this tool instead.
So removal of a.out support is back in the plans for
the 5.18 kernel. Perhaps the effort will succeed this time, though there
are still no guarantees; there may be a.out users who are unaware of the
coming apocalypse and haven't yet made their objections known. If those
users are unable or unwilling to use Cook's wrapper utility, the excision
of a.out support may, once again, have to be deferred to some future kernel
release.
| Index entries for this article | |
|---|---|
| Kernel | a.out |
Posted Mar 24, 2022 15:50 UTC (Thu)
by dullfire (guest, #111432)
[Link]
Or, if the programs properties were just right, you could get even fancier and convert the program to elf (of course you could not take advantage of elf features) in memory, before actually "exec-ing" it.
Posted Mar 24, 2022 16:09 UTC (Thu)
by wtarreau (subscriber, #51152)
[Link] (20 responses)
$ cat /proc/version
Waiting for this PC-Engines ALIX to die but it doesn't want to. It has only known ELF anyway, I migrated to ELF with kernel 2.0 (except for floppy disks for which I kept the smaller a.out a bit longer).
Jonathan, like you I was one of those installing libc.so.4 and libc.so.5 side by side on Slackware to try to figure if ELF would bring anything (by then it only brought slightly slower program execution IMHO but it was the future). Not everything worked out of the box, but overall it used to be much more seamless than the following migration to glibc2 where the low-level network API completely changed and required many network programs to be heavily modified!
Posted Mar 24, 2022 16:38 UTC (Thu)
by corbet (editor, #1)
[Link] (4 responses)
Posted Mar 24, 2022 16:43 UTC (Thu)
by smoogen (subscriber, #97)
[Link]
Posted Mar 25, 2022 11:46 UTC (Fri)
by jd (guest, #26381)
[Link] (2 responses)
Anyone remember the problems involving Gnu Make during that transition?
Posted Mar 26, 2022 4:54 UTC (Sat)
by wtarreau (subscriber, #51152)
[Link] (1 responses)
I don't remember problems *specific* to that transition, GNU Make broke something with *every single minor update*. For more than 10 years I've had to have a collection of 3.77, 3.79, 3.80, 3.81, 3.82 and 3.83 on my machines and chose the working one depending on what I was trying to build :-/ It seems to have stabilized a bit since 4.1 or so, let's keep fingers crossed.
Posted Mar 28, 2022 23:15 UTC (Mon)
by anonymous_commenter (guest, #117657)
[Link]
Debian 1.3 —> Debian 2
There was also, many blue moons ago, the transition from nsswitch et al to libpam. That one was worse for me.
Posted Mar 24, 2022 18:33 UTC (Thu)
by hrw (subscriber, #44826)
[Link] (12 responses)
Alix was fun platform. That Geode LX cpu which was either 586 or 686 depending who asked.
Always can run newer stuff but having system running so old software i would keep it alive until death (their or mine).
Posted Mar 24, 2022 22:51 UTC (Thu)
by nix (subscriber, #2304)
[Link] (11 responses)
(I really must replace it one of these days: having a single point of failure that is 13 years old is worrisome. I have a Turris Omnia I keep meaning to set up, but the new system is so crazy weird I haven't dared to try to actually replace it yet. It's a shame nobody makes anything like the net5501 with lots and lots of network ports any more :( I need at least five...)
Posted Mar 25, 2022 1:31 UTC (Fri)
by pabs (subscriber, #43278)
[Link]
Posted Mar 25, 2022 6:37 UTC (Fri)
by hrw (subscriber, #44826)
[Link]
https://www.aliexpress.com/item/1005003174267310.html has 6 ports each with Intel i211 network card.
I use similar 4 port one with OpenWRT as a router. Works fine.
Posted Mar 25, 2022 7:33 UTC (Fri)
by pbonzini (subscriber, #60935)
[Link]
Posted Mar 25, 2022 10:14 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
Posted Mar 25, 2022 12:49 UTC (Fri)
by nix (subscriber, #2304)
[Link] (1 responses)
Posted Mar 25, 2022 12:58 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link]
They recently added Docker support and it allowed me to scratch the last itch, to host my own version of speedtest to easily monitor the WiFi performance (https://github.com/librespeed/speedtest).
Posted Mar 28, 2022 21:48 UTC (Mon)
by abufrejoval (guest, #100159)
[Link] (4 responses)
Actually with USB3 NICs, just about any board will do, even if lack off offload support may require more CPU power.
I've been using a 8-port Jetway NF592-Q170 with a Kaby Lake i7-7700T for a few years, but that's a slightly different price (and performance) point.
Posted Apr 11, 2022 21:29 UTC (Mon)
by nix (subscriber, #2304)
[Link] (3 responses)
But maybe I've been misled by the raspberry pi's famously terrible USB ports :)
Posted Apr 12, 2022 9:46 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (1 responses)
USB High Speed or USB Super Speed? High speed (480 Mbit/s) has a CRC-5 to protect packet headers, and a CRC-16 to protect the data, while Super Speed (5 Gbit/s or above) uses the same CRC-32 as Ethernet for data, and a different CRC-16 to USB High Speed data for packet headers.
Super Speed is thus much more capable of detecting errors in transfer than High Speed.
Posted Apr 19, 2022 13:30 UTC (Tue)
by nix (subscriber, #2304)
[Link]
Posted Apr 12, 2022 10:27 UTC (Tue)
by james (subscriber, #1325)
[Link]
I would argue that this means laptop manufacturers are sufficiently happy with USB NICs to risk having them on the desks of millions of decision-makers, if it wasn't for my experience with a dodgy Surface Pro 3 and dock, where a jolt to the desk would temporarily disconnect the USB connection and drop all SSH sessions. I replaced that with a USB NIC and was much happier (until I persuaded work to give me a proper laptop instead).
Posted Mar 26, 2022 0:36 UTC (Sat)
by mss (subscriber, #138799)
[Link] (1 responses)
Posted Mar 26, 2022 5:01 UTC (Sat)
by wtarreau (subscriber, #51152)
[Link]
It was a bit more complicated in fact. It would only implement CMOV between registers and not from memory. However, compilers would almost never emit CMOV from memory, so generally speaking that was OK to build with -march=i686. Once in a while you'd find a faulty program though, that needed to be rebuilt for i586. What a painful period! And there was a real benefit in trying to use CMOV by then since branch predictors were still non-existent.
Posted Mar 24, 2022 17:05 UTC (Thu)
by glaubitz (subscriber, #96452)
[Link] (4 responses)
That is not correct though. Debian is also still supporting the Alpha architecture among other older architectures under the umbrella of the Debian Ports project.
See: https://cdimage.debian.org/cdimage/ports/ and: https://www.ports.debian.org/
Disclaimer: I'm one of the primary maintainers of Debian Ports.
Posted Mar 25, 2022 5:23 UTC (Fri)
by rdeforest (guest, #153619)
[Link] (3 responses)
Has there really been no news in the Debian Ports world since May of 2019, or is none of the news since then worthy of the project's front page?
Posted Mar 25, 2022 5:28 UTC (Fri)
by glaubitz (subscriber, #96452)
[Link] (2 responses)
Thanks, I appreciate the praise. It's also fun to work on these old machines and we're making good progress even with getting Rust supported on m68k ;-).
> Has there really been no news in the Debian Ports world since May of 2019, or is none of the news since then worthy of the project's front page?
We don't really have an active web team for Debian Ports, most of the discussion happens on the mailing list.
As you can see from the folder with the CD images, we are more or less regularly push out new Debian unstable snapshots.
Posted Mar 25, 2022 8:21 UTC (Fri)
by taladar (subscriber, #68407)
[Link] (1 responses)
Might be worth at least posting that somewhere prominently on the web page so people don't assume the project is dead.
Posted Mar 25, 2022 9:03 UTC (Fri)
by beagnach (guest, #32987)
[Link]
Posted Mar 24, 2022 17:20 UTC (Thu)
by pebolle (guest, #35204)
[Link] (3 responses)
Nitpick: that's not a wrapper but a loader, isn't it?
Anyhow, would it be possible to write an a.out to ELF converter? Ie, a program that is fed an a.out executable and creates an ELF executable (basically this loader and the a.out dumped into an ELF executable).
Posted Mar 24, 2022 20:21 UTC (Thu)
by gutschke (subscriber, #27910)
[Link] (2 responses)
I haven't looked too closely. But there might be some need to add a bit of assembly "glue". Shouldn't be much, but that would be architecture-specific. Maybe, that's why a wrapper turned out to be easier to code.
The only other remaining issue is the fact that a.out wants to be loaded to an address that is very low. In fact, it's so low that it is usually not allowed to be mapped from user space. So, you'd quite possibly end up with ELF binaries that fail to start until you reconfigure your system, and you won't get any informative error messages. The wrapper addresses that problem a little nicer and prompts you what to do.
If I think hard enough, I can imagine solutions for all these problems. But they get increasingly more complex, for something that really doesn't justify this amount of engineering effort. If the wrapper works, that's likely the most expedient and pragmatic solution.
Posted Mar 24, 2022 23:48 UTC (Thu)
by Trou.fr (subscriber, #26289)
[Link]
Posted Apr 1, 2022 8:27 UTC (Fri)
by anton (subscriber, #25547)
[Link]
Posted Mar 24, 2022 17:51 UTC (Thu)
by halla (subscriber, #14185)
[Link]
I remember trying that...
Posted Mar 24, 2022 20:29 UTC (Thu)
by cuviper (subscriber, #56273)
[Link] (3 responses)
Posted Mar 24, 2022 21:49 UTC (Thu)
by JoeBuck (subscriber, #2330)
[Link] (2 responses)
Posted Mar 24, 2022 22:36 UTC (Thu)
by gerdesj (subscriber, #5446)
[Link] (1 responses)
A shitload of documentation. lol here's some C, wack it into a file called x.c, run this (gcc command) and then run ./a.out.
Posted Mar 27, 2022 17:10 UTC (Sun)
by NYKevin (subscriber, #129325)
[Link]
Posted Mar 25, 2022 5:57 UTC (Fri)
by bsdimp (guest, #18082)
[Link]
The code posted assumes the a.out system call ABI is the same as the ELF one. This wasn't true for my venix emulaor so i had to interpret the system calls. It was easy for 16bit code runnimg in a 32bit binary, but would be harder for 32bit code. Otherwise it is quite similar to what I wrote.
You can find my FOSDEM 2022 presentation here https://fosdem.org/2022/schedule/event/venix/
Posted Mar 25, 2022 6:45 UTC (Fri)
by felixfix (subscriber, #242)
[Link] (1 responses)
I'd apologize for the off-topic reference, but one can never have too many Fats Waller links, can one?
Posted Mar 28, 2022 15:39 UTC (Mon)
by rsidd (subscriber, #2582)
[Link]
Posted Mar 27, 2022 17:34 UTC (Sun)
by jdulaney (subscriber, #83672)
[Link] (1 responses)
Posted Mar 30, 2022 9:32 UTC (Wed)
by immibis (subscriber, #105511)
[Link]
Posted Apr 6, 2022 18:24 UTC (Wed)
by anton (subscriber, #25547)
[Link]
If anybody wants to do their own experiments, here are some a.out binaries and libraries. Not all of them worked on Debian 8, but at least gforth-0.3.0 did as root or if I used
A way out for a.out
A way out for a.out
Linux version 2.4.37-wt1-utm10.1-alix (willy@wtap) (gcc version 3.3.6) #1 Tue Dec 9 12:50:07 CET 2008
And here I'd thought I had successfully suppressed the painful memories from the glibc2 transition...thanks a lot, man...:)
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
This one recently reviewed on ServeTheHome falls one port short, but if you insist on Linux (vs. pfSense/BSD) you can add another RealTek 8156 base USB3 2.5 Gbit/s NIC.
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
Docking stations tend to be USB-C or Thunderbird these days (looks like Dell don't sell anything else), so naturally the NICs are USB 3. As far as I know, Surface Pro docking stations (with a custom connector) have USB NICs inside, too.A way out for a.out
A way out for a.out
Waiting for this PC-Engines ALIX to die but it doesn't want to.
I still remember working with ALIX predecessor board from PC-Engines - WRAP.
Still had NSC logo on the chip.
Alix was fun platform. That Geode LX cpu which was either 586 or 686 depending who asked.
If my memory serves me right I remember Geode being treated as a 686 in the userspace as it supported the CMOV series of x86 instructions, which was apparently the defining difference between a 586 and a 686 (at least from the userspace perspective).
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
In particular, it appears as if the only binaries anybody cares about are statically linked already.
Those a.out binaries I care about are dynamically linked (pretty much everything was at the time). However, Linux broke these user space programs with the 5.1 release (so much for "we don't break user space").
Maybe, that's why a wrapper turned out to be easier to code.
Which wrapper turned out to be easier to code? The only wrapper I am aware of, Siegfried Pammer's run-aout turned out to be a lot more work and less capable than I expected. You can read all about it in his thesis, but among the difficulties was that run-aout has to emulate the uselib() system call (because the libraries are also a.out binaries). Pammer deserves high praise for tackling all these obstacles.
My conclusion from all of this is that it would probably have been easier to put an a.out loader into kernel space, as an out-of-tree module if the kernel maintainers insist on eradicating a.out support from the official tree.
The only other remaining issue is the fact that a.out wants to be loaded to an address that is very low. In fact, it's so low that it is usually not allowed to be mapped from user space. So, you'd quite possibly end up with ELF binaries that fail to start until you reconfigure your system, and you won't get any informative error messages.
Yes, some of the a.out binaries have so low starting addresses that they either required starting it as root (apparently whatever makes mmap fail does not apply to root) or first setting
sudo sysctl -w vm.mmap_min_addr="4096"
However, there are also a.out binaries that still worked with the default mmap_min_addr (of IIRC kernel 4.15).
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
A way out for a.out
Sooo close!
Sooo close!
A way out for a.out
A way out for a.out
I have now tried Kees Cook's aout program:
A way out for a.out
./aout /usr/local/bin/gforth-0.3.0
/lib/ld.so: cache '/etc/ld.so.cache' is corrupt
/usr/local/bin/gforth-0.3.0: can't load library '/usr/lib/libm.so.4'
Exec format error
/usr/local/bin/gforth-0.3.0: can't find library 'libm.so.4'
It's interesting that the old dynamic linker can no longer handle Debian 11 ld.so.cache (still worked on Debian 8), but putting a link in /usr/lib got around that problem. But then of course uselib cannot deal with the format of libm.so.4.6.27, which is a.out.
sudo sysctl -w vm.mmap_min_addr="4096"
I also tried the suggested alternative of rebuilding gforth-0.3.0. Some of the early hurdles were relatively easy to overcome, but eventually I had a binary to run, and it crashed; my guess is that the reason is that modern libcs no longer use RWX permissions for malloc()ed memory. Fixing this would be a somewhat more involved, and it may be getting away from my purpose of running the old program: to be able to compare its behaviour to that of modern programs (e.g., recent versions of gforth).