Killing off /dev/kmem

Posted Apr 5, 2021 17:31 UTC (Mon) by josh (subscriber, #17465)
In reply to: Killing off /dev/kmem by michaelkjohnson
Parent article: Killing off /dev/kmem

Did you call it "procps" because it's a ps that uses /proc rather than other methods?

Yes, procps from /proc ps

Posted Apr 5, 2021 20:30 UTC (Mon) by michaelkjohnson (subscriber, #41438) [Link] (9 responses)

Yes, that was the source of the name.

The original ps for Linux, often requiring rebuild after a new kernel build, if any of the structures had changed, used /dev/kmem. And after procps was released, the original ps was sometimes referred to as "kmem ps" to differentiate.

The original proc filesystem did not have enough functionality for a full replacement version of ps. I modified it to have all the necessary data for ps and uptime, and then wrote procps as a suite of programs that used the new functionality.

The output was formatted compactly (keep in mind this was when a 386sx16 was a decent machine) and I separated the stat and statm files because of the expense of producing the statm data, then in ps I kept track of whether statm needed to be read in order to produce the output.

As far as I know, my original procps was the original implementation of ps that defaulted to sorted output. All versions of ps that directly read /dev/kmem, as far as I know, listed the data in the order it happened to find it in the kernel memory it was digging through, and I was tired of juggling sort arguments when invoking ps.

I believe my original procps was also among the first, if not the first, to just recognize both BSD or SysV command line arguments and do what you meant, rather than requiring you to remember which syntax you needed to use on this particular system.

In any case, I don't know whether I was actually the first to introduce either or both of sorting internally and honoring both BSD and SysV arguments, or if one or both were previously invented and I unknowingly reimplemented ideas that already existed.

Yes, procps from /proc ps

Posted Apr 6, 2021 11:40 UTC (Tue) by lyda (subscriber, #7429) [Link] (4 responses)

Can't remember exactly why, but I had to look for a process in C on SCO once and discovered that the two ways to do it were to pull it out of /dev/kmem or to parse ps output. Massive gaping hole in libc in my mind. The /proc fs is a good unix-y solution.

I can imagine switching to it saved a massive amount of hassle.

Yes, procps from /proc ps

Posted Apr 7, 2021 17:05 UTC (Wed) by quanstro (guest, #77996) [Link] (2 responses)

procfs goes back to at least 8ed in 1984, and was included (and expanded) in plan 9 from the beginning.

Linux proc influence

Posted Apr 8, 2021 0:58 UTC (Thu) by michaelkjohnson (subscriber, #41438) [Link] (1 responses)

My confident recollection is that the initial implementation of the Linux proc filesystem was explicitly inspired by plan 9's proc filesystem.

Linux proc influence

Posted Apr 9, 2021 17:49 UTC (Fri) by quanstro (guest, #77996) [Link]

i recall that as well. at the time, nobody had access to plan 9. so linux /proc was inspired by _papers_ about plan9's /proc. one of the things linux missed---perhaps because the plan 9 implementation was not visible---was a dirt cheep way to have lots of little single-job file systems. so linux /proc acquired a few warts. there is more to implement in a linux file system than 9p. but perhaps the 9p subset is good enough for most cases.

Yes, procps from /proc ps

Posted Apr 8, 2021 3:52 UTC (Thu) by k8to (guest, #15413) [Link]

Given variations in proc, ps is still the portable solution, sadly.

how I remember the history

Posted Apr 6, 2021 13:09 UTC (Tue) by acahalan (guest, #151496) [Link] (3 responses)

Somebody wrote the original ps.

Based on that, Michael K Johnson wrote the procps.

Somebody else ended up maintaining procps for a while, adding color to the output, but then not much happened.

Michael K Johnson, then at Red Hat, decided (was told?) to maintain procps. He reverted to the pre-color version of the code. He put out a call for help, and Albert Cahalan responded with the suggestion that ps support both BSD and SysV syntax like OSF/1 (later renamed Tru64 then Digital UNIX) and AIX did. This would have been 1996 probably, or perhaps 1997. Sorted output was possible, but I don't believe it was the default. It should not be the default, mainly because ps is often used when a system is low on memory but also because partial output is desirable when running on a failing kernel. Sorting with the "O" option appears to have a BSD origin.

Albert Cahalan rewrote procps, initially just to prove that it would be possible to go beyond what OSF/1 and AIX could do, parsing mixed BSD and SysV options. (OSF/1 could only do one or the other, not mixed) There was then some human conflict relating to "ps -aux" printing a warning. Craig Small over at Debian started using Albert Cahalan's new code. This code definitely did not sort by default.

Michael K Johnson turned over a CVS repository to Rick van Riel and Ingo Molnar, excluding Albert Cahalan without explanation. This was almost certainly in 1997. Albert Cahalan then put a version 3.x.x on sourceforge, where he maintained procps for about a decade. At some point the 2.x.x version was made unreliable, grouping processes as threads if they happened to share various attributes as procps non-atomically looked at them. Albert Cahalan instead enhanced the /proc filesystem by adding the /proc/*/task/ directories and the thread counts. All distributions, including Red Hat, switched over to Albert Cahalan's procps 3.x.x code.

After about a decade maintaining procps, Albert Cahalan became too busy due to a large family. Also, he was demotivated because he found that it was impossible to stop Red Hat from hacking things up in ways that would add bugs and ill-considered compatibility troubles. This led to Craig Small, the Debian package maintainer, joining up with some other people to start the 4.x.x version series elsewhere.

Well, we remember some different things...

Posted Apr 7, 2021 2:10 UTC (Wed) by michaelkjohnson (subscriber, #41438) [Link] (1 responses)

I found some old tarballs to refresh my memory. ☺

Branko Lankester built kmem ps that came earlier.

In the earliest procps version I found (0.7), I already tried to honor at least SysV arguments e and f for people whose fingers had been trained on SysV, but provided BSD-style output regardless. Your rewrite implemented multiple personalities, which was naturally much better.

It looks like I introduced sorting output in version 0.93 in April 1994, later than I recalled, but before I was aware of you doing work on procps. That version definitely sorts by default, and the "o" option toggles sorting. I also clearly failed to update the man page along with that new feature.

Your memory of the transition is different from mine. I was doing a poor job of being maintainer (slow to apply patches and do new releases) but I certainly didn't "revert" color support, though I suspect it was there in a patch or fork that I hadn't adopted. A fork was the obvious response to an unresponsive maintainer, so no complaints there! I did finally step back formally.

Well, we remember some different things...

Posted Apr 8, 2021 21:17 UTC (Thu) by Kamilion (subscriber, #42576) [Link]

Wow, big thanks to Albert Cahalan and Michael K. Johnson for showing up and taking the time to explain to us johnny-come-latelys.

Took me a moment to look at the poster's names and realize they were the very people involved.

how I remember the history

Posted Apr 8, 2021 3:58 UTC (Thu) by k8to (guest, #15413) [Link]

The warning on ps -aux led to

export I_WANT_A_BROKEN_PS=shutup

Being part of my .profile on Linux for many years. At some point around 2012 I realized I didn't need it anymore.