fix uniq -c [LWN.net]

fix uniq -c

Posted Feb 12, 2025 19:06 UTC (Wed) by NYKevin (subscriber, #129325)
In reply to: fix uniq -c by stijn
Parent article: Rewriting essential Linux packages in Rust

You may dislike that interpretation of "field," but it is specified in the POSIX standard[1]:

> A field is the maximal string matched by the basic regular expression:
> [[:blank:]]*[^[:blank:]]*

But note that this is describing how the -f flag works, not the -c flag (which is also the case for the man page you quote).

As far as I can tell, POSIX makes no allowance whatsoever for right-justifying the -c output, and in fact specifies the opposite:

> If the -c option is specified, the output file shall be empty or each line shall be of the form:
> "%d %s", <number of duplicates>, <line>

(%d means "a number in decimal," not "a number in decimal, but possibly with some whitespace in front of it.")

If a uniq implementation right-justifies its -c output, that is either a bug or a deliberate non-conformance to the standard. I would suggest reporting it upstream if you have not already done so.

[1]: https://pubs.opengroup.org/onlinepubs/9799919799/utilitie...

to post comments

fix uniq -c

Posted Feb 13, 2025 13:55 UTC (Thu) by stijn (subscriber, #570) [Link]

> But note that this is describing how the -f flag works, not the -c flag (which is also the case for the man page you quote).

Yep I know. It is a horror I noted in passing.

> You may dislike that interpretation of "field," but it is specified in the POSIX standard[1]:

I very much dislike that POSIX standard then. POSIX probably codified existing behaviour, but anyway it does not matter. Clearly existing options have to continue to carry the meaning that they have, I'm not tilting at that windmill.

Coreutils combined with utilities such as datamash can be quite powerful in composing filters, maps or computations in a streaming paradigm, but it does require having a meaningful definition of field. One workhorse in scientific computing is the dataframe encoded as tab-separated data with column headers. This format (bar column headers) is already half-embraced by a lot of unix utilities, I hope this progresses further. A good check is always 'does it work for the empty array / empty string', and this is required to make the tab-separated format usable.

> As far as I can tell, POSIX makes no allowance whatsoever for right-justifying the -c output, and in fact specifies the opposite:

In the savannah gnu git repo the code for uniq has 'if (count_occurrences) printf ("%7jd ", linecount + 1);'

http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;...

I'm not interested in left-justification either - datamash -g 1 count 1 is a more usable alternative.

fix uniq -c

Posted Feb 15, 2025 22:59 UTC (Sat) by pdewacht (subscriber, #47633) [Link]

The "-c" behavior would be a bug in POSIX I assume, since historic Unix uses the equivalent of "%4d %s" and I can't find anybody who has adopted the strict POSIX reading.