Rewriting the GNU Coreutils in Rust [LWN.net]

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 7:20 UTC (Wed) by eru (subscriber, #2753)
Parent article: Rewriting the GNU Coreutils in Rust

Good idea, as long as the results are 100% compatible and at least performant as the originals. But hope they resist the temptation to "enhance". These utilities are part of the basic vocabulary of Linux and friends, and each program is generally designed to do one thing well. No bells and whistles, please.

to post comments

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 8:08 UTC (Wed) by Sesse (subscriber, #53779) [Link] (5 responses)

I thought the same, but I actually find rg more usable than grep. I'm not 100% sold on its grouping output, and sometimes, it skips files it shouldn't, but doing “rg foo” and having it do -r by default, skip .o files etc. is actually better than the behavior I came to get used to over 20 years with GNU grep.

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 10:07 UTC (Wed) by eru (subscriber, #2753) [Link]

I don't have a problem with adding improved commands under a new name, like "rg", as long "grep" still activates a command with the "classic" behaviour.

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 21:23 UTC (Wed) by wahern (subscriber, #37304) [Link] (3 responses)

> I'm not 100% sold on its grouping output, and sometimes, it skips files it shouldn't, but doing “rg foo” and having it do -r by default, skip .o files etc. is actually better than the behavior I came to get used to over 20 years with GNU grep.

That sounds very nice from the standpoint of interactive use. But for non-interactive scripting such behaviors violate numerous best practices, and are neither particularly convenient, consistent, nor safe. This is a common story for tools like ripgrep.

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 21:28 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

> But for non-interactive scripting such behaviors violate numerous best practices, and are neither particularly convenient, consistent, nor safe. This is a common story for tools like ripgrep.

Why is this a problem? It is a easy toggle to turn on/off

Rewriting the GNU Coreutils in Rust

Posted Jun 11, 2021 23:12 UTC (Fri) by jccleaver (guest, #127418) [Link] (1 responses)

> Why is this a problem? It is a easy toggle to turn on/off

Because a system administrator may spend lots of time iterating between executing command lines, writing those command lines into a shell script, and debugging said command lines.

Predictability is vitally important, especially if it's 3am and I'm trying to figure out why this shell script that had been working suddenly breaks in the script but works fine when executed by hand.

There's a reason code deltas for interactive operation were few and far between, and mostly invisible (eg, stripping color). When the utility didn't handle it, "standard" shell aliases (eg, "rm -i") could do the trick. Plenty of utilities in the last decade seem to think that I *always* want a huge, complex TUI for getting data, or think it knows better about how I want my output presented.

I have high confidence that grep in an interactive shell and grep in a shell script will do essentially the same thing (except for things that apply to all scripting, like proper handling of globs). I have little confidence a "modern" grep replacement won't have some weird hacky behavior that was put in because it looks cool and will cost me an hour of hair-pulling early in the am trying to track it down.

Rewriting the GNU Coreutils in Rust

Posted Jun 12, 2021 3:17 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

> Because a system administrator may spend lots of time iterating between executing command lines, writing those command lines into a shell script, and debugging said command lines.

I suspect from your response that you haven't used rg.

I have written plenty of scripts using both grep and occasionally rg as part of my work and I consider either behavior very much predictable and documented. If anything grep behavior is slightly less predictable when switching between say a laptop running Mac and a server running Linux because of the variations between grep.

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 13:22 UTC (Wed) by taladar (subscriber, #68407) [Link]

It depends on the type of enhancement.

More readable documentation and --help output might certainly be welcome, as might be some optional verbose/debug flags that give more readable error messages that explain why something isn't working.

On the other hand if we are talking about making things incompatible for scripting use I would certainly agree that that is the wrong type of "enhancement".

Rewriting the GNU Coreutils in Rust

Posted Jun 9, 2021 14:24 UTC (Wed) by mbunkus (subscriber, #87248) [Link]

ripgrep (much like all the other enhanced searcher programs out there such as Ack or Silver Surfer) isn't supposed to be a drop-in replacement for grep. It's supposed to do something similar, but in a better way and way faster. Yeah, ideally it would replace grep, but not as a drop-in but as a better alternative.

This is a different goal than rewriting programs or collections such as Coreutils.

Rewriting the GNU Coreutils in Rust

Posted Jun 10, 2021 13:50 UTC (Thu) by moltonel (subscriber, #45207) [Link]

Being usable as a drop-in replacement of coreutils is a stated critical requirement of uutils, but I sure hope that uutils will eventually gain user-facing well-thoutght-out features as well.

In my youthful days I'd read all the coreutils changelogs and get excited about new features. Yesterday's bells are today's must-have. A lot of the entries in http://www.gnu.org/software/coreutils/rejected_requests.html look really reasonable and useful to me.

Rewriting the GNU Coreutils in Rust

Posted Jun 10, 2021 19:33 UTC (Thu) by rgmoore (✭ supporter ✭, #75) [Link] (1 responses)

I disagree mildly. It's vital that a replacement for something important like coreutils be 100% compatible, but that should be limited to doing the same things when given commands that are valid for the old version. They should be free to add new features as long as they are only accessed in a way that wouldn't happen with the old software, e.g. by using currently unused command line switches. But new features should take a back seat to getting compatibility complete.

Rewriting the GNU Coreutils in Rust

Posted Jun 11, 2021 9:42 UTC (Fri) by james (guest, #1325) [Link]

I disagree mildly.

So do I!

It's vital that a replacement for something important like coreutils be 100% compatible...

Not that coreutils was ever 100% compatible unless POSIXLY_CORRECT was set. I see no reason why a reimplementation shouldn't be afforded the same luxury (GNULY_CORRECT?) if they identified unfortunate edge behaviour.

Having said that, the original Unix utilities were written by competent hackers. The GNU reimplementations were also written by competent hackers. There can't be many places where existing behaviour could be improved, but not in a backwards-compatible way, and where the compatibility break would be worth it.