A shell in Rust could be really cool [LWN.net]

A shell in Rust could be really cool

Posted Feb 12, 2025 15:25 UTC (Wed) by Cyberax (✭ supporter ✭, #52523)
In reply to: A shell in Rust could be really cool by koverstreet
Parent article: Rewriting essential Linux packages in Rust

Who "we"? Shell overuse is really a phenomenon seen in low-level packages, where people for some reason don't want to reach out to perl/python/whatever.

to post comments

A shell in Rust could be really cool

Posted Feb 12, 2025 16:10 UTC (Wed) by stijn (subscriber, #570) [Link]

In bioinformatics there are many pipelines where the units are processes/scripts/programs. They might be sequence aligners, statistics packages, clustering programs and a whole lot more. Typically these do not come as libraries attached to a language but as separate programs. These days they are usually orchestrated with tools such as Nextflow, Snakemake, Cromwell and others. There is still a fair bit of use for shell scripting when manipulating the outputs (which are usually big dataframes). It gets tiresome very quickly to write a python script with parameters that loads a large dataframe into memory when all you want to do is some stream data processing. The downside is the marshalling/unmarshalling of data. I wouldn't mind something like Apache Arrow formatted data that I could compose with unix pipes in a shell-like environment. It might exist, I haven't checked as there is more than just one obstacle.