Faces of Open Source

Faces of Open Source

It's sad to learn of your passing, but nice to learn about the person behind some wonderful tools I've used for so long. Thank you Lorinda! But in the writeup I'm struck by all the tools that I don't use and didn't know exist:

Lorinda had a hand in "typo", too, a Morris invention that found gross spelling mistakes by statistical analysis. Sorting the words of a document by the similarity of their trigrams to those in the rest of the document tended to bring typos to the front of the list. [...]

[...] It all began as a one-woman skunk-works project. Noticing the very slow progress in natuaral-language processing, she identified a useful subtask that could be carved out of the larger problem: identifying parts of speech. Using a vocabulary of function words (articles, pronouns, prepositions and conjunctions) and rules of inflection, she was able to classify parts of speech in running text with impressive accuracy.

When Rutgers professor William Vesterman proposed a style-assessing program, with measures such as the frequencies of adjectives, subordinate clauses, or compound sentences, Lorinda was able to harness her "parts" program to implement the idea in a couple of weeks. Subsequently Nina MacDonald, with Lorinda's support, incorporated it into a larger suite that checked and made suggestions about other stylistic issues such as cliches, malapropisms, and redundancy.

Another aspect of text processing that Lorinda addressed was topic identification. Terms (often word pairs) that occur with abnormal frequency are likely to describe the topic at hand. She used this idea to construct first drafts of indexes. One in-house application was to the Unix manual, which up until that time had only a table of contents, but no index. [...]

I am missing these kinds of tools. How did we end up without these being common utilities like "dc" and "bc"? I would love to be able to apply these kinds of tools to text files on the command-line, or inside of vim.