Development [LWN.net]

Improving writing with proselint

By Jonathan Corbet
March 9, 2016

Here at LWN, we put a lot of effort into the quality of our writing, so our attention is naturally drawn to tools that might help the writing process. Recently, links to the proselint tool made the rounds, so it made sense to take a look. What we found was a tool that shows some potential, but quite a bit of work will be needed to get it to where it aspires to be.

What is this project's goal? The word from the front page of its web site is:

With time, proselint will become fast, precise, exhaustive, and authoritative. Eventually, it will be unthinkable to write without it by your side.

Installation of proselint is simple; like many Python utilities, it seems to have bypassed the ordinary distribution channels and can be installed directly with pip. It is a command-line tool; running it on a text file will, by default, result in any warnings being written to the standard output. This is clearly not the expected use, though; it is more likely that proselint will be integrated into other tools that can present its advice in a more friendly manner. To help make this integration easy, proselint can be asked to format its output as JSON rather than plain text.

As a result, it is easy to use proselint as part of the Emacs Flycheck mode, for example; it is a simple matter of adding a few lines to the .emacs file. Once a buffer has been put into flycheck-mode, the text therein will be occasionally checked for problematic English, and the offending text will be highlighted; hovering over that text with the pointer will yield the associated message.

Your editor went searching for some high-quality text to test this tool against. After pondering the classics from Project Gutenberg, the GNU Manifesto, and Trump speech transcripts, the search settled a little closer to home — the kernel's documentation directory. Playing with the Flycheck integration in Emacs quickly yielded results like this:

[Emacs with proselint]

That is good advice indeed, but, in truth, most of what comes out of proselint is less helpful. By far the bulk of the warnings are either (1) using both one and two spaces after a period, or (2) failure to use curly quotes. Neither warning is all that helpful for a writer who was hoping for, as the web site promises, "the world’s greatest writers and editors by your side, where they whisper suggestions on how to improve your prose". Instead, it feels a bit more like one of the world's worst elementary-school teachers criticizing you in front of the entire class about irrelevant details.

But, then, proselint is a new tool, so there is hope for the future. The success of such a tool depends on two things: the rules it uses to analyze text, and the flexibility of the engine that interprets those rules. The rules themselves are written as Python code; here, for example, is the core of the rule that generated the message shown above:

    @memoize
    def check_very_damn(text):
        err = "twain.damn"
        msg = ("Substitute 'damn' every time you're "
               "inclined to write 'very;' your editor will delete it "
               "and the writing will be just as it should be.")
        regex = "very"

        return existence_check(text, [regex], err, msg, max_errors=1)

There are various other functions for performing checks on text, but they all have the same core nature: they are applying regular expressions to the text as a way of triggering rules. That makes rules relatively easy to write, but also limits how smart the tool can actually be. It can readily pick out specific words or combinations of words, but will be hard pressed to offer advice that requires a deeper understanding of the subject text.

A look at the code shows that some thought has been put into making the core engine smarter; there are stubs for functions that can extract specific amounts of context or try to determine what the topic of a range of text is, for example. But stubs is all they are at this point; a lot of work will need to be filled in before this tool can come close to the ambitious goals its developers have set.

One can only wish them luck. There is a lot of advice for writers out there, but it can be hard to even read it all, much less keep it all in mind when actually writing or editing a piece of text. If it were possible to distill a significant part of that advice into a form where it could be applied automatically, the result might be better writing all around. That is a result that one could be damn happy about.

Comments (5 posted)

Brief items

Quotes of the week

Given the readability issues caused by overuse of lambda, there's a school of thought that sees "I like lambda functions, but don't like the lambda keyword, so I avoid using lambda expressions in Python" as a desirable characteristic of the status quo.

— Nick Coghlan

In this happy case the source code for ZFS is available, but what about the future, when we aren't so lucky and someone asserts in court that the "you know, the software license was really about the spirit of sharing and that means we are allowed to use it -- and not be held to the pesky details as written in the license".

— Tom Marble is skeptical of the Software Freedom Law Center's analysis of ZFS.

Volunteers don't generally donate their time to solve problems they don't have (and employers don't typically donate their employees' time to solve problems they or their customers don't have), which means "I don't personally have that problem, so I deny its very existence" is almost never a compelling line of argument in a community-driven open source project.

— Nick Coghlan

Comments (3 posted)

Mozilla unveils Firefox OS based IoT projects (LinuxGizmos.com)

Over at LinuxGizmos, Eric Brown notes some new "Internet of Things" (IoT) projects from Mozilla that were described in a recent blog post by Ari Jaaksi, Mozilla Senior VP for Connected Devices. "The first projects include a Project Start Home framework for a home automation system, as well as a Project Link personal user agent and Vaani voice interface that would work within such a framework. Finally, there’s a crowdsourced Project SensorWeb for tracking air pollution. Interestingly, the term “Firefox OS” is not used in the latest announcement, despite the reference to Firefox OS Connected Devices in the previous post. Still, all the projects appear to use Firefox OS or Mozilla’s underlying Boot to Gecko (b2g) codebase. Mozilla is seeking testers, developers, and advisers, for all these open source projects."

Comments (37 posted)

ownCloud 9.0 released

Version 9.0 of ownCloud has been released, with many performance improvements, bug fixes, cleanup, and new features. "[Full Federation] is one of the main goals of ownCloud, since the beginning, to enable everyone to run their own server but still collaborate and share with others. Sharing between different ownCloud servers is possible for a while now. But this is easier than ever with cross-server user name auto complete, trusted servers and more. Once you’ve shared with another ownCloud server, it will be added as a trusted server, exchanging user names. This will enable ownCloud to auto complete names from users of all shared servers. Admins have control over these features, so they can add trusted servers manually and disable the automatic addition."

Comments (6 posted)

LLVM 3.8 released

Version 3.8 of the LLVM compiler suite has been released. "This release contains the work of the LLVM community over the past six months: deprecated autoconf build, shrink-wrapping on by default, overhauled MSVC-compatible exception handling, updated Kaleidoscope tutorial, emutls, OpenMP supported by default, as well as improved optimizations, many bug fixes, and more." See the LLVM release notes and the Clang release notes for lots of details.

Full Story (comments: none)

Firefox 45.0 released

Firefox 45.0 has been released. This release features instant browser tab sharing through Hello, Tabs synced via Firefox Accounts from other devices are now shown in dropdown area of Awesome Bar when searching, Synced Tabs button in button bar, introduces a new preference (network.dns.blockDotOnion) to allow blocking .onion at the DNS level, and more. See the release notes for details.

Comments (15 posted)

Newsletters and articles

Development newsletters from the past week

Comments (none posted)

Borg, Omega, and Kubernetes (ACM Queue)

Five Google developers share the lessons from ten years of container development in this ACM Queue article. "To cope with these kinds of requirements, configuration-management systems tend to invent a domain-specific configuration language that (eventually) becomes Turing complete, starting from the desire to perform computation on the data in the configuration (e.g., to adjust the amount of memory to give a server as a function of the number of shards in the service). The result is the kind of inscrutable 'configuration is code' that people were trying to avoid by eliminating hard-coded parameters in the application's source code. It doesn't reduce operational complexity or make the configurations easier to debug or change; it just moves the computations from a real programming language to a domain-specific one, which typically has weaker development tools (e.g., debuggers, unit test frameworks, etc)."

Comments (25 posted)

Page editor: Nathan Willis
Next page: Announcements>>