PyCon: Evangelizing Python

Python core developer Raymond Hettinger's PyCon 2013 keynote had elements of a revival meeting sermon, but it was also meant to spread the "religion" well beyond those inside the meeting tent. Hettinger specifically tasked attendees to use his "What makes Python awesome?" talk as a sales tool with management and other Python skeptics. While he may have used the word "awesome" a few too many times in the talk, Hettinger is clearly an excellent advocate of the language from a technical—not just cheerleading—perspective.

He started the talk by noting that he teaches "Python 140 characters at a time" on Twitter (@raymondh). He has been a core developer for twelve years, working on builtins, the standard library, and a few core language features. For the last year and a half, Hettinger has had a chance to "teach a lot of people Python". Teaching has given him a perspective on what is good and bad in Python.

Context for success

Python has a "context for success", he said, starting with its license. He and many others would never have heard of Python if it were not available under an open source license. It is also important for a "serious language" to have commercial distributions and the support that comes with those.

Python also has a "Zen", he said, which is also true of some other languages, like Ruby, but "C++ does not have Zen". Community is another area where Python excels. "C is a wonderful language", but it doesn't have a community, Hettinger said.

The PyPI repository for Python modules and packages is another important piece of the puzzle. Python also has a "killer app", in fact it has more than one. Zope, Django, and pandas are all killer apps, he said.

Windows support is another important attribute of Python. While many in the audience may be "Linux weenies" and look down on Windows users, most of the computers in the world are running Windows, so it is important for Python to run there too, he said. There are lots of Python books available, unlike some other languages. Hettinger is interested in Go, but there aren't many books on that language.

All of these attributes make up a context for success, and any language that has them is poised to succeed. But, he asked, why is he talking about the good points of Python at PyCon, where everyone there is likely to already know much of what he is saying? It is because attendees will often be in a position to recommend or defend Python. Hettinger's goal is for attendees to be able to articulate what is special about the language.

High-level qualities

The Python language itself has certain qualities that make it special, he said, starting with "ease of learning". He noted that David Beazley runs classes where students are able to write "amazing code" by the end of the second day. One of the exercises in those classes is to write a web log summarizing tool, which shows how quickly non-programmers can learn Python.

Python allows for a rapid development cycle as well. Hettinger used to work at a high-frequency trading company that could come up with a trading strategy in the morning and be using it by the afternoon because of Python. Though he was a good Java programmer, he could never get that kind of rapid turnaround using Java.

Readability and beauty in a language is important, he said, because it means that programmers will want to program in the language. Python programmers will write code on evenings and weekends, but "I never code C++ on the weekend" because it is "not fun, not beautiful". Python is both, he said.

The "batteries included" philosophy of Python, where the standard library is part of the language, is another important quality. Finally, one of Hettinger's favorite Python qualities is the protocols that it defines, such as the database and WSGI protocols. The database protocol means that you can swap out the underlying database system, switching to or from MySQL, Oracle, or PostgreSQL without changing the code to access the database. Once you know how to access one of them through Python, you know how to access them all.

As an example of the expressiveness and development speed of the language, Hettinger put up a slide with a short program. In a class he was teaching, someone asked how he would deduplicate a disk full of photos, and in five minutes he was able to come up with a fifteen-line program to do so. It is a real testament to the language that he could write that program live in class, but even more importantly, he can teach others to do the same. That one slide shows "a killer feature of the language: its productivity, and its beauty and brevity", he said.

But, there is a problem with that example. A similar slide could be created for Ruby or Perl, with roughly the same brevity. That would be evidence for the "all scripting languages are basically the same, just with different syntax" argument that he hears frequently from software executives. But all scripting languages are not the same, he said. That may have been true in 2000, but "we've grown since then"; there are lots of features that separate Python from the pack.

Winning language features

First up on Hettinger's list of "winning language features" is the required indentation of the language. It was an "audacious move" to make that choice for the language, but it contributes to the "clean, uncluttered" appearance of the code. He claimed that Python was the first to use indentation that way, though he later received a "Miranda warning" from an audience member as the Miranda language uses indentation and predates Python. People new to the language sometimes react negatively to the forced indentation, but it is a net positive. He showed some standard examples of where C programs can go wrong because the indentation doesn't actually match the control flow, which is impossible with Python. Python "never lies with its visual appearance", which is a winning feature, he said.

The iterator protocol is one of his favorite parts of the language. It is a "design pattern" that can be replicated in languages like Java and C++, but it is "effortless to use" in Python. The yield statement can create iterators everywhere. Because iterators are so deeply wired into the language, they can be used somewhat like Unix pipes. So the shell construct:

    cat filename | sort | uniq

can be expressed similarly in Python as:

    sorted(set(open(filename)))

This shows how iterators can be used as composable filters. In addition, Python has a level of expressiveness that is similar to SQL, so:

    sum(shares*price for symbol, shares, price in port)

will sum the number of shares times the price for all of the entries in port, which is much like the SQL equivalent:

    SELECT SUM(shares*price) FROM port;

Languages that don't have for loops that are as powerful as Python's cannot really compete, he said.

One of his favorite things to teach about Python are list comprehensions. The idea came from mathematical set building notation. They "profoundly improve the expressiveness of Python", Hettinger said. While list comprehensions might at first appear to violate the "don't put too much on one line" advice given to new programmers, it is actually a way to build up a higher-level view. The examples he gave can fairly easily be expressed as natural language sentences:

    [line.lower() for line in open(filename) if 'INFO' in line]

which creates a list of lower-cased lines that contain "INFO". The second seems directly derived from math notation:

    sum([x**3 for x in range(10000)])

which sums a list of the cubes of the first 10,000 integers (starting at zero). Since list comprehensions can generally be expressed as single sentences, it is reasonable to write them that way in Python.

The generators feature is a "masterpiece" that was stolen from the Icon language. Now that Python has generators, other languages are adding them as well. Generators allow Python functions to "freeze their execution" at a particular point and to resume execution later. Using generators makes both iterators and coroutines easier to implement in a "clean, readable, beautiful" form. Doing things that way is something that Python has "that others don't". His simple example showed some of the power of the feature:

    def pager(lines, pagelen=60):
        for lineno, line in enumerate(lines):
            yield line
            if lineno % pagelen == 0:
                yield FORMFEED

Generator expressions come from Hettinger's idea of combining generators and list comprehensions. Rather than requiring the creation of a list, generators can be used in expressions directly:

    sum(x**3 for x in range(10000))

From that idea, dictionary and set comprehensions are obvious extensions, he said. Generator expressions are one way to combat performance problems in Python code because they have a small memory footprint and are thus cache friendlier, he said.

But generators have a problem: they are a "bad date". Like a date that can only talk about themselves, generators can only talk, not listen. That led to the idea of two-way generators. Now generators can accept inputs in the form of send(), throw(), and close() methods. It is a feature that is unique to Python, he said, and is useful for implementing coroutines. It also helps "tame" some of the constructs in Twisted.

Decorators have an interesting history in Python. They don't really add new functionality that can't be done other ways, so the first few times they were proposed, they were turned down. But they kept being proposed, so Guido van Rossum (Python's benevolent dictator for life) used a tried and true strategy to make the problem go away: he said that if everyone could agree on a syntax for decorators, he would consider adding them. For the first time ever, the entire community came together and agreed on a syntax. It presented that agreement to Van Rossum, who agreed: "you shall have decorators, but not the syntax you asked for".

In retrospect, the resistance to decorators (from Van Rossum and other core developers) was wrong, Hettinger said, as they have turned out to be a "profound improvement to the language". He pointed to the lightweight web frameworks (naming itty, Flask, and CherryPy) as examples of how decorators can be used to create simple web applications. His one slide example of an itty-based web service uses decorators for routing. Each new service is usually a matter of adding three lines or so:

    @get('/freespace')
    def compute_free_disk_space(request):
        return subprocess.check_output('df')

The code above creates a page at /freespace that runs df and returns its output as a web page.

"Who's digging Python now?", he asked with a big grin, as he did in spots throughout the talk—to much applause. The features he had mentioned are reasons to pick Python over languages like Ruby, he said. While back in 2000, Python may have been the equivalent of other scripting languages, that has clearly changed.

There are even more features that make Python compelling, such as the with statement. Hettinger thinks that "context managers" using with may turn out to be as important to programming as was the invention of the subroutine. The with statement is a tool for making code "clean and beautiful" by setting up a temporary context where the entry and exit conditions can be ensured (e.g. files closed or locks unlocked) without sprinkling try/finally blocks all over. Other languages have a with, but they are not at all the same as Python's. The best uses for it have not yet been discovered, he said, and suggested that audience members "prove to the world that they are awesome", so that other languages get them.

The last winning feature that he mentioned was one that he initially didn't want to be added: abstract base classes. Van Rossum had done six months of programming in Java and "came back" with abstract base classes. Hettinger has come to embrace them. Abstract base classes help clarify what a sequence or a mapping actually is by defining the interfaces used by those types. They are also useful for mixing in different classes to better organize programs and modules.

There is something odd that comes with abstract base classes, though. Python uses "duck typing", which means that using isinstance() is frowned upon. In fact, novice Python programmers spend their first six months adding isinstance() calls, he said, and then spend the next six months taking them back out.

With abstract base classes, there is an addition to the usual "looks like a duck, walks like a duck, quacks like a duck" test because isinstance() can lie. That leads to code that uses: "well, it said it was a duck, and that's good enough for me", he said with a laugh. He thought this was "incredibly weird", but it turns out there are some good use cases for the feature. He showed an example of using the collections.Set abstract base class to create a complete list-based set just by implementing a few basic operations. All of the normal set operations (subset and superset tests, set equality, etc.) are simply inherited from the base class.

Hettinger wrapped up his keynote with a request: "Please take this presentation and go be me". He suggested that attendees present it to explain what Python has that other languages are missing, thus why Python should be chosen over a language like Ruby. He also had "one more thing" to note: the Python community has a lot of both "established superstars" as well as "rising young superstars". Other languages have "one or two stars", he said, but Python has many; just one more thing that Python has that other languages don't.

---

(Log in to post comments)

    import Db
    db = Db.connect(whatever)

    for x,y in db.select("select x,y from foo where bar=${baz}", baz=123):
        do_something(x,y)

def check_age(data):
    d = json.loads(data)
    if d['age'] < 18:
        print "You must be 18 or older to continue."
    else:
        print "OK!"

>>> check_age('{"age": 13}')
You must be 18 or older to continue.

>>> check_age('{"age": "13"}')
OK!

   if (foo)
       if (bar)
            baz();
   else
       quux();

{
some_lock_t lock(x);
x.do_something();
}

x.do_something_locked<some_lock_t>();

some_locking_scaffold(x).do_something();