Alternative syntax for Python's lambda

The Python lambda keyword, which can be used to create small, anonymous functions, comes from the world of functional programming, but is perhaps not the most beloved of Python features. In part, that may be because it is somewhat clunky to use, especially in comparison to the shorthand notation offered by other languages, such as JavaScript. That has led to some discussions on possible changes to lambda in Python mailing lists since mid-February.

Background

This is far from the first time lambda has featured in discussions in the Python community; the search for a more compact and, perhaps, more capable, version has generally been the focus of the previous discussions. Even the name "lambda" is somewhat obscure; it comes from the Greek letter "λ", by way of the lambda calculus formal system of mathematical logic. In Python, lambda expressions can be used anywhere a function object is needed; for example:

    >>> (lambda x: x * 7)(6)
    42

In that example, we define an anonymous function that "multiplies" its argument by seven, then call it with an argument of six. But, of course, Python has overloaded the multiplication operation, so:

    >>> (lambda x: x * 7)('ba')
    'bababababababa'

Meanwhile, lambda can be used in place of def, though it may be of dubious value to do so:

    >>> incfunc = lambda x: x + 1
    >>> incfunc(37)
    38

    # not much different from:
    >>> def incfunc(x):
    ...     return x + 1
    ... 
    >>> incfunc(37)
    38

Lambdas are restricted to a single Python expression; they cannot contain statements, nor can they have type annotations. Some of the value of the feature can be seen in combination with some of the other functional-flavored parts of Python. For example:

    >>> list(filter(lambda x: x % 2 == 1, range(17)))
    [1, 3, 5, 7, 9, 11, 13, 15]

There we use the filter() function to create an iterator, then use list() to produce a list of the first eight odd numbers, with a lambda providing the test function. Over time, though, other Python features supplanted these uses for lambda expressions; the same result as above can be accomplished using a list comprehension:

    >>> [ x for x in range(17) if x % 2 == 1 ]
    [1, 3, 5, 7, 9, 11, 13, 15]

The most obvious use of lambda may be as key parameters to list.sort(), sorted(), max()/min(), and the like. That parameter can be used to extract a particular attribute or piece of each object in order to sort based on that:

    >>> sorted([ ('a', 37), ('b', 23), ('c', 73) ], key=lambda x: x[1])
    [('b', 23), ('a', 37), ('c', 73)]

Arrow operators?

A thread about "mutable defaults" on the python-list mailing list made its way over to python-ideas when James Pic posted a suggestion for an alternate lambda syntax. His ultra-terse syntax did not strike a chord with the others participating in the thread, but it led "Random832" to suggest looking at the "->" or "=>" arrow operators used by other languages, such as C#, Java, and JavaScript.

It's worth noting that all three of these are later additions to their respective languages, and they all have earlier, more difficult, ways of writing nested functions within expressions. Their designers saw the benefit of an easy lambda syntax, why don't we?

Former Python benevolent dictator Guido van Rossum agreed: "I'd prefer the JavaScript solution, since -> already has a different meaning in Python return *type*. We could use -> to simplify typing.Callable, and => to simplify lambda." He also answered the question by suggesting that the "endless search for for multi-line lambdas" may have derailed any kind of lambda-simplification effort. Van Rossum declared multi-line lambda expressions "un-Pythonic" in a blog post back in 2006, but in the thread he said that it is not too late to add some kind of simplified lambda.

Steven D'Aprano was concerned about having two separate "arrow" operators. "That will lead to constant confusion for people who can't remember which arrow operator to use." He said that the "->" symbol that is already in use for the annotation of return types could also be used to define anonymous functions. It is also a well-known idiom:

There are plenty of popular and influential languages that use the single line arrow -> such as Maple, Haskell, Julia, CoffeeScript, Erlang and Groovy, to say nothing of numerous lesser known and obscure languages.

He also posted a lengthy analysis of how both uses of the single-line arrow could coexist, though it turned out that there is a parsing ambiguity if type annotations are allowed in "arrow functions" (i.e. those defined using the single-line arrow). One can perhaps be forgiven for thinking that the example he gives is not entirely Pythonic, however:

    (values:List[float], arg:int=0 -> Type) -> expression

[...] In case anyone is still having trouble parsing that:

    # lambda version
    lambda values, arg=0: expression

    # becomes arrow function
    (values, arg=0) -> expression

    # add parameter annotations
    (values:List[float], arg:int=0) -> expression

    # and the return type of the function body (expression)
    (values:List[float], arg:int=0 -> T) -> expression

The obscurity of the name "lambda" also came up. Brendan Barnwell lamented the choice of the name as confusing, while Ned Batchelder called it "opaque":

People who know the background of lambda can easily understand using a different word.  People who don't know the background are presented with a "magic word" with no meaning.  That's not good UI.

But, as D'Aprano pointed out, it is far from the only jargon that Python (and other language) programmers will need to pick up:

[It's] no more "magic" than tuple, deque, iterator, coroutine, ordinal, modulus, etc, not to mention those ordinary English words with specialised jargon meanings like float, tab, zip, thread, key, promise, trampoline, tree, hash etc.

While Paul Sokolovsky is a big fan of the lambda keyword, he does think that differentiating between the two uses of the arrow notation (the existing use for return types and a possible future use for defining short functions) is important. He thinks it would be better to use two different arrows; for defining functions, he is in favor of the double-line arrow (=>) instead.

Proof of concept

To demonstrate the idea, he came up with some proof-of-concept code that implements the double-line arrow as a lambda replacement. Here are some examples that he gave:

    >>> f = (a, b) => a + b    # not actual Python syntax
    >>> print(f(1, 2))
    3
    >>> print(list(map((x) => x * 2, [1, 2, 3, 4])))    # nor this
    [2, 4, 6, 8]

Cade Brown did not like the double-line arrow, on general principles, but Sokolovsky reiterated his belief that the two uses of arrows should be distinct; he also thinks that using the same symbol as JavaScript has some advantages. D'Aprano, however, is not convinced that following JavaScript's notation is necessarily the right thing for Python, nor does he think there is a need to separate the two uses of arrows. As might be guessed, others disagree; it was, to a certain extent, a bikeshedding opportunity, after all.

For his part, Van Rossum was not really opposed to using the single-line arrow for function definitions if there were no technical barriers to overlapping the two uses. No one seemed to think that allowing type annotations for lambda functions, which are generally quite self-contained, was truly needed. On the other hand, both David Mertz and Ricky Teachey were opposed to adding new syntax to handle lambdas, though Teachey thought it would make more sense if it could be used for both unnamed and named functions:

But if we could expand the proposal to allow both anonymous and named functions, that would seem like a fantastic idea to me.

Anonymous function syntax:

(x,y)->x+y

Named function syntax:

f(x,y)->x+y

That was something of a step too far for Van Rossum, though. There is already a perfectly good way to create named functions, he said:

Proposals like this always baffle me. Python already has both anonymous and named functions. They are spelled with 'lambda' and 'def', respectively. What good would it do us to create an alternate spelling for 'def'?

[...] I can sympathize with trying to get a replacement for lambda, because many other languages have jumped on the arrow bandwagon, and few Python first-time programmers have enough of a CS background to recognize the significance of the word lambda. But named functions? Why??

Greg Ewing hypothesized that it simply comes from the desire for brevity: "In the situations where it's appropriate to use a lambda, you want something very compact, and 'lambda' is a rather long and unwieldy thing to have stuck in there." D'Aprano added that it may come from mathematical notation for defining functions, which is not necessarily a good match with Python's def:

So there is definitely some aesthetic advantage to the arrow if you're used to maths notation, and if Python had it, I'd use it.

But it doesn't scale up to multi-statement functions, and doesn't bring any new functionality into the language, so I'm not convinced that its worth adding as a mere synonym for def or lambda or both.

While there was some support for the idea, and Sokolovsky is particularly enthusiastic about it, so far there have been no plans mentioned for a Python Enhancement Proposal (PEP). Adopting an arrow syntax for lambda expressions may just be one of those topics that pops up occasionally in Python circles; maybe, like the recurring request for a Python "switch", it will evolve into something that gets added to the language (as with pattern matching). On the other hand, lambda may be one of those corners of the language that is not used frequently enough to be worth changing. Only time will tell.

a = def (x: int, y: bool): return x if z else 0

x = foo(1,
    2)

x = foo ( 1 , 2 ) NEWLINE.

x = foo ( 1 , NEWLINE INDENT 2 ) NEWLINE DEDENT.

x = foo ( 1 , NEWLINE1 INDENT1 2 DEDENT1 ) NEWLINE

x = foo(if True:
          2
        else:
          3)

x = foo ( if True : NL1 IN1 2 NL1 DE1 else : NL1 ID1 3 NL1 DE1 ) NL1

x = foo ( [arbitrary tokens] ) NL

if True : NL IN 2 NL DE else : NL ID 3 NL DE

if True: # if True : NL
    2 # IN 2 NL
else: # DE else : NL
    3 # IN 3 NL
# DE

<Ctrl+k>*l

:help digraphs

int& (*fpi)(int*) = [](auto* a)->auto& { return *a; };

>>> import ast
>>> print(ast.dump(ast.parse('z |x, y| x + y - 2', mode='eval'), indent=4))
Expression(
    body=Tuple(
        elts=[
            BinOp(
                left=Name(id='z', ctx=Load()),
                op=BitOr(),
                right=Name(id='x', ctx=Load())),
            BinOp(
                left=Name(id='y', ctx=Load()),
                op=BitOr(),
                right=BinOp(
                    left=BinOp(
                        left=Name(id='x', ctx=Load()),
                        op=Add(),
                        right=Name(id='y', ctx=Load())),
                    op=Sub(),
                    right=Constant(value=2)))],
        ctx=Load()))

>>> map(lambda x: x + 1, range(5))

>>> range(5).map(lambda x: x + 1)