LWN: Comments on "Python exception groups"

Python exception groups

mathstuf — Tue, 16 Mar 2021 23:45:50 +0000

Generics are on their way, so that's not likely to stay true for long. Modules were certainly some infrastructural changes (though admittedly not a *language* change).

Python exception groups

jazzy — Tue, 16 Mar 2021 22:17:48 +0000

Go (Golang) has not changed much since Go 1 was ”frozen” almost ten years ago.

Python exception groups

milesrout — Sun, 14 Mar 2021 03:39:47 +0000

Another way to put it is that every sublanguage tends towards the complexity of a full language over time when allowed to. Languages with pattern matching quickly realise that the pattern matching language needs to grow to become as complex as the language describing the data it is matching. The language for pattern matching on exceptions needs to reflect the complexity of the exceptions being matched, and the exceptions themselves will grow over time to reflect the complexity of the code generating them. Code nested recursively naturally leads to exceptions nested recursively (exception chaining). Code executed in parallel naturally leads to groups of parallel exceptions (exception groups).

In an extensible language like Common Lisp, these features can be implemented in a library. Of course, these sorts of features aren't trivial to implement correctly, but they _can_ be implemented correctly, and you don't need to modify the language specification and release a new compiler/interpreter to do so.

The same thing happened with async/await: what was once a relatively simple feature quickly gained its own parallel set of control flow in the form of async for/async with/await f()/etc. Async might as well be completely orthogonal to normal programming concerns like branching, looping, calling functions, etc. But it was designed in such a way as to make it impossible to just use standard constructs. The inevitable result was that the feature grew its own parallel set of constructs.

Python exception groups

tchernobog — Fri, 12 Mar 2021 17:32:15 +0000

I start to postulate a rule for programming language evolution: given enough time, every language will tend asymptotically to C++.

When more and more syntax is added, there is a point that you need a 1600 pages book to cover it ("Learning Python", anyone?).
Okay, fair enough, a lot is not syntax but standard library. But still...

We saw this for Java, we saw this for Ruby, Javascript too.

I see these additions as good, but I'd have liked a bit more honesty and foresight in retrospect. Instead of wanting to "keep it simple" for the first few language releases or so, and then retrofitting object orientation and well-typedness into it to cover those use cases that could have been covered from day one (and breaking the world with python 3), just plan for them from the beginning. Or having Java generics built upon type erasure. And so on. Just recognize that programmers have complex needs, build on the shoulders of giants to avoid the same mistakes, and plan accordingly, instead of hiding the head in the sand.

Python exception groups

LtWorf — Fri, 12 Mar 2021 07:21:19 +0000

That's basically how it works now.

The actual exception is a TypedloadValueError that has a field with a list of exceptions (which might have their own exceptions).

So I guess this is meant more for a map() or similar.

Python exception groups

NYKevin — Thu, 11 Mar 2021 22:53:51 +0000

> Plus, if a type being loaded has nested objects that use Union at multiple levels, can the except* statement deal with a tree of exceptions?

Yes, no, sort of. It's complicated.

Yes: You can raise a tree of exceptions by raising something like ExceptionGroup([FooError(), ExceptionGroup([BarError(), ...]), ...]) and so on, and the same error can appear in multiple different branches, or even multiple times in the same branch. All of those exceptions will preserve their original tracebacks and other contextual information. PEP 654 explicitly calls that out as a supported use case, and makes cogent arguments for why it should be supported. Don't expect this to go away unless the proposal gets axed altogether.

No/sort of: If someone writes, say, except *(FooError, BarError) as err, then they will get an ExceptionGroup which has been filtered to contain all instances of FooError and BarError, preserving the original tree structure, but with non-matching exceptions (and the resulting empty branches) deleted. They are then expected to consume that tree by hand; if you want to handle each exception individually, you need to write the tree traversal code yourself, or use .subgroup() on a callback function with side effects, which is honestly kind of horrifying.

The expectation, so far as I can tell, is that this will be used for top-level "can't do any meaningful recovery" situations, where you're basically just going to log it, maybe do some minimal cleanup, and then return to the main event loop (or whatever you have instead of a "main event loop"). If you actually want to take these things apart and do nontrivial recovery, then the PEP assumes you will catch those errors before they get coalesced into groups in the first place (for example, if you're throwing a group out of asyncio.gather(), then the individual coroutines should handle recoverable errors themselves, so that asyncio.gather() never even sees them).

Python exception groups

iabervon — Thu, 11 Mar 2021 22:37:21 +0000

That feels to me like Union should give a ValueError whose cause is the group of ValueErrors from various types, rather than raising the group. It's not either an int or a float because: (a) this exception indicates it's not an int and (b) this exception indicates it's not a float.

I think raising a group would make sense for a function that loads multiple values, and the group would have one exception per thing that should be fixed. When the Union fails, the fix isn't to make the value an int and also make it a float; you'd only want to make it one or the other.

Python exception groups

Funcan — Thu, 11 Mar 2021 15:41:28 +0000

The solution to backward-compatable syntax issues is python4, right?

Python exception groups

LtWorf — Thu, 11 Mar 2021 07:40:49 +0000

I can see it being useful for my typedload library, that when failing to load a Union type raises an exception that contains all the exceptions happened in the various attempts to use the types in the Union.

However this would complicate things quite a bit because

load('a', int)

and

load('a', Union[float, int])

would no longer both raise a ValueError.

Plus, if a type being loaded has nested objects that use Union at multiple levels, can the except* statement deal with a tree of exceptions?