Finally continuing the discussion over continue in finally

In 2019, the Python community had a lengthy discussion about changing the rules (that some find counterintuitive) on using break, continue, or return statements in finally blocks. These are all ways of jumping out of a finally block, which can interrupt the handling of a raised exception. At the time, the Python developers chose not to change things, because the consensus was that the existing behavior was not a problem. Now, after a report put together by Irit Katriel, the project is once again considering changing the language.

Like several other languages, Python has a try statement that allows catching exceptions. The optional finally block of a try statement allows the programmer to write code that should always run, regardless of whether an exception occurred in the try statement. This facility is frequently used to ensure a resource is always cleaned up, even if an exception is thrown. The Python documentation clearly describes what happens when an exception is thrown and a finally block that includes a control-flow statement executes: "If the finally clause executes a return, break or continue statement, the saved exception is discarded". But some people see this behavior as counterintuitive.

When Batuhan Taskaya originally proposed PEP 601 ("Forbid return/break/continue breaking out of finally"), forbidding control-flow statements in finally blocks, they called the behavior "not at all obvious". At the time, a handful of other Python developers agreed. Brandt Bucher shared an example of one of the kinds of potentially surprising code that is currently allowed:

    >>> def f() -> bool:
    ...     while True:
    ...         try:
    ...             return True
    ...         finally:
    ...             break
    ...     return False
    ...
    >>> f()
    False

This code would normally exit from f() upon reaching the return statement — except that the finally block runs and breaks out of the enclosing loop, letting execution continue into the rest of the function. Therefore the value returned from inside the try block is lost, and the function returns False. A similar example that throws an exception from inside the try block would show that the exception is lost as well.

Other people did not think that this behavior was a problem. Paul Moore called the motivation for the proposal weak, and asked whether there were any real-world examples of people using control-flow statements inside a finally block incorrectly. Serhiy Storchaka supplied two examples that they thought were bugs in the standard library — but Guido van Rossum indicated that only one of those was actually a bug.

Ultimately, the Python steering council voted unanimously not to adopt PEP 601, but suggested that the rule should instead be added to PEP 8 ("Style Guide for Python Code") as a matter of good coding practice.

Revisiting the topic

In November 2024, Katriel reopened the discussion by posting a link to a report that examines the 8,000 most popular Python packages. Katriel wrote a script to examine nearly 121 million lines of Python code. The script found 203 cases where there were control-flow statements in a finally block that could cause the block to be exited in a way that could potentially suppress exceptions. She examined the examples and determined that 46 were correct, 149 were clearly incorrect and 8 were difficult to classify. Notably, almost all of the correct uses appear in tests for linters that check that the linter can identify when a control-flow statement is used like this.

Katriel analyzed the incorrect cases in more detail, noting that 27 of them had actually been fixed in the development branch of the relevant library. She reported the remaining cases to the developers of affected packages. Of the 73 packages where she opened issues, only two indicated that the code was working as intended. With these facts in hand, Katriel and Alyssa Coghlan proposed PEP 765 ("Disallow return/break/continue that exit a finally block").

The new PEP would issue a SyntaxWarning for any use of a return, break, or continue statement that would exit a finally block. The PEP would essentially call for the Python interpreter to warn about any means of exiting a finally block other than letting control flow reach the end of the block normally. This would be a syntax warning so that it shows up when Python code is first parsed, not when the code is actually executed. This means that the warning would appear to library authors, but not to most library users, who usually use pre-compiled Python bytecode. These users might see the warning at installation time, but not while running their Python scripts.

Van Rossum called the report "an excellent piece of research", and suggested that "perhaps we should do more PEPs based on such investigations". He was, however, "personally sad to see this syntactic corner case disappear". He believes that having control-flow statements within finally blocks which work as they do now is an example of how Python has many small features that can be seamlessly composed together.

Katriel responded that there are already similar exceptions to the rules in Python. Specifically, except* clauses already disallow control-flow statements. The overall response to the proposal was fairly positive, however. Several people chimed in to express their support, and Tim Peters, recently back from his three-month suspension, went so far as to call the existing behavior "a bug magnet on the face of it".

Not everyone was as convinced by Katriel's research. Robin Becker thought that the existing behavior was obvious, and said in a later message that there was nothing special about finally blocks suppressing exceptions. Peters disagreed, quoting the Zen of Python, which he authored:

Errors should never pass silently.
Unless explicitly silenced.

That led Steve Dower to propose an alternative: "why are we forbidding the construct rather than just making exceptions re-raise on exit from finally, regardless of how we leave the block?" He did later clarify that he wasn't really arguing for that alternative, but just wanted an explanation of why the PEP didn't seem to consider the possibility.

Katriel said that Dower's proposal would introduce backward-compatibility problems; code that was previously correct would raise unexpected exceptions. Dower wasn't satisfied with that explanation, saying that adding a syntax warning would also disrupt existing code. He eventually asked: "So is it better if the error is still raised? Or better if everyone who uses your module gets a warning (that breaks their own tests/users) whether that error ever occurs or not?"

Katriel thought the latter was preferable, because it is easier to debug a syntax warning that points to the exact line causing the issue, rather than a random no-longer-suppressed exception that may occur without warning. In the end, neither Katriel nor Dower was able to convince the other, but Katriel did add a section to the PEP summarizing Dower's proposed alternative and why she did not think it was a good idea. Ultimately, Dower wanted a solution that "minimises breakage", since he is going to have to justify any changes to his users, who are already "unjustifiably nervous" about updating to Python 3.12.

Daniël van Noord asked Dower what it would take for him to support the PEP. Dower replied that Python has "many millions of users and billions of lines of code" — so adding a new warning is going to impact people, no matter how rare the circumstances. He won't support a change unless that can somehow be avoided.

Katriel pointed out that the choice to issue a warning instead of an error was deliberate — existing Python code would continue to work, just possibly with a warning. That led to an extended discussion about under which circumstances the warning would be shown, whether that was too disruptive, and various related concerns. Several people said that they thought putting a control-flow statement inside a finally block should be an error, instead of a warning, despite that being more backward incompatible than Katriel's proposed change.

Ultimately, Katriel submitted the PEP to the Python Steering Council. So regardless of the arguments in favor (the fact that the feature is almost never used correctly) and the arguments against (any amount of backward incompatibility is a pain), it will lie with the council to decide whether the PEP is accepted. The council faces yearly elections — and being so close to the end of its term, it decided to leave the decision to next year's council. (The election for which ends on December 9.) Van Rossum, near the end of the discussion, said that he has mixed feelings about the proposal. "I honestly don't know what I would have done when I was BDFL [benevolent dictator for life]." What the steering council will decide remains to be seen, but it seems clear that either choice is going to make some people unhappy.