The return of None-aware operators for Python

Posted Jan 5, 2024 20:44 UTC (Fri) by NYKevin (subscriber, #129325)
In reply to: The return of None-aware operators for Python by marcH
Parent article: The return of None-aware operators for Python

I would argue that Rust's None-aware operators are much better than what is proposed for Python, simply because they cause an implicit return rather than silently propagating a None. The latter is reasonable if you have the full machinery of monads at your disposal (so Haskell is correct to implement Maybe-bind that way), but Python does not (and will never) have monads.

Because of Python's laissez-faire multi-paradigm attitude, it's actually quite difficult to design a good implementation of None-aware operators. You can't really use the Rust solution, because Python is dynamically-typed and likes to signal errors with exceptions rather than sentinel values (i.e. you can't reasonably define it to propagate Result<V, E> or the like, since Result is not even a thing in Python). But the TypeError or ValueError that you tend to get from None is usually the wrong exception to throw, and propagating None as if it was NaN will make it difficult to get a traceback. The idiomatic behavior, in some cases, is to eagerly check for None and raise an exception.

Maybe this syntax could work?

x = y ?? raise FooError('y should not be None.')

But that is going to be problematic. Raise is a statement, not an expression, so you'd need to make a special case to allow it in this one context, or you'd need to convert it into an expression. And then people will also want to write x = y ?? z, so you need to allow for that as well.

I have no idea how this is supposed to be extended for ?. and ?[], because where are you supposed to put the raise?

The return of None-aware operators for Python

Posted Jan 7, 2024 14:21 UTC (Sun) by cpitrat (subscriber, #116459) [Link] (13 responses)

> x = y ?? raise FooError('y should not be None.')

May I interest you in assert?

The return of None-aware operators for Python

Posted Jan 8, 2024 22:06 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (12 responses)

assert is not properly used for any purpose other than as a "live comment" (i.e. a comment that actually gets executed). This is because the -O flag disables asserts, so the implication is that your code is expected to be correct even when asserts are not run. To be more explicit about this, a proper assert should be describing an invariant which the surrounding code is intended to guarantee (and which it actually does guarantee, assuming it has been written correctly). Since the invariant is already upheld by the surrounding code, you don't have to check it for correctness, so the assert is just for documentation purposes (to inform the next person who reads the code that the invariant exists and is important).

(Also, you can't specify the exception type, but that's small potatoes in comparison.)

The return of None-aware operators for Python

Posted Jan 8, 2024 22:13 UTC (Mon) by khim (subscriber, #9252) [Link] (4 responses)

> assert is not properly used for any purpose other than as a "live comment" (i.e. a comment that actually gets executed). This is because the -O flag disables asserts, so the implication is that your code is expected to be correct even when asserts are not run.

I have just tested and that doesn't happen. What version of python are you using??? I have never observed that effect in Python, but there are many implementations, maybe one of them does that, but for me it's reason not to use it rather then change use of assert.

The return of None-aware operators for Python

Posted Jan 8, 2024 22:22 UTC (Mon) by mb (subscriber, #50428) [Link]

You don't pass -O

$ cat t.py
assert 2+2==5
$ python3.11 t.py
[...]
AssertionError
$ python3.11 -O t.py

The return of None-aware operators for Python

Posted Jan 8, 2024 23:19 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (1 responses)

This has been documented as standard behavior at https://docs.python.org/3/using/cmdline.html#cmdoption-O basically forever (I believe it was probably added at some point in Python 2.x and then carried over into 3). I do not know why the link you provide appears to show Python ignoring this flag, but it is likely a problem with either Godbolt or your use of it, because this is not new or exotic behavior.

The return of None-aware operators for Python

Posted Jan 8, 2024 23:26 UTC (Mon) by NYKevin (subscriber, #129325) [Link]

I'm mistaken. This has been standard behavior since version 1.5, released in 1998, according to this old documentation: https://docs.python.org/release/1.5/ref/ref-8.html#HEADIN...

As far as I can tell, 1.4 did not have an assert statement, so the assert statement has never been unconditional in any released version of Python.

The return of None-aware operators for Python

Posted Jan 8, 2024 23:40 UTC (Mon) by ABCD (subscriber, #53650) [Link]

It seems that godbolt.org doesn't pass the "compiler arguments" to the interpreter the way you expected. If you use the PYTHONOPTIMIZE=1 environment variable instead, you get the expected behavior: https://godbolt.org/z/z8Wo5qYT4

The return of None-aware operators for Python

Posted Jan 8, 2024 23:36 UTC (Mon) by marcH (subscriber, #57642) [Link] (1 responses)

> a "live comment" (i.e. a comment that actually gets executed)

> so the assert is just for documentation purposes (to inform the next person who reads the code that the invariant exists and is important).

Small contradiction here.

I didn't know about the -O flag and I've always "run" asserts and every time one is hit it is massively more useful than a comment!

> so the implication is that your code is expected to be correct even when asserts are not run.

Agreed.

The return of None-aware operators for Python

Posted Jan 8, 2024 23:37 UTC (Mon) by NYKevin (subscriber, #129325) [Link]

> Small contradiction here.

Sorry, my brain is smaller than yours, can you please elaborate?

The return of None-aware operators for Python

Posted Jan 9, 2024 11:28 UTC (Tue) by cpitrat (subscriber, #116459) [Link]

I'm not sure why I was sure it was possible to choose the exception type, not only the message. And I had forgotten about -O deactivating it.

But you can easily define your own raiseIfNone.

The return of None-aware operators for Python

Posted Jan 9, 2024 15:11 UTC (Tue) by atnot (subscriber, #124910) [Link] (3 responses)

There's another super annoying issue with python's assert and it's that because assert is just a single keyword that checks a bool, you get zero information from it beyond "AssertionError". Unlike the asserts in many other languages or ecosystem, you don't get any info about what assertion failed, what the left or right hand side of the comparison was or any sort of message describing the errot. If you're lucky, you can maybe get the source location from the stack trace but that's about it. It's just never worth using them.

The return of None-aware operators for Python

Posted Jan 9, 2024 15:28 UTC (Tue) by kleptog (subscriber, #1183) [Link]

assert has a second argument where you can give a more descriptive message, and include the arguments if you like. But you have to remember to do it.

The return of None-aware operators for Python

Posted Jan 9, 2024 19:13 UTC (Tue) by NYKevin (subscriber, #129325) [Link]

Those things are all provided in unittest, and in contexts other than unit testing, I never find myself reaching for such functionality in the first place.

The return of None-aware operators for Python

Posted Jan 10, 2024 16:28 UTC (Wed) by laarmen (subscriber, #63948) [Link]

pytest actually does some voodoo that supercharges Python standard assert, including displaying subexpressions. Granted, it won't help if your assertion fails during normal runtime as opposed to test runs, but there might be a way to reuse that particular bit outside of the test framework?