The return of None-aware operators for Python
The return of None-aware operators for Python
Posted Jan 5, 2024 9:17 UTC (Fri) by marcH (subscriber, #57642)Parent article: The return of None-aware operators for Python
This is spot on; best and most accurate comment on the entire topic. Many times when writing code like "42 if x is None else x" I wondered "Wait, why can x be None in the first place?" and sometimes I did change the code so x could not be None any more.
Python's None is nowhere near as bad NULL in C/C++ (the so-called "billion dollar mistake") but it's still not great. So even if the new shortcuts were great, intuitive and very readable (which is debatable and debated), they would still only offer a very slightly lazier way to hide what is often another design issue.
Posted Jan 5, 2024 13:10 UTC (Fri)
by khim (subscriber, #9252)
[Link] (22 responses)
Huh? Python's Sure, but the only bulletproof solution is to drop dynamic typing and add mandatory types. And then it wouldn't be a Python anymore.
Posted Jan 5, 2024 16:11 UTC (Fri)
by marcH (subscriber, #57642)
[Link]
It's better because it gives you a clear stack trace instantly.
> Sure, but the only bulletproof solution is to drop dynamic typing and add mandatory types. And then it wouldn't be a Python anymore.
OK I should really have avoided this bad, pointless and distracting comparison with C...
Posted Jan 5, 2024 17:02 UTC (Fri)
by NYKevin (subscriber, #129325)
[Link] (3 responses)
This is a simplification. Python does provide support for static typing if you can be bothered to use it. If you run a FOSS project (or a closed-source project, for that matter), you can mandate that all code must pass a static type analysis (e.g. with mypy) before it may be merged. But that's up to you as a user of Python - the language won't force you to do it (just like it won't force you to write unit tests, run a linter, use a formatter, etc.).
Posted Jan 8, 2024 7:02 UTC (Mon)
by LtWorf (subscriber, #124958)
[Link] (2 responses)
And the typing cannot express some things, for example a function that takes T as a parameter and returns an instance of T as a result… there's been an issue open since years to support this, but in general it isn't supported.
Posted Jan 8, 2024 8:39 UTC (Mon)
by gdiscry (subscriber, #91125)
[Link] (1 responses)
Do you mean something like this? (Python 3.12 syntax for brevity, but previous versions only require small tweaks) If not, I'm curious about what you meant. Nevertheless, it's true that not all APIs can be correctly annotated in Python and it's frustrating when that happens. But when it can, it's really nice to avoid defensive programming at runtime.
Posted Jan 10, 2024 15:44 UTC (Wed)
by LtWorf (subscriber, #124958)
[Link]
https://github.com/python/mypy/issues/9773
https://mail.python.org/archives/list/typing-sig@python.o...
Posted Jan 5, 2024 18:00 UTC (Fri)
by marcH (subscriber, #57642)
[Link] (16 responses)
But you have a point: Option must be explicit and cannot be everywhere.
OK, enough Apples to Oranges comparisons, I said I would stop...
Posted Jan 5, 2024 20:44 UTC (Fri)
by NYKevin (subscriber, #129325)
[Link] (14 responses)
Because of Python's laissez-faire multi-paradigm attitude, it's actually quite difficult to design a good implementation of None-aware operators. You can't really use the Rust solution, because Python is dynamically-typed and likes to signal errors with exceptions rather than sentinel values (i.e. you can't reasonably define it to propagate Result<V, E> or the like, since Result is not even a thing in Python). But the TypeError or ValueError that you tend to get from None is usually the wrong exception to throw, and propagating None as if it was NaN will make it difficult to get a traceback. The idiomatic behavior, in some cases, is to eagerly check for None and raise an exception.
Maybe this syntax could work?
x = y ?? raise FooError('y should not be None.')
But that is going to be problematic. Raise is a statement, not an expression, so you'd need to make a special case to allow it in this one context, or you'd need to convert it into an expression. And then people will also want to write x = y ?? z, so you need to allow for that as well.
I have no idea how this is supposed to be extended for ?. and ?[], because where are you supposed to put the raise?
Posted Jan 7, 2024 14:21 UTC (Sun)
by cpitrat (subscriber, #116459)
[Link] (13 responses)
May I interest you in assert?
Posted Jan 8, 2024 22:06 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link] (12 responses)
(Also, you can't specify the exception type, but that's small potatoes in comparison.)
Posted Jan 8, 2024 22:13 UTC (Mon)
by khim (subscriber, #9252)
[Link] (4 responses)
I have just tested and that doesn't happen. What version of python are you using??? I have never observed that effect in Python, but there are many implementations, maybe one of them does that, but for me it's reason not to use it rather then change use of
Posted Jan 8, 2024 22:22 UTC (Mon)
by mb (subscriber, #50428)
[Link]
$ cat t.py
Posted Jan 8, 2024 23:19 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link] (1 responses)
Posted Jan 8, 2024 23:26 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link]
As far as I can tell, 1.4 did not have an assert statement, so the assert statement has never been unconditional in any released version of Python.
Posted Jan 8, 2024 23:40 UTC (Mon)
by ABCD (subscriber, #53650)
[Link]
Posted Jan 8, 2024 23:36 UTC (Mon)
by marcH (subscriber, #57642)
[Link] (1 responses)
> so the assert is just for documentation purposes (to inform the next person who reads the code that the invariant exists and is important).
Small contradiction here.
I didn't know about the -O flag and I've always "run" asserts and every time one is hit it is massively more useful than a comment!
> so the implication is that your code is expected to be correct even when asserts are not run.
Agreed.
Posted Jan 8, 2024 23:37 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link]
Sorry, my brain is smaller than yours, can you please elaborate?
Posted Jan 9, 2024 11:28 UTC (Tue)
by cpitrat (subscriber, #116459)
[Link]
But you can easily define your own raiseIfNone.
Posted Jan 9, 2024 15:11 UTC (Tue)
by atnot (subscriber, #124910)
[Link] (3 responses)
Posted Jan 9, 2024 15:28 UTC (Tue)
by kleptog (subscriber, #1183)
[Link]
Posted Jan 9, 2024 19:13 UTC (Tue)
by NYKevin (subscriber, #129325)
[Link]
Posted Jan 10, 2024 16:28 UTC (Wed)
by laarmen (subscriber, #63948)
[Link]
Posted Jan 6, 2024 0:40 UTC (Sat)
by tialaramex (subscriber, #21167)
[Link]
Let's make our own, custom user-defined generic sum type we'll call it Perhaps<T> and it'll have two values, Huh, and Well(T). Ours works exactly the same way and sure enough the Huh value of Perhaps<&T> will also be represented by an all-zeroes bit pattern.
This behaviour is guaranteed by the language (it is named the Guaranteed Niche Optimisation and is crucial to the language's design) but similar behaviours are delivered in practice for more exotic arrangements. For example Rust's char type needs 4 bytes, but it only handles Unicode "scalar values" (basically think codepoints unless you really care about the minutia of Unicode) so there are a *lot* of unused values. Accordingly, a sum type with Tafkap, SimpsonsMeme and Other(char) will fit in the same 4 bytes as the char alone, because Rust will just squeeze Tafkap and SimpsonsMeme in as bit patterns which aren't valid for char. You aren't promised this will work, but the Rust compiler is going to do it anyway because it's faster and smaller and easy.
† Rust only "really" has one loop, the one introduced by the keyword loop. But for and while both work fine, because the compiler just transforms them into loops, this "de-sugaring" is actually spelled out in the documentation, and the de-sugaring of for (since it's a modern iterator for-each not a C-style for) needs all of IntoIterator, Iterator, Some and None - and so they have to be langitems, annotated core library features which must exist or the language won't work.
In Python things are very different, None is a completely different type than whatever you expected, just as the null pointer is a distinct type in C++.
> Python's None is nowhere near as bad NULL in C/C++ (the so-called "billion dollar mistake") but it's still not great.
The return of None-aware operators for Python
None
is worse than NULL
. At least with NULL you know that you have to deal with pointers to hit that corner-case. None
may be everywhere in Python.The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
And the typing cannot express some things, for example a function that takes T as a parameter and returns an instance of T as a result… there's been an issue open since years to support this, but in general it isn't supported.
def create_instance[T](cls: type[T]) -> T:
...
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
> assert is not properly used for any purpose other than as a "live comment" (i.e. a comment that actually gets executed). This is because the -O flag disables asserts, so the implication is that your code is expected to be correct even when asserts are not run.
The return of None-aware operators for Python
assert
.The return of None-aware operators for Python
assert 2+2==5
$ python3.11 t.py
[...]
AssertionError
$ python3.11 -O t.py
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python
The return of None-aware operators for Python