LWN: Comments on "The PEPs of Python 3.9"

The PEPs of Python 3.9

quietbritishjim — Fri, 29 May 2020 14:05:37 +0000

This is a great comment, thanks. I still found the term "visual order" confusing though, so I had a Google around for it. Here are two things to clarify.

The first thing I got confused by is that I assumed that "visual order" would still mean the rightmost character to be "first" if you're looking at a purely right-to-left language. After all, if you asked a native Arabic speaker what the visually first character is then they would point over on the right hand side. So when you said "For purely right-to-left text, logical order is the opposite of visual order", that would be a contradiction in terms. But it seems that in practice this term is usually used to mean what I'd call "visually left-to-right order", so "first" is always leftmost regardless of the text.

Second, I thought you only brought up "visual order" to explain the difference between how strings are stored vs. how they're rendered. I didn't realise that some people actually have stored text in visually left-to-right order *in memory*. So `mystr[0]` would be the logically last character in a right-to-left language, because that's on the left. Yuck!

The PEPs of Python 3.9

flussence — Sat, 23 May 2020 07:52:24 +0000

Raku saves the day (again):

dd (S/ ef $ //, S/ ef $$ //) for "abcdef", "abcdef\n", "abcdef\n\n";
("abcd", "abcd")
("abcdef\n", "abcd\n")
("abcdef\n\n", "abcd\n\n")

$ and $$ here are invariant replacements for $, \Z and \z that aren't affected by regex postfix switches (and don't have nightmare edge cases with blank lines apparently).

There's only one — hopefully obvious — way to do it, one that also doesn't require remembering what thesaurus word a function name uses or whether or not it accepts list args…

The PEPs of Python 3.9

gerdesj — Fri, 22 May 2020 23:37:54 +0000

Thank you for that explanation. Your comment speaks volumes and I can almost see the blood dripping through clenched teeth.

String manipulation function definitions are quite tricky, when the definition of string is hard.

The PEPs of Python 3.9

NYKevin — Fri, 22 May 2020 18:27:50 +0000

> the logical order is still first-word-first, and the visual order is complicated.

If anyone is curious about the visual order, the Unicode Consortium has written down the gory details here: https://unicode.org/reports/tr9/ (TL;DR: "Complicated" is an understatement, but basically, it tries to figure out which pieces of text are "embedded" in surrounding text of the opposite directionality, and then lays things out to preserve the visual order of each level of embedding. It also spends a great deal of complexity on "guessing" the directionality of neutral characters such as punctuation, digits, and whitespace.)

The PEPs of Python 3.9

smcv — Fri, 22 May 2020 16:52:01 +0000

Text is normally represented in "logical order", where the first letter of the first word appears first in memory, regardless of whether it will be displayed to the left or the right of subsequent letters. It stays in logical order during editing and manipulation, until something like Pango turns it into pixels.

For purely right-to-left text, logical order is the opposite of "visual order". For mixed left-to-right and right-to-left text (for example an English web page containing some Arabic words or vice versa), the logical order is still first-word-first, and the visual order is complicated.

Writing content in visual order basically can't work unless the text width is known and fixed (that is, the lines of text are hard-wrapped, as they would be in a terminal emulator).

For example in HTML: https://www.w3.org/International/questions/qa-visual-vs-l...

For RTL text in logical order, left- and right-oriented API names like Python's lstrip() and rstrip() or BASIC's Left() and Right() do the opposite of what their names would suggest: lstrip() deletes the first characters of the string, which are the first letters of the first word (even though they would be displayed on the right), while rstrip() deletes the last characters (even though they would be displayed on the left).

Talking about a prefix or suffix (like GLib's g_str_has_prefix(), g_str_has_suffix), or the start and end (like Python's str.startswith() and str.endswith()), or numeric positions (like Python's str[:3] or str[5:]) makes more sense than "left" and "right" when working with logical order.

The PEPs of Python 3.9

excors — Fri, 22 May 2020 00:41:48 +0000

left()/right() sound even more confusing, because the 'left' character of "עִבְרִית‎" might be the one that's on either the left or the right of the screen depending on your text editor.

The PEPs of Python 3.9

gerdesj — Thu, 21 May 2020 20:53:18 +0000

I may have accidentally stumbled behind the wrong bike shed but what does pre/post fix mean in languages that don't run L->R.

Also, what's wrong with left(), right() and/or mid() which are already in use in other languages and hence "obvious"?

The PEPs of Python 3.9

NYKevin — Thu, 21 May 2020 17:02:29 +0000

AttributeError (because it's removesuffix).
If we fix that, then it would return the string unchanged, because "def" is not a suffix of "abcdef\n\n" (according to the endswith() method). Newline is just another character and doesn't get magical treatment here. If you want newlines to be ignored, you have to call strip("\n") (or lstrip or rstrip) to remove them.

See the specification if you're curious about any other aspects here.

The PEPs of Python 3.9

kay — Thu, 21 May 2020 16:04:31 +0000

so what's the result with "abcdef\n\n".removepostfix("def")?

The PEPs of Python 3.9

smitty_one_each — Thu, 21 May 2020 14:41:15 +0000

I just fear that the PEG parser and the waning of Guido's benevolent dictatorship could lead to a shift away from Monty Python and toward Steely Dan references in future releases.

The PEPs of Python 3.9

excors — Thu, 21 May 2020 14:23:55 +0000

$_ = "abcdef";     s/ef$//;  # "abcd"
$_ = "abcdef\n";   s/ef$//;  # "abcd\n"
$_ = "abcdef\n\n"; s/ef$//;  # "abcdef\n\n"

Sometimes regexes aren't quite as simple as they seem.

The PEPs of Python 3.9

smurf — Thu, 21 May 2020 11:07:38 +0000

Plus, a reasonable regexp engine should be able to replace its invocation with "removeprefix" if that's the RE's effect.

On the other hand, too much magic behavior (plus heaps of TMTOWTDI) is exactly why I'm using Python instead of Perl these days.

The PEPs of Python 3.9

sytoka — Thu, 21 May 2020 07:08:21 +0000

s/^abc//
s/ef$//

Sometime, regex are simpler than human language ;-)