Arch Linux drops Python 2 [LWN.net]

Arch Linux drops Python 2

Posted Sep 23, 2022 15:54 UTC (Fri) by Amolith (guest, #137875) [Link] (9 responses)

Hopefully this starts a sort of chain reaction; Python 2 should be dropped from every distro :)

Arch Linux drops Python 2

Posted Sep 23, 2022 15:56 UTC (Fri) by amacater (subscriber, #790) [Link] (8 responses)

It already has been dropped by Debian, Ubuntu, Red Hat and derivatives, I think.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:11 UTC (Fri) by plugwash (subscriber, #29694) [Link] (5 responses)

Debian have dropped most python 2 modules, but they haven't yet dropped python 2 itself.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:24 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (4 responses)

Only RHEL9+ doesn't ship with Python2

https://access.redhat.com/solutions/4455511

Fedora only dropped Python2 by default (there is python2.7 package still in the repo) and unlike Arch didn't symlink Python to Python3.

Arch Linux drops Python 2

Posted Sep 23, 2022 19:42 UTC (Fri) by tome (subscriber, #3171) [Link] (3 responses)

> Fedora only dropped Python2 by default (there is python2.7 package still in the repo) and unlike Arch didn't symlink Python to Python3.

/usr/bin/python is a symlink to /usr/bin/python3 on all my fedoras, and it was done by package upgrades, not by me manually. It might depend on the fact that somewhere along the way I removed all python2 packages, but I can't remember in what order things were done. I'll look in my dnf histories if anyone is interested.

Arch Linux drops Python 2

Posted Sep 23, 2022 20:02 UTC (Fri) by tome (subscriber, #3171) [Link] (2 responses)

Update: The fedora package python-unversioned-command.noarch is installed on these boxes. Dnf history shows no record of that package being installed manually.

For the historical record...

Posted Sep 24, 2022 16:14 UTC (Sat) by mattdm (subscriber, #18) [Link]

Overall plan: https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPy...

Fedora Linux 31 change: https://fedoraproject.org/wiki/Changes/Python_means_Python3

Arch Linux drops Python 2

Posted Sep 24, 2022 16:38 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

> Update: The fedora package python-unversioned-command.noarch is installed on these boxes. Dnf history shows no record of that package being installed manually.

This prompted me to look again. Apparently, the defaults vary depending on how you are using it, in the official containers published by the project, this package isn't installed by default and therefore "python" just says command not installed and works similar to RHEL8 (which also doesn't have /usr/bin/python by default either). If you have been upgrading from Fedora prior to when this change was introduced, it doesn't appear to be installed. However in a fresh installation of Fedora 36 it does show up.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:19 UTC (Fri) by stephane (subscriber, #57867) [Link] (1 responses)

Not at all, still present in Debian, Ubuntu, RH and Fedora Rawhide
https://packages.debian.org/search?keywords=python2

Arch Linux drops Python 2

Posted Sep 23, 2022 16:26 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

> Not at all, still present in Debian, Ubuntu, RH and Fedora Rawhide

True for all others but RHEL 9 did drop Python2 first. It was released in May this year.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:12 UTC (Fri) by ceplm (subscriber, #41334) [Link] (4 responses)

I wonder what they do with Chromium and Gimp. These are the only ones which keep openSUSE to drop Python 2 from Factory (https://is.gd/Gpi7Aw).

Arch Linux drops Python 2

Posted Sep 23, 2022 16:20 UTC (Fri) by Vorpal (guest, #136011) [Link] (2 responses)

I checked. Gimp is being built without python support. I don't see anything obvious in the chromium PKGBUILD, it seems to depend on Python 3 already: https://github.com/archlinux/svntogit-packages/blob/packa... ("python" in makedepends).

Arch Linux drops Python 2

Posted Sep 23, 2022 19:44 UTC (Fri) by ceplm (subscriber, #41334) [Link] (1 responses)

Which is surprising considering https://bugs.chromium.org/p/chromium/issues/detail?id=942720 is still live and happily opened.

Arch Linux drops Python 2

Posted Sep 26, 2022 10:45 UTC (Mon) by khim (subscriber, #9252) [Link]

Chromium build doesn't require Python 2 anymore, although some optional tests still require it.

Arch Linux drops Python 2

Posted Sep 23, 2022 20:21 UTC (Fri) by jmm (subscriber, #34596) [Link]

Chromium in Debian is built using Python 3 since Chromium 97. But the qt5 webkit packages still include an old copy which needs Python 2

Arch Linux drops Python 2

Posted Sep 23, 2022 16:18 UTC (Fri) by zwol (guest, #126152) [Link] (25 responses)

Now they just need to fix the bug where they ship a 'python' command that runs Python 3.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:21 UTC (Fri) by Vorpal (guest, #136011) [Link] (23 responses)

Not really a bug is it? Scripts should use python3 or python2 explicitly. Plan "python" is for the end user to use.

Arch Linux drops Python 2

Posted Sep 23, 2022 16:39 UTC (Fri) by zwol (guest, #126152) [Link] (22 responses)

No, it's a bug.

There is an enormous corpus of scripts (often written decades ago, and never updated since) that use /usr/bin/python or /usr/bjn/env python on the #! line and expect to get v2. There is also an enormous population of *people* who type "python" without thinking about it and expect to get v2. Therefore, when a system has no Python 2 interpreter available, /usr/bin/python MUST NOT[rfc2119] exist and the default PATH MUST NOT contain any executable named "python".

(Yes, PEP 394 says something different. PEP 394 is wrong.)

Arch Linux drops Python 2

Posted Sep 23, 2022 17:31 UTC (Fri) by randomguy3 (subscriber, #71063) [Link] (6 responses)

In RFC2119, MUST NOT, etc, are things an implementation must or must not do in order to conform to a specific standard. What standard are you saying Arch is intending to (and failing to) implement?

Arch Linux drops Python 2

Posted Sep 23, 2022 17:54 UTC (Fri) by Wol (subscriber, #4433) [Link] (5 responses)

Not that I've read any of this, but all these python2 scripts - do they PREdate the standard?

Because if so, then the standard is out of touch with reality, not reality out of touch with the standard :-)

And given that a standard requiring the use of python2 or python3 doesn't make sense in a pre-python3 world, I suspect the former not the latter.

(Oh - and even SIMPLE python2 scripts require effort to convert to python3. I have one such 3rd-party script on my system (that I know about ...))

Cheers,
Wol

Arch Linux drops Python 2

Posted Sep 24, 2022 18:11 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (4 responses)

Python 2 has been end of life for over two years (since 2020-01-01), and deprecated for much longer. At a certain point, IMHO it becomes acceptable (for distros, libraries, upstreams, etc.) to say "if you're still on Python 2, then compatibility is your problem. We won't support it anymore, nor will we go out of our way to make life easy for you." As early as 2011, they wrote PEP 404 explicitly spelling out that "[t]he official upgrade path from Python 2.7 is to Python 3." It's not as if people haven't had adequate warning that something like this was eventually going to happen.

Arch Linux drops Python 2

Posted Sep 24, 2022 20:11 UTC (Sat) by malmedal (subscriber, #56172) [Link] (3 responses)

It's not a problem for things I write for myself.

But I've written various programs for non-programmers and sometimes been surprised to find they were still using them many years later. I'll have to avoid python for this use-case.

Best for longevity has been Tcl, I've programs written early nineties that still works almost unchanged. I believe the only changes were
adding "package Tk" and "package BLT" at one point.

Arch Linux drops Python 2

Posted Sep 24, 2022 21:06 UTC (Sat) by cjcox (guest, #60378) [Link] (2 responses)

Waiting for Python 4 to make everyone's Python 3 garbage.... people will sing a different tune then.

Arch Linux drops Python 2

Posted Sep 25, 2022 2:00 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (1 responses)

You're going to be waiting a long time. Python 3 happened under an exceptional set of circumstances, and given all the pain it caused, nobody is eager to do it again. It's possible that they will make a Python 4, and it's even possible that it will have some minor backwards incompatibility with Python 3. What it will not have is "You have to rewrite all of your string-processing code from the ground up."

Arch Linux drops Python 2

Posted Sep 25, 2022 2:43 UTC (Sun) by jhoblitt (subscriber, #77733) [Link]

Python 4 is out. It is called Go.

Only half joking.

Arch Linux drops Python 2

Posted Sep 23, 2022 23:15 UTC (Fri) by Vorpal (guest, #136011) [Link]

Which standard is this that says that python should not point to python3? PEP 394 as you mentioned clearly allows for the Arch behaviour. Please tell me which PEP contradicts this. From a quick search of PEP titles I cannot find any PEP that overrides PEP 394. "rfc2119" that you quote is just the definition of "MUST NOT", so not relevant. Frankly, unless you can back up that statement I call BS.

And those scripts that expect python to be python2 would be broken anyway on a system without python2, so there is functionally no difference (it is not like any of the scripts are going to actually work but do the wrong thing, they are going to fail outright, or just happen to work, unless explicitly written to create different behaviour).

Oh and I haven't seen a python2 script in years now, so I doubt that much of that "enormous corpus" is still actively used by typical end users. Sure there will be some old server or industrial control machine running some old CentOS version that is no longer supported with python2 scripts. But they are most probably not running Arch anyway.

Arch Linux drops Python 2

Posted Sep 23, 2022 23:32 UTC (Fri) by gerdesj (subscriber, #5446) [Link] (9 responses)

"There is an enormous corpus of scripts (often written decades ago, and never updated since)"

You hold up "never updated since" [decades ago] as something for the world to worry about. Don't think so. If you can't be arsed to update them then they can't complain if the world moves on.

Within that sort of timescale (decades) I've seen fuse boxes (consumer units) have nails and screws substituted for fuse wire to avoid "tripping". That of course leads to far worse potential failures, starting with 240V/16A or 240V/32A crossing a heart and other exciting electrical related outcomes.

If your scripts are important then you patch them according to changing conditions and keep them alive. They need care and feeding. If you are heartless enough to just dump and forget them, then that's your problem.

Python, by a very polite decree, delivered over several years, has decided that it would like to be invoked by its v3 incarnation instead of v2 and so it should be if you want to continue mainstream. Feel free to go your own way - you always have choice. v2 is still available with minimal hassle.

Arch Linux drops Python 2

Posted Sep 24, 2022 14:14 UTC (Sat) by Vipketsh (guest, #134480) [Link] (8 responses)

You write as someone who has never seen a large scale real world deployments -- in theory you are correct, in practice you are wrong. Anyone who has knows full well that there are a plethora of ancient scripts in any deployment which can not be changed without herculean effort for a whole slew of reasons, some sane some moronic but that is not the engineer's choice. Denying this reality is trying to change history that Python2 never actually existed.

> Python, by a very polite decree, delivered over several years, has decided [...]

Whether that's polite or not is relative. For something generally fast changing like network facing scripts that may be considered "polite", but there are other use cases. For example, in the chip-design industry I'm in, it is not uncommon to have to open projects from 10-20 years in the past and do some work on it -- maybe run a simulation to check a customer complaint or maybe even make a fix. If it so happened that some script was written in python2, which was very reasonable at the time, today it does not work and Python's decision can hardly be considered "polite".

One could argue that "just use the same environment as back then", and engineers generally would prefer that, but that is often not possible because the commercial EDA tool vendors need lots of convincing to hand out licenses to tools that old so one is often forced to new tools and environments. EDA vendors typically realise the problem and tend to have quite good backwards compatibility in their tools, but that doesn't solve the script issue.

In the global picture, the only way open source can make general forward progress is if one is able to continue building instead of having to always race to keep up with the latest fads just because someone decides that history needs to be rewritten. Do you think Python would be any good today if every decade it would have to rewrite its interpreter in the programming language of the day ? I'm truly curious because the Python guys seem to think its perfectly fine when they force everyone else to do the same.

Arch Linux drops Python 2

Posted Sep 24, 2022 18:34 UTC (Sat) by Wol (subscriber, #4433) [Link]

> One could argue that "just use the same environment as back then", and engineers generally would prefer that, but that is often not possible because the commercial EDA tool vendors need lots of convincing to hand out licenses to tools that old so one is often forced to new tools and environments. EDA vendors typically realise the problem and tend to have quite good backwards compatibility in their tools, but that doesn't solve the script issue.

That assumes the vendor CAN hand out a licence ...

I've just seen on one of my mailing lists, a request for UniData 5 point something install media. I don't know how old that is, but the current vendor (Rocket) doesn't have a copy. They've taken over most of the Pick vendors, and in the process a lot of history has probably been lost. Trying to get hold of commercial products from that long ago is probably very difficult simply to find an extant copy!

Cheers,
Wol

Arch Linux drops Python 2

Posted Sep 24, 2022 22:32 UTC (Sat) by gerdesj (subscriber, #5446) [Link] (6 responses)

"You write as someone who has never seen a large scale real world deployments" Nice start to a reasoned response.

My nascent Civil Engineering career morphed into IT around 30 odd years back. You will have heard of some of my little firm's customers - strange but likely true.

I own a treasured screwdriver with a wooden handle and a simple flat blade - you may do too. It still works - mine was my great granddad's and was made in the 1880s or so. The handle has many years of skin oils and various other oils and who knows what else infused into it and it has a phenomenal patina which has perfectly preserved it. The metal bit is a masterpiece of forging and heat treatment. The whole thing is chromium infused steel for rust resistance. The shaft is roughly like modern mild steel - it is slightly ductile, which means it deals well with torsion. The blade part has been heat treated in two parts. The whole "blade" has been hardened a bit and the leading edge has been hardened more. The part that you stick in a flat edge screw is as good today as it was 140 odd years back.

I also have my dad's ratchet screwdriver. This one has a wooden handle too and is only 60 odd years old. It works fine too and just needs a light oil every few years or so.

I have several electric drill drivers, one of which you have to ensure you get a leg in the way to avoid wrenching your wrist and a wide set of modern cross head/flat head/insulated screwdrivers and all the rest.

Guess which set of tools I use these days? I maintain the old ones and they still work but a De Walt wrist destroyer with a decent bit is my go to screwdriver.

The above is simply a fun digression but I'm sure we can whistle up an analogy. I think you are missing multiple points:

1. "Entitlement" - Python devs should do what I want them to for my use case.
2. "Laziness" - I can't be bothered to maintain my stuff.
3. "Inertia" - Its always done like this and I want it to last forever.
4. "Lack of imagination" - We can't be arsed with change.

You can always grab a Python2 and maintain it yourself if your code is that important. It's a simple cost/benefit analysis.

Whilst we are at it, we should note that this article is about Arch dumping Python2 not Centos or whatevs it is is called today. I run Arch on my personal laptop and work desktop - I doubt you do on your work gear and so the whole argument is rather moot. I do suggest you plan ahead though.

Arch Linux drops Python 2

Posted Sep 25, 2022 18:21 UTC (Sun) by Wol (subscriber, #4433) [Link] (5 responses)

> My nascent Civil Engineering career morphed into IT around 30 odd years back. You will have heard of some of my little firm's customers - strange but likely true.

As someone who worked in Civil Engineering, I'm not surprised. MOST big firms actually have precious few employees. Most of the work is done by subcontractors. So yes, I probably have heard of quite a few of your customers ...

Before I moved into the HR world, I worked for Babcocks, Costain, Merlin, ... (that I can remember).

Cheers,
Wol

Arch Linux drops Python 2

Posted Sep 26, 2022 23:20 UTC (Mon) by gerdesj (subscriber, #5446) [Link] (4 responses)

"As someone who worked in Civil Engineering, I'm not surprised. MOST big firms actually have precious few employees. Most of the work is done by subcontractors. So yes, I probably have heard of quite a few of your customers ..."

I never even got that far in civ eng. I graduated in 1991, from Plymouth. I applied to Plymouth Poly, attended Poly South West and a year later it was Uni Plymouth. As you will recall, 1991 was not a good year for building stuff in the UK! I rocked a UB40 for a fair old while. I can still whistle up a decent 1-2-3 mix of conc and some of my garden structures are rather more formally designed and analysed than the usual.

My comment about hearing about my customers is for my very little IT company 8) Two of them are rather global and another one: you (as a Brit) will know well.

My main point about Python devs deprecating P2 is that I think it is behaving in an entitled way to try and insist that P2 be maintained forever. Arch is a great platform to test the waters for the change and give off an indication that it will finally happen across the board.

I only trained to become an engineer but I run my work life by what I learned in a few years. It might be nice to see IT grow up and learn that maintenance is part of the design of a system. Some bloke farting around for Facebook once famously said something along the lines of: "work fast and break things". He might now be a multi zillionaire but perhaps he ought to try painting a bridge. You will know that the Forth rail bridge was famously a job never finished. I believe it is now covered in some sort of plastic - but at least that is a proper solution.

My main problem with complaints about deprecating P2 and actually removing it is "entitlement". Either enjoy a better way to coat your bridge (you might need a new applicator and learn how to use it) or keep your old paints and brushes around - at least you have the recipe for the stuff.

Arch Linux drops Python 2

Posted Sep 27, 2022 9:03 UTC (Tue) by Vipketsh (guest, #134480) [Link] (3 responses)

> insist that P2 be maintained forever

For what it's worth, my "change history" comment was referring to the Python people asking for the 'python' command to mean python3. That closes the door to being able to install python2 (originally what 'python' did) and having things just work. It also leaves no indication to anyone new to the Python world that "oh, this is written in a different Python". This causes pain all around for no reason what so ever. Also their stance towards anyone wanting to maintain python2 (a comment below) is simply hostile.

That is the reality of running big systems: there will always be ancient stuff running for "reasons" requiring weird interpreters and that includes Python2. The right thing to do is to admit this is the case, not close the door to anyone wanting to do "this crazy thing" and accept patches if someone puts in the effort to keep Python2 running in modern environments. See ? I'm not asking for anyone to maintain things, just keep the door open to allow people to do necessary things needed in the real world and not make the problem worse.

I also take big offense when people downplay the work needed to fix breakage and try to sell the millionth incompatibility as "this is absolutely necessary" (a big fat lie). That is real time not spent doing real work and across thousands of projects burning many thousands of man hours. The simple reality is that the Python2->3 transition was not liked by many and thus was simply a bad idea. Instead of admitting that was the case the Python crew seem to just push harder that python2 never in fact happened.

From my point of view actions of the Python people are just asking for bad words to be sent in their direction and just aren't making things better.

> It might be nice to see IT grow up and learn that maintenance is part of the design of a system.

This I very much agree with. Even if your foundations don't change (e.g. if Python2->3 never happened), maintenance is very much required. Of course I think this goes both ways: organisations putting up the money/resources for maintenance but also software understanding their dependents will need to be maintained and thus not unnecessarily making things difficult for them.

Arch Linux drops Python 2

Posted Sep 27, 2022 21:14 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (2 responses)

> For what it's worth, my "change history" comment was referring to the Python people asking for the 'python' command to mean python3. That closes the door to being able to install python2 (originally what 'python' did) and having things just work. It also leaves no indication to anyone new to the Python world that "oh, this is written in a different Python". This causes pain all around for no reason what so ever. Also their stance towards anyone wanting to maintain python2 (a comment below) is simply hostile.

My attitude towards this is pretty much identical to the attitude that the C standards committee displayed towards IBM, when the latter asked for trigraphs to be kept in the language for backcompat with EBCDIC: Meh, that's your problem. You can do the symlink dance yourself, or configure your sysadmin automation to do it for you.

I don't mean to be rude. I'm simply stating a fact. You cannot rationally expect people who are not on your payroll to do work that you consider necessary. Reviewing patches is not free. Maintaining VCS branches is not free. Compiling interpreter binaries is not free. All of these things require human attention, computing power, or both. While this may not be a lot of work, it's their decision whether they want to do it or not. It should also be emphasized that you can still download Python 2.7 today, if you really want to: https://www.python.org/downloads/release/python-2718/

Arch Linux drops Python 2

Posted Sep 27, 2022 22:12 UTC (Tue) by Wol (subscriber, #4433) [Link] (1 responses)

> My attitude towards this is pretty much identical to the attitude that the C standards committee displayed towards IBM, when the latter asked for trigraphs to be kept in the language for backcompat with EBCDIC: Meh, that's your problem. You can do the symlink dance yourself, or configure your sysadmin automation to do it for you.

"to be kept" ... in other words, the standards committee was MAKING work for other people. Same here - if people want "python" to *just* *keep* *working*, the python guys are MAKING work for other people.

I get your point - why should the python guys do work for other people. But the point in the other direction is just as valid - why are the Python guys making work for other people?

Cheers,
Wol

Arch Linux drops Python 2

Posted Sep 27, 2022 22:15 UTC (Tue) by NYKevin (subscriber, #129325) [Link]

An interpreter that contains security flaws doesn't work, for any reasonable interpretation of "work." Therefore, keeping Python working would require one of two things:

1. Python does not implement any security boundaries whatsoever. This entails the removal of quite a lot of the standard library (e.g. all of the HTTP and SSL code), which probably breaks a lot of legacy scripts anyway.
2. Someone patches those flaws when they are found.

Arch Linux drops Python 2

Posted Sep 25, 2022 3:20 UTC (Sun) by milesrout (guest, #126894) [Link]

Personally, I have expected 'python' to run python 3 for about 10 years. And unless my memory betrays me, that's exactly what it has done on Arch for about that long.

> (Yes, PEP 394 says something different. PEP 394 is wrong.)

I would politely suggest that maybe you are in fact wrong.

Arch Linux drops Python 2

Posted Sep 26, 2022 12:15 UTC (Mon) by agateau (subscriber, #57569) [Link] (2 responses)

I felt similarly for a while, until I found out the official Python 3 Windows installer, does not install a python3.exe binary, so any shebang explicitly referring to python3 fails on Windows.

Shebang lines for Python on Windows

Posted Sep 26, 2022 13:20 UTC (Mon) by rschroev (subscriber, #4164) [Link] (1 responses)

Windows itself does not interpret shebang lines. These days (since Python 3.3) the Python installer installs a utility, the "Python Launcher for Windows" (py.exe), which is configured as the default file association for Python scripts. The launcher does interpret the shebang line (limited to a few common formats) and launches the appropriate python executable.

#!/usr/bin/env python launches the default Python
#!/usr/bin/env python2 launches the most recent Python 2
#!/usr/bin/env python3.9 launches Python 3.9

See https://docs.python.org/3/using/windows.html#python-launc... and more specifically https://docs.python.org/3/using/windows.html#shebang-lines

In a nutshell, this makes it possible to have shebang lines that work on both Windows and Unix-like operating systems.

Shebang lines for Python on Windows

Posted Sep 26, 2022 14:05 UTC (Mon) by agateau (subscriber, #57569) [Link]

Interesting. I never used py.exe. That can be handy, but it would not have helped in my case. We were using Python in our CI and the assumption was: since we start the script from Git Bash, the shebang is going to work properly, because Bash knows about shebangs. And indeed it does, but we still could not write "#!/usr/bin/env python3" because there was no python3.exe file (until we created one by copying python.exe to python3.exe, meh).

I still believe being able to reliably depend on the availability of a python3 binary on all supported platforms would have helped a lot.

Arch Linux drops Python 2

Posted Sep 23, 2022 17:42 UTC (Fri) by riking (subscriber, #95706) [Link]

# apt install python-is-python-3

Arch Linux drops Python 2

Posted Sep 24, 2022 11:54 UTC (Sat) by azumanga (subscriber, #90158) [Link] (20 responses)

I will always be annoyed that while I can compile Fortran 77, C89 and C++ 98, and there is no suggestion (that I have seen) that support for these should be removed from compilers, the Python leadership is so anti-Python 2 they killed it, and threatened to sue anyone who wanted to carry on supporting it in any way which could be "confused as official".

Arch Linux drops Python 2

Posted Sep 24, 2022 16:44 UTC (Sat) by vstinner (subscriber, #42675) [Link] (19 responses)

Maintaining Python 2.7 is fine, but so far nobody stepped in. Linux distributions are still maintaining it (security fixes).

The PSF is against "Python 2.8" (see PEP 404): something between Python 2.7 and Python 3: that's why the tauthon project couldn't use the name "python".

Supporting old flavors of Fortran and C is cheaper than supporting Python 2.7 which comes with a whole HTTP server, TLS security, portable API to spawn subprocesses, ... : the big "standard library" (more than 300 modules in Python 3.10 for example). The stdlib is part of Python success and moving it outside Python is not planned.

Just supporting TLS is a big maintenance burden since SSL evolved from v2 to v3, then TLS v1.0, v1.1 and now v1.2. Each protocol requires subtle API changes and some old protocols are no longer supported in newer OpenSSL versions (ex: SSLv2). OpenSSL API 3.0 is backward incompatible with OpenSSL API 1.1.1 which is incompatible with OpenSSL API 1.0. When you use Python, you don't have to worry about that, it's hidden by higher Python API (ssl module).

If someone wants to support Python 2.7 for the next 10 years: please go ahead. Sadly, basically all third party modules dropped Python 2 support. On Fedora and RHEL, an old pip version is used, but the PyPI server security evolved (is now stricter) and is now incompatible with old Python 2.7 SSL/TLS client security...

For a third party project supporting a wide range of Python version can be too expensive for a small team. The testing matrix becomes too big. So it's common that they cut eaggerly support for old Python versions. These days, Python 3.6 is being removed. Python 3.6 no longer gets security fixes: https://devguide.python.org/versions/

Arch Linux drops Python 2

Posted Sep 25, 2022 6:45 UTC (Sun) by cyperpunks (subscriber, #39406) [Link] (17 responses)

RHEL 9 comes with Python 3.9 as default, from page above we see:

Ver   GA           EOL
3.9   2020-10-05   2025-10
3.10  2021-10-04   2026-10
3.11  2022-10-03   2027-10
3.12  2023-10-03   2028-10

RHEL 9 has 10 years of support to 2032-06.

Let's say production deployment is done today with Python 3.9, that ends in 3 years, in 2025-10.

Before that date whole stack needs porting to 3.12. Support for 3.12 ends in 2028-10.

New port to Python 3.16 is required. Python 3.16 will be supported to 2032.

Conclusion: to use any Python application safely for 10 years requires two major porting efforts.

Arch Linux drops Python 2

Posted Sep 25, 2022 7:45 UTC (Sun) by WolfWings (subscriber, #56790) [Link] (16 responses)

I'd strongly disagree with calling them 'porting' efforts, because unlike the 2.x -> 3.x changes things are generally very mild, and all that I can see at a glance have warnings when stepping incrementally through the versions, and are specifying edge cases or removing support for what most would call "Do what I mean not what I wrote" such as "0in x" as an example in 3.10 requiring you to update it to "0 in x" instead now.

If you have to maintain a project for 10 years? That includes updating it for security fixes, and that includes updating your code that calls other libraries or runs on scripting languages that release security fixes in that timeframe.

Or else freeze it in the digital equivalent of carbonite and hang it on the wall, including all the required executables, etc.

So if anything I'd think in 10 years support of a project using Python? You'd need to schedule a roughly annual security update of the code, stepping through each python version.

And if that's too huge a burden... then either the project was implemented in a way that can't support 10 years of maintenance (lack of test cases, etc), or there's other fundamental problems and blaming project issues on Python is akin to demanding some random 50-line library author to present entire cybersecurity documenting, 2FA validating, etc, just because some 'process' at your company requires that documentation.

Arch Linux drops Python 2

Posted Sep 25, 2022 8:49 UTC (Sun) by ballombe (subscriber, #9523) [Link] (3 responses)

And yet we have perl programs that have been running unmodified for 25 year now...

Arch Linux drops Python 2

Posted Sep 26, 2022 13:13 UTC (Mon) by hkario (subscriber, #94864) [Link] (1 responses)

And I have multiple software projects that run with no modifications needed on both Python 2.6 and Python 3.11, exact same source is executed on both.

So what's your point?

Arch Linux drops Python 2

Posted Sep 26, 2022 14:45 UTC (Mon) by Wol (subscriber, #4433) [Link]

When were your projects last modified? THAT is the point.

If your Python 2.6 source was written before 2.7 was released, and still works fine on 3, then great. If you had to modify the source to work with 3, then *you* have clearly missed the point. It's not clear from what you say which is the case.

Unfortunately, I have a python utility that was written for (and runs fine on) 2.7. It crashes on - I guess 3.8? 3.9? I'm not sure which one was current when I tried it.

It's someone else's source, I don't "do" Python, so I can't ditch 2.7 until someone else fixes it for me ...

Cheers,
Wol

Arch Linux drops Python 2

Posted Sep 26, 2022 18:12 UTC (Mon) by dvdeug (guest, #10998) [Link]

Given that Perl 5.26 dropped support for ISO 8859-x source code, it's not universal. Given that Perl 5.6 (2000) added (experimental) Unicode support, it's likely your code has problems handling certain text; since that version also added 64-bit support and support for >2GB files, those could also be problematic points. The world is a pretty different place; merely running doesn't mean they usefully work.

Arch Linux drops Python 2

Posted Sep 25, 2022 16:16 UTC (Sun) by cyperpunks (subscriber, #39406) [Link] (1 responses)

Python is both a "compiler", runtime environment and very large set of standard modules.

Due to this, Python is more like an operating system than compiler in the tradional sense.

When a Python goes release EOL, your "operating system" is simply not supported any longer,
this means you can't safely run any Python script using this particular version of Python.

The only way to continue in sane manner to switch to a supported Python release.

All new Python major release comes with new issues, if you are in doubt just check this tracking bug in Fedora when moving to Python 3.10 (from Python 3.9):

https://fedoraproject.org/wiki/Changes/Python3.10
https://bugzilla.redhat.com/show_bug.cgi?id=1890881

Any software written today needs a life time of more than 10 years, all software I maintain or use today have or will have a life time way longer than 10 years. As software goes older, the more value does it add: think Linux kernel, GCC, LLVM, valgrind etc. You don't want to rewrite those from scratch.

Python is used by many large projects today, just look at all the AI projects, web framework, package managers etc etc.

It's a not problem that Python evolves and creates new releases, the problem is that life time of each major release is way too short. I would set 8 years as mininum, 10 years as ok and 15 as excellent.

Arch Linux drops Python 2

Posted Sep 25, 2022 20:29 UTC (Sun) by k8to (guest, #15413) [Link]

It kind of depends what you're delivering, IMO. If it's a thing you have to keep updating anyway for various reasons, like a part of a service product or a shipped product that you need to keep updating to ensure it's secure etc, the python status quo isn't horrid.

For things that have no real forcing function to ensure and/or naturally arrange for ongoing maintenance, it's *very* awkward.

Arch Linux drops Python 2

Posted Sep 26, 2022 18:35 UTC (Mon) by khim (subscriber, #9252) [Link] (9 responses)

> I'd strongly disagree with calling them 'porting' efforts, because unlike the 2.x -> 3.x changes things are generally very mild

It doesn't matter how small they are: because if python dynamic nature even minor changes requires copious amount of testing.

That's what python developers don't understand: it's not the required amount of changes that is the problem but the need to spend a lot of efforts trying to see if anything is broken or not.

Changes to python were similar in scale to C++ transition to C++11 or C++20 and these releases are huge.

> and all that I can see at a glance have warnings when stepping incrementally through the versions

Stepping incrementally through versions just increases the pain: instead of couple of significant porting efforts you now have ten of them!

> If you have to maintain a project for 10 years? That includes updating it for security fixes, and that includes updating your code that calls other libraries or runs on scripting languages that release security fixes in that timeframe.

How many time do we need to repeat the same thing before people realize that people don't do that? You can not both push your language as “language for laymens” and expect it to only be used by software developers who would be treating it as something different from fridge or a lawnmower. Do you update your fridge for security fixes or replace pieces of your lawnmower (except when they break and need to be fixed)?

Arch Linux drops Python 2

Posted Sep 27, 2022 1:57 UTC (Tue) by WolfWings (subscriber, #56790) [Link] (8 responses)

That's what python developers don't understand: it's not the required amount of changes that is the problem but the need to spend a lot of efforts trying to see if anything is broken or not.

Python devs do understand that, they just disagree with that statements assumption that testing is complex, difficult, or time consuming. Anything that's intended to be used and run for that span of time should have testing in place and be broken into small enough pieces that such testing is simply part of development, quick to do, etc. Python has had an included and stable unittest subsystem since 2.1. Literally over two decades now.

Stepping incrementally through versions just increases the pain: instead of couple of significant porting efforts you now have ten of them!

The difference is one of magnitude. It's less overall work to have a smaller annual tune-up than jump multiple minor versions only updating every few years. Just like it's cheaper to change your oil and spark plugs regularly than to wait until the engine is seizing and trying to correct everything then.

How many time do we need to repeat the same thing before people realize that people don't do that? You can not both push your language as “language for laymens” and expect it to only be used by software developers who would be treating it as something different from fridge or a lawnmower. Do you update your fridge for security fixes or replace pieces of your lawnmower (except when they break and need to be fixed)?

Both can be and are true. Python is a language a layperson can readily pick up and do effective work with, and another layperson can see what's happening in simpler scripts easily as well. But writing something that will be used by others for ten years as you're saying requires more planning and dedication to upkeep. You have to replace the blades on a lawnmower, or repair the engine if the spark plug dies, or if it's let sit too long flush the gas tank, etc. But a pair of scissors? Those might require re-sharpening every few years at most. Both involve sharp blades and can very much be constructed initially by a layperson if given the right tools and parts. But there's a huge different in overall complexity and long-term maintenance involved.

Arch Linux drops Python 2

Posted Sep 27, 2022 9:11 UTC (Tue) by Vipketsh (guest, #134480) [Link] (7 responses)

> should have testing in place and be broken into small enough pieces that such testing is simply part of development, quick to do, etc.

Wonderful theory. Meanwhile in the real world, never have I seen code written and tested not show bugs when first put into production. In other words: no matter how well you test your code you can not find all possible bugs. Meaning there will always be hard and painful things to find, especially when behaviour of things you call change -- after all you are not testing the functions provided by others.

I'm in the world of designing hardware where there is a universal requirement to thoroughly test things before it even smells the real world (i.e. implemented in an FPGA) thus, compared to software, things are well tested by the time you get there. In the industry there are also tons of metrics which many customers expect you to use to prove how well your code is tested -- and the expectations are not small (e.g. 100% of all code lines executed, among others). Furthermore for any minor release they expect you prove everything again. Yet, the most important thing every customer expects is for you to show that your code has been implemented and used in the real world. Why ? Because everyone understands that there is no substitute for production and there are always things which pop out only then.

In summary: no matter how much you test you can not avoid long painful debug sessions inside production environments.

> writing something that will be used by others for ten years [...] requires more planning and dedication to upkeep

What an ideal world you are living in. I have never seen a SW or HW development project where the plan was to maintain things for 10+ years. The one and only plan is always to get the first version of the project out the door by the deadline. The rest will be solved later. I would love for this to change, but this is reality for many reasons.

Quite frankly, is it even possible to plan 10 years in the future in software development ? I have not seen a single project promise 10 years of maintenance -- anything you may rely on today will very likely be gone by the time 10+ years is up.

Arch Linux drops Python 2

Posted Sep 27, 2022 11:36 UTC (Tue) by kleptog (subscriber, #1183) [Link] (3 responses)

> Wonderful theory. Meanwhile in the real world, never have I seen code written and tested not show bugs when first put into production. In other words: no matter how well you test your code you can not find all possible bugs. Meaning there will always be hard and painful things to find, especially when behaviour of things you call change -- after all you are not testing the functions provided by others.

Sure, but when upgrading across versions of python, the issue is not that things break in subtle ways. Either it works, or it blows up. As such, simple smoke-tests across your application are sufficient to detect any issues. And python makes it extremely easy to write these kinds of tests because you can literally mock everything trivially. Trying to test individual functions in a large C application can be very annoying as you somehow have to split off a chunk of the application such that it can be compiled and linked separately.

The issues that are most missed during Python version upgrades are the error paths, because they tend to be less well tested. These days though static analysis is pretty good at catching these kinds of errors.

As for the issue that drove this thread: changing /usr/bin/python to mean python3, that's literally grepping your source for #! and adding a 2. Hardly weeks of work. Or even easier, just use a virtualenv and then python can mean whatever you like..

Arch Linux drops Python 2

Posted Sep 27, 2022 12:28 UTC (Tue) by pizza (subscriber, #46) [Link] (2 responses)

> Sure, but when upgrading across versions of python, the issue is not that things break in subtle ways. Either it works, or it blows up. As such, simple smoke-tests across your application are sufficient to detect any issues

See, this tells me you've never had to deal with an even moderately complex python codebase.

As the language is dynamic, unless your "simple smoke test" results in the interpreter parsing _every single file_ then it's not even the most basic of smoke tests. Heck, even invoking every single method isn't sufficient because that doesn't guarantee that the _callers_ are doing the right thing in all cases.

Add to that that any python codebase inevitably pulls in a bazillion external pip modules, and those can and do break on a whim. So you either pin every specific version forever (which is particularly fun for python2 stuff now -- and heck, even older python3 codebases) or you'll have random "build" or runtime failures from one day's deployment to the next.

If you're not _constantly_ developing a given python codebase, it'll fall into ruin within a couple of years; That's the sad, objective truth -- My last two employers had to learn that the hard way (and they're in industries where 10+ year support windows are considered short)

Arch Linux drops Python 2

Posted Sep 27, 2022 14:40 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

> If you're not _constantly_ developing a given python codebase, it'll fall into ruin within a couple of years. That's the sad, objective truth.

That's overstating it quite a bit. If you have a complex Python codebase, it may need some maintenance to keep up with newer versions but how much and how often will vary depending on the features you use and the libraries you are relying on. Sometimes you do live in an ecosystem that is already heavily reliant on a language and there isn't much you can do about it but if you are starting new, I would recommend using alternatives like Go if the codebase is expected to live long and remain fairly untouched.

Arch Linux drops Python 2

Posted Sep 27, 2022 14:52 UTC (Tue) by Wol (subscriber, #4433) [Link]

> > Sure, but when upgrading across versions of python, the issue is not that things break in subtle ways. Either it works, or it blows up. As such, simple smoke-tests across your application are sufficient to detect any issues

> See, this tells me you've never had to deal with an even moderately complex python codebase.

Reality depends on the viewpoint of the observer :-)

And this discussion seems to emphasise that most strongly - "it works for me" is fine until you start dealing with a complex problem ... what did Einstein say? "Make things as simple as possible (but no simpler)".

Cheers,
Wol

Very Long Term Support

Posted Sep 28, 2022 7:38 UTC (Wed) by cladisch (✭ supporter ✭, #50193) [Link] (2 responses)

> Quite frankly, is it even possible to plan 10 years in the future in software development ?

https://www.sqlite.org/lts.html says:
> The intent of the developers is to support SQLite through the year 2050.
>
> At this writing, 2050 is still 34 years in the future. Nobody knows what will happen in that time, and we cannot absolutely promise that SQLite will be viable or useful that far out. But we can promise this: we plan as if we will be supporting SQLite until 2050. […]

Very Long Term Support

Posted Sep 28, 2022 13:21 UTC (Wed) by tzafrir (subscriber, #11501) [Link] (1 responses)

SQLite is a small and focused project.

A slightly larger project is Common Infrastructure Project, that maintains a (or some?) specific basic Linux system for 20 years. It includes a kernel, glibc, bash, mawk, and even minimal perl (although I would not be surprised if people try to get rid of perl). Surely not python.

BTW: two Debian packages not included there are:

sqlite - command line interface for SQLite 2
sqlite3 - Command line interface for SQLite 3

Very Long Term Support

Posted Sep 28, 2022 17:28 UTC (Wed) by geert (subscriber, #98403) [Link]

Civil Infrastructure Platform (https://www.cip-project.org/), I assume?

Arch Linux drops Python 2

Posted Sep 25, 2022 21:45 UTC (Sun) by mpg (subscriber, #70797) [Link]

> Just supporting TLS is a big maintenance burden since SSL evolved from v2 to v3, then TLS v1.0, v1.1 and now v1.2.

This is really a side point (and it only strengthens your general argument), but I think you mean 1.3 - RFC 8446 is dated August 2018, so it's been 4 years now. (And real-world deployment is not too bad compared to the time it took with previous versions.)

Arch Linux drops Python 2

Posted Sep 24, 2022 18:23 UTC (Sat) by apolinsky (subscriber, #19556) [Link]

/usr/bin/python on Slackware 15 and Slackware64-15 point to python2.7