Arch Linux drops Python 2
Arch Linux drops Python 2
Posted Sep 24, 2022 11:54 UTC (Sat) by azumanga (subscriber, #90158)Parent article: Arch Linux drops Python 2
Posted Sep 24, 2022 16:44 UTC (Sat)
by vstinner (subscriber, #42675)
[Link] (19 responses)
The PSF is against "Python 2.8" (see PEP 404): something between Python 2.7 and Python 3: that's why the tauthon project couldn't use the name "python".
Supporting old flavors of Fortran and C is cheaper than supporting Python 2.7 which comes with a whole HTTP server, TLS security, portable API to spawn subprocesses, ... : the big "standard library" (more than 300 modules in Python 3.10 for example). The stdlib is part of Python success and moving it outside Python is not planned.
Just supporting TLS is a big maintenance burden since SSL evolved from v2 to v3, then TLS v1.0, v1.1 and now v1.2. Each protocol requires subtle API changes and some old protocols are no longer supported in newer OpenSSL versions (ex: SSLv2). OpenSSL API 3.0 is backward incompatible with OpenSSL API 1.1.1 which is incompatible with OpenSSL API 1.0. When you use Python, you don't have to worry about that, it's hidden by higher Python API (ssl module).
If someone wants to support Python 2.7 for the next 10 years: please go ahead. Sadly, basically all third party modules dropped Python 2 support. On Fedora and RHEL, an old pip version is used, but the PyPI server security evolved (is now stricter) and is now incompatible with old Python 2.7 SSL/TLS client security...
For a third party project supporting a wide range of Python version can be too expensive for a small team. The testing matrix becomes too big. So it's common that they cut eaggerly support for old Python versions. These days, Python 3.6 is being removed. Python 3.6 no longer gets security fixes: https://devguide.python.org/versions/
Posted Sep 25, 2022 6:45 UTC (Sun)
by cyperpunks (subscriber, #39406)
[Link] (17 responses)
Let's say production deployment is done today with Python 3.9, that ends in 3 years, in 2025-10.
Before that date whole stack needs porting to 3.12. Support for 3.12 ends in 2028-10.
New port to Python 3.16 is required. Python 3.16 will be supported to 2032.
Conclusion: to use any Python application safely for 10 years requires two major porting efforts.
Posted Sep 25, 2022 7:45 UTC (Sun)
by WolfWings (subscriber, #56790)
[Link] (16 responses)
If you have to maintain a project for 10 years? That includes updating it for security fixes, and that includes updating your code that calls other libraries or runs on scripting languages that release security fixes in that timeframe.
Or else freeze it in the digital equivalent of carbonite and hang it on the wall, including all the required executables, etc.
So if anything I'd think in 10 years support of a project using Python? You'd need to schedule a roughly annual security update of the code, stepping through each python version.
And if that's too huge a burden... then either the project was implemented in a way that can't support 10 years of maintenance (lack of test cases, etc), or there's other fundamental problems and blaming project issues on Python is akin to demanding some random 50-line library author to present entire cybersecurity documenting, 2FA validating, etc, just because some 'process' at your company requires that documentation.
Posted Sep 25, 2022 8:49 UTC (Sun)
by ballombe (subscriber, #9523)
[Link] (3 responses)
Posted Sep 26, 2022 13:13 UTC (Mon)
by hkario (subscriber, #94864)
[Link] (1 responses)
So what's your point?
Posted Sep 26, 2022 14:45 UTC (Mon)
by Wol (subscriber, #4433)
[Link]
If your Python 2.6 source was written before 2.7 was released, and still works fine on 3, then great. If you had to modify the source to work with 3, then *you* have clearly missed the point. It's not clear from what you say which is the case.
Unfortunately, I have a python utility that was written for (and runs fine on) 2.7. It crashes on - I guess 3.8? 3.9? I'm not sure which one was current when I tried it.
It's someone else's source, I don't "do" Python, so I can't ditch 2.7 until someone else fixes it for me ...
Cheers,
Posted Sep 26, 2022 18:12 UTC (Mon)
by dvdeug (guest, #10998)
[Link]
Posted Sep 25, 2022 16:16 UTC (Sun)
by cyperpunks (subscriber, #39406)
[Link] (1 responses)
Due to this, Python is more like an operating system than compiler in the tradional sense.
When a Python goes release EOL, your "operating system" is simply not supported any longer,
The only way to continue in sane manner to switch to a supported Python release.
All new Python major release comes with new issues, if you are in doubt just check this tracking bug in Fedora when moving to Python 3.10 (from Python 3.9):
https://fedoraproject.org/wiki/Changes/Python3.10
Any software written today needs a life time of more than 10 years, all software I maintain or use today have or will have a life time way longer than 10 years. As software goes older, the more value does it add: think Linux kernel, GCC, LLVM, valgrind etc. You don't want to rewrite those from scratch.
Python is used by many large projects today, just look at all the AI projects, web framework, package managers etc etc.
It's a not problem that Python evolves and creates new releases, the problem is that life time of each major release is way too short. I would set 8 years as mininum, 10 years as ok and 15 as excellent.
Posted Sep 25, 2022 20:29 UTC (Sun)
by k8to (guest, #15413)
[Link]
For things that have no real forcing function to ensure and/or naturally arrange for ongoing maintenance, it's *very* awkward.
Posted Sep 26, 2022 18:35 UTC (Mon)
by khim (subscriber, #9252)
[Link] (9 responses)
It doesn't matter how small they are: because if python dynamic nature even minor changes requires copious amount of testing. That's what python developers don't understand: it's not the required amount of changes that is the problem but the need to spend a lot of efforts trying to see if anything is broken or not. Changes to python were similar in scale to C++ transition to C++11 or C++20 and these releases are huge. Stepping incrementally through versions just increases the pain: instead of couple of significant porting efforts you now have ten of them! How many time do we need to repeat the same thing before people realize that people don't do that? You can not both push your language as “language for laymens” and expect it to only be used by software developers who would be treating it as something different from fridge or a lawnmower. Do you update your fridge for security fixes or replace pieces of your lawnmower (except when they break and need to be fixed)?
Posted Sep 27, 2022 1:57 UTC (Tue)
by WolfWings (subscriber, #56790)
[Link] (8 responses)
Posted Sep 27, 2022 9:11 UTC (Tue)
by Vipketsh (guest, #134480)
[Link] (7 responses)
Wonderful theory. Meanwhile in the real world, never have I seen code written and tested not show bugs when first put into production. In other words: no matter how well you test your code you can not find all possible bugs. Meaning there will always be hard and painful things to find, especially when behaviour of things you call change -- after all you are not testing the functions provided by others.
I'm in the world of designing hardware where there is a universal requirement to thoroughly test things before it even smells the real world (i.e. implemented in an FPGA) thus, compared to software, things are well tested by the time you get there. In the industry there are also tons of metrics which many customers expect you to use to prove how well your code is tested -- and the expectations are not small (e.g. 100% of all code lines executed, among others). Furthermore for any minor release they expect you prove everything again. Yet, the most important thing every customer expects is for you to show that your code has been implemented and used in the real world. Why ? Because everyone understands that there is no substitute for production and there are always things which pop out only then.
In summary: no matter how much you test you can not avoid long painful debug sessions inside production environments.
> writing something that will be used by others for ten years [...] requires more planning and dedication to upkeep
What an ideal world you are living in. I have never seen a SW or HW development project where the plan was to maintain things for 10+ years. The one and only plan is always to get the first version of the project out the door by the deadline. The rest will be solved later. I would love for this to change, but this is reality for many reasons.
Quite frankly, is it even possible to plan 10 years in the future in software development ? I have not seen a single project promise 10 years of maintenance -- anything you may rely on today will very likely be gone by the time 10+ years is up.
Posted Sep 27, 2022 11:36 UTC (Tue)
by kleptog (subscriber, #1183)
[Link] (3 responses)
Sure, but when upgrading across versions of python, the issue is not that things break in subtle ways. Either it works, or it blows up. As such, simple smoke-tests across your application are sufficient to detect any issues. And python makes it extremely easy to write these kinds of tests because you can literally mock everything trivially. Trying to test individual functions in a large C application can be very annoying as you somehow have to split off a chunk of the application such that it can be compiled and linked separately.
The issues that are most missed during Python version upgrades are the error paths, because they tend to be less well tested. These days though static analysis is pretty good at catching these kinds of errors.
As for the issue that drove this thread: changing /usr/bin/python to mean python3, that's literally grepping your source for #! and adding a 2. Hardly weeks of work. Or even easier, just use a virtualenv and then python can mean whatever you like..
Posted Sep 27, 2022 12:28 UTC (Tue)
by pizza (subscriber, #46)
[Link] (2 responses)
See, this tells me you've never had to deal with an even moderately complex python codebase.
As the language is dynamic, unless your "simple smoke test" results in the interpreter parsing _every single file_ then it's not even the most basic of smoke tests. Heck, even invoking every single method isn't sufficient because that doesn't guarantee that the _callers_ are doing the right thing in all cases.
Add to that that any python codebase inevitably pulls in a bazillion external pip modules, and those can and do break on a whim. So you either pin every specific version forever (which is particularly fun for python2 stuff now -- and heck, even older python3 codebases) or you'll have random "build" or runtime failures from one day's deployment to the next.
If you're not _constantly_ developing a given python codebase, it'll fall into ruin within a couple of years; That's the sad, objective truth -- My last two employers had to learn that the hard way (and they're in industries where 10+ year support windows are considered short)
Posted Sep 27, 2022 14:40 UTC (Tue)
by rahulsundaram (subscriber, #21946)
[Link]
That's overstating it quite a bit. If you have a complex Python codebase, it may need some maintenance to keep up with newer versions but how much and how often will vary depending on the features you use and the libraries you are relying on. Sometimes you do live in an ecosystem that is already heavily reliant on a language and there isn't much you can do about it but if you are starting new, I would recommend using alternatives like Go if the codebase is expected to live long and remain fairly untouched.
Posted Sep 27, 2022 14:52 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
> See, this tells me you've never had to deal with an even moderately complex python codebase.
Reality depends on the viewpoint of the observer :-)
And this discussion seems to emphasise that most strongly - "it works for me" is fine until you start dealing with a complex problem ... what did Einstein say? "Make things as simple as possible (but no simpler)".
Cheers,
Posted Sep 28, 2022 7:38 UTC (Wed)
by cladisch (✭ supporter ✭, #50193)
[Link] (2 responses)
https://www.sqlite.org/lts.html says:
Posted Sep 28, 2022 13:21 UTC (Wed)
by tzafrir (subscriber, #11501)
[Link] (1 responses)
A slightly larger project is Common Infrastructure Project, that maintains a (or some?) specific basic Linux system for 20 years. It includes a kernel, glibc, bash, mawk, and even minimal perl (although I would not be surprised if people try to get rid of perl). Surely not python.
BTW: two Debian packages not included there are:
sqlite - command line interface for SQLite 2
Posted Sep 28, 2022 17:28 UTC (Wed)
by geert (subscriber, #98403)
[Link]
Posted Sep 25, 2022 21:45 UTC (Sun)
by mpg (subscriber, #70797)
[Link]
This is really a side point (and it only strengthens your general argument), but I think you mean 1.3 - RFC 8446 is dated August 2018, so it's been 4 years now. (And real-world deployment is not too bad compared to the time it took with previous versions.)
Arch Linux drops Python 2
RHEL 9 comes with Python 3.9 as default, from page above we see:
Arch Linux drops Python 2
Ver GA EOL
3.9 2020-10-05 2025-10
3.10 2021-10-04 2026-10
3.11 2022-10-03 2027-10
3.12 2023-10-03 2028-10
RHEL 9 has 10 years of support to 2032-06.Arch Linux drops Python 2
Arch Linux drops Python 2
Arch Linux drops Python 2
Arch Linux drops Python 2
Wol
Arch Linux drops Python 2
Arch Linux drops Python 2
this means you can't safely run any Python script using this particular version of Python.
https://bugzilla.redhat.com/show_bug.cgi?id=1890881
Arch Linux drops Python 2
> I'd strongly disagree with calling them 'porting' efforts, because unlike the 2.x -> 3.x changes things are generally very mild
Arch Linux drops Python 2
Arch Linux drops Python 2
That's what python developers don't understand: it's not the required amount of changes that is the problem but the need to spend a lot of efforts trying to see if anything is broken or not.
Python devs do understand that, they just disagree with that statements assumption that testing is complex, difficult, or time consuming.
Anything that's intended to be used and run for that span of time should have testing in place and be broken into small enough pieces that such testing is simply part of development, quick to do, etc.
Python has had an included and stable unittest subsystem since 2.1. Literally over two decades now.
Stepping incrementally through versions just increases the pain: instead of couple of significant porting efforts you now have ten of them!
The difference is one of magnitude.
It's less overall work to have a smaller annual tune-up than jump multiple minor versions only updating every few years.
Just like it's cheaper to change your oil and spark plugs regularly than to wait until the engine is seizing and trying to correct everything then.
How many time do we need to repeat the same thing before people realize that people don't do that? You can not both push your language as “language for laymens” and expect it to only be used by software developers who would be treating it as something different from fridge or a lawnmower. Do you update your fridge for security fixes or replace pieces of your lawnmower (except when they break and need to be fixed)?
Both can be and are true.
Python is a language a layperson can readily pick up and do effective work with, and another layperson can see what's happening in simpler scripts easily as well.
But writing something that will be used by others for ten years as you're saying requires more planning and dedication to upkeep.
You have to replace the blades on a lawnmower, or repair the engine if the spark plug dies, or if it's let sit too long flush the gas tank, etc.
But a pair of scissors? Those might require re-sharpening every few years at most.
Both involve sharp blades and can very much be constructed initially by a layperson if given the right tools and parts. But there's a huge different in overall complexity and long-term maintenance involved.
Arch Linux drops Python 2
Arch Linux drops Python 2
Arch Linux drops Python 2
Arch Linux drops Python 2
Arch Linux drops Python 2
Wol
Very Long Term Support
> The intent of the developers is to support SQLite through the year 2050.
>
> At this writing, 2050 is still 34 years in the future. Nobody knows what will happen in that time, and we cannot absolutely promise that SQLite will be viable or useful that far out. But we can promise this: we plan as if we will be supporting SQLite until 2050. […]
Very Long Term Support
sqlite3 - Command line interface for SQLite 3
Very Long Term Support
Arch Linux drops Python 2