Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Posted Aug 1, 2017 16:34 UTC (Tue) by fratti (guest, #105722)Parent article: Fedora ponders the Python 2 end game
Posted Aug 1, 2017 16:44 UTC (Tue)
by josh (subscriber, #17465)
[Link] (25 responses)
Posted Aug 1, 2017 16:45 UTC (Tue)
by josh (subscriber, #17465)
[Link]
Posted Aug 1, 2017 16:55 UTC (Tue)
by fratti (guest, #105722)
[Link] (8 responses)
Posted Aug 1, 2017 17:10 UTC (Tue)
by josh (subscriber, #17465)
[Link] (2 responses)
Posted Aug 1, 2017 17:12 UTC (Tue)
by fratti (guest, #105722)
[Link] (1 responses)
Posted Aug 1, 2017 17:19 UTC (Tue)
by josh (subscriber, #17465)
[Link]
Posted Aug 2, 2017 12:50 UTC (Wed)
by ballombe (subscriber, #9523)
[Link] (4 responses)
Posted Aug 2, 2017 14:55 UTC (Wed)
by josh (subscriber, #17465)
[Link] (3 responses)
Posted Aug 3, 2017 20:17 UTC (Thu)
by hmh (subscriber, #3838)
[Link] (2 responses)
Posted Aug 3, 2017 20:38 UTC (Thu)
by karkhaz (subscriber, #99844)
[Link] (1 responses)
$ find / -maxdepth 1 -type l -exec ls -l {} \;
Posted Aug 4, 2017 5:33 UTC (Fri)
by josh (subscriber, #17465)
[Link]
Posted Aug 1, 2017 17:00 UTC (Tue)
by karkhaz (subscriber, #99844)
[Link] (7 responses)
Posted Aug 1, 2017 17:12 UTC (Tue)
by josh (subscriber, #17465)
[Link] (6 responses)
Posted Aug 1, 2017 17:18 UTC (Tue)
by liw (subscriber, #6379)
[Link] (4 responses)
System-installed software shouldn't break when a user installs things early on in their $PATH.
Posted Aug 1, 2017 17:22 UTC (Tue)
by josh (subscriber, #17465)
[Link]
But there's a limit to how much you can protect the user from their own attempts to break things.
(That said, I've certainly seen more than a few reports of brokenness that ultimately got tracked down to a user's local installation of python in /usr/local or $HOME, so I mostly agree.)
Posted Aug 1, 2017 23:39 UTC (Tue)
by flussence (guest, #85566)
[Link] (2 responses)
The package manager is /usr/bin/emerge. That's a wrapper script (written in python, invoked by a C binary that chases symlinks to figure out which version of python to call it with) that chases symlinks to figure out where the real program is (which is also a python script, but can be installed for one or more of python2/3.x/pypy simultaneously).
Now and again there's support requests from users who mess up their system by manually installing python packages as root outside the OS's control, and have ended up with something in the dependency chain installing a /usr/local/bin/emerge (also a python script) that does something completely different, but is higher priority in $PATH, and gives vague errors that don't make it immediately obvious that the wrong thing's being run.
Posted Aug 2, 2017 3:38 UTC (Wed)
by wahern (subscriber, #37304)
[Link] (1 responses)
POSIX created the getconf(1) utility just so you could do PATH="$(getconf PATH)" to get the default system provided PATH guaranteed to provide the POSIX utilities. It's one of the rare instances where they didn't just codify existing practice. Perhaps for that reason it's an underused and virtually unknown command. Theoretically Linux distributions could extend it so that something like getconf PYTHON2 prints the default python2 interpreter (if available). That doesn't by itself solve the problem of a simple shebang line, but I think it's the least distributions could do to meet developers half-way.[1]
For Lua scripts I often abuse a coincidence of shell and Lua syntax so I can do
where _=[[ begins a multiline string in Lua but is a harmless assignment in shell. Because the semantics of shell syntax effectively require parsing the file line-by-line, the shell code never reaches the terminating ]] or subsequent Lua code as long as you invoke exec or exit before reaching ]]. I'm not familiar with Python syntax but perhaps something similar can be done.
[1] Distributions could standardize on the "python2" command name, but you still run into the problem of a bad PATH variable. Ideally you'd be able to do something like #!/usr/bin/env command -p python2, where the -p switch to the command utility only searches the default PATH from getconf PATH. And where env(1) locates command(1), as command(1) is usually a built-in and even when it's available as an independent utility I don't know if you can expect it to be in /bin or /usr/bin--not in the same way that env(1) is reliably at the absolute path /usr/bin/env. But Linux is one of the few Unix systems that concatenate all the shebang command argument words before invoking the interpreter, so the above will work on macOS and some other kernels, but not on Linux.
Posted Aug 2, 2017 8:26 UTC (Wed)
by itvirta (guest, #49997)
[Link]
> Ideally you'd be able to do something like
Except that you can't do that in Linux, the kernel only passes one argument from the hashbang line.
I.e. the line above would run "
Posted Aug 2, 2017 15:21 UTC (Wed)
by drag (guest, #31333)
[Link]
It's only part of the solution. If you hardcode the paths then it pretty much forces users to edit your program to suite their environment.
$ which python2
$ which python3
The situation here is that distributions really don't solve these issues for end users. If users want to be able to take full advantage of the python ecosystem for writing and running their software they are going to have to use a alternative solution.
If you only really to write complex python software for a particular Linux distribution (for a enterprise environment, for example) then trying to only use dependencies in the distribution is fine and is very likely to work. But if your goal is to have something that works on a wide variety of Linux distributions, then you end up stuck with a more 'DIY' approach. Making your software available via pip is really what is going to make it easiest for people to use your software at this point.
Posted Aug 2, 2017 15:10 UTC (Wed)
by drag (guest, #31333)
[Link] (4 responses)
This one of the reasons why people who care about portability between Linux distributions and want to be productive end up using things like pip, pvenv, and pyenv instead of distribution-provided dependencies for anything complex.
> so you can't always use "#!/usr/bin/python2" (or for that matter "#!/usr/bin/python3"), either.
Unless you are writing scripts meant to be shipped part of the operating system then you really really need to avoid using any sort of hard-coded paths to python interpreters. There is absolutely meaningful differences you need to take into consideration with this sort of 'part of the OS' versus 'on the OS'. If you are writing as part of the OS distribution.. use hard coded paths. If you are writing programs as a user of the OS then use '#!/usr/bin/env python'.
Posted Aug 2, 2017 15:39 UTC (Wed)
by karkhaz (subscriber, #99844)
[Link] (3 responses)
The Debian python packaging guidelines doesn't mention it, and neither even does the Arch Linux one---remarkable because /usr/bin/python on Arch is a symlink to /usr/bin/python3 (which itself is a symlink to /usr/bin/python3.6). There was a bit of breakage back when Arch did that, I suppose similar issues to when Debian switched their /bin/sh to a proper POSIX shell.
Posted Aug 2, 2017 16:42 UTC (Wed)
by liw (subscriber, #6379)
[Link] (1 responses)
Posted Aug 2, 2017 18:44 UTC (Wed)
by drag (guest, #31333)
[Link]
You want to avoid having these programs be broken by user's shell environments as much as possible.
Posted Aug 3, 2017 12:45 UTC (Thu)
by ewan (guest, #5533)
[Link]
Posted Aug 4, 2017 4:12 UTC (Fri)
by anatolik (guest, #73797)
[Link] (1 responses)
Which one does not do it and why? There is a PEP that says all python distributions should provide "versioned python" binary. Even macports has pythonX nowdays.
Posted Aug 22, 2017 8:21 UTC (Tue)
by mgedmin (subscriber, #34497)
[Link]
Posted Aug 2, 2017 6:33 UTC (Wed)
by joib (subscriber, #8541)
[Link] (1 responses)
Posted Aug 13, 2017 14:59 UTC (Sun)
by Wol (subscriber, #4433)
[Link]
It would certainly make life easier if all distro-supplied stuff used a shebang explicitly to call the version it wanted. But that's a lot of work catching all instances ...
Cheers,
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
lrwxrwxrwx 1 root root 7 Mar 26 22:57 /lib64 -> usr/lib
lrwxrwxrwx 1 root root 7 Mar 26 22:57 /lib -> usr/lib
lrwxrwxrwx 1 root root 7 Mar 26 22:57 /sbin -> usr/bin
lrwxrwxrwx 1 root root 7 Mar 26 22:57 /bin -> usr/bin
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
#!/bin/sh
_=[[
# shell script code to locate a Lua interpreter
]]
-- Lua code
Fedora ponders the Python 2 end game
#!/usr/bin/env command -p python2
/usr/bin/env" with "command -p python2" as one argument
(plus the name of the script as the second argument).
Or well, you could, if this "env" would split the argument to pieces itself.
Fedora ponders the Python 2 end game
/home/user/.pyenv/shims/python2
/home/user/.pyenv/shims/python3
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
I'm now wondering whether the collective wisdom in this thread is codified anywhere?
It's somewhat codified in PEP-0394, which advises on how python should be installed (in short, don't make 'python' point to Python 3) and how to cope with the fact that it might (the less than completely helpful suggestion to make anything that just asks for 'python' be compatible with both Python 2 and Python 3).
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Fedora ponders the Python 2 end game
Wol
