When working with multiple Python versions installed on the same machine, on Windows I use pyenv, but on Linux I prefer to do an altinstall. This means downloading the Python source code and building it. It may sound scary, but it's pretty simple. Once done you'll have a new Python installation in /usr/local/bin - /usr/local/lib that will not interfere with the system/default one (the one that came "from factory" with your Linux distribution, that is used by the Operating System itself and that should not be modified) that is installed on /usr/bin - /usr/lib. Just use use python3.xx to invoke that newly installed version, and python3 to invoke the system Python.
What feels odd is that it's not something that is explained in so many places. The official Python documentation just mentions this. The more detailed instructions that I've always followed are here, and as it explains installation is that simple as this:
sudo apt install build-essential zlib1g-dev \ libncurses5-dev libgdbm-dev libnss3-dev \ libssl-dev libreadline-dev libffi-dev wget https://www.python.org/downloads/release/python-3xxx/Python-3.xx.x.tar.xz tar xf Python-3.xx.x.tar.xz ./configure make altinstall
The first step is particularly important. Python comes with python modules that use native modules, and to compile those native modules they need some -dev packages installed on your system (these source packages contain mainly C header files), otherwise the compilation of those modules will fail and you'll have an incomplete installation that will cause errors when trying to import those missing modules. If you plan to use sqlite on your system, given that the sqlite module that is part of the Python distribution depends on a native module, in order to compile it you must add this: libsqlite3-dev to the list of dependencies to install that I listed above.
My work laptop (the one provided by my employer I mean) is still a Windows one. I have no problem with that, I used to have good knowledge of Windows internals, and even now that I'm more of a Linux person (all my personal computers are Linux based) I still consider that Windows architecture is really good (though I've come to distaste the updates system, the restore points, the UI...). That said, I'm using WSL2 more and more these days. I have Python3.13 installed as an altinstall on it and it's been working perfectly fine for testing on linux stuff that I develop on Windows. The other day I went one step further and wanted to debug that code on linux. Your Windows VS Code can work with folders on your WSL2 installation just in the same way it works with code on a remote linux machine. The WSL extension works in combination with the Remote SSH extension, installing to your $HOME/.vscode-server/ folder in WSL2 the code it needs on the linux side (same as it does when working with any remote Linux server). I think all this remote development is something that a few years back one could not dream about.
With VS Code and the WSL extension combined, VS Code’s UI runs on Windows, and all your commands, extensions, and even the terminal, run on Linux. You get the full VS Code experience, including autocomplete and debugging, powered by the tools and compilers installed on Linux.
The thing is that when trying to run under the debugger my code on WSL2 I was confronted with this
debugpy/launcher/debuggee.py", line 6, in module
import ctypes
File "/usr/local/lib/python3.10/ctypes/__init__.py", line 8, in module
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
Initially I was thinking it would be some problem of the debugger itself, some issue with the amazing "remote development experience" that was making it fail to find that module, but just jumping into a WLS2 terminal, opening a Python3.13 REPL and trying to import _ctypes was causing the same error. So that _ctypes module was really missing on my Python3.13 WSL2 altinstallation.
Jumping to my main Ubuntu personal laptop, with also a Python3.13 altinstallation and importing _ctypes I got:
$ python3.13 Python 3.13.0 (main, Nov 9 2024, 16:10:52) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import _ctypes >>> _ctypes.__file__ '/usr/local/lib/python3.13/lib-dynload/_ctypes.cpython-313-x86_64-linux-gnu.so'
lib-dynload seems to be where native modules get installed (I can see also for example the sqlite.so), si if the ctypes.so is missing is that some necessary -dev package was missing when I did my altinstall on WSL2. For the sake of experiment I decided to just copy the _ctypes___.so from my laptop to the windows WSL laptop. Doing that, I got another missing module, libffi, that is imported by _ctypes. Doing a sudo apt list --installed | grep libffi I see that there's not a libffi-dev package installed on my WSL2, so somehow when a time ago I installed the different -dev packages needed to compile Python I missed to install it (so the Python compilation could not create that libffi.so into lib-dynload), and the issue had not hit me until now. To fix the problem I installed libffi-dev, uninstalled python3.13 and did a new altinstall. It works like a charm now.
There does not seem to be an automatic mechanism to uninstall a Python version installed as altinstall (a Python install takes little space and indeed I assume that I could just have do a new install without removing the existing one and it would get correctly updated), but anyway, as explained here I removed this list of folders/files:
directory /usr/local/lib/python3.13
directory /usr/local/include/python3.13
file /usr/local/lib/libpython3.13.a
file /usr/local/lib/pkgconfig/python-3.13-embed.pc
6 files /usr/local/bin/*3.13*
While checking this thing of the missing native module (.so) I also used these commands:
lsof -p [PID] | grep .so to see the shared objects loaded by a process (lsof was an old friend of mine)
readelf -d (this was new to me. It gives you information about an elf binary file (executable or shared object, the equivalent to a windows PE file), and among that information you can see the shared objects needed by that binary, eg:
readelf -d _ctypes.cpython-313-x86_64-linux-gnu.so $ readelf -d _ctypes.cpython-313-x86_64-linux-gnu.so Dynamic section at offset 0x21cf8 contains 25 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libffi.so.8] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000c (INIT) 0x7000
No comments:
Post a Comment