I’m trying to install LibreNMS on Debian 12 , everything is ok expect this error
FAIL: Python3 module issue found: 'Required packages: [‘PyMySQL!=1.0.0’, ‘python-dotenv’, ‘redis>=4.0’, ‘setuptools’, ‘psutil>=5.6.0’, ‘command_runner>=1.3.0’] Package not found: The ‘command_runner>=1.3.0’ distribution was not found and is required by the application ’
Fix:
pip3 install -r /opt/librenms/requirements.txt
I tried the fix then I get
error: externally-managed-environment
pip3 install -r /opt/librenms/requirements.txt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Pytho n installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
librenms@pol01:~$ pwd
/opt/librenms
librenms@pol01:~$ whoami
librenms
librenms@pol01:~$ uname -a
Linux pol01 6.1.0-9-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.27-1 (2023-05-08) x86_64 GNU/Linux
librenms@pol01:~$ cat /etc/debian_version
12.0
librenms@pol01:~$ pip3 install -U -r /opt/librenms/requirements.txt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
@systeembeheerder@murrant venv might be the only way that PEP 668 expects a program to use pip3 out of docker containers from now on (though I have no citation to back this claim).
Still, this patch is not enough. You need to replace all the calls to the python and the pip3 binary to the venv instances of these binaries.
They might be right that this is the only sane way to handle local admin-installed dependencies without breaking the system randomly. Still, the doc on how to proceed in detail is heavily lacking. They tell to run in a venv via source/activate but this is not doable for interactive use which is how programs are supposed to handle managing their dependencies if the system install does not already.
I know of no project that handles PEP 668 currently. So devs are left to guess how the PEP 668 is expected to be implemented by programs depending on pip dependencies.
For one the path to the venv will vary fro one install to the other so we cannot hardcode the path in the shebang of all python scripts shipped by LibreNMS.
@systeembeheerder the command to force install in the librenms user home is pip3 install --user --break-system-packages -r /opt/librenms/requirements.txt
venv looks like a solution, however it is also yet another customization you need to remember/document, however tiny it may be. Isn’t the answer that just like more and more other software the recommended way of running LibreNMS is in a docker container?
@murrant the issue with this python PEP668 is with dependencies not shipped by the distribution.
venv is not the best option in my opinion but the one I prefer seems not available in python. That is ship the dependencies into a vendor directory.
Or plain avoid depending on non-universally available dependencies.
The issue of having non-widespread dependencies is common to all languages.
To me the issue with PEP668 is that telling to run the program in the venv via source/activate is of no help to developers. We cannot expect the user to always open a terminal and enter the program command interactively to run a program. Either there is another way and it would have been way better to document long ago. Or there was no vision that this will break non-interactive programs (well you can point each shebang to the venv python and pip3 binaries. But as I told we cannot know beforehand where the user will install the program thus the venv path).
I have to admit I am not involved in the LibreNMS internals at all. Just came across the problem while upgrading to Debian 12 last night, then found this thread and tried around.
The step “Switch to user librenms and run composer_wrapper” is already part of the installation documentation, so the solution might be to just insert the pip3-command above in this part of the documentation.
Of course deeper architectural changes like you suggested (get rid of Python code) well might be worth a consideration, too. But I guess that’s a long-term thing, while more and more users will try using Debian 12 in the near future.
(On a side-note: I am just happy to finally be able to run Debian 12, as I didn’t like to fetch PHP from a different package source. More and more PHP applications require PHP 8.x.)
The composer scripts already call pip. It might be worth adding the --break-system-packages to that call in the short term as installing in the user context shouldn’t break system packages at all.