Python2 segmentation fault

Hello,

I’m getting segmentation faults caused by the poller-wrapper.py dependency on python2. I’m running Arch Linux, and if I remove the mysql-python plugin (python2) and replace it with python-mysqlclient (python3), the problem goes away, but then LibreNMS can no longer poll devices.

This is the output from ./validate.php:

user@server:/opt/librenms$ sudo ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.61
DB Schema | 2020_02_10_223323_create_alert_location_map_table (159)
PHP       | 7.3.15
MySQL     | 10.4.12-MariaDB-log
RRDTool   | 1.7.2
SNMP      | NET-SNMP 5.8
====================================

[OK]    Composer Version: 1.10.1
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
user@server:/opt/librenms$

This is the cronic output received via email:

From [email protected]  Sat Mar 14 18:15:10 2020
Return-Path: <[email protected]>
X-Original-To: librenms
Delivered-To: [email protected]
Received: by server.example.com (Postfix, from userid 975)
	id 6F5AE60E86; Sat, 14 Mar 2020 18:15:10 +0800 (+08)
From: "(Cron Daemon)" <[email protected]>
To: [email protected]
Subject: Cron <librenms@server>    /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <PATH=/usr/local/sbin:/usr/local/bin:/usr/bin>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/opt/librenms>
X-Cron-Env: <LOGNAME=librenms>
X-Cron-Env: <USER=librenms>
Message-Id: <[email protected]>
Date: Sat, 14 Mar 2020 18:15:10 +0800 (+08)

/opt/librenms/cronic: line 14: 21703 Segmentation fault      (core dumped) "$@" > $OUT 2> $TRACE
Cronic detected failure or error output for the command:
/opt/librenms/poller-wrapper.py 16

RESULT CODE: 139

ERROR OUTPUT:

STANDARD OUTPUT:

This is the start of the core dump:

Mar 14 18:20:08 server kernel: python2[21944]: segfault at 5b631a28 ip 00007f707c1a30ac sp 00007fff387bad20 error 4 in libc-2.31.so[7f707c141000+14c000]
Mar 14 18:20:08 server kernel: Code: 43 28 00 00 00 00 48 8b 54 24 08 48 89 ef 48 89 43 10 48 83 cf 01 48 89 7b 08 48 89 53 18 48 89 2c 2b 48 85 c9 74 87 48 89>
Mar 14 18:20:08 server systemd[1]: Started Process Core Dump (PID 22064/UID 0).
Mar 14 18:20:09 server CROND[21912]: pam_unix(crond:session): session closed for user librenms
Mar 14 18:20:09 server postfix/pickup[18938]: B59A360E87: uid=975 from=<librenms>
Mar 14 18:20:09 server postfix/cleanup[22074]: B59A360E87: message-id=<[email protected]>
Mar 14 18:20:09 server postfix/qmgr[625]: B59A360E87: from=<[email protected]>, size=977, nrcpt=1 (queue active)
Mar 14 18:20:09 server postfix/local[22079]: B59A360E87: to=<[email protected]>, orig_to=<librenms>, relay=local, delay=0.14, delays=0.09/0.04/0/0.0>
Mar 14 18:20:09 server postfix/qmgr[625]: B59A360E87: removed
Mar 14 18:20:10 server systemd-coredump[22065]: Process 21944 (python2) of user 975 dumped core.
                                              
                                              Stack trace of thread 21944:
                                              #0  0x00007f707c1a30ac malloc_consolidate (libc.so.6 + 0x870ac)
                                              #1  0x00007f707c1a4600 _int_free (libc.so.6 + 0x88600)
                                              #2  0x00007f707bfe69d1 dict_dealloc.lto_priv.0 (libpython2.7.so.1.0 + 0xf19d1)
                                              #3  0x00007f707bfeacc8 PyDict_SetItem (libpython2.7.so.1.0 + 0xf5cc8)
                                              #4  0x00007f707bfe6b58 _PyModule_Clear (libpython2.7.so.1.0 + 0xf1b58)
                                              #5  0x00007f707bfa0f5e PyImport_Cleanup (libpython2.7.so.1.0 + 0xabf5e)
                                              #6  0x00007f707c025688 Py_Finalize (libpython2.7.so.1.0 + 0x130688)
                                              #7  0x00007f707c022b9f Py_Main (libpython2.7.so.1.0 + 0x12db9f)
                                              #8  0x00007f707c143023 __libc_start_main (libc.so.6 + 0x27023)
                                              #9  0x000055f15a83b05e _start (python2.7 + 0x105e)

Are there any plans to move to python3?

Thanks.

There are no plans, whatever the community contributes is what happens :slight_smile: I’m sure many people would be happy to see those scripts updated to support python3 and python2. Most of the other python scripts support python3 already.

You could use this instead of the cron scripts since python2 is broken on your system. Dispatcher Service (RC) - LibreNMS Docs

Using the dispatcher service fixes the core dump problem on my system, but is it possible to tune the billing job? If I don’t use billing, can I disable the billing calculations?

$config['service_billing_frequency']           = 300;
$config['service_billing_calculate_frequency'] = 60;

Thanks.

service_billing_enabled