Polling broken in gd17eb4f

Hi guys,

Something in the latest or a recent daily.sh appears to have broken at least two LibreNMS installs we look after. Poller.php is failing to run at all, and there’s a number of errors in the librenms.log file -

./daily.sh
Re-running /opt/librenms/daily.sh as librenms user
Updating to latest codebase OK
Updating Composer packages OK
Updating SQL-Schema OK
Updating submodules OK
Cleaning up DB OK
Fetching notifications OK
Caching PeeringDB data OK
root@HOSTNAME:/opt/librenms# ./validate.php

Component Version
LibreNMS 1.43-15-gd17eb4f
DB Schema 262
PHP 7.0.30-0ubuntu0.16.04.1
MySQL 10.0.36-MariaDB-0ubuntu0.16.04.1
RRDTool 1.5.5
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct
[FAIL] The poller (HOSTNAME) has not completed within the last 5 minutes, check the cron job.
[WARN] Some devices have not been polled in the last 5 minutes. You may have performance issues.
[FIX] Check your poll log and see: http://docs.librenms.org/Support/Performance/
Devices:
and 606 more…
[FAIL] Some devices have not completed their polling run in 5 minutes, this will create gaps in data.
[FIX] Check your poll log and see: http://docs.librenms.org/Support/Performance/
Devices:
and 36 more…

After a full power-down and reboot of the VM, we get this first off in the librenms.log file -

[2018-09-03 22:15:01] production.ERROR: PDOException: SQLSTATE[HY000] [2002] No such file or directory in /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php:68 Stack trace: #0 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php(68): PDO->__construct(‘mysql:host=loca…’, ‘librenms’, ‘XXXXXXXXX’, Array) #1 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php(44): Illuminate\Database\Connectors\Connector->createPdoConnection(‘mysql:host=loca…’, ‘librenms’, ‘XXXXXXXX’, Array) #2 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/MySqlConnector.php(24): Illuminate\Database\Connectors\Connector->createConnection(‘mysql:host=loca…’, Array, Array) #3 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/ConnectionFactory.php(183): Illuminate\Database\Connectors\MySqlConnector->connect(Array) #4 [internal function]: Illuminate\Database\Connectors\ConnectionFactory->Illuminate\Database\Connectors{closure}() #5 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(883): call_user_func(Object(Closure)) #6 /opt/librenms/LibreNMS/DB/Eloquent.php(110): Illuminate\Database\Connection->getPdo() #7 /opt/librenms/LibreNMS/DB/Eloquent.php(78): LibreNMS\DB\Eloquent::isConnected() #8 /opt/librenms/app/Providers/AppServiceProvider.php(24): LibreNMS\DB\Eloquent::initLegacyListeners() #9 [internal function]: App\Providers\AppServiceProvider->boot() #10 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(29): call_user_func_array(Array, Array) #11 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(87): Illuminate\Container\BoundMethod::Illuminate\Container{closure}() #12 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(31): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure)) #13 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/Container.php(539): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL) #14 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(788): Illuminate\Container\Container->call(Array) #15 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(771): Illuminate\Foundation\Application->bootProvider(Object(App\Providers\AppServiceProvider)) #16 [internal function]: Illuminate\Foundation\Application->Illuminate\Foundation{closure}(Object(App\Providers\AppServiceProvider), 14) #17 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(772): array_walk(Array, Object(Closure)) #18 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/BootProviders.php(17): Illuminate\Foundation\Application->boot() #19 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(208): Illuminate\Foundation\Bootstrap\BootProviders->bootstrap(Object(Illuminate\Foundation\Application)) #20 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(267): Illuminate\Foundation\Application->bootstrapWith(Array) #21 /opt/librenms/includes/init.php(101): Illuminate\Foundation\Console\Kernel->bootstrap() #22 /opt/librenms/alerts.php(31): require(’/opt/librenms/i…’) #23 {main} [] []

Any thoughts?

Andy

Ignore this! We also export LibreNMS stats to a docker instance of Influx/Grafana which was powered down for maintenance. The fact it wasn’t online caused the poller to stop completing.

Andy