Poller Failure, Ubuntu Server

OK, this one is quite odd. On a few machines (running Ubuntu server, not desktop - more on that below) … discovery works fine, but I get the errors below for poller (.php),

In Connection.php line 664:

  SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: '' (SQL: select exists(select * from `alert_schedule` inner join `alert_schedulables` on `
  alert_schedule`.`schedule_id` = `alert_schedulables`.`schedule_id` where ((`recurring` = 0 and `start` <= NOW() and `end` >= NOW()) or (`recurring` =
   1 and (`start_recurring_dt` <= date_format(NOW(), '%Y-%m-%d') and (`end_recurring_dt` >= date_format(NOW(), '%Y-%m-%d') or `end_recurring_dt` is nul
  l or `end_recurring_dt` = 0000-00-00 or `end_recurring_dt` = )) and `start_recurring_hr` <= date_format(NOW(), '%H:%i:%s') and `end_recurring_hr` >=
  date_format(NOW(), '%H:%i:%s') and (`recurring_day` like CONCAT('%', date_format(NOW(), '%w'), '%') or `recurring_day` is null or `recurring_day` = )
  )) and ((`alert_schedulable_type` = device and `alert_schedulable_id` = 64) or (`alert_schedulable_type` = device_group and 0 = 1))) as `exists`)


In PDOStatement.php line 119:

  SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: ''


In PDOStatement.php line 117:

  SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: ''

Now, the LibreNMS machine has just been updated, to Ubuntu v19.10 (and php v7.3) - this may be part of the issue? validate.php output below,

====================================
Component | Version
--------- | -------
LibreNMS  | 1.56-44-ge40a33631
DB Schema | 2019_02_05_140857_remove_config_definition_from_db (145)
PHP       | 7.3.8-1
MySQL     | 8.0.17-0ubuntu2
RRDTool   | 1.7.1
SNMP      | NET-SNMP 5.7.3
====================================

[OK]    Composer Version: 1.9.0
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
[WARN]  Your local git contains modified files, this could prevent automatic updates.

But what is a bit odd - on the trouble machines, the date output is as follows,
Sat Oct 19 21:07:14 CDT 2019

But on the LibreNMS machine,
Sat 19 Oct 2019 09:06:15 PM CDT

Thoughts?

Thanks!

OK, I take that back - I think all machines are failing now (poller). Trying to dig into it!

FYI, it seems to be in Alerts =>

#### Start Alerts ####
SQL[select * from `devices` where `devices`.`device_id` = ? limit 1 [64] 0.57ms]

SQL[select `device_groups`.*, `device_group_device`.`device_id` as `pivot_device_id`, `device_group_device`.`device_group_id` as `pivot_device_group_id` from `device_groups` inner join `device_group_device` on `device_groups`.`id` = `device_group_device`.`device_group_id` where `device_group_device`.`device_id` = ? [64] 0.6ms]

SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: '' (SQL: select exists(select * from `alert_schedule` inner join `alert_schedulables` on `alert_schedule`.`schedule_id` = `alert_schedulables`.`schedule_id` where ((`recurring` = 0 and `start` <= NOW() and `end` >= NOW()) or (`recurring` = 1 and (`start_recurring_dt` <= date_format(NOW(), '%Y-%m-%d') and (`end_recurring_dt` >= date_format(NOW(), '%Y-%m-%d') or `end_recurring_dt` is null or `end_recurring_dt` = 0000-00-00 or `end_recurring_dt` = )) and `start_recurring_hr` <= date_format(NOW(), '%H:%i:%s') and `end_recurring_hr` >= date_format(NOW(), '%H:%i:%s') and (`recurring_day` like CONCAT('%', date_format(NOW(), '%w'), '%') or `recurring_day` is null or `recurring_day` = ))) and ((`alert_schedulable_type` = device and `alert_schedulable_id` = 64) or (`alert_schedulable_type` = device_group and 0 = 1))) as `exists`) {"exception":"[object] (Illuminate\\Database\\QueryException(code: HY000): SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: '' (SQL: select exists(select * from `alert_schedule` inner join `alert_schedulables` on `alert_schedule`.`schedule_id` = `alert_schedulables`.`schedule_id` where ((`recurring` = 0 and `start` <= NOW() and `end` >= NOW()) or (`recurring` = 1 and (`start_recurring_dt` <= date_format(NOW(), '%Y-%m-%d') and (`end_recurring_dt` >= date_format(NOW(), '%Y-%m-%d') or `end_recurring_dt` is null or `end_recurring_dt` = 0000-00-00 or `end_recurring_dt` = )) and `start_recurring_hr` <= date_format(NOW(), '%H:%i:%s') and `end_recurring_hr` >= date_format(NOW(), '%H:%i:%s') and (`recurring_day` like CONCAT('%', date_format(NOW(), '%w'), '%') or `recurring_day` is null or `recurring_day` = ))) and ((`alert_schedulable_type` = device and `alert_schedulable_id` = 64) or (`alert_schedulable_type` = device_group and 0 = 1))) as `exists`) at /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php:664, Doctrine\\DBAL\\Driver\\PDOException(code: HY000): SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: '' at /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:119, PDOException(code: HY000): SQLSTATE[HY000]: General error: 1525 Incorrect DATE value: '' at /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:117)"}

Trying to wind my way through it, see what is going on.

OK, found where it is! if I make the (temporary) change below, my problems are resolved (return 0 for isMaintenance). Thoughts?

    public static function isMaintenance($device_id)
    {
        return 0;
        $device = Device::find($device_id);
        return !is_null($device) && $device->isUnderMaintenance();
    }

Thanks!

Basically, the $device->isUnderMaintenance() is returning true. So you probably have a maintenance defined in the GUI. If not visible in GUI, check in the DB.

Hi,

The issue isn’t about returning true vs. false - rather the error messages (and abort) when it tries to run this routine (isUnderMaintenance()) … or am I missing the point?

Thanks!

Correct. My bad.

SQL seems wrong here :
or end_recurring_dt = ))

Equal should be followed with a value.

So you need definitly to check what you have in GUI, and if nothing is visible in maintenances, then check the DB.

Please let us know what you find exactly, so we can improve the sanity checks to avoid this situation later.

Good eye! I missed that. OK, here is what I find,

  1. GUI - nothing there. In fact, on the Scheduled Maintenance page … nothing, and it says “Showing 0 to 0 of 0 entries” (as expected)
  2. DB - checked alert_schedule, it shows " This table contains no data.". Also as expected. alert_schedulables is also the same. Not sure what else to check, but all looks empty (as I would have expected, I haven’t added anything on purpose).

Thoughts?

Thanks!

Weird. I have no more ideas unfortunately. Let see if somebody else have suggestions.

Hi,

Sorry to pester, but just checked - this failure still exists. My hack works, but it’s not pretty of course … ;-). Any thoughts?

Thanks!

I have the same issue. After seeing this post I too broke the alerts after mysql 8 upgrade. But thank you for the temporary fix in Post 4

if end_recurring_dt = )) is the possible culprit where is the SQL line being generated?

Running ubuntu 18.04, validate output:
====================================
Component | Version
--------- | -------
LibreNMS | 1.59
DB Schema | 2019_12_16_140000_create_customoids_table (152)
PHP | 7.2.24-0ubuntu0.18.04.1
MySQL | 8.0.18
RRDTool | 1.7.0
SNMP | NET-SNMP 5.7.3
====================================

[OK]    Composer Version: 1.9.1
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
[WARN]  Your local git contains modified files, this could prevent automatic updates.
        [FIX]:
        You can fix this with ./scripts/github-remove
        Modified Files:
         LibreNMS/Alert/AlertUtil.php