Stopped Getting Alerts

We had an issue where we stopped getting alerts this weekend and through some troubleshooting this morning I found the following.

$ ./alerts.php
Failed to acquire lock alerts

I ended up rebooting that server and that fixed the issue and we received the backlog of alerts that we missed but I’m unsure why it happened. Could someone possibly explain how this works so that I can figure out way to monitor for this issue in the future?

$ ./validate.php

Component Version
LibreNMS 1.45-39-g93433ce4b
DB Schema 270
PHP 7.0.27-0+deb9u1
MySQL 10.1.26-MariaDB-0+deb9u1
RRDTool 1.6.0


[OK] Composer Version: 1.7.3
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

We create locks for certain things so that other processes don’t overrun. The locks should be cleared up unless something crashes in between runs. Next time check /opt/librenms/.alerts.lock to see if it exists and has an old timestamp.

1 Like

Perfect thanks!