Hi everyone,
Having a major problem with the database. All of sudden ibdata1 started rapidly growing (1GB per hour) and despite stopping syslog and doing all possible purges I was not able to resolve this, so I’ve restored the Libre instance from a snapshot and now not able to update the SQL-Schema after adding extra 20GB to the filled disk:
[root@librenms librenms]# sudo -u librenms ./daily.sh
Could not connect to database, check logs/librenms.log.
Updating to latest codebase OK
Updating Composer packages OK
Updated from d303174 to 36b4d16 OK
Updating SQL-Schema
It just hangs forever and nothing happens.
Validate shows the following:
[root@librenms librenms]# sudo -u librenms ./validate.php
| Component | Version |
|---|---|
| LibreNMS | 1.47-19-g36b4d16 |
| DB Schema | 277 |
| PHP | 7.0.31 |
| MySQL | 5.5.56-MariaDB |
| RRDTool | 1.4.8 |
| SNMP | NET-SNMP 5.7.2 |
====================================
[OK] Composer Version: 1.8.0
[OK] Dependencies up-to-date.
[OK] Database connection successful
[FAIL] Your database schema (277) is older than the latest (278).
[FIX]:
Manually run ./daily.sh, and check for any errors.
[WARN] Your local git contains modified files, this could prevent automatic updates.
[FIX]:
You can fix this with ./scripts/github-remove
Modified Files:
bootstrap/cache/.gitignore
storage/app/.gitignore
storage/app/public/.gitignore
storage/debugbar/.gitignore
storage/framework/cache/.gitignore
storage/framework/sessions/.gitignore
storage/framework/testing/.gitignore
storage/framework/views/.gitignore
storage/logs/.gitignore
WebUI is down and storage is still filling up at approx 1GB per hour:
[root@librenms librenms]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/cl_bchqmon01-root 70G 52G 19G 75% /
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 9.2M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/sda1 1014M 288M 727M 29% /boot
/dev/mapper/cl_bchqmon01-home 26G 34M 26G 1% /home
tmpfs 782M 0 782M 0% /run/user/996
tmpfs 782M 0 782M 0% /run/user/0
Could somebody please assist? I’ve tried to update the SQL schema manually from here: https://github.com/librenms/librenms/blob/master/sql-schema/278.sql but it just hangs the same way as when running daily.sh