production.ERROR: ErrorException: unserialize - Librenms inaccessible now

Help
1

Long story short, I got noticed that daily.sh had failed, started going through steps to resolve, running validate.php, daily.sh, etc…and have ended up with this error “production.ERROR: ErrorException: unserialize” Worst I backed up the /opt/librenms directory, then did a full restore from that morning’s backup. Nothing but errors so I deleted the restored version and reverted back to the original. At least with this I am getting just one error. But now my install is inaccessible. Not a developer so I have no idea on how to proceed, but praying this is a simple fix.

[2019-02-12 19:31:48] production.ERROR: ErrorException: unserialize(): Error at offset 0 of 40 bytes in /opt/librenms/vendor/laravel/framework/src/Illuminate/Encryption/Encrypter.php:138
Stack trace:
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(8, 'unserialize(): ...', '/opt/librenms/v...', 138, Array)
#1 /opt/librenms/vendor/laravel/framework/src/Illuminate/Encryption/Encrypter.php(138): unserialize('Qz4CeyxvsxEYdIP...')
#2 /opt/librenms/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(95): Illuminate\Encryption\Encrypter->decrypt(Array)
#3 /opt/librenms/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(76): Illuminate\Cookie\Middleware\EncryptCookies->decryptCookie('eyJpdiI6IlVuWXR...')
#4 /opt/librenms/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(59): Illuminate\Cookie\Middleware\EncryptCookies->decrypt(Object(Illuminate\Http\Request))
#5 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Illuminate\Cookie\Middleware\EncryptCookies->handle(Object(Illuminate\Http\Request), Object(Closure))
#6 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#7 /opt/librenms/app/Http/Middleware/CheckInstalled.php(46): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#8 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): App\Http\Middleware\CheckInstalled->handle(Object(Illuminate\Http\Request), Object(Closure))
#9 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#10 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(102): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#11 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(574): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#12 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(533): Illuminate\Routing\Router->runRouteWithinStack(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request))
#13 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(511): Illuminate\Routing\Router->dispatchToRoute(Object(Illuminate\Http\Request))
#14 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(176): Illuminate\Routing\Router->dispatch(Object(Illuminate\Http\Request))
#15 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}(Object(Illuminate\Http\Request))
#16 /opt/librenms/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#17 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Fideloper\Proxy\TrustProxies->handle(Object(Illuminate\Http\Request), Object(Closure))
#18 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#19 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(30): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#20 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#21 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#22 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(30): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#23 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#24 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#25 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#26 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle(Object(Illuminate\Http\Request), Object(Closure))
#27 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#28 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php(46): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#29 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(148): Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode->handle(Object(Illuminate\Http\Request), Object(Closure))
#30 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#31 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(102): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#32 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(151): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#33 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(116): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter(Object(Illuminate\Http\Request))
#34 /opt/librenms/html/index.php(53): Illuminate\Foundation\Http\Kernel->handle(Object(Illuminate\Http\Request))
2

Please, post the output of ./validate.php

3

The initial problem has gone away, I was able to log in with Edge, and after complete shutdown of Chrome LibreNMS displayed. Everything appears to be working at least from the GUI standpoint, but I do get the error on unable to run daily.sh (so I am back to where I began).

Running validate.php gives me this:

====================================
Component | Version
--------- | -------
LibreNMS  | 1.48.1-32-g25954cc
DB Schema | 2019_01_16_195644_add_vrf_id_and_bgpLocalAs (131)
PHP       | 7.2.14
MySQL     | 10.0.38-MariaDB-wsrep
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================
[OK]    Composer Version: 1.8.4
[OK]    Dependencies up-to-date.
In functions.php line 2457:
Undefined index: Create Table

in logs, nothing shown now in error, and librenms.log shows normal activity with no errors, everything appears to be working except I am unable to update

4

Can you paste the output of ./daily.sh?

5

First time I ran it I got the following:

Re-running /opt/librenms/daily.sh as librenms user
Updating to latest codebase                        FAIL
error: The following untracked working tree files would be overwritten by merge:
        config/hashing.php
        config/logging.php
        config/tinker.php
Please move or remove them before you can merge.
Aborting
Updating Composer packages                         OK
Updating SQL-Schema                                OK
Updating submodules                                OK
Cleaning up DB                                     OK
Fetching notifications                             OK
Caching PeeringDB data                             OK

So I moved the 3 files to a different tmp directory and ran daily.sh again. I got the following output:

Re-running /opt/librenms/daily.sh as librenms user
Updating to latest codebase                        OK
Updating Composer packages                         OK
Updating SQL-Schema                                OK
Updating submodules                                OK
Cleaning up DB                                     OK
Fetching notifications                             OK
Caching PeeringDB data                             OK

Now the web UI displays:

Whoops, looks like something went wrong. Check your librenms.log.

Check your log for more details. (librenms.log)

And nothing is logged in either librenms.log or error_log. I again tried different browsers, different machines, nothing. The UI just doesnt work now.

6

If I run validate.php now I get:

./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.48.1-49-g3f1489b
DB Schema | Not Connected (0)
PHP       | 7.2.14
MySQL     | ?
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[FAIL]  Missing dependencies! [FIX] ./scripts/composer_wrapper.php install --no-dev
         doctrine/event-manager
         symfony/polyfill-php72
         symfony/contracts
         egulias/email-validator
         psr/simple-cache
         psr/container
         opis/closure
         laravel/slack-notification-channel
         zendframework/zend-diactoros
         php-http/promise
          and 6 more...
[FAIL]  Outdated dependencies [FIX] ./scripts/composer_wrapper.php install --no-dev
         doctrine/cache
         doctrine/dbal
         doctrine/inflector
         vlucas/phpdotenv
         symfony/css-selector
         symfony/var-dumper
         symfony/routing
         symfony/process
         symfony/http-foundation
         symfony/event-dispatcher
          and 15 more...
[OK]    Composer Version: 1.8.4
[FAIL]  Missing dependencies!
        [FIX]:
        composer install --no-dev
        Dependencies:
         doctrine/event-manager
         symfony/polyfill-php72
         symfony/contracts
         egulias/email-validator
         psr/simple-cache
         psr/container
         opis/closure
         laravel/slack-notification-channel
         zendframework/zend-diactoros
         php-http/promise
         php-http/httplug
         php-http/guzzle6-adapter
         lcobucci/jwt
         nexmo/client
         laravel/nexmo-notification-channel
          and 1 more...

If I run the fix ./scripts/composer_wrapper.php install --no-dev I end up after when I run validate.php:

 ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.48.1-49-g3f1489b
DB Schema | 2019_01_16_195644_add_vrf_id_and_bgpLocalAs (131)
PHP       | 7.2.14
MySQL     | 10.0.38-MariaDB-wsrep
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.8.4
[OK]    Dependencies up-to-date.

In functions.php line 2457:
Undefined index: Create Table

BUT I can log in again now.

7

I’ve added a workaround for the issue you were running into with validate. (which is hiding the real issue)

Can you run ./daily.sh and ./validate.php again?

8

AWESOME! it looks like everyone is working now, running both daily.sh and then validate.php show no errors now, and logging in with test alarms is successful.

Thank you so much for your help, I really appreciate it.