Can't login to Web interface

Hi. I run a number of LibreNMS servers, since the last auto-update I can no longer access the web interface of any of them. The login page displays, I try the admin account I setup, the circle goes on and on and then eventually stops, staying on the login screen.

The admin password is correct and the servers continue to successfully poll devices etc. The Web interface is accessed via port 80 http.

They have been running for months now fine.

If I enter an incorrect password, the system reports the login error. Entering the correct password just circles and eventually stops staying on the login screen.

All these servers are running on the latest CentOS 7 fully patched, and validate fine:

-bash-4.2$ ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.40
DB Schema | 251
PHP       | 7.2.5
MySQL     | 5.5.56-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.6.5
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct

I’ve tried to trouble-shoot looking at nginx logs, librenms logs, etc and can’t find any issues.

I’ve tried different Web browsers, cleared cookies and caches, etc, all same result.

This is a major issue, if any has any ideas to trouble-shoot such a problem please advise.

A few extra checks

sudo chown -R librenms:librenms /opt/librenms

Are you using the default auth mechanism? or are you using auth modules like https://github.com/librenms/librenms/blob/master/doc/Extensions/Authentication.md

mariadb is running?

./daily.sh , any errors?

Hi Chas,

Thanks for your reply.

Yes I re-ran the chown.

Am using default auth mechanism.

MariaDB that comes with CentOS 7 is used:

mariadb-5.5.56-2.el7.x86_64
mariadb-libs-5.5.56-2.el7.x86_64
mariadb-server-5.5.56-2.el7.x86_64

The daily has no errors:

-bash-4.2$ ./daily.sh
Updating to latest release                         OK
Updating Composer packages                         OK
Updating SQL-Schema                                OK
Updating submodules                                OK
Cleaning up DB                                     OK
Fetching notifications                             OK
Caching PeeringDB data                             OK

Unfortunately still can’t get into the web interface.

Anything else I could look at ?

Very strange

Did you ever use two factor authentication?

systemctl status mariadb (running?)

Try add a new admin user like this:
./adduser.php testuser password 10

in Chrome: right click->inspect->console (now login and see if any errors appear in this javascript console)

Try a different browser

Are you on the daily update or the monthly update?

(If daily, then there has only been 3 closed pull requests in the last 3 days. it seems unlikely that the last 3 would have caused an issue)

Fixed missing var declaration for description search in FDB tables
Device: Adding sensors to omnitron iconverter
Fix storing metrics for SMART

Could be worth checking recent package upgrades on your OS, did php update etc.

Also i take it this isn’t related to the Laravel upgrade from 1 month ago? New Requirements for 1.40 you applied php mbstring etc?

Hi,

No, never used two factor authentication.

MariaDB running:

 systemctl status mariadb
â—Ź mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-05-24 00:25:19 UTC; 2 weeks 4 days ago
 Main PID: 9150 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─9150 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─9361 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/v...

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

Adding user:

-bash-4.2$ ./adduser.php testuser password 10
User testuser added successfully

Went to inspect in Chrome and logged in with both admin and testuser, no errors.

Tried Firefox, Chrome, Edge.

Update period:

## To enable stable tree updates only to LibreNMS
$config['update_channel'] = 'release';

For the OS, the last major update was done to go from CentOS 7.4 to 7.5 on 21 May 2018. webtatic’s php72w was also updated at that time. It’s possible that could have affected it.

I was able to use the system as it’s always logged in for me, I got a complaint with others tried to login using the admin credentials, saying they can’t. Once I rebooted my Windows workstations (3 of them) then I couldn’t log in either. So the session cookies for my browser were active and kept me logged in.

It could have been that or the updates 21 May, I can’t be sure.

Yes I prepared for that update and the package is installed:

php72w-cli-7.2.5-1.w7.x86_64
php72w-common-7.2.5-1.w7.x86_64
php72w-fpm-7.2.5-1.w7.x86_64
php72w-gd-7.2.5-1.w7.x86_64
php72w-intl-7.2.5-1.w7.x86_64
php72w-mbstring-7.2.5-1.w7.x86_64
php72w-mysql-7.2.5-1.w7.x86_64
php72w-pdo-7.2.5-1.w7.x86_64
php72w-process-7.2.5-1.w7.x86_64
php72w-snmp-7.2.5-1.w7.x86_64
php72w-xml-7.2.5-1.w7.x86_64

Any compatibility issues with LibreNMS and PHP 7.2 ?

Hi,

I ran the inspect through Firefox and got this:

Error loading this URI: Could not load the source for http://eluxer.net/code?id=105&subid=51807_6464_.
[Exception... "Component returned failure code: 0x80470002 (NS_BASE_STREAM_CLOSED) [nsIInputStream.available]"  nsresult: "0x80470002 (NS_BASE_STREAM_CLOSED)"  location: "JS frame :: resource://devtools/shared/base-loader.js -> resource://devtools/shared/DevToolsUtils.js :: onResponse :: line 562"  data: no]
Stack: onResponse@resource://devtools/shared/base-loader.js -> resource://devtools/shared/DevToolsUtils.js:562:23
onStopRequest@resource://gre/modules/NetUtil.jsm:126:17
Line: 562, column: 0

for:

eluxer.net

The “console” section in Firefox keeps flashing:

code:1:1104

continuously.

Does that mean anything to you?

Also when I press F5 refresh of the page in Firefox I get this:

https://pasteboard.co/Hplz57R.jpg

then it goes away back to the flashing in/out of code:1:1104.

Hmm, i think that error might be related to some other problem

I had a google for eluxer seems to be associated to malware/browser malware

#https://www.virustotal.com/#/domain/eluxer.net

#https://www.reddit.com/r/webdev/comments/7zqcdl/question_what_can_i_do_about_an_impostor_site/?st=jia0xsfy&sh=537b1a8b

If you don’t see it in chrome, maybe a firefox thing?

Although probably not related to your WebGUI problem, worth trying to access LibreNMS from another computer to rule out.

As your on the monthly update, i think your issue is related to the laravel update, it’s just odd that you’re not presented with any web errors. Please run these checks::
Look in your .env file vim /opt/librenms/.env Do you have an APP-KEY set?
Run ./scripts/composer_wrapper.php install --no-dev any errors?
Check sestatus output, if current mode: enforcing, then you will need to enter some further permissions

Hi Chas,

I spent a couple of hours today trying to trouble-shoot this, from fiddling with php-fpm (changing from socket to tcp connection, trying other things) to downgrading from php 7.2 to 7.1, same issues unfortunately. I also verified session files are created in /var/lig/php/session and although running nginx (and group ownership is apache for those session files), I changed the group ownership to nginx and still had the same problem.

Yes I checked .env and an APP_KEY is set, base64:long string

I ran the composer_wrapper and got:

-bash-4.2$ ./scripts/composer_wrapper.php install --no-dev
> LibreNMS\ComposerHelper::preInstall
Loading composer repositories with package information
Installing dependencies from lock file
Nothing to install or update
Generating autoload files
> LibreNMS\ComposerHelper::postInstall
> Illuminate\Foundation\ComposerScripts::postInstall
> php artisan optimize
Generating optimized class loader
The compiled services file has been removed.

I have sestatus active, however I have used sealaert -a /var/log/audit.log to clear up any selinux issues, all is allowed.

I have also tried turning SELinux off, rebooting, same problem unfortunately.

What I notice is that even though I’m on the login screen, it refreshes at the set time it’s meant to refresh the browser window (if I was actually logged in), which makes no sense doing this on a login screen.

I’ll verify with other team members if they can access the interface, they have different machines, but the original report about this problem they sent to me was because they couldn’t get past the login screen, while I was logged in (and never log out), accessing the environment daily. Only when I rebooted my workstation did I then experience the problem they experienced (this problem now).

Prior to that I was in and configuring, using, maintaining, etc without any issues for weeks.

Out of interest, try and login on the librenms server itself.

e.g
yum install w3m
w3m localhost
Use arrow keys to navigate fields
press enter on the user name/password fields, enter credentials
image
hit login

I would also run the the selinux commands just in case it does something magical

Hi Chas. Thanks for all the suggestions so far… really want to regain access to the interface again.

I went through that, installed w3m and tried the login, same thing, just logs in and gives the login screen again.

I also re-ran the SELinux commands, then tried again to login successfully via the GUI.

Unfortunately, still does the same thing.

I’m thinking of getting rid of Nginx and replacing it with Apache, what are your thoughts on this?

The install is simply based off the link you provided for installing LibreNMS on CentOS 7, so it should be the same as everyone else in the world that did it that way.

Do you have any weird redirects in your config?

You could try the apache route, or post your nginx configuration in here.

I’m on CentOS 7 with nginx version: nginx/1.14.0 and the same version as you 7.2.5-1.w7

You could also try commenting out this line is config.php
# $config['base_url'] and try access your librenms by IP address

If this line is set incorrectly it will prevent the web interface being usable from any other hostname

Also check if your filesystem is full df -H , and available memory remaining on your box free -m

I saw another user had something similar with a full disk

Hi Jarod. Not as far as my checks go. Just standard stuff with the /etc/nginx/conf.d/librenms.conf file from Nginx showing:

server {
 listen      80;
 server_name _our.internal-ip.no.hostname_;
 root        /opt/librenms/html;
 index       index.php;

 charset utf-8;
 gzip on;
 gzip_types text/css application/javascript text/javascript application/x-javascript image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;
 location / {
  try_files $uri $uri/ /index.php?$query_string;
 }
 location /api/v0 {
  try_files $uri $uri/ /api_v0.php?$query_string;
 }
location ~ \.php {
  include fastcgi.conf;
  fastcgi_split_path_info ^(.+\.php)(/.+)$;
  fastcgi_pass unix:/var/run/php-fpm/php7.2-fpm.sock;
 }
 location ~ /\.ht {
  deny all;
 }
}

And our /etc/nginx/nginx.conf file:

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

##    server {
##        listen       80 default_server;
##        listen       [::]:80 default_server;
##        server_name  _;
##        root         /usr/share/nginx/html;
##
##        # Load configuration files for the default server block.
##        include /etc/nginx/default.d/*.conf;
##
##        location / {
##        }
##
##        error_page 404 /404.html;
##            location = /40x.html {
##        }
##
##        error_page 500 502 503 504 /50x.html;
##            location = /50x.html {
##        }
##    }

# Settings for a TLS enabled server.
#
#    server {
#        listen       443 ssl http2 default_server;
#        listen       [::]:443 ssl http2 default_server;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate "/etc/pki/nginx/server.crt";
#        ssl_certificate_key "/etc/pki/nginx/private/server.key";
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers HIGH:!aNULL:!MD5;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        location / {
#        }
#
#        error_page 404 /404.html;
#            location = /40x.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#            location = /50x.html {
#        }
#    }

}

Hi Chas,

I’ve posted the nginx config for your perusal.

Our Nginx packages:

nginx-1.12.2-2.el7.x86_64
nginx-all-modules-1.12.2-2.el7.noarch
nginx-filesystem-1.12.2-2.el7.noarch
nginx-mod-http-geoip-1.12.2-2.el7.x86_64
nginx-mod-http-image-filter-1.12.2-2.el7.x86_64
nginx-mod-http-perl-1.12.2-2.el7.x86_64
nginx-mod-http-xslt-filter-1.12.2-2.el7.x86_64
nginx-mod-mail-1.12.2-2.el7.x86_64
nginx-mod-stream-1.12.2-2.el7.x86_64

Our PHP packages:

php72w-cli-7.2.5-1.w7.x86_64
php72w-common-7.2.5-1.w7.x86_64
php72w-fpm-7.2.5-1.w7.x86_64
php72w-gd-7.2.5-1.w7.x86_64
php72w-intl-7.2.5-1.w7.x86_64
php72w-mbstring-7.2.5-1.w7.x86_64
php72w-mysql-7.2.5-1.w7.x86_64
php72w-pdo-7.2.5-1.w7.x86_64
php72w-process-7.2.5-1.w7.x86_64
php72w-snmp-7.2.5-1.w7.x86_64
php72w-xml-7.2.5-1.w7.x86_64
php-composer-ca-bundle-1.0.2-1.el7.noarch
php-composer-semver-1.4.2-1.el7.noarch
php-composer-spdx-licenses-1.4.0-1.el7.noarch
php-fedora-autoloader-1.0.0-1.el7.noarch
php-jsonlint-1.6.1-1.el7.noarch
php-justinrainbow-json-schema5-5.2.7-1.el7.noarch
php-paragonie-random-compat-2.0.12-1.el7.noarch
php-password-compat-1.0.4-1.el7.noarch
php-PsrLog-1.0.2-2.el7.noarch
php-seld-cli-prompt-1.0.3-1.el7.noarch
php-seld-phar-utils-1.0.1-1.el7.noarch
php-symfony-browser-kit-2.8.12-2.el7.noarch
php-symfony-class-loader-2.8.12-2.el7.noarch
php-symfony-common-2.8.12-2.el7.noarch
php-symfony-config-2.8.12-2.el7.noarch
php-symfony-console-2.8.12-2.el7.noarch
php-symfony-css-selector-2.8.12-2.el7.noarch
php-symfony-debug-2.8.12-2.el7.noarch
php-symfony-dependency-injection-2.8.12-2.el7.noarch
php-symfony-dom-crawler-2.8.12-2.el7.noarch
php-symfony-event-dispatcher-2.8.12-2.el7.noarch
php-symfony-expression-language-2.8.12-2.el7.noarch
php-symfony-filesystem-2.8.12-2.el7.noarch
php-symfony-finder-2.8.12-2.el7.noarch
php-symfony-http-foundation-2.8.12-2.el7.noarch
php-symfony-http-kernel-2.8.12-2.el7.noarch
php-symfony-polyfill-1.5.0-1.el7.noarch
php-symfony-process-2.8.12-2.el7.noarch
php-symfony-var-dumper-2.8.12-2.el7.noarch
php-symfony-yaml-2.8.12-2.el7.noarch

Thanks for the base_url check, I remember that setting, we just put in our internal IP into it in the form:

$config['base_url'] = "http://ur.internal-ip.no.hostname";

We don’t have internal DNS so hostnames are typically only used via hosts files (we don’t technically need hostnames for our servers). This means the above has always just been http:// of the IP.

I commented that out, restarted both:

systemctl restart php-fpm && systemctl restart nginx

and unfortunately still no login.

I’ve also just tried stopping firewalld, SELinux, rebooted, tried again, same problem.

I’ve checked the disk space and free memory, plenty of available space and memory.

I’ve checked the librenms user can login to the DB also.

Hi. I found the problem…

I commented out:

## To enable secure cookies ref: https://docs.librenms.org/#General/Security/
##$config['secure_cookies'] = true;

in config.php, and I am able to access the console again.

Why is secure cookies breaking this now you think?

1 Like

Good that you found the problem, apparently your only meant to set that to true if your using HTTPS, but doesn’t explain why it was working before the laravel update.

1 Like

Agreed. I don’t remember reading in that install guide you’re only meant to use that for https, which is why I thought the more secure the better and set it up on those machines.

Maybe that should be updated in the docs if I’m correct there?

But as discussed, this always worked before so I can’t explain why. Nevertheless, with the re-gaining of the access to the GUI very happy again I can continue to work on the environment.

Appreciate your time and patience and help with all this. Thank you.

2 Likes

I think the processing of config.php changed a little.

For that particular setting, we check if https and enable/disable.

We used to do this after config.php is loaded, so your setting was overriden. But there was an architectural issue with that.

Now config.php is loaded last, so your setting is actually applied.

Excellent. It’s good to have an understanding why this occurred for the LibreNMS servers that are only on HTTP.

Hopefully the thread can assist others too who get the problem and can now fix it.

Thank you.