404 File Not Found

toriol · 14 April 2020 21:14

I recent started getting 404 file found found errors. It’s very similar to the following post:
WebGUI suddently stopped working(404 Not Found)
It happens with almost all links (add device, manage groups, services, etc.)
There are few links that do work: accessing the default dashboard, my settings, manage users)
I have my installation to update automatically, so I don’t know exactly when this happened.
I’ve run the recommended commands:

su - librenms
cd /opt/librenms
git pull
./daily.sh
./validate.sh

[root@aliap01 ~]# su - librenms
Last login: Tue Apr 14 15:51:15 CDT 2020 on pts/0
-bash-4.2$ cd /opt/librenms
-bash-4.2$ git pull
Already up-to-date.
-bash-4.2$ ./daily.sh
Updating to latest codebase                        OK
Updating Composer packages                         OK
Updating SQL-Schema                                OK
Updating submodules                                OK
Cleaning up DB                                     OK
Fetching notifications                             OK
Caching PeeringDB data                             OK
-bash-4.2$ ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.62-65-g7dfc226
DB Schema | 2020_01_10_075852_alter_mpls_lsp_paths_table (162)
PHP       | 7.2.24
MySQL     | 5.5.56-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.10.5
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
[WARN]  IPv6 is disabled on your server, you will not be able to add IPv6 devices.
-bash-4.2$

The installation is comprised of a database server and an application server. Both run RHEL7. The database server is MariaDB 5.5.56 from RHEL. The application server runs nginx 1.12.2 from EPEL and php 7.2 from webtatic. Both servers are setup to update the OS and applications automatically.

Any suggestions?

toriol · 15 April 2020 16:40

I enabled the debug mode and loaded one of the non-working pages. Here’s what I got:

/opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php#992Symfony\Component\HttpKernel\Exception\NotFoundHttpException

#0 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/helpers.php(46): Illuminate\Foundation\Application->abort(404, '', Array)

#1 /opt/librenms/app/Http/Controllers/LegacyController.php(55): abort(404)

#2 [internal function]: App\Http\Controllers\LegacyController->index(Object(Illuminate\Http\Request), Object(Illuminate\Session\Store), 'addhost')

#3 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call_user_func_array(Array, Array)

#4 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller->callAction('index', Array)

#5 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Route.php(219): Illuminate\Routing\ControllerDispatcher->dispatch(Object(Illuminate\Routing\Route), Object(App\Http\Controllers\LegacyController), 'index')

#6 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Route.php(176): Illuminate\Routing\Route->runController()

#7 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(680): Illuminate\Routing\Route->run()

#8 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\Routing\Router->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#9 /opt/librenms/app/Http/Middleware/VerifyTwoFactor.php(38): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#10 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): App\Http\Middleware\VerifyTwoFactor->handle(Object(Illuminate\Http\Request), Object(Closure))

#11 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#12 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(41): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#13 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Routing\Middleware\SubstituteBindings->handle(Object(Illuminate\Http\Request), Object(Closure))

#14 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#15 /opt/librenms/vendor/laravel/framework/src/Illuminate/Auth/Middleware/Authenticate.php(43): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#16 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Auth\Middleware\Authenticate->handle(Object(Illuminate\Http\Request), Object(Closure))

#17 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#18 /opt/librenms/app/Http/Middleware/LegacyExternalAuth.php(45): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#19 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): App\Http\Middleware\LegacyExternalAuth->handle(Object(Illuminate\Http\Request), Object(Closure))

#20 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#21 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(75): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#22 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Foundation\Http\Middleware\VerifyCsrfToken->handle(Object(Illuminate\Http\Request), Object(Closure))

#23 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#24 /opt/librenms/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#25 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\View\Middleware\ShareErrorsFromSession->handle(Object(Illuminate\Http\Request), Object(Closure))

#26 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#27 /opt/librenms/app/Http/Middleware/LoadUserPreferences.php(31): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#28 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): App\Http\Middleware\LoadUserPreferences->handle(Object(Illuminate\Http\Request), Object(Closure))

#29 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#30 /opt/librenms/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(56): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#31 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Session\Middleware\StartSession->handle(Object(Illuminate\Http\Request), Object(Closure))

#32 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#33 /opt/librenms/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#34 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse->handle(Object(Illuminate\Http\Request), Object(Closure))

#35 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#36 /opt/librenms/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(66): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#37 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Cookie\Middleware\EncryptCookies->handle(Object(Illuminate\Http\Request), Object(Closure))

#38 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#39 /opt/librenms/app/Http/Middleware/CheckInstalled.php(46): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#40 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): App\Http\Middleware\CheckInstalled->handle(Object(Illuminate\Http\Request), Object(Closure))

#41 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#42 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(104): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#43 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(682): Illuminate\Pipeline\Pipeline->then(Object(Closure))

#44 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(657): Illuminate\Routing\Router->runRouteWithinStack(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request))

#45 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(623): Illuminate\Routing\Router->runRoute(Object(Illuminate\Http\Request), Object(Illuminate\Routing\Route))

#46 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Router.php(612): Illuminate\Routing\Router->dispatchToRoute(Object(Illuminate\Http\Request))

#47 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(176): Illuminate\Routing\Router->dispatch(Object(Illuminate\Http\Request))

#48 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(30): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}(Object(Illuminate\Http\Request))

#49 /opt/librenms/vendor/darkghosthunter/larapoke/src/Http/Middleware/LarapokeGlobalMiddleware.php(18): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#50 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): DarkGhostHunter\Larapoke\Http\Middleware\LarapokeGlobalMiddleware->handle(Object(Illuminate\Http\Request), Object(Closure))

#51 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#52 /opt/librenms/vendor/barryvdh/laravel-debugbar/src/Middleware/InjectDebugbar.php(65): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#53 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Barryvdh\Debugbar\Middleware\InjectDebugbar->handle(Object(Illuminate\Http\Request), Object(Closure))

#54 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#55 /opt/librenms/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#56 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Fideloper\Proxy\TrustProxies->handle(Object(Illuminate\Http\Request), Object(Closure))

#57 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#58 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#59 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))

#60 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#61 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#62 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))

#63 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#64 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#65 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle(Object(Illuminate\Http\Request), Object(Closure))

#66 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#67 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php(62): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#68 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(163): Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode->handle(Object(Illuminate\Http\Request), Object(Closure))

#69 /opt/librenms/vendor/laravel/framework/src/Illuminate/Routing/Pipeline.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))

#70 /opt/librenms/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(104): Illuminate\Routing\Pipeline->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))

#71 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(151): Illuminate\Pipeline\Pipeline->then(Object(Closure))

#72 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(116): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter(Object(Illuminate\Http\Request))

#73 /opt/librenms/html/index.php(53): Illuminate\Foundation\Http\Kernel->handle(Object(Illuminate\Http\Request))

#74 {main}

Perhaps there was an update to one of the libraries whihc changed some functionality that wasn’t anticipated?

PipoCanaja · 15 April 2020 16:57

Hi,
Seems that you are the first (and only) one to report this issue. And I cannot reproduce it. I have 6 instances on different OSes with no issue.
Anybody ?

toriol · 15 April 2020 17:49

Just to be thorough, I retested using firefox and chome in private mode. I got the same error. I then thought perhaps the corporate proxy was the issue (transparent proxy, protocol inspection) so I installed firefox locally on the application server and retested. I still got the same error.

I do notice the 404 error page is different from the regular 404 error page.

On the left is the normal 404. On the right is the custom 404.

I don’t know if this makes a difference or anything, but it’s another piece of information to work with. Hopefully it’s not a red herring.

I also thought perhaps the device didn’t exist, so I verified in the database that the devices table is populated and that the corresponding device id does exist as a row (in the screenshot I used device id 98).

PipoCanaja · 15 April 2020 22:06

Your URL looks funny. For addhost for instance, it should be. by default :
http://my.server.name/addhost
In your case, there is an “index.php” that comes from I don’t know where, and may indicate that you have a custom nginx/apache config, or reverse proxy etc etc.
Running a browser locally on the server is a good start. And the custom 404 error is the expected result.
https://librenms.hq.wclc.com/addhost should be the addhost page in your case. Does it display correctly ? If not, what error in the web server logs ?

toriol · 16 April 2020 14:38

Thank you PipoCanaja! The hint that the URI is supposed to be /addhost as opposed to /index.php/addhost. I had to remove the following line from librenms.conf:
rewrite ^/$ https://librenms.hq.wclc.com/index.php redirect;
Now the installation is working correctly.

PipoCanaja · 16 April 2020 15:00

Great !