Packaging LibreNMS for FreeBSD

I am working with the FreeBSD maintainer to package LibreNMS 1.41 and we are hitting problems.

These issues did not arise with 1.35, the latest version ported to FreeBSD.

I hope to document the issues in case the developers wish to change the way LibreNMS is doing things. I wish to highlight the problems associated with packaging the application. Not every environment permits the use of git to update or the use of composer to install the dependencies. We are looking at finding a way to create a package given those constraints.

One issuer with running composer at install time, there are many scripts installed by composer which assume path names. FreeBSD calls these SHEBANG fixes. e.g. stuff might be in /usr/local/bin etc, not /bin or /sbin. Everything not related to the base operating systems goes into /usr/local/ not /.

We are trying to find a way to get around that by running composer at package building time. We need to consider the license issues of distributing a tarball containing the vendor code.

I’ll keep you updated as things progress. Thank you.

@dvl My question is what were you doing before when we shipped all this in the git folder? 1.35 used composer autoload, it was just in the git repository. (I’m not tying to start a disagreement, I legitimately want to know).

As far as I know nothing installed by composer uses fixed paths. Also, everything installed by composer should reside inside the vendor folder, inside the librenms folder.

If it makes it easier, I can run composer and upload a tar ball for you…

1 Like

I think the major difference is the 1.35 tarball contained a vendor directory in the tarball:

[[email protected]:~/tmp/librenms-1.35/vendor] $ ls
amenadiel composer easybook palanik php-amqplib ramsey slim tecnickcom
autoload.php dapphp ezyang pear phpmailer rmccue symfony xjtuwangke

I think that’s the difference. It was provided and we just used that. We could patch it correctly as it was just there at install time.

The difference now is we can’t download the tarball, extract the code, patch it and supply it as a package. NOTE: these are not manual steps, they are complete automated by the FreeBSD port tree.

When you run composer, what command are you running? Perhaps we can just specify that in our Makefile.

php composer.phar install or ./script/composer_wrapper.php install

Note that this will create .env file which has some things in that need to be generated for each install.

I had time to play with this tonight. The first attempt failed, because git was not installed. After installing git

Must this command be run within a copy of the git repo?

$ ./scripts/composer_wrapper.php install
Error: Failed to download https://getcomposer.org/installer

LibreNMS\ComposerHelper::preInstall
fatal: not a git repository (or any parent up to mount point /usr)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /usr)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

Problem 1
- The requested PHP extension ext-mysqli * is missing from your system. Install or enable PHP’s mysqli extension.
Problem 2
- The requested PHP extension ext-session * is missing from your system. Install or enable PHP’s session extension.
Problem 3
- The requested PHP extension ext-gd * is missing from your system. Install or enable PHP’s gd extension.
Problem 4
- Installation request for amenadiel/jpgraph 3.6.19 -> satisfiable by amenadiel/jpgraph[3.6.19].
- amenadiel/jpgraph 3.6.19 requires ext-gd * -> the requested PHP extension gd is missing from your system.
Problem 5
- Installation request for nikic/php-parser v3.1.5 -> satisfiable by nikic/php-parser[v3.1.5].
- nikic/php-parser v3.1.5 requires ext-tokenizer * -> the requested PHP extension tokenizer is missing from your system.
Problem 6
- Installation request for phpunit/php-token-stream 1.4.12 -> satisfiable by phpunit/php-token-stream[1.4.12].
- phpunit/php-token-stream 1.4.12 requires ext-tokenizer * -> the requested PHP extension tokenizer is missing from your system.
Problem 7
- Installation request for phpunit/php-code-coverage 4.0.8 -> satisfiable by phpunit/php-code-coverage[4.0.8].
- phpunit/php-code-coverage 4.0.8 requires ext-xmlwriter * -> the requested PHP extension xmlwriter is missing from your system.
Problem 8
- Installation request for squizlabs/php_codesniffer 2.9.1 -> satisfiable by squizlabs/php_codesniffer[2.9.1].
- squizlabs/php_codesniffer 2.9.1 requires ext-tokenizer * -> the requested PHP extension tokenizer is missing from your system.
Problem 9
- nikic/php-parser v3.1.5 requires ext-tokenizer * -> the requested PHP extension tokenizer is missing from your system.
- psy/psysh v0.9.5 requires nikic/php-parser ~1.3|~2.0|~3.0|~4.0 -> satisfiable by nikic/php-parser[v3.1.5].
- Installation request for psy/psysh v0.9.5 -> satisfiable by psy/psysh[v0.9.5].

To enable extensions, verify that they are enabled in your .ini files:
-
- /usr/local/etc/php/ext-20-ctype.ini
- /usr/local/etc/php/ext-20-curl.ini
- /usr/local/etc/php/ext-20-dom.ini
- /usr/local/etc/php/ext-20-filter.ini
- /usr/local/etc/php/ext-20-hash.ini
- /usr/local/etc/php/ext-20-intl.ini
- /usr/local/etc/php/ext-20-json.ini
- /usr/local/etc/php/ext-20-mbstring.ini
- /usr/local/etc/php/ext-20-openssl.ini
- /usr/local/etc/php/ext-20-simplexml.ini
- /usr/local/etc/php/ext-20-xml.ini
- /usr/local/etc/php/ext-30-phar.ini
You can also run php --ini inside terminal to see which files are used by PHP in CLI mode.

I think I have it:

[[email protected]:~] $ git clone [email protected]:librenms/librenms.git
[[email protected]:~] $ cd librenms/
[[email protected]:~/librenms] $ git checkout tags/1.42.01
[[email protected]:~/librenms] $ ./scripts/composer_wrapper.php install

More information at https://gist.github.com/dlangille/42c99b82a74774f287d3853bfa6001c1

I think my next step is to create a librenms-vendor-1.41.01.tar.gz and host it somewhere. I suspect I can do this on the FreeBSD infrastructure. These steps would be necessary for each new release I assume.

I will report back again after I try building with that I have.

Yeah, almost guaranteed libraries will change with every release.

Thank you.

This morning, I incorporated the new vendor directory into my working copy of the FreeBSD port. It builds, and I see that after installing it, I have some work to do with respect to dependencies. I suspect I have to review Problems 1…9 listed above. I’ll come back to this soon…

FWIW, this is what I get when I browse to the new website:

Error: Missing dependencies! Run the following command to fix:

./scripts/composer_wrapper.php install --no-dev

Of note, that is not the exact command I ran to build the vendor directory. Should my steps be amended to include --no-dev?

Over lunch, I tried again:

I went back and redid the vendor directory creation. I added the requisite packages to this otherwise empty server:

$ sudo pkg install php56-mysqli php56-session php56-gd php56-tokenizer php56-xmlwriter php56-curl php56-xml php56-pdo

Then I ran the composer command:

[[email protected]:~/librenms] $ ./scripts/composer_wrapper.php install --no-dev
> LibreNMS\ComposerHelper::preInstall
Loading composer repositories with package information
Installing dependencies from lock file
Package operations: 64 installs, 0 updates, 0 removals
- Installing symfony/dotenv (v3.4.13): Loading from cache
- Installing amenadiel/jpgraph (3.6.20): Loading from cache
- Installing dapphp/radius (2.5.4): Loading from cache
- Installing doctrine/inflector (v1.1.0): Loading from cache
- Installing doctrine/instantiator (1.0.5): Loading from cache
- Installing easybook/geshi (v1.0.8.19): Loading from cache
- Installing erusev/parsedown (1.7.1): Loading from cache
- Installing ezyang/htmlpurifier (v4.10.0): Loading from cache
- Installing vlucas/phpdotenv (v2.4.0): Loading from cache
- Installing symfony/css-selector (v3.4.13): Loading from cache
- Installing tijsverkoyen/css-to-inline-styles (2.2.1): Loading from cache
- Installing symfony/polyfill-mbstring (v1.8.0): Loading from cache
- Installing symfony/var-dumper (v3.4.13): Loading from cache
- Installing symfony/routing (v3.2.14): Loading from cache
- Installing symfony/process (v3.4.13): Loading from cache
- Installing symfony/polyfill-ctype (v1.8.0): Loading from cache
- Installing paragonie/random_compat (v2.0.17): Loading from cache
- Installing symfony/polyfill-php70 (v1.8.0): Loading from cache
- Installing symfony/http-foundation (v3.4.13): Loading from cache
- Installing symfony/event-dispatcher (v3.4.13): Loading from cache
- Installing psr/log (1.0.2): Loading from cache
- Installing symfony/debug (v3.4.13): Loading from cache
- Installing symfony/http-kernel (v3.4.13): Loading from cache
- Installing symfony/finder (v3.4.13): Loading from cache
- Installing symfony/console (v3.4.13): Loading from cache
- Installing swiftmailer/swiftmailer (v5.4.9): Loading from cache
- Installing ramsey/uuid (3.8.0): Loading from cache
- Installing symfony/translation (v3.2.14): Loading from cache
- Installing nesbot/carbon (1.32.0): Loading from cache
- Installing mtdowling/cron-expression (v1.2.1): Loading from cache
- Installing monolog/monolog (1.23.0): Loading from cache
- Installing league/flysystem (1.0.45): Loading from cache
- Installing laravel/framework (v5.4.36): Loading from cache
- Installing fico7489/laravel-pivot (2.0.6): Loading from cache
- Installing guzzlehttp/promises (v1.3.1): Loading from cache
- Installing psr/http-message (1.0.1): Loading from cache
- Installing guzzlehttp/psr7 (1.4.2): Loading from cache
- Installing guzzlehttp/guzzle (6.3.3): Loading from cache
- Installing influxdb/influxdb-php (1.14.7): Loading from cache
- Installing jakub-onderka/php-console-color (0.1): Loading from cache
- Installing nikic/php-parser (v3.1.5): Loading from cache
- Installing jakub-onderka/php-console-highlighter (v0.3.2): Loading from cache
- Installing dnoegel/php-xdg-base-dir (0.1): Loading from cache
- Installing psy/psysh (v0.9.6): Loading from cache
- Installing laravel/tinker (v1.0.7): Loading from cache
- Installing laravel/laravel (v5.4.30): Loading from cache
- Installing myclabs/deep-copy (1.7.0): Loading from cache
- Installing oriceon/toastr-5-laravel (dev-master 8a03502): Cloning 8a03502cc7 from cache
- Installing slim/slim (2.6.3): Loading from cache
- Installing palanik/corsslim (v1.1.0): Loading from cache
- Installing pear/console_color2 (0.1.2): Loading from cache
- Installing pear/console_table (v1.3.1): Loading from cache
- Installing php-amqplib/php-amqplib (v2.0.2): Loading from cache
- Installing webmozart/assert (1.3.0): Loading from cache
- Installing phpdocumentor/reflection-common (1.0.1): Loading from cache
- Installing phpdocumentor/type-resolver (0.4.0): Loading from cache
- Installing phpdocumentor/reflection-docblock (3.3.2): Loading from cache
- Installing phpmailer/phpmailer (v5.2.26): Loading from cache
- Installing phpunit/php-token-stream (1.4.12): Loading from cache
- Installing rmccue/requests (v1.7.0): Loading from cache
- Installing symfony/yaml (v2.8.43): Loading from cache
- Installing tecnickcom/tcpdf (6.2.17): Loading from cache
- Installing wpb/string-blade-compiler (3.4.x-dev 30c974a): Cloning 30c974a305 from cache
- Installing xjtuwangke/passwordhash (dev-master a7bcd97): Cloning a7bcd9705a from cache
Generating autoload files
> LibreNMS\ComposerHelper::postInstall
> Illuminate\Foundation\ComposerScripts::postInstall
> php artisan optimize
Generating optimized class loader
The compiled services file has been removed.
[[email protected]:~/librenms] $

After installing and restarting Apache, the website reports:

### Error: Missing dependencies! Run the following command to fix:

./scripts/composer_wrapper.php install --no-dev

Running validate.php:

$ ./validate.php 
PHP Warning:  Module 'tokenizer' already loaded in Unknown on line 0
PHP Warning:  Module 'snmp' already loaded in Unknown on line 0
PHP Warning:  Module 'session' already loaded in Unknown on line 0
PHP Warning:  Module 'mysql' already loaded in Unknown on line 0
PHP Warning:  Module 'mcrypt' already loaded in Unknown on line 0
PHP Warning:  Module 'json' already loaded in Unknown on line 0
PHP Warning:  Module 'gd' already loaded in Unknown on line 0
PHP Warning:  Module 'filter' already loaded in Unknown on line 0
PHP Warning:  Module 'ctype' already loaded in Unknown on line 0
PHP Fatal error:  Class 'LibreNMS\Validator' not found in /usr/local/www/librenms/validate.php on line 96
====================================
Component | Version
--------- | -------
LibreNMS  | 
DB Schema | ?
PHP       | 5.6.37
MySQL     | ?
RRDTool   | 1.7.0
SNMP      | NET-SNMP 5.7.3
====================================


Fatal error: Class 'LibreNMS\Validator' not found in /usr/local/www/librenms/validate.php on line 96

FYI, the vendor directory contains:

$ ls vendor
amenadiel    composer     easybook     guzzlehttp   palanik      php-amqplib  psr          rmccue       symfony      xjtuwangke
autoload.php dapphp       ezyang       influxdb     pear         phpmailer    ramsey       slim         tecnickcom

The class is there:

$ ls LibreNMS/
Alert                          DB                             OS                             RRDRecursiveFilterIterator.php queuemanager.py
Alerting                       Device                         OS.php                         Util                           service.py
Authentication                 Exceptions                     ObjectCache.php                ValidationResult.php
Component.php                  IRCBot.php                     Plugins.php                    Validations
ComposerHelper.php             Interfaces                     Proc.php                       Validator.php
Config.php                     Model.php                      RRD                            __init__.py

As taken from my notes at https://gist.github.com/dlangille/e91b77ca4a32dfcbcdd7515ffb263e74

All packaged now. Thanks for hlpeing.

1 Like