Daily.sh removed required library, hit RuntimeException

First off, validation output.

====================================

Component Version
LibreNMS 1.43-119-g6242f94
DB Schema 268
PHP 7.1.21
MySQL 5.5.56-MariaDB
RRDTool 1.4.8
SNMP NET-SNMP 5.7.2

====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Problem report:

On September 19th, at around 00:15 EST, the daily.sh script ran and deleted what was apparently a necessary library. It then hit a RuntimeException and failed to restore the library, resulting in failed updates until our monitoring system reported an HTTP 500 response when attempting a health check. I see the RuntimeException in daily logs but am uncertain if this would cause the script to skip restoration of the removed libraries (in this case, myclabs/DeepCopy was the culprit).

This was fixed by using a big hammer to clone the myclabs/DeepCopy repository and place the sources where the update scripts expect them, then running daily.sh again.

I have two questions. 1.) What happened, and 2.) could I have just used the composer_wrapper script to fix the missing dependencies?

Relevant log snippet follows.

[ Wed, 19 Sep 2018 00:15:14 ] http://www.librenms.org/notifications.rss (33)
[ Wed, 19 Sep 2018 00:15:14 ] misc/notifications.rss (40)
[ Wed, 19 Sep 2018 00:15:14 ] Updating DB Done
Returned: 0
Caching PeeringDB data
Peering DB integration disabled
Returned: 0
Updating to latest codebase

Returned: 0
Updating Composer packages

LibreNMS\ComposerHelper::preInstall
Loading composer repositories with package information
Installing dependencies from lock file
Package operations: 0 installs, 16 updates, 7 removals

  • Removing webmozart/assert (1.3.0)
  • Removing phpunit/php-token-stream (1.4.12)
  • Removing phpdocumentor/type-resolver (0.4.0)
  • Removing phpdocumentor/reflection-docblock (3.3.2)
  • Removing phpdocumentor/reflection-common (1.0.1)
  • Removing myclabs/deep-copy (1.7.0)
  • Removing doctrine/instantiator (1.0.5)
  • Updating vlucas/phpdotenv (v2.4.0 => v2.5.1): Downloading (100%)
  • Updating symfony/css-selector (v3.4.14 => v3.4.15): Loading from cache
  • Updating symfony/var-dumper (v3.4.14 => v3.4.15): Loading from cache
  • Updating symfony/process (v3.4.14 => v3.4.15): Downloading (100%)
  • Updating symfony/http-foundation (v3.4.14 => v3.4.15): Downloading (100%)
  • Updating symfony/event-dispatcher (v3.4.14 => v3.4.15): Loading from cache
  • Updating symfony/debug (v3.4.14 => v3.4.15): Downloading (100%)
  • Updating symfony/http-kernel (v3.4.14 => v3.4.15): Downloading (100%)
  • Updating symfony/finder (v3.4.14 => v3.4.15): Loading from cache
  • Updating symfony/console (v3.4.14 => v3.4.15): Loading from cache
  • Updating league/flysystem (1.0.45 => 1.0.47): Downloading (100%)
  • Updating fico7489/laravel-pivot (2.0.6 => 2.3.7): Downloading (100%)
  • Updating psy/psysh (v0.9.7 => v0.9.8):

[RuntimeException]
Could not delete /var/www/librenms/vendor/psy/psysh/.editorconfig:

Updating SQL-Schema
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
Returned: 255
Updating submodules

Returned: 0
Cleaning up DB
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep
:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
Returned: 255
Fetching notifications
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
Returned: 255
Caching PeeringDB data
PHP Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66

Fatal error: require(): Failed opening required ‘/var/www/librenms/vendor/composer/…/myclabs/deep-copy/src/DeepCopy/deep_copy.php’ (include_path=’.:/usr/share/pear:/usr/share/php’) in /var/www/librenms/vendor/composer/autoload_real.php on line 66
Returned: 255