Standalone Oxidized SSH Issue

Welcome to the Oxidized help section.

You are free to ask any questions surrounding Oxidized, it does not need to be specific to LibreNMS integration.

GitHub
Gitter

Hello all. Thank you in advance for your assistance. It is greatly appreciated.

I am very new to LibreNMS/Oxidized and in an effort to utilize config backup/diff more immediately I’m attempting a standalone implementation of Oxidized. At this stage I have completed an Oxidized build and am trying to gather the backup from a single device.

I am receiving an SSH error that I am unable to remedy and am hoping you can assist.

*** The device is a Cisco CSR8kV. From the Oxidized server I am able to SSH directly to the CSR manually. When I attempt it via Oxidized I receive the following debug after which the job repeats:

I, [2022-09-12T14:38:30.792166 #9709]  INFO -- : Oxidized starting, running as pid 9709
I, [2022-09-12T14:38:30.792484 #9709]  INFO -- : lib/oxidized/nodes.rb: Loading nodes
D, [2022-09-12T14:38:30.792549 #9709] DEBUG -- : resolving DNS for { HOSTNAME }...
D, [2022-09-12T14:38:30.792565 #9709] DEBUG -- : IPADDR { CSR_Lo0 }
D, [2022-09-12T14:38:30.792603 #9709] DEBUG -- : node.rb: resolving node key 'model', with passed global value of '' and node value 'ios'
D, [2022-09-12T14:38:30.792619 #9709] DEBUG -- : node.rb: setting node key 'model' to value 'junos' from global
D, [2022-09-12T14:38:30.792634 #9709] DEBUG -- : node.rb: returning node key 'model' with value 'ios'
D, [2022-09-12T14:38:30.792644 #9709] DEBUG -- : lib/oxidized/node.rb: Loading model "ios"
D, [2022-09-12T14:38:30.793398 #9709] DEBUG -- : lib/oxidized/model/model.rb Added all to the commands list
D, [2022-09-12T14:38:30.793420 #9709] DEBUG -- : lib/oxidized/model/model.rb Added secret to the commands list
D, [2022-09-12T14:38:30.793431 #9709] DEBUG -- : lib/oxidized/model/model.rb Added show version to the commands list
D, [2022-09-12T14:38:30.793440 #9709] DEBUG -- : lib/oxidized/model/model.rb Added show vtp status to the commands list
D, [2022-09-12T14:38:30.793447 #9709] DEBUG -- : lib/oxidized/model/model.rb Added show inventory to the commands list
D, [2022-09-12T14:38:30.793454 #9709] DEBUG -- : lib/oxidized/model/model.rb Added show running-config to the commands list
D, [2022-09-12T14:38:30.793534 #9709] DEBUG -- : node.rb: resolving node key 'input', with passed global value of 'ssh' and node value ''
D, [2022-09-12T14:38:30.793552 #9709] DEBUG -- : node.rb: returning node key 'input' with value 'ssh'
D, [2022-09-12T14:38:30.835147 #9709] DEBUG -- : node.rb: resolving node key 'output', with passed global value of 'file' and node value ''
D, [2022-09-12T14:38:30.835180 #9709] DEBUG -- : node.rb: returning node key 'output' with value 'file'
D, [2022-09-12T14:38:30.835483 #9709] DEBUG -- : node.rb: resolving node key 'username', with passed global value of '' and node value ''
D, [2022-09-12T14:38:30.835508 #9709] DEBUG -- : node.rb: setting node key 'username' to value '{ USERNAME }' from global
D, [2022-09-12T14:38:30.835524 #9709] DEBUG -- : node.rb: returning node key 'username' with value '{ USERNAME }'
D, [2022-09-12T14:38:30.835534 #9709] DEBUG -- : node.rb: resolving node key 'password', with passed global value of '' and node value ''
D, [2022-09-12T14:38:30.835544 #9709] DEBUG -- : node.rb: setting node key 'password' to value '{ PASSWORD }' from global
D, [2022-09-12T14:38:30.835555 #9709] DEBUG -- : node.rb: returning node key 'password' with value '{ PASSWORD }'
I, [2022-09-12T14:38:30.835583 #9709]  INFO -- : lib/oxidized/nodes.rb: Loaded 1 nodes
D, [2022-09-12T14:38:30.984764 #9709] DEBUG -- : lib/oxidized/core.rb: Starting the worker...
Puma starting in single mode...
* Version 3.11.4 (ruby 3.0.2-p107), codename: Love Song
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:8888
Use Ctrl-C to stop
D, [2022-09-12T14:38:31.985929 #9709] DEBUG -- : lib/oxidized/worker.rb: Jobs running: 0 of 1 - ended: 0 of 1
D, [2022-09-12T14:38:31.986076 #9709] DEBUG -- : lib/oxidized/worker.rb: Added /{ HOSTNAME } to the job queue
D, [2022-09-12T14:38:31.986101 #9709] DEBUG -- : lib/oxidized/worker.rb: 1 jobs running in parallel
D, [2022-09-12T14:38:31.986400 #9709] DEBUG -- : lib/oxidized/job.rb: Starting fetching process for { HOSTNAME } at 2022-09-12 14:38:31 UTC
D, [2022-09-12T14:38:31.986586 #9709] DEBUG -- : lib/oxidized/input/ssh.rb: Connecting to { HOSTNAME }
D, [2022-09-12T14:38:31.986676 #9709] DEBUG -- : AUTH METHODS::["none", "publickey", "password", "keyboard-interactive"]
D, [2022-09-12T14:38:31.987126 #9709] DEBUG -- net.ssh.transport.session[474]: establishing connection to { CSR_Lo0 }:22
D, [2022-09-12T14:38:31.988651 #9709] DEBUG -- net.ssh.transport.session[474]: connection established
I, [2022-09-12T14:38:31.988700 #9709]  INFO -- net.ssh.transport.server_version[488]: negotiating protocol version
D, [2022-09-12T14:38:31.988716 #9709] DEBUG -- net.ssh.transport.server_version[488]: local is `SSH-2.0-Ruby/Net::SSH_5.2.0 x86_64-linux-gnu'
D, [2022-09-12T14:38:31.990511 #9709] DEBUG -- net.ssh.transport.server_version[488]: remote is `SSH-2.0-Cisco-1.25'
I, [2022-09-12T14:38:31.990663 #9709]  INFO -- net.ssh.transport.algorithms[49c]: sending KEXINIT
D, [2022-09-12T14:38:31.990844 #9709] DEBUG -- socket[4b0]: queueing packet nr 0 type 20 len 1324
D, [2022-09-12T14:38:31.990892 #9709] DEBUG -- socket[4b0]: sent 1328 bytes
D, [2022-09-12T14:38:31.990952 #9709] DEBUG -- socket[4b0]: read 480 bytes
D, [2022-09-12T14:38:31.991006 #9709] DEBUG -- socket[4b0]: received packet nr 0 type 20 len 476
I, [2022-09-12T14:38:31.991039 #9709]  INFO -- net.ssh.transport.algorithms[49c]: got KEXINIT from server
I, [2022-09-12T14:38:31.991097 #9709]  INFO -- net.ssh.transport.algorithms[49c]: negotiating algorithms
D, [2022-09-12T14:38:31.991179 #9709] DEBUG -- net.ssh.transport.algorithms[49c]: negotiated:
* kex: ecdh-sha2-nistp521
* host_key: ssh-rsa
* encryption_server: aes256-ctr
* encryption_client: aes256-ctr
* hmac_client: hmac-sha2-512
* hmac_server: hmac-sha2-512
* compression_client: none
* compression_server: none
* language_client: 
* language_server: 
D, [2022-09-12T14:38:31.991201 #9709] DEBUG -- net.ssh.transport.algorithms[49c]: exchanging keys
E, [2022-09-12T14:38:31.992784 #9709] ERROR -- : { CSR_Lo0 } raised OpenSSL::PKey::PKeyError with msg "pkeys are immutable on OpenSSL 3.0", { CSR_Lo0 } saved
D, [2022-09-12T14:38:31.992807 #9709] DEBUG -- : lib/oxidized/node.rb: Oxidized::SSH failed for { HOSTNAME }
D, [2022-09-12T14:38:31.992827 #9709] DEBUG -- : lib/oxidized/job.rb: Config fetched for { HOSTNAME } at 2022-09-12 14:38:31 UTC
W, [2022-09-12T14:38:32.987381 #9709]  WARN -- : /{ HOSTNAME } status no_connection, retry attempt 1
D, [2022-09-12T14:38:32.987436 #9709] DEBUG -- : lib/oxidized/worker.rb: Jobs running: 0 of 1 - ended: 0 of 1

*** The following is what I have in router.db as a host entry:

{ HOSTNAME }:{ CSR_Lo0 }:ios

*** This is the config file in play (quotes added to PW b/c of special characters, does appear to allow the PW through correctly):


username: { HOSTNAME }
password: “{ PASSWORD }”
model: junos
resolve_dns: true
interval: 3600
use_syslog: false
debug: true
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
rest: 127.0.0.1:8888
next_adds_job: false
vars:
auth_methods: [ “none”, “publickey”, “password”, “keyboard-interactive” ]
groups: {}
models: {}
pid: “/home/oxidized/.config/oxidized/pid”
crash:
directory: “/home/oxidized/.config/oxidized/crashes”
hostnames: false
stats:
history_size: 10
input:
input:
default: ssh
debug: true
ssh:
secure: false
ftp:
passive: true
utf8_encoded: true
output:
default: file
file:
directory: “/home/oxidized/.config/oxidized/configs”
source:
default: csv
csv:
file: “/home/oxidized/.config/oxidized/router.db”
delimiter: !ruby/regexp /:confused:
map:
name: 0
ip: 1
model: 2
model_map:
juniper: junos
cisco: ios

Hello again. I’ve been trying to locate any info in log files that can assist but nothing of value is being returned.

Might anyone have a suggestion as to what is going on with the failed SSH returns we’re experiencing?

Thanks again.