Oxidized not pulling full config

Help Oxidized
1

I’ve got Oxidized up and running (finally), but it’s not pulling the full config from my Cisco switches. It’s pulling software version information, VTP info, stacking module ports, and stuff like that but not the actual config. I have debugging set to “true” in the Oxidized config, I have a location for the log files configured, but I’m not seeing ANY logs. How do I go about debugging this?

2

Do you have this set in your oxidized config
enable: true

and did you specify your enable password under

vars:
  enable: passw0rd

Also you should see logs so perhaps debug that first, did you restart oxidized service, force config refresh from oxidized-web, no log output ? does the oxidized user have permissions to write logs in that specified folder?

3

I added the two lines you mentioned. No change in behavior.

I have restarted the service with each change to the config file using these commands:

sudo systemctl daemon-reload
sudo systemctl enable oxidized.service
sudo systemctl start oxidized
sudo systemctl status oxidized

The oxidized user should have write permissions to the logs folder. This is the output of ls -l for the /home/oxidized/.config/oxidized directory:

-rw-r–r-- 1 root root 1083 Jun 11 08:47 config
-rw-rw-r-- 1 oxidized oxidized 1513 Jun 7 09:51 crash
drwxr-xr-x 6 oxidized oxidized 4096 Jun 11 08:52 default.git
drwxrwxr-x 2 oxidized oxidized 4096 Jun 7 13:50 logs
-rw-r–r-- 1 oxidized oxidized 6 Jun 7 13:21 pid

Below is my config file (passwords obscured, of course).

username: **********
password: **********
model: ios
interval: 3600
log: /home/oxidized/.config/oxidized/logs
debug: true
use_syslog: false
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
rest: 0.0.0.0:8888
next_adds_job: false
enable: true
vars:
enable: **********
models: {}
pid: “/home/oxidized/.config/oxidized/pid”

input:
default: ssh, telnet
debug: /home/oxidized/.config/oxidized/logs/log_input
ssh:
secure: false

output:
default: git
git:
user: root
email: **********
repo: “/home/oxidized/.config/oxidized/devices.git”

source:
default: http
http:
url: http:// ********** /api/v0/oxidized
map:
name: hostname
model: os
group: group
headers:
X-Auth-Token: ‘**********’

model_map:
cisco: ios
juniper: junos
unifi: airos
ciscosb: ciscosmb

hooks:
post-commit:
type: exec
events: [node_success, post_store]
cmd: ‘cd /home/oxidized/.config/oxidized/devices.git && git push -f -q /home/oxidized/.config/oxidized/devices.git master’

And this is the contents of /etc/systmd/system/oxidized.service

[Unit]
Description=Oxidized - Network Device Configuration Backup Tool
After=network-online.target multi-user.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/oxidized
User=oxidized
KillSignal=SIGKILL
Environment=“OXIDIZED_HOME=/home/oxidized”

[Install]
WantedBy=multi-user.target

4

Change

input:
default: ssh, telnet
debug: /home/oxidized/.config/oxidized/logs/log_input
ssh:
secure: false

to

input:
  default: ssh, telnet
  debug: true
  ssh:
    secure: false

Then restart, and force a config run from the web to the cisco switch.

For the cisco switch what model is it using in oxidized web? IOS ?

Is your oxidized configuration file indented properly ? otherwise some things wont work, for example

vars:
enable: **********

should be

vars:
  enable: **********
5

The indentation is correct (as far as I can tell), it just didn’t transfer over correctly when I copied an pasted here.

I made the config change you suggested. No behavior change. The web interface shows a green status indicator and the “Last Update” time changes, but it never loads the running config. It seems to stop after the vtp status or inventory depending on the switch.

Still no log files in the logs directory.

6

Screenshot to verify indentations.

image
image907×816 13.3 KB

7

Hmm looks ok to me too.

Anything useful in your journal?
journalctl -u oxidized --since "today" -f

Try increase your Timeout to 60 and restart, incase the job takes a while to grab the running-config.

What versions of oxidized are you running? gem list | grep oxidized

8

I do see some stuff in the journal.

Jun 11 03:53:49 nms oxidized[124352]: sh: 1: cd: can’t cd to /home/oxidized/.config/oxidized/devices.git
Jun 11 03:53:49 nms oxidized[124352]: E, [2019-06-11T03:53:49.987116 #124352] ERROR – : Exec: “cd /home/oxidized/.config/oxidized/devices.git && git push -f -q /home//oxidized/.config/oxidized/devices.git master” failed with exit value 2
Jun 11 03:53:49 nms oxidized[124352]: #<Thread:0x0000561e36e66bd0@/var/lib/gems/2.5.0/gems/oxidized-0.26.3/lib/oxidized/hook/exec.rb:32 run> terminated with exception (report_on_exception is true):
Jun 11 03:53:49 nms oxidized[124352]: /var/lib/gems/2.5.0/gems/oxidized-0.26.3/lib/oxidized/hook/exec.rb:50:in block in run_cmd!': "cd /home/oxidized/.config/oxidized/devices.git && git push -f -q /home//oxidized/.config/oxidized/devices.git master" failed with exit value 2 (RuntimeError) Jun 11 03:53:49 nms oxidized[124352]: from /usr/lib/ruby/2.5.0/timeout.rb:93:in block in timeout’
Jun 11 03:53:49 nms oxidized[124352]: from /usr/lib/ruby/2.5.0/timeout.rb:33:in block in catch' Jun 11 03:53:49 nms oxidized[124352]: from /usr/lib/ruby/2.5.0/timeout.rb:33:in catch’
Jun 11 03:53:49 nms oxidized[124352]: from /usr/lib/ruby/2.5.0/timeout.rb:108:in timeout' Jun 11 03:53:49 nms oxidized[124352]: from /var/lib/gems/2.5.0/gems/oxidized-0.26.3/lib/oxidized/hook/exec.rb:44:in run_cmd!’
Jun 11 03:53:49 nms oxidized[124352]: from /var/lib/gems/2.5.0/gems/oxidized-0.26.3/lib/oxidized/hook/exec.rb:34:in `block in run_hook’
Jun 11 03:53:49 nms oxidized[124352]: E, [2019-06-11T03:53:49.987343 #124352] ERROR – : Hook post-commit (#Exec:0x0000561e35d4d038) failed (#<RuntimeError: “cd /home/oxidized/.config/oxidized/devices.git && git push -f -q /home//oxidized/.config/oxidized/devices.git master” failed with exit value 2>) for event :node_success

A “devices.git” directory didn’t exist so I just now manually created it.

9

Everything is working now! Unfortunately, I don’t know exactly what caused it start working.

I created the “devices.git” directory that didn’t exist but was in the config, but that didn’t change any behavior.

I updated Ubuntu and it requested a reboot, so I did that and that broke full functionality of Oxidized. Trying to start Oxidized manually, I saw an error that “logs” was a directory instead of a file and couldn’t be opened for writing. So I changed the log file in the config

from
“log: /home/oxidized/.config/oxidized/logs”

to
“log: /home/oxidized/.config/oxidized/logs/log”

I can’t recall anything else I may have done, but after that it all started working correctly.

1 Like