We have about 20+ machines running powerdns auth and were running SNMP with powerdns.py. This was working well and graphing was coming through normally.
All of a sudden over the last 24 hours, all machines stopped responding to the OID in question. Manually walking it would get "Timeout: No Response from ". Even locally.
The rest of the SNMP tree was fine and didn’t alert, it is just the one script. We also have dnsdist loaded too via extend which works fine.
I have determined it is powerdns.py being the cause. Inserting an error in the code, throughout parts of the script give me an error in the walk, but it does respond.
iso.3.6.1.4.1.8072.1.3.2.4.1.2.8.112.111.119.101.114.100.110.115.1 = STRING: "Traceback (most recent call last):"
iso.3.6.1.4.1.8072.1.3.2.4.1.2.8.112.111.119.101.114.100.110.115.2 = STRING: " File \"/etc/snmp/powerdns.py\", line 25, in <module>"
iso.3.6.1.4.1.8072.1.3.2.4.1.2.8.112.111.119.101.114.100.110.115.3 = STRING: " xxxx"
iso.3.6.1.4.1.8072.1.3.2.4.1.2.8.112.111.119.101.114.100.110.115.4 = STRING: "NameError: name 'xxxx' is not defined"
However, adding one after the final line print(json.dumps(output)), does nothing so it is at this point in which it gets stuck.
Running the script via command line untouched outputs as expected.
These machines don’t have auto updates turned on as far as I am aware so apt running and doing something is unlikely and overall, they’ve been untouched since this was added about a month ago.
So far, I have rebuilt one machine with Ubuntu 20 and the same thing happens.
I’m at a bit of a loss as to where to go now so any ideas would be appreciated.