I am not quite sure how this happened but I ended up with a four duplicated devices added to LNMS via discovery. I found them like this:
select device_id,sysName,hostname,serial from devices where serial in (select serial from devices group by serial having count(serial)>1) order by serial;
What is the best way to clean up the duplicated devices?
Do I just pick one and delete it?
Since they were added via discovery what is to prevent them from returning?
We’d need to see some of that data to see why but hostname field is unique so you can’t have the same name in there, sysName can have duplicates but only if $config['allow_duplicate_sysName'] = true; (it’s false by default).
I have confirmed that I do NOT allow duplicate sysNames. I have pasted below the sql output. I assume that there is not much point in cleanup until we determine source of the duplicates. I ran a discovery.php -h all -d -v outputted to a file last night. It finished about 30 minutes ago so I havent had a chance to look at it yet. Let me know if you have any ideas.
+-----------+----------------------------------+---------------+-------------+
| device_id | sysName | hostname | serial |
+-----------+----------------------------------+---------------+-------------+
| 1080 | fbdc-nexus-5k-voip-dr-2.work.com | 10.66.26.3 | FOC2021R11F |
| 1081 | fbdc-nexus-5k-voip-dr-2 | 10.66.27.11 | FOC2021R11F |
| 203 | wb-np-7n-4510r.work.com | 10.132.248.58 | SPE13260089 |
| 204 | wb-np-7n-4510r.work.com | 10.132.56.4 | SPE13260089 |
| 1109 | dc-nexus-5k-d5 | 10.100.0.40 | SSI142520VS |
| 1111 | dc-nexus-5k-d5 | 10.110.200.40 | SSI142520VS |
| 1110 | dc-nexus-5k-d6.work.com | 10.110.200.41 | SSI144007FN |
| 1112 | dc-nexus-5k-d6 | 10.100.0.41 | SSI144007FN |
+-----------+----------------------------------+---------------+-------------+Ok I dont know if this is related but it does seems odd to me but then I am just learning how everything works. Tell me if it is normal or if it is unrelated and we can throw this into another thread.
So as I mentioned I ran a discovery.php -h all -d -v last night. It looked like it ran fine. Here is the trailer.
/opt/librenms/discovery.php all 2017-02-04 11:21:03 - 1118 devices discovered in 38161 secs
SNMP: Get[21999/15902.60s] Walk [85806/17619.25s]
MySQL: Cell[248471/89.62s] Row[120822/88.18s] Rows[66227/36.91s] Column[1118/0.40s] Update[211156/656.26s] Insert[743941/354.69s] Delete[3077/4.54s]
When looking at the value for last_discovered on my devices though the majority of them show to have an old discovery date. Shouldnt it update the date on all the devices?
select distinct date(last_discovered),count(date(last_discovered)) from devices group by date(last_discovered) order by last_discovered;
+-----------------------+------------------------------+
| date(last_discovered) | count(date(last_discovered)) |
+-----------------------+------------------------------+
| 2017-01-17 | 542 |
| 2017-01-18 | 76 |
| 2017-01-20 | 1 |
| 2017-01-23 | 2 |
| 2017-01-30 | 1 |
| 2017-02-02 | 1 |
| 2017-02-04 | 505 |
+-----------------------+------------------------------+So I was not sure what to do but what I ended up doing was to delete both instances of all of the duplicated devices. I then started discovery again. When I checked it today I now have a bunch of duplicates again.
I focused on finding devices 1271 & 1272 in my logs and I have a copy of the discovery with -d -v options of one devices that causes those two entries.
+-----------+------------------------------+----------------+-------------+
| device_id | sysName | hostname | serial |
+-----------+------------------------------+----------------+-------------+
| 1263 | dc-nexus-5k-a6-new.work.com | 10.110.200.14 | FOC1851R18P |
| 1264 | dc-nexus-5k-a6-new | 10.100.0.14 | FOC1851R18P |
| 1259 | dc-nexus-5k-b6-new.work.com | 10.110.200.24 | FOC1852R0TY |
| 1260 | dc-nexus-5k-b6-new | 10.100.0.24 | FOC1852R0TY |
| 1249 | dc-nexus-7k-f3 | 10.100.0.200 | JAF1752AKHA |
| 1009 | dc-nexus-7k-f3.work.com | 10.255.0.200 | JAF1752AKHA |
| 1250 | dc-nexus-7k-f7 | 10.100.0.201 | JAF1752AKHJ |
| 1012 | dc-nexus-7k-f7.work.com | 10.255.0.201 | JAF1752AKHJ |
| 1256 | dc-nexus-5k-b6.work.com | 10.110.200.21 | SSI143606YD |
| 1257 | dc-nexus-5k-b6 | 10.100.0.21 | SSI143606YD |
| 1098 | dc-nexus-5k-f3-ring.work.com | 10.110.200.220 | SSI1437092M |
| 1248 | dc-nexus-5k-f3-ring | 10.100.0.220 | SSI1437092M |
| 1127 | dc-nexus-5k-a6.work.com | 10.110.200.11 | SSI1437099Q |
| 1261 | dc-nexus-5k-a6 | 10.100.0.11 | SSI1437099Q |
| 1271 | dc-nexus-5k-d6.work.com | 10.110.200.41 | SSI144007FN |
| 1272 | dc-nexus-5k-d6 | 10.100.0.41 | SSI144007FN |
+-----------+------------------------------+----------------+-------------+It appears, based on the “hostname”, that you are discovering devices on multiple interfaces. It’s also seeing different hostnames, i.e. dc-nexus-5k-a6-new.work.com and dc-nexus-5k-a6-new. Make sure your autodiscovery DNS suffix is correct (work.com in this case). You may also want to disable autodiscovery on some subnets to prevent duplicates.
Ok so I am hopeful this is fixed. The change that was made (thanks laf!) was:
$config[‘addhost_alwayscheckip’] = true;
When looking for what added dc-nexus-5k-d6 (10.110.200.41 and 10.100.0.41) I found out it was when discovery was done of 10.100.0.42. Here is relevant lines from the discovery output of device 10.100.0.42
http://pastebin.com/qJ24CNXk
I deleted all of the duplicate devices and re-ran discovery of 10.100.0.42. This time it only added the device once as 10.100.0.41.
http://pastebin.com/KnrPcCsf
I also ran discovery of 10.100.0.40 since it was the cause of the second entry.
http://pastebin.com/0KB5aY0X
After both discoveries I no longer have duplication…