Introduction:
Hey there! Ever found yourself scratching your head over a quirky network issue that just won't quit? I recently had my fair share of head-scratching moments when network broadcast storms started causing serious disruptions in our switches and wireless access points. In this post, I'm breaking down the steps I took to unravel this persistent, network mystery.
Setting the Scene:
Picture this – network switches and access points dropping off the radar every few nights. To get to the bottom of it, I needed to gather some intel without losing sleep, especially as these problems seemed to occur in the middle of the night. First move? Configuring Home Assistant to ping, with the Ping integration, a crucial switch and give me a notification on my phone when things go south. It became my silent night watchman, marking the time when the broadcast storm kicked in.
The Quest for Network Clarity:
LibreNMS, a network monitoring system, was my next stop. Although it did show spikes in network bandwidth, the specifics were elusive. SNMP logs lacked the juicy details needed for a deep investigation.
Enter rsyslogd:
To beef up my data game, I brought in rsyslogd on a Proxmox server. This Linux container was configured to capture logs from the admin VLAN, where switches and access points were. This did capture a lot of data, however, drowning in logs was not my idea of fun. It was too hard to find the root cause of these issues.
