Tag Archives: Monitoring

Syncronizing Xymon’s ‘bb-hosts’ Configurations

I’ve been using Xymon (formerly known as “Hobbit”) for a long time.  In most situations, I have Xymon running in a redundant configuration, with two or more instances of Xymon working together to monitor a network.

Even though Xymon works very well, a single change to the primary server’s configuration file (the “bb-hosts” file) means that you have to make the same change to all other ‘bb-hosts’ files in all other Xymon instances.

There are some creative ways to eliminate the drudgery of updating all these files any time a change to the primary file is necessary.  One method, for example would be to have the master file exported via NFS to all the other Xymon server instances, and each of those instances would sym-link to that primary ‘bb-hosts’ file from their local mount of that NFS export.

I don’t like the NFS export idea, because if the primary server has a problem, and the NFS export is no longer available, all instances of Xymon would break – badly.

Instead, I’ve opted for automatically synchronizing the ‘bb-hosts’ file across all Xymon instances via the use of apache, cron, a sym-link, and a simple bash script.

Here’s  how it works:

  • On the primary Xymon instance, sym-link ‘/home/xymon/server/etc/bb-hosts’ to ‘/var/www/bb-hosts’.
  • On the other instances of Xymon, run a bash script which grabs the primary server’s ‘bb-hosts’ via HTTP, which does some simple comparisons, and over-writes the local Xymon ‘bb-hosts’ if changes are detected.
  • Automat this script with cron.

Perhaps the trickiest part of doing this is the actual script used to grab, compare, and over-write the ‘bb-hosts’ file for the other instances of Xymon.  The script I’ve written below grabs the primary ‘bb-hosts’ file, and does a simple MD5 comparison with md5sum, and if it detects a change in the ‘bb-hosts’ file, it will send an e-mail to notify me that this change has occurred, along with details on what has changed.

Here’s the script:

wget http://somewebhost.domain.com/bb-hosts -qO "$REMOTE_BB_HOSTS"
LOCAL_MD5=`md5sum $LOCAL_BB_HOSTS  | cut -d " " -f 1`
REMOTE_MD5=`md5sum  $REMOTE_BB_HOSTS  | cut -d " " -f 1`
#echo "$LOCAL_MD5"
#echo "$REMOTE_MD5"
if [ "$LOCAL_MD5" != "$REMOTE_MD5" ]; then
        echo "Generated by $0" > $BB_HOSTS_DIFFS;
        mail -s "Xymon: monitor-02 bb-hosts updated" alertme@email.com < $BB_HOSTS_DIFFS;

If you need a way to keep your Xymon ‘bb-hosts’ files in sync, something along the lines of the above script just may be what you’re looking for. If you’re currently accomplishing the same thing in an interesting way, please post a comment and let me know!

Nationwide Blackberry Outage – 22/23 December 2009

Well, my Blackberry is officially offline because of a nationwide blackberry outage currently taking place. I use my Blackberry to receive messages from monitoring systems at VMware, so I’m severely pissed. I haven’t received any e-mails for several hours! The only way I knew that something was wrong, is that I have a sanity e-mail sent to myself every 2 hours. When I don’t see that e-mail, something evil is happening.

For a brief moment, I thought I was going to have to call and yell at Rogers, but they’re not at fault this time.

Real time updates on the situation via Twitter is nice, but a working Blackberry would be nicer.