The WatchTower Project
User Manual
The WatchTower ProjectUser Manual | |
| Top |
The WatchTower Project was designed primarily to assist System Administrators in the active monitoring of servers and the services they provide. It consists of a few simple files which monitor specified servers and ports on which services run, and to alert administrators on different levels when the status of one or more of these services change. These levels are currently SMS messages, E-Mail notification and Log Entry.
The WatchTower Project is designed primarily to be run as a cron job, such that it is scheduled to run every so often (eg 10 minutes) and monitor the services specified in the configuration file. It then records its current state and compares it to the last recorded state. If a service has changed state, the action specified in the configuration file is undertaken.
This document will briefly go through all the configuration options and requirements for running the WatchTower Project, as well as how to get it up and running. It does not, however detail how to set up programs such as Gnokii; this documentation assumes programs like gnokii,nmap and the nmap Perl module are installed and functioning correctly.
| Top |
Currently, The WatchTower Project requires the following programs to be installed and present in the path to be run at a bare minimal:For all features (including SMS and Mail Alerts) the following programs are required to be installed and present in the path:
- perl 5.6.1 or later (available from http://perl.com)
- nMap 2.54 or later (available from http://insecure.org)
- Perl NMap::Scanner 0.01 Module (available from http://search.cpan.org/~maxschube/Nmap-0.01/)
Earlier versions of these programs may work, but have not been tested.
- Mail 8.1 or later
- Gnokii 0.5.1 or later (available from http://gnokii.org)
- the Cron daemon
| Top |
There are three key WatchTower Project files:The following sections will describe the format of these files and how to use them correctly.
- watchtower.conf, which holds the rules and global configuration of the WatchTower Project.
- watchtower.status, which contains a crude database of sorts, holding the result of the scans from the previous WatchTower Project scan.
- watchtower.pl, the main executable for the WatchTower Project, which reads and manipulates the above two files.
watchtower.conf
Purpose: Configuration File
Recommended placement: /etc/watchtower.conf
Notes: To be edited by the end user.
This file is the first thing watchtower.pl looks for. Currently this file must belong in the /etc directory under the name of watchtower.conf. The format of the configuration file is similar to many other configuration files.
The file consists of one or more sections, which are indicated by placing square brackets ([ and ]) around an alphanumeric string (i.e. [my_rule_1]). the underscore, Alphabetic and numeric characters are all that is allowed within these section headers. These headers define what the WatchTower Project refers to as a Rule.
There is a special header that is reserved for the configuration of the WatchTower Project itself, and that is the [CONFIGURATION] header. this header is given special treatment and expects diffferent values, explained later.
All other rules will accept two types of values - scan and alert. more than one scan or alert value may be specified.
The [CONFIGURATION] section accepts 6 different values:
- SCAN has the syntax scan=<ip-addr>:<port> [SYN|FIN] or scan=<ip-addr> STAT. The first notation scans an IP address and a port to check a particular service is running. It defaults to using SYN scans, thus SYN is a noiseword. Windows-based machines do not respond properly to FIN scans, so FIN should not be specified if the target is a windows machine. The second notation scans a specified host to see if it is either up or down, thus not requiring a port. This can get through some firewalls when SYN and FIN scans cannot.
having more than one SCAN definition per section allows the rules to be logically ANDed together, so the entire rule will fail if one or more of the specified scans fail.
- ALERT has the syntax alert=<SMS|EMAIL|LOG>. This tells the WatchTower Project how to report failed or restored rules. SMS instructs the WatchTower Project to send an SMS to all the numbers listed in the [CONFIGURATION] section. Email sends an e-mail to all the recipients listed in the [CONFIGURATION] section. Log enters a log entry in the log file specified in the [CONFIGURATION] section.
That Said, here is a very basic sample configuration:
- sms_numbers which has the syntax sms_numbers=<ph-num>[,ph-num[...]]>. It essentially takes a list of one or more phone numbers seperated by a comma. These numbers will be SMS'd by any rules which fail, and are told to SMS errors.
- email_addresses which has the syntax email_addresses=<mail-addr>[,mail-addr[...]]>. It essentially takes a list of one or more e-mail addresses seperated by a comma. These addresses will be mailed by any rules which fail, and are told to mail errors.
- log_file which has the syntax log_file=<filename>. The file specified is appended to each time a rule fails, which is instructed to write a log.
- sms_all which has the syntax sms_all=<YES|NO>. This allows you to send SMS whenever any rule fails, so you don't have to specify ALERT=SMS in every rule.
- email_all which has the syntax email_all=<YES|NO>. The same as SMS_ALL, except for e-mail.
- log_all which has the syntax log_all=<YES|NO>. The same as SMS_ALL, except for logs.
[CONFIGURATION] sms_numbers= 0441555555 email_addresses=admins@domain.com; me@domain.com log_file=/var/log/watchtower.log log_all=yes [file_server] scan=fileserv.domain.com:139 FIN scan=fileserv.domain.com:21 FIN alert=sms [windows_logon] scan=logon.domain.com:139 alert=sms alert=email [external_www] scan=www.anotherdomain.com STAT scan=www2.anotherdomain.com STAT alert=emailThe sms_all and email_all values are omitted from the [CONFIGURATION] section, but can be added as sms_all=no and email_all=no if it makes you feel better.
All alerts which are destined to be SMS'd will be sent to the hypotheteical phone number 0441555555, likewise for all e-mail alerts which will be sent to admins@domain.com and me@domain.com. The log file exists in /var/log as watchtower.log. All alerts will be logged to this file.
There are 3 rules defined. The first rule, named file_server, scans a machine named fileserv.domain.com twice- once on port 139 (netbios) and once on port 21 (FTP). Any alerts for this rule will be sent to the log file (because of the log_all option) and also to the SMS number specified.
The second rule is named windows_logon and scans just one machine (logon.domain.com) on port 139. However, syn scanning is used because it is a windows machine. All alerts will be sent to email and sms as well as the logs.
The third rule named external_www scans two machines by a status scan (by specifying STAT). we're assuming we're behind a firewall that won't let us scan outside our network, so we're using STAT scans to check the status of the two web servers. All alerts will be e-mailed as well as logged.
watchtower.status
Purpose: Keep record of rule status
Recommended placement: /var/log/watchtower/watchtower.status
Notes: Not to be edited by end user. File completely rewritten after each scan.
The watchtower.status file is a simple file which records the status for each rule after the WatchTower Project run so it can be checked on subsequent scans for state changes. Each line has the syntax:
<rule-name>:<0|1>where 0 represents a rule failure and 1 represents a rule pass. This file is used internally by the WatchTower Project to determine whether or not to send alerts when a rule fails, and to determine when a rule is restored.
watchtower.pl
Purpose: Main executable
Recommended placement: /usr/local/bin
Notes: Not to be edited by end user. Perl Script.
This is the main WatchTower Project executable. It is written in perl. Hence, there may be an adjustment that needs to be made to the first line of the script:
#!/usr/bin/perlmay have to be changed to reflect the path to your perl installation. For example, if perl is found in /usr/local/bin, you'll have to change this to#!/usr/local/bin/perl. This is the only change you will need to make to this file.It is called simply by typing ./watchtower.pl from its installed directory. If everything is working correctly, it will open the config file, and display the rules it is scanning.
| Top |
Firstly, ensure that the prerequisite components mentioned earlier are installed and working. Secondly, you will need to edit your own /etc/watchtower.conf file. You should define rules as you see fit, and use IP addresses instead of DNS names if you do not have DNS servers. Also, if you plan on using a log file, ensure that the directory for the log file exists.
Once the configuration file is edited appropriately, you may run it by simply typing ./watchtower.pl from its installed directory. If all works well, you should see something similar to the following:
This indicates that the WatchTower project is installed and configured correctly. You will have to wait until the end of the scanning and for a rule to fail before you can test gnokii is working in conjunction with the script. Once the WatchTower project is confirmed to work, you may want to configure it as a cron job to run every so often to scan your network for all its available services. editing the cron jobs is done usually by typing crontab -e.A typical cron entry may be as follows:
# MB Run our watchtower project script every 10 minutes 0-59/10 * * * * /usr/local/etc/watchtower.pl > /var/log/watchtower/watchtower.lastrunThis particular configuration will run the watchtower project script every 10 minutes, and redirect the output to /var/log/watchtower/watchtower.lastrun. To view this, you can simply type cat /var/log/watchtower/watchtower.lastrun. If output is not directed, the result of the scan will most likely be e-mailed to the root user of the system you have installed it on. This is all that is involved in running the WatchTower Project.
| Top |
BugsTodo
- The script will fail if the directory for the log file does not exist.
- Rules will fail if the system cannot resolve the IP address for the DNS names specified in the config file - can be avoided by using IP addresses instead.
- Command line options for such things as automatically redirecting output to a specified file, specifying path to config file for multiple configurations
- Sets of e-mail and sms values, so that only certain groups of people receive notification depending on which rules are affected
| Top |
You can reach me via e-mail at: watchtower@mattbradford.com. Happy to receive proposed features and comments.
| Top |