MONIT, unix system management utility

Open source UNIX system and monitoring management utility (MONIT)

* Monit is a handy utility for managing and monitoring processes, files and directories and devices on a UNIX system. It performs automatic maintenance, repairs with the ability to take pro-active steps and generate alerts. Configuration can be performed via a text file or a web interface. Auto actions include alert, restart, start, stop, exec and unmonitor. Monit can be integrated with open-source heatbeat package for monitoring service failures.

* monitrc is the configuration file, which contains a global and per-service sections. Processes and daemons can be monitored by their PID file and re-started or alerted to the administrator. Remote services can be monitored by a TCP or UDP connection. monit also understand most common internet protocols that helps to check responses instead of only open port checks. Local resource usage can also be tracked and action taken in response. Files, directories and devices can be monitored for checksum verifications, timestamp records, sizes, permissions and owernships. External programs can be executed in case of monitoring failures.

* A sample process check is as follows:
check process cron with pidfile /var/run/cron.pid
group system
start program = "/etc/init.d/cron start"
stop program = "/etc/init.d/cron stop"
if 5 restarts within 5 cycles then timeout
depends on cron_rc

* A sample file check is as follows:
check file cron_rc with path /etc/init.d/cron
group system
if failed checksum then unmonitor
if failed permission 755 then unmonitor
if failed uid root then unmonitor
if failed gid root then unmonitor

* A sample device check is as follows:
check device disk1 with path /dev/hda1
start = "/bin/mount /dev/hda1"
stop = "/bin/umount /dev/hda1"
if space usage > 90% then alert
if space usage > 99% then stop

* A sample protocol check is as follows:
check host www.test.com with address www.test.com
if failed port 80 protocol http and request "/help/about.txt"
then alert with mail-format { subject: test.com is down }
alert test@test.com

* A sample protocol test with send/expect strings is as follows:

check host tildeslash.com with address tildeslash.com
if failed port 80
send "GET / HTTP/1.0\r\nHost: tildeslash.com\r\n”
send “Connection: close\r\n"
expect "HTTP/[0-9\.]{3} 200 .*"
then ...

* It has a neat concept of service dependency, where an action, such as unmonitor can be escalated to a dependent service. Normally file and service checks are linked in this way. Option is specified by using 'depends on [service check]' token.

The idea of Vendor Management Office (VMO)

* Vendor management office (VMO) is a group within an enterprise IT department that manages vendor relationship. It is responsible for both front end activities such as contract negotiations and overseeing RFPs as well as maintaining invoices and assessing performance. In a nutshell, VMO works to get the best deals out of existing and new IT vendors. VMO makes vendor aware of their competitors, their offerings and pricings, making them improve their bids.

* IT management experience coupled with legal and financial expertise are the key skills required for a VMO manager.

* Establishment of a VMO may fundamentally alter existing work flow for execution of new projects. IT managers may have to create project briefs and submit them to the VMO, which in turn finds the best deal in terms of price, quality and vendor commitment and expertise. Project briefs consist of project description, explaining business needs and technology sought. VMO crafts their own purchase agreements instead of relying on vendor-drafted contracts.

* Chief advantage of VMO is cost optimization. It can help to cut costs in the long term by getting latest prices from vendors, identifying poor vendors and restructuring existing overtly costly deals. It accomplishes all these and more by tracking spending.

* VMO can also have its pitfalls. It may not be relevant for smaller companies dealing with one or two vendors or extremely large businesses with de-centralized management of their global presence. VMO can also take out business owners from the purchasing decisions so they need to be effectively communicated and involved.

Veritas Volume Manager how-to

Volume virtual device consists of plexes which in turn consists of one or multiple sub-disks. Sub-disks are carved from VM disks, which are physical disks under the control of VxVM. VM disk contains public and private regions.

In non-layered volume, sub-disks are mapped to VM disks. In layered volume sub-disks are mapped to underlying volumes. Columns consist of one or more sub-disks. VxVM manipulates stripes in terms of columns and stripe width.


VM disk initialization destroys data on disk. Disk encapsulation allows VM control without destroying data. It is used for importing non-VM disks. vxdiskadm command asks for hot-relocation use (hotuse), cds (cross-platform data sharing), disk name etc during disk initialization. Encapsulation of disk would require a reboot of system. VM will place entries in /etc/vfstab file. vxdiskadd command can be used in place of vxdiskadm. Volumes in encapsulated root disk are not re-sizable. Encapsulation requires free space on disk to accommodate private regions. Normally root disks that are to be mirrored are encapsulated.

rootvol in bookdg disk group must be used for booting.

vxrecover command is used for volume recovery operations. To start recovery in the background after starting volumes, use vxrecover -g [diskgroup] -sb

Cross-platform data sharing disk groups can be ported between different OS, like Solaris, HP-UX, Linux, AIX etc.

There is a concept of disk group version number. Version number can be upgraded by vxdg upgrade command. Newer versions support new features.

vxdctl command is the interface to vxconfigd daemon which writes changes to configuration database on disk. Configuration database is stored in the private region of various devices in each disk group for redundancy. To find the disks with configuration database, use, vxdg list sunonedg |egrep "config disk.*clean online". vxprint -ht command is used to print the contents of configuration database. In VxVM 4.x and later, vxconfigbackupd daemon backs up configuration database and disk headers, whenever there is a change. It keep backup copies in /etc/vx/cbr/bk directory. vxconfigrestore command can be used to restore the backup. It allows the administrator to verify the restored config before commiting it.

Disk private region (disk header) contains diskid to identify disk, disk group id to identify the disk group, a set of flags to indicate status and intended use of disk and a hostid. It can be printed by passing veritas device name to vxdisk, e.g vxdisk list c1t1d0

vxrelocd daemon detects failures and relocates sub-disks. By default, it is configured to send a mail to root user in case of device and veritas object failure. A forwarder can be defined for root to forward mails to another account. vxreattach command reattach disk drives that have once again become accessible. Deport and import of disk group can enable a disabled disk group.

Useful commands:

To determine mapping between physical disk device paths and VM disks labels,
vxdisk path

To monitor currently running vxvm tasks,
vxtask list

To scan and configure new disk devices
vxdiskconfig

vxdisk can also be used for scanning disks, 'vxdisk scandisks'


To change naming scheme from enclosure-based to operating system based and vice versa,

vxddladm set namingscheme=[ebn|osn]