fmodchk - Generate alerts when file ages or file age differences satisfy user supplied criteria.
usage: fmodchk [ options ] [ File ... ]
-r,--alerts-to Addr[,Addr[,...]]
-R,--trusted-alerts-to Addr[,Addr[,...]]
-c,--config-file ConfigFile
-d,--difference
-D,--signed-difference
-e,--errors-to Addr[,Addr[,...]]
-f,--mail-from Addr
-h,--help
-l,--label LabelList
-P,--alert-patrol
-t,--threshold [ComparisonOperator]DurationSpec
-T,--alert-comments Text
-w,--response-window DurationSpec
fmodchk generates alerts when a violation is detected, i.e., the age of a file (current time minus time of last modification) or the difference in age between two files satisfies an expression (violation criteria). The expression is of one of these forms.
FILE_AGE OPERATOR AGE_OR_DURATION FILE_AGE_DIFFERENCE OPERATOR AGE_OR_DURATION
where OPERATOR is a comparison operator (e.g., '>') and AGE_OR_DURATION is a threshold specified by the user. Both the operator and threshold value have defaults, for which, see the usage statement (execute: fmodchk --help).
To fmodchk, a file is anything with an inode, including directories. Symbolic links are followed.
Examples of situations for which fmodchk might generate an alert:
Exactly one of the following options must be specified in batch mode (stdin not associated with a tty):
-r,--alerts-to Addr[,Addr[,...]]
Send alert notification emails to the given comma-delimited email address list. Pathnames in the fmodchk-generated alert text will be basenames only.
-R,--trusted-alerts-to Addr[,Addr[,...]]
Same as --alerts-to except that full pathnames are used in the fmodchk-generated alert text.
-c,--config-file ConfigFile
Get options from configuration file ConfigFile rather than from the command line. --config-file is incompatible with all options except for --label. Configfile must be a regular file and its size may not exceed a default maximum value (for which, see usage statement: fmodchk --help).
See the CONFIGURATION FILE section for details on the configuration file.
-d,--difference
Indicate the difference in ages between two files' is being examined. (The default is to examine the age of each of a set of files.) Requires two file arguments. The absolute value of the age difference will be compared to a value given by the --threshold option or to a default if --threshold is not used.
-D,--signed-difference
Same as --difference except that a signed difference will be
compared. The order of the files on the command line is
significant. The difference will be calculated as the first file's
age minus the second file's age. age(A) - age(B) > 0 will indicate
that A is older than B, where A is given before B on the command
line. age(A) - age(B) < 0 will indicate that A is younger than B.
-e,--errors-to Addr[,Addr[,...]]
Send runtime warnings/errors encountered by this program (not alerts), to the given comma-delimited email address list. The default list to which errors are sent is in the usage statement (fmodchk --help). This option is ignored when not in batch mode.
-f,--mail-from Addr
Use Addr as the value of the "From: " header in the alert email messages. The default address to which errors are sent is in the usage statement (fmodchk --help). This option is ignored when not in batch mode.
-h,--help
List usage.
-l,--label LabelList
Process only configuration file blocks named by comma-delimited LabelList. The default is to process all configuration blocks that don't specify 'label-invocation-only'.
This option may be used multiple times. Requires use of --config-file option. Incompatible with all other options.
To specify a label that actually contains a comma, escape the comma using a backslash, e.g., '\,'.
-P,--alert-patrol
Indicate that the local bmc patrol is to be informed of alerts.
-t,--threshold [ComparisonOperator]DurationSpec
Specify DurationSpec and optionally ComparisonOperator for file age or age difference comparison. See the usage statement (fmodchk --help) for a list of valid comparison operators. See usage statement also for default values for DurationSpec and ComparisonOperators.
Note that the condition specified in this option is the condition for which an alert will be sent. It is not the steady-state, non-violation condition.
This option is ignored if --response-window is used.
-T,--alert-comments Text
Append Text to the alert message. Any backslash-n sequences ('\n') will be converted to newlines.
It is strongly suggested that alert comments be used for all alerts. Example configuration file snippet:
alert-comments << EndComments
***
The lack of a young enough archive file for this inbound Foo
archive directory may mean that we didn't receive a file on time
from Foo & Associates. Please investigate.
***
EndComments
By default, the alert text contains only age or age difference data. The alert comments are used to give context.
NOTE: Do not include information (such as full pathnames) in alert comments that you would not want sent in regular (untrusted) alert messages.
-w,--response-window DurationSpec
Indicate the duration of the response window whose start time is the time of last modification of a stimulus file. Requires --signed-difference. Implies the threshold '>0'. Duration spec must evaluate to a value greater than 0 seconds.
A duration specification is an indication of an amount of time, from a few seconds to a number of days, hours, minutes, and seconds. Duration specifications are given using one of the following three forms:
One or more days digits, a literal '+', and two each of hours, minutes, and seconds digits, separated by colons. Example: 100+12:30:00
One or more hours digits and two each of minutes and seconds digits separated by colons. Example: 2:00:00
One or more seconds digits. Example: 7200
A 'file' is considered by fmodchk to be anything that has an inode. Files may be specified as pathnames or globs. Each glob is expanded into a set of matching pathnames. In addition, the following form may be used to specify a file:
AGE_SELECTOR(GLOB)
GLOB is a file glob, and must be enclosed in parentheses as shown. AGE_SELECTOR may be one of the following, using any combination of uppercase and/or lowercase letters:
{ 'o', 'oldest', 'y', 'youngest' }
Each instance of the use of the age selectors 'oldest' and 'youngest' will cause a single pathname to be specified, which will be generated by expanding the glob and then choosing the oldest or youngest file, respectively.
fmodchk uses Log::Transcript, so all logging is done to stderr. Additionally a session transcript may be emailed under certain conditions.
An fmodchk configuration file is a text file whose maximum size is given in the usage message (fmodchk --help).
If an fmodchk command line contains the --config-file option, fmodchk will get all of its configuration information, including a list of checks to be performed, from the active configuration blocks in the named file. If --config-file is not used, then the check specified on the command line will be performed.
If both --config-file and --label are used, then only the active configuration blocks whose block names are identified by the given label list are used. ("Label" is synonymous with "configuration block name".) If --config-file is used without --label, all active configuration blocks in the configuration file are processed.
A configuration file is logically a set of tasks to be performed by fmodchk. These tasks contain the checks to be performed and other attributes of the task, e.g., to whom to email alerts.
The configuration file is a text file in which apache-like blocks of configuration text are present. A block has the form:
<blockname>
.
.
.
</blockname>
Blocks can be nested.
Key-value statements occur in blocks. A key value statement has the form:
key<whitespace>value
The value may contain whitespace. Everything after the whitespace that follows the key is considered to be the value for that key.
Blocknames and keywords may be specified in either upper- or lowercase. They will be lowercased when parsed. The case of values, however, will not be modified. For example, the following configuration snippet:
<fOO>
Bar This Is the Value For Bat
BAZ MUMBLE mumble
</Foo
will be stored in memory as the following perl data structure:
foo => {
bar => 'This Is the Value For Bat',
baz => 'MUMBLE mumble',
}
fmodchk requires that all configuration data in the file be enclosed in a top level block called "fmodchk":
<fmodchk>
.
.
.
</fmodchk>
Within this block may be zero or more "task" blocks, i.e., blocks that represent tasks, or checks that are to be performed. The exception to this is the master suspend block.
A task block consists of key-value statements and blocks from which argument vectors ("argv's"), or, arrays of options, are built. The long form of the options described in the usage statement (minus the "--" prefixes) are used as key values in a task block. In addition to these options, the keys "file" and "label-invocation-only" may be used in task blocks.
The "file" key is used to specify a file, glob, or age selector. See section SPECIFYING FILES for more information about the values for the "file" keyword.
The "file" keyword may appear multiple times to specify multiple files:
file /usr/local/foo file /usr/local/bar file /usr/local/bat file /usr/local/baz
So, what would be given on the command line as:
$ fmodchk --alerts-to foo@bar.domain --threshold '> 1+00:00:00' \
--alert-patrol --alert-text "check job FOO" '/tmp/junk*'
Would be specified as the following task block:
<my_block>
alerts-to foo@bar.domain
threshold > 1+00:00:00
alert-patrol yes
alert-text check job FOO
file /tmp/junk*
</my_block>
The 'label-invocation-only' key is a boolean key used in a task block to indicate that the block may be considered for processing (if it is active) only if specified on the command line in the LabelList option argument to the --label option. If the --label option is not used, the block will not be processed. If the --label option is used, but the block's name is not in the given label list, the block will not be processed. Configuration blocks without this boolean key (or with a false value for it) represent tasks which can be performed at any time. That is, they are not time sensitive tasks. Tasks which are label-invocation-only tasks are time sensitive, and this boolean key is used to make sure they are only processed when explicitly called.
Note that values for boolean options (e.g., "difference", or "signed-difference") may the following boolean values:
true false 1 0 yes no on off
NOTE: If a boolean key is given without a value, e.g.:
<my_block>
.
.
.
alert-patrol
.
.
.
<my_block>
a true value will be assigned to it. Using the example block above, Patrol would be alerted.
Suspend blocks
fmodchk processes only active configuration blocks. A block is active (vs. suspended) if one of the following is true:
Suspend blocks are used to toggle the active / inactive status of either the entire configuration file or of selected blocks. When a suspend block contains an active suspend interval, its enclosing block is inactive (suspended), i.e., not considered for processing.
A suspend block may occur in the top level "fmodchk" block (the master suspend block), or in any task block. It has the form:
<suspend>
from TIMESPEC
to TIMESPEC
</suspend>
where TIMESPEC is a string which represents a date and time. fmodchk parses a TIMESPEC by first removing all non-digit characters and then calling Date::Manip::ParseDate. The following are valid fmodchk TIMESPEC formats:
YYYYMMDDHHMMSS
YYYYMMDDHHMM
YYYYMMDDHH
YYYYMMDD
YYYYMM
YYYY
Any TIMESPEC that collapses into one of these forms when all non-digit characters are removed, may be used.
At least one of the keys "to" and "from" must be specified. If both are specified, the date and time represented by "to" must be later than the date and time represented by "from".
A suspend block defines a time interval in the local time zone. If the current time falls in that interval, the interval is said to be active. And when a suspend interval is active, its enclosing block is inactive (suspended) and is not considered for processing.
When both keys are given, the interval starts at the "from" time and ends at the "to" time. When only "from" is given, the interval starts at the "from" time and has no end. When only "to" is given, the interval ends at the "to" time but has no beginning. For example:
<suspend>
from 20050701
to 2006 01 01
</suspend>
specifies a time interval beginning July 1, 2005 at 0:00 local time and ends January 1, 2006 at 0:00.
<suspend>
from 2004-03-05 18:45:00
</suspend>
specifies an interval beginning March 5, 2004 at 6:45 pm local time but having no end.
<suspend>
to 2004/03/05 18.45.00
</suspend>
specifies an interval ending March 5, 2004 at 6:45 pm local time but having no beginning.
A suspend block, like any text in the configuration can be commented out, or have individual fields commented out.
Comments and ignored fields
Throughout this documentation, language such as "the configuration
file contains [something]" is usually used to mean that it contains
[something] and the line(s) that contain [something] are not
commented out.
Config::General, which parses the configuration file, recognizes comments as existing from a pound sign (#) to the end of the line. It also recognizes C-style comments ( /* */ ). If a pound sign is required in a value, it can be escaped with a backslash. Pound signs don't need to be escaped in here documents.
Anything commented out as mentioned above will not become part of any data structure resulting from parsing the configuration file. That is, fmodchk will not be aware of its existence.
There are two names that will become part of parsed configuration data structures, but will be ignored by fmodchk. fmodchk will ignore any key-value statement whose key is "comment" or "comments" and will ignore any "comment" or "comments" blocks. These key-value statements and blocks will become a part of the configuration data (unlike the comments in the file using # and /* */, which are ignored by the parser). The value in this is that information can be included in the configuration blocks themselves which will be logged when the block is logged.
Line continuation
Lines may be continued by quoting the newline with a backslash. Using:
mykey foo \
bar \
bat \
baz \
mumble
the value for mykey will become "foo bar bat baz mumble".
Here documents
Using:
mykey <<EOD
line 1
line 2
line 4
EOD
the value for mykey will become:
line 1 line 2
line 4
An amount of whitespace equal to that occuring before the delimiter ("EOD") will be removed from each line in the here doc.
Suspend blocks
(One suspend block per containing block.)
<suspend>
from 2005-07-01 18:00
to 2005.07.02 00.00
</suspend>
<suspend>
from 2006
</suspend>
<suspend>
to 2006/02/28
</suspend>
label-invocation-only
label-invocation-only
label-invocation-only yes
label-invocation-only 1
label-invocation-only true
-r,--alerts-to
alerts-to foo@bar.domain,some-list@bork.domain,root@gleep.domain
-R,--trusted-alerts-to
trusted-alerts-to baz@bat.domain,admin@bloop.domain,user37@localhost
-d,--difference
difference
difference yes
difference 1
difference true
-D,--signed-difference
Any of:
signed-difference
signed-difference yes
signed-difference 1
signed-difference true
-e,--errors-to
errors-to mumble@mumble.domain,error-folks@fnerp.domain
-f,--mail-from
mail-from fmodchk@my.domain
-P,--alert-patrol
alert-patrol
alert-patrol yes
alert-patrol 1
alert-patrol true
-t,--threshold
Whitespace between operator and value O.K. but not required.
threshold ==0
threshold <=600
threshold <= 8640000
threshold > 30+00:00:00
threshold >= 240:00:00
-T,--alert-comments
alert-comments << EndComments
***
The lack of a young enough archive file for this inbound Foo
archive directory may mean that we didn't receive a file on time
from Foo & Associates. Please investigate.
***
EndComments
file arguments
file /some/path/*foo*
file /some/other/path/bar.{c,h}
file /usr/local/bat
file /
file oldest(/partners/foo/archive/*)
file youngest(/partners/widgets_inc/archive/*)
There are five kinds of tests identified currently which can be implemented using fmodchk:
Age check
In this example, we want to send a trusted alert (full pathname
information included) to foo@bar.domain if directory /usr/local is less
than sixty days old. We want the email to appear to come from
<bat@baz.domain>, rather than the default address (for which see usage
statement: fmodchk --help). We want BMC Patrol to be made aware
of any alerts. We'd like the recipients of the alert to contact the
help desk at (555)-555-5555. Rather than having runtime error
transcripts emailed to the default email address(es) (for which, see
usage statement), we want error transcripts to be sent to
<thomas.tallis@motets.domain>.
We could use the following invocation in a crontab entry (must be all on one line in the crontab; broken across multiple lines here only for readability):
/FULL/PATH/TO/fmodchk \
--trusted-alerts-to foo@bar.domain \
--errors-to thomas.tallis@motets.domain \
--mail-from bat@baz.domain \
--alert-patrol \
--alert-comments 'Please contact help desk at (555) 555-5555' \
--threshold '< 60+00:00:00' \
/usr/local
Alternatively, we could specify a configuration file in the invocation:
/FULL/PATH/TO/fmodchk --config-file /PATH/TO/fmodchk.conf
and include the following block in fmodchk.conf:
<usr-local-age-check>
trusted-alerts-to foo@bar.domain
errors-to thomas.tallis@motets.domain
mail-from bat@baz.domain
alert-patrol yes
alert-comments Please contact help desk at (555) 555-5555
threshold < 60+00:00:00
file /usr/local
</usr-local-age-check>
The name you use for the block can be any identifier. It is recommended that you use something descriptive and unique to the task.
Recent modification check
In this example, we want to send an untrusted alert to mumble@bar.domain if the youngest file in directory /data/flows/foo_associates/inbound/archive is over two hours old. In this case, we want fmodchk to execute only a single time sensitive task, and only at a certain time. Therefore, in the invocation, we direct fmodchk to use only one block in a configuration file:
/FULL/PATH/TO/fmodchk \
--config-file /some/directory/fmodchk.configuration \
--label foo-associates-inbound-archive-recent-modification
In /some/directory/fmodchk.configuration, we then have:
<foo-associates-inbound-archive-recent-modification>
label-invocation-only
alerts-to mumble@bar.domain
threshold > 2:00:00
alert-comments <<EndOfAlertComments
We haven't archived an inbound file from Foo & Associates
for over two hours. Please investigate.
EndOfAlertComments
file youngest(/data/flows/foo_associates/inbound/archive/*)
</foo-associates-inbound-archive-recent-modification>
Note that because of the inclusion of the label-invocation-only key in the block, this block will ever be executed only if its name is in a label list given with the --label option.
Note that label-invocation-only was not given a value. When this happens for a boolean key, the value is taken to be true.
Unsigned difference check
Send a trusted alert to william.byrd@composers.domain if the absolute value of the age difference between directory / and directory /etc is less than or equal to three and a half days. Cron entry:
/FULL/PATH/TO/fmodchk \
--trusted-alerts-to william.byrd@composers.domain \
--difference \
--threshold '<= 3+12:00:00' \
'{/,/etc}'
Note that on this particular invocation, we arbitrarily chose to use a file glob (using curly brace expansion) to specify the directories. We could just as easily have listed the directories individually:
/ /etc
Alternatively, here is a configuration block that defines the same task:
<root-etc-unsigned-difference>
trusted-alerts-to william.byrd@composers.domain
difference
threshold <= 3+12:00:00
file {/,/etc}
<root-etc-unsigned-difference>
Signed difference check
Send a trusted alert to unreliable.pager@archpageless.domain if /etc/foo.config is older than /etc/bar.config by more then ten seconds.
/FULL/PATH/TO/fmodchk \
--trusted-alerts-to unreliable.pager@archpageless.domain \
--signed-difference \
--threshold '> 10' \
'/etc/{foo,bar}.config'
Again, we arbitrarily chose to use brace expansion rather than list both files separately.
Alternatively, here is a configuration block that defines the same task:
<etc-foo-bar-config-signed-difference>
trusted-alerts-to unreliable.pager@archpageless.domain
signed-difference
threshold > 10
file /etc/{foo,bar}.config
</etc-foo-bar-config-signed-difference>
Stimulus-response check
Assume that whenever we send Squidly Sushi & Bait a *.bait file, we expect a *.switch file in response within eighteen hours. Assume we have separate inbound and outbound archive directories for Squidly files. Then we will detect that Squidly has not sent a response that meets our time requirements when both of the following are true:
Here is a cron invocation that would accomplish this:
/FULL/PATH/TO/fmodchk \
--trusted-alerts-to squidly-monitors@bait.and.switch.domain \
--signed-difference \
--response-window 18:00:00 \
youngest(/data/flows/squidly/outbound/archive/*.bait) \
youngest(/data/flows/squidly/inbound/archive/*.switch)
Note that --threshold is not used (and it is ignored if given). The threshold used when --response-window is given is '<0'. When doing a stimulus-response check, list the stimulus file first and the response file second. That is, detect a violation if the signed difference:
age( stimulus file ) - age( response file ) < 0
Alternatively, here is a configuration block that defines the same task:
<squidly-bait-switch-stimulus-response>
trusted-alerts-to squidly-monitors@bait.and.switch.domain
signed-difference
response-window 18:00:00
file youngest(/data/flows/squidly/outbound/archive/*.bait)
file youngest(/data/flows/squidly/inbound/archive/*.switch)
</squidly-bait-switch-stimulus-response>
The following are modules used by fmodchk that are not in the standard Perl distribution:
Available at <http://www.cpan.org>:
Config::General Date::Manip File::Glob Mail::Sendmail
Developed by author:
Log::Transcript
See documentation for the modules listed in the DEPENDENCIES section.
David Alban