Soto Systems LLC

by Ramses Soto-Navarro ramses@sotosystems.com, 4/27/2021

Overview
The Script
Crontab

Overview

Brief explanation of a log check bash script which alerts when there are too many log errors per day. It parses a set of words to search at the end of the day; counts the number of occurrences; then if a max number is surpassed, send an Email alert with a count of each set of words. A choice of sending via Email or displaying on console is provided.

The Script

For cron to work well, it is recommended to export the unique variables:

#!/bin/bash
#
# Find if too many log errors per day; then Email.

SNAME=`basename $0`

SYNTAX="$SNAME < -e email results | -d display only | -h help>"

if [ $# -lt 1 ] ; then
	echo $SYNTAX
	exit 1
fi

CHOICE=$1
export MYLOG_TAGS="error|fail|warn|panic|critical|deny|denied|unknown|refuse|illegal|bad|reject|unapproved" 
export MYLOG_MAX=4
export MYLOG_DATE=`date +%Y-%m-%d`
export MYLOG_FILE=/var/log/messages
export MYLOG_TMP1=/tmp/$SNAME-$RANDOM.txt
export MYLOG_TMP2=/tmp/$SNAME-$RANDOM-2.txt
export MYLOG_MSG=/tmp/$SNAME-$RANDOM-3.txt
export MYLOG_MAIL="
admin@example.com
"

f_email () {
cat $MYLOG_MSG | mail -s "Alert: suspicious central logs" $MYLOG_MAIL
}

f_display () {
cat $MYLOG_MSG
}

grep $MYLOG_DATE $MYLOG_FILE > $MYLOG_TMP1
for a in `echo $MYLOG_TAGS | sed 's/|/ /g'` ; do echo -ne "$a: " && grep -i $a $MYLOG_TMP1 | wc -l ; done | sed /0$/d > $MYLOG_TMP2
export MYLOG_NUM=`sort -n -k2 $MYLOG_TMP2 | tail -n 1 | awk '{print $2}'`
if [ $MYLOG_NUM -gt $MYLOG_MAX ]; then
	echo -ne "nALERT: suspicious central log entries found on $HOSTNAME:$MYLOG_FILE: nn" > $MYLOG_MSG
	logger -t mylogcheck.sh "Suspicious central log entries found: $MYLOG_NUM"
	sort -n -k 2 "$MYLOG_TMP2" | grep $MYLOG_NUM >> $MYLOG_MSG
	case $CHOICE in
		-e)	f_email 
			;;
		-d)	f_display 
			;;
		*)	echo $SYNTAX 
			exit 1
			;;
	esac
fi

rm -f $MYLOG_TMP1 $MYLOG_TMP2 $MYLOG_MSG

unset MYLOG_MAX
unset MYLOG_DATE
unset MYLOG_FILE
unset MYLOG_TMP1
unset MYLOG_TMP2
unset MYLOG_MSG
unset MYLOG_MAIL
unset MYLOG_NUM

Crontab

# crontab -e

45 23 * * * /root/bin/mylogcheck.sh -e

Email Sample

root@server1.example.com
Tue 4/27/2021 2:12 PM

ALERT: suspicious central log entries found on server1:/var/log/messages:

bad: 5
error: 5

The End.