Nagios Install on Red Hat 8
Thursday, April 15, 2021
Add a comment
by Ramses Soto-Navarro ramses@sotosystems.com, 4/15/2021
Overview
Required Packages
Install Fping
Create Accounts
Install Nagios Core
Install Nagios Plugins
Configure Nagios
Configure Apache
Configure Email
Firewall Rules
Configure Nodes
Add Nodes to Monitor
Logrotate
Configure SNMP Defaults
Test Commands
Node Stress Test
OID List
TODO
Overview
Brief notes on Nagios install on Red Hat Linux 8.x. Provides: SNMP monitoring, Email alerts, web interface. The audience is experienced Linux administrators.
Required Packages
# yum install man wget openssh-clients rsync traceroute nmap nc telnet ftp elinks ntp bind-utils httpd gcc gcc-c++ php glibc glibc-common man perl gd gd-devel libjpeg-devel libpng-devel postfix openssl-devel gnutls gnutls-devel perl-CPAN libdbi libdbi-devel libdbi-dbd-mysql samba-client net-snmp-utils bind-utils perl-DBD-MySQL mod_auth_mysql openldap-clients openldap-devel perl-LDAP php-ldap libgcrypt-devel libopenssl-devel gettext-runtime automake net-snmp perl-Net-SNMP cpan Crypt::DES Digest::HMAC Digest::SHA1 Net::SNMP Crypt:Rijndael # yum groupinstall "MYSQL Database Client" (Optional) # yum install mysql mysql-server mysql-test mysql-devel mysql-libs mysql-connector-odbc php-mysql
Install Fping
# cd /usr/src # wget http://fping.org/dist/fping-5.0.tar.gz # tar zxf fping-5.0.tar.gz # cd fping-5.0/ # ./configure --disable-ipv6 --enable-ipv4 # make # make install # fping mis26
Create Accounts
Create Nagios account:
# groupadd nagios # useradd -m -g nagios nagios # passwd nagios # grep apache /etc/passwd # usermod -a -G nagios apache
Install Nagios Core
# wget https://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.4.6/nagios-4.4.6.tar.gz # tar zxvf nagios-4.4.6.tar.gz # cd nagios-4.4.6/ # ./configure --with-nagios-group=nagios --with-command-group=nagios # make all # make install # make install-init # make install-commandmode # make install-config # make install-webconf # make install-exfoliation # make install-daemoninit # make install-classicui # ldconfig
Install Nagios Plugins
# cd /usr/src # wget http://nagios-plugins.org/download/nagios-plugins-2.3.3.tar.gz # tar zxf nagios-plugins-2.3.3.tar.gz # cd nagios-plugins-2.3.3/ # ./configure --with-cgiurl=/ --with-nagios-user=nagios --with-nagios-group=nagios --with-openssl # make # make install # chown -R nagios.nagios /usr/local/nagios
Configure Nagios
Edit CGI configuration:
# sed -i 's/nagiosadmin/nagios/g' /usr/local/nagios/etc/cgi.cfg ... url_html_path=/ ...
Edit PHP configuration:
# vi /usr/local/nagios/share/config.inc.php ... $cfg['cgi_base_url']='/cgi-bin'; ...
Edit contacts Email:
# sed -i 's/nagiosadmin/nagios/g' /usr/local/nagios/etc/objects/contacts.cfg # vi /usr/local/nagios/etc/objects/contacts.cfg ... email ramses@sotosystems.com ; ...
Edit templates:
# vi /usr/local/nagios/etc/objects/templates.cfg
define host {
name linux-server
use generic-host
check_period 24x7
check_interval 2
retry_interval 1
max_check_attempts 10
check_command check-host-alive
notification_period workhours
notification_interval 120
notification_options d,u,r
contact_groups admins
register 0
}
Edit notification time periods:
define timeperiod {
name workhours
timeperiod_name workhours
alias Normal Work Hours
monday 07:00-18:00
tuesday 07:00-18:00
wednesday 07:00-18:00
thursday 07:00-18:00
friday 07:00-18:00
}
Edit commands:
# vi /usr/local/nagios/etc/objects/commands.cfg
define command {
command_name notify-host-by-email
command_line /usr/bin/printf "%b" "Nagios $NOTIFICATIONTYPE$nHost: $HOSTNAME$nState: $HOSTSTATE$nAddress: $HOSTADDRESS$nInfo: $HOSTOUTPUT$nnDate/Time: $LONGDATETIME$n" | /usr/bin/mail -s "$HOSTSTATE$ $HOSTNAME$" $CONTACTEMAIL$
}
define command {
command_name notify-service-by-email
command_line /usr/bin/printf "%b" "Nagios $NOTIFICATIONTYPE$nnService: $SERVICEDESC$nHost: $HOSTALIAS$nAddress: $HOSTADDRESS$nState: $SERVICESTATE$nnDate/Time: $LONGDATETIME$nnAdditional Info:nn$SERVICEOUTPUT$n" | /usr/bin/mail -s "$SERVICESTATE$ $HOSTALIAS$/$SERVICEDESC$" $CONTACTEMAIL$
}
...
define command {
command_name check_snmp_load
command_line $USER1$/check_snmp_load.pl -H $HOSTADDRESS$ -C public -T netsl -w $ARG1$ -c $ARG2$
}
define command {
command_name check_snmp_storage
command_line $USER1$/check_snmp_storage.pl -H $HOSTADDRESS$ -C public -m $ARG1$ -w $ARG2$ -c $ARG3$ $ARG4$
}
define command {
command_name check_snmp_mem
command_line $USER1$/check_snmp_mem.pl -H $HOSTADDRESS$ -C public -w $ARG1$ -c $ARG2$
}
Configure Apache
Create the Apache user and password:
# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagios
Configure Apache virtual host:
# vi /etc/httpd/conf.d/nagios.example.com.conf <VirtualHost *:80> ServerName nagios.example.com ServerAlias nagios.example.com ServerAdmin ramses@example.com ErrorLog /var/log/httpd/nagios.example.com.err CustomLog /var/log/httpd/nagios.example.com.log combined DocumentRoot /usr/local/nagios/share ScriptAlias /cgi-bin "/usr/local/nagios/sbin" <Directory "/usr/local/nagios/sbin"> # SSLRequireSSL Options ExecCGI AllowOverride None <IfVersion >= 2.3> <RequireAll> Require all granted # Require host 127.0.0.1 AuthName "Nagios Access" AuthType Basic AuthUserFile /usr/local/nagios/etc/htpasswd.users Require valid-user </RequireAll> </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all # Order deny,allow # Deny from all # Allow from 127.0.0.1 AuthName "Nagios Access" AuthType Basic AuthUserFile /usr/local/nagios/etc/htpasswd.users Require valid-user </IfVersion> </Directory> Alias /nagios "/usr/local/nagios/share" <Directory "/usr/local/nagios/share"> # SSLRequireSSL Options None AllowOverride None <IfVersion >= 2.3> <RequireAll> Require all granted # Require host 127.0.0.1 AuthName "Nagios Access" AuthType Basic AuthUserFile /usr/local/nagios/etc/htpasswd.users Require valid-user </RequireAll> </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all # Order deny,allow # Deny from all # Allow from 127.0.0.1 AuthName "Nagios Access" AuthType Basic AuthUserFile /usr/local/nagios/etc/htpasswd.users Require valid-user </IfVersion> </Directory> </VirtualHost>
Restart Apache:
# systemctl restart httpd
Configure Email
Configure mailer:
# vi /etc/postfix/main.cfg inet_interfaces = $myhostname, localhost relayhost = mx.example.com # systemctl enable postfix # systemctl start postfix
Configure Mailrc:
# vi /home/nagios/.mailrc set from=nagios@nagios.example.com
Firewall Rules
Add firewall rules:
# firewall-cmd --zone=public add-service=http --permament # firewall-cmd --zone=public add-service=https --permament # systemctl reload firewalld
Configure Nodes
Configure each node client with the following:
* SNMP Listen on IPV4 161/udp only, read only access.
* Monitor all disks for 10% or less free space.
* Compatible SNMPv1 public community string.
* Firewall rules accepts only Nagios server and localhost.
* Reachable only by Nagios server and localhost.
* public = SNMP community string.
* No encryption or authentication; keep it simple.
Install SNMP on the host:
# yum install net-snmp
Edit the node SNMP settings so that 192.168.1.45 is the address of the Nagios server. Read-only Security will be accepted via static IP address access and firewall rules. Enable SNMP.
# vi /etc/snmp/snmpd.conf rocommunity public 192.168.1.45 rocommunity public 127.0.0.1 agentAddress udp:161 syscontact sysadmins@example.com includeAllDisks 10% load 15 10 5 # systemctl enable snmpd # systemctl start snmpd
Firewall rule; accept only communication from nagios server 192.168.1.45. List the rules.
# systemctl enable firewalld # systemctl start firewalld
Configure node firewall to accept only connections from Nagios server and localhost:
# firewall-cmd --zone=public --remove-service snmp # firewall-cmd --add-rich-rule='rule family="ipv4" source address="192.168.1.45" port protocol="udp" port="161" accept' # firewall-cmd --add-rich-rule='rule family="ipv4" source address="127.0.0.1" port protocol="udp" port="161" accept' # firewall-cmd --runtime-to-permanent # firewall-cmd --reload # firewall-cmd --list-all --zone=public
Add Nodes to Monitor
Create the alert monitoring host file for each node to monitor.
# mkdir -p /usr/local/nagios/etc/objects/hosts
# vi /usr/local/nagios/etc/objects/hosts/devapp.broward.edu.cfg
define host {
use linux-server
host_name node1.example.com
alias node1.example.com
address 10.215.6.80
}
# Ping
define service {
use generic-service
host_name node1.example.com
service_description Ping
check_command check_ping!100.0,20%!500.0,60%
notifications_enabled 1
}
# SSH
define service {
use generic-service
host_name node1.example.com
service_description SSH
check_command check_ssh
notifications_enabled 1
}
# Load
define service{
use generic-service
host_name node1.example.com
service_description Load
check_command check_snmp_load!5,4,3!7,6,5
notifications_enabled 1
}
# Memory
define service{
use generic-service
host_name node1.example.com
service_description Mem
# Memory: -warning ram%,swap% ! -critical ram%,swap%
check_command check_snmp_mem!95,60!99,80
#check_command check_snmp_mem!80,60!99,80
#check_command check_snmp_mem!90,60!99,80
notifications_enabled 1
}
# Storage /
define service{
use generic-service
host_name node1.example.com
service_description /
check_command check_snmp_storage!"^/$"!70!90!
notifications_enabled 1
}
# Swap
define service{
use generic-service
host_name node1.example.com
service_description Swap
check_command check_snmp_storage!Swap!60!90
notifications_enabled 1
}
# Storage /usr
define service{
use generic-service
host_name node1.example.com
service_description /usr
check_command check_snmp_storage!"^/usr$"!70!90!
notifications_enabled 1
}
# Storage /specialdir
define service{
use generic-service
host_name node1.example.com
service_description /specialdir
check_command check_snmp_storage!"^/specialdir$"!80!90!
notifications_enabled 1
}
# Storage /var
define service{
use generic-service
host_name node1.example.com
service_description /var
check_command check_snmp_storage!"^/var$"!70!90!
notifications_enabled 1
}
# Storage /home
define service{
use generic-service
host_name node1.example.com
service_description /home
check_command check_snmp_storage!"^/home$"!70!90!
notifications_enabled 1
}
# HTTP
define service{
use generic-service
host_name node1.example.com
service_description HTTP
check_command check_http!2!5
notifications_enabled 1
}
Add the configuration to Nagios:
# vi /usr/local/nagios/etc/nagios.cfg ... cfg_file=/usr/local/nagios/etc/objects/hosts/node1.example.com.cfg ...
Restart Nagios:
# systemctl restart nagios # systemctl status nagios
Logrotate
Create the log rotations:
# vi /etc/logrotate.d/nagios
/usr/local/nagios/var/nagios.log {
monthly
compress
dateext
maxage 90
rotate 10
missingok
notifempty
size +40960k
create 640 nagios.nagios
sharedscripts
}
Configure SNMP Defaults
From the Nagios server configure the root account with default settings. Verify that the host SNMP can be reached with the defaults.
# mkdir -p ~/.snmp # vi ~/.snmp/snmp.conf defVersion 1 defCommunity public # snmpwalk node1.example.com sysName
Test Commands
Test one Nagios command:
# /usr/local/nagios/libexec/check_http -h # /usr/local/nagios/libexec/check_http -4 -H node1.example.com -w 2 -c 5
Check logs parsing the dates to human readable:
cat /usr/local/nagios/var/nagios.log | perl -pe 's/(d+)/localtime($1)/e'
Test PHP:
# vi /usr/local/nagios/share/test.php <?php phpinfo(); ?>
Test mail:
# su - nagios $ echo "test" | mail -s "test" ramses@sotosystems.com
Testing connecting from the nagios server and from localhost:
# snmpwalk -v 1 -c public node1.example.com sysName # snmpwalk -v 1 -c public localhost sysName
Node Stress Test
Stress samples for testing CPU, memory, disk, SSH.
# yum install lksctp-tools # rpm -ivh https://centos.pkgs.org/8/getpagespeed-x86_64/stress-1.0.4-23.el8.x86_64.rpm.html # rpm -ivh https://centos.pkgs.org/8/epel-testing-x86_64/stress-ng-0.12.00-1.el8.x86_64.rpm.html
Test storage space by filling a mount (increase size to accomodate):
# df -hP # dd if=/dev/zero of=/tmp/nullfile.img bs=1G count=10 # df -hP
Test load. Raise the value for faster stress:
# uptime # stress --cpu 2 # uptime # tail -f /usr/local/nagios/var/nagios.log
On another screen and while looking at the Nagios web GUI, run:
# ../../libexec/check_snmp_load.pl -H node1.example.com -C public -w 3 -c 6 2 CPU, average load 14.0% > 6% : CRITICAL
Test memory:
# stress-ng --vm-bytes $(awk '/MemAvailable/{printf "%dn", $2 * 0.9;}' < /proc/meminfo)k --vm-keep -m 1
# watch free -h
OID List
List of some OID’s in the MIB to search for:
hostname: .1.3.6.1.2.1.1.5 1 minute Load: .1.3.6.1.4.1.2021.10.1.3.1 5 minute Load: .1.3.6.1.4.1.2021.10.1.3.2 15 minute Load: .1.3.6.1.4.1.2021.10.1.3.3 Percentage of user CPU time: .1.3.6.1.4.1.2021.11.9.0 Raw user cpu time: .1.3.6.1.4.1.2021.11.50.0 Percentages of system CPU time: .1.3.6.1.4.1.2021.11.10.0 Raw system cpu time: .1.3.6.1.4.1.2021.11.52.0 Percentages of idle CPU time: .1.3.6.1.4.1.2021.11.11.0 Raw idle cpu time: .1.3.6.1.4.1.2021.11.53.0 Raw nice cpu time: .1.3.6.1.4.1.2021.11.51.0 Total Swap Size: .1.3.6.1.4.1.2021.4.3.0 Available Swap Space: .1.3.6.1.4.1.2021.4.4.0 Total RAM in machine: .1.3.6.1.4.1.2021.4.5.0 Total RAM used: .1.3.6.1.4.1.2021.4.6.0 Total RAM Free: .1.3.6.1.4.1.2021.4.11.0 Total RAM Shared: .1.3.6.1.4.1.2021.4.13.0 Total RAM Buffered: .1.3.6.1.4.1.2021.4.14.0 Total Cached Memory: .1.3.6.1.4.1.2021.4.15.0 Path where the disk is mounted: .1.3.6.1.4.1.2021.9.1.2.1 Path of the device for the partition: .1.3.6.1.4.1.2021.9.1.3.1 Total size of the disk/partion (kBytes): .1.3.6.1.4.1.2021.9.1.6.1 Available space on the disk: .1.3.6.1.4.1.2021.9.1.7.1 Used space on the disk: .1.3.6.1.4.1.2021.9.1.8.1 Percentage of space used on disk: .1.3.6.1.4.1.2021.9.1.9.1 Percentage of inodes used on disk: .1.3.6.1.4.1.2021.9.1.10.1 System Uptime: .1.3.6.1.2.1.1.3.0
TODO
* Configure HTTPS with signed certificate.
The End.