Soto Systems LLC

by Ramses Soto-Navarro, ramses@sotosystems.com

---

Overview
Install
Quick Start
Daily Script
List Archives
Extract Restore
Delete Archive
Mount Archive
Export Tarball
Install SSHFS
Remote Restore
Prune

---

Overview

Brief notes about Borg; an executable for backups with many cool features: deduplication, high compression, mountable file system. Follow the logic; for experienced Linux administrators.

Install

Download latest from: https://github.com/borgbackup/borg/releases

# cd /usr/local/bin
# wget https://github.com/borgbackup/borg/releases/download/1.1.16/borg-linux64
# chmod 0755 borg-linux64
# ln -s borg-linux64 borg
# borg -h

Quick Start

Create backup repository; leave password blank for testing. Backup /u1/ directory into the /backup/ archive called Monday. Next day will be a lot quicker, since only new data is stored (deduplication). Show the stats. Create the cronjob to run daily.

$ borg init -e repokey /backup
$ borg create --compression zlib /backup::Monday /u1
$ borg create --compression zlib --stats /backup::Tuesday /u1

$ crontab -e

# m h  dom mon dow   command
# Borg daily 11PM.

* 23 * * * 1-5 /root/bin/myborg-backup.sh

Daily Script

Create a backup script to backup remote nodes via pull. Initialize the archive. Create configuration script for each node to backup. Last, a backup plan script to backup all the nodes one after the other via a cronjob. NOTE: first fuse-sshfs needs to be installed, as shown in sections at the bottom.

# borg init -e repokey /backup/node1

# vi ~/bin/myborg.sh

#!/bin/bash
#
# Borg backup script.
# by Ramses Soto-Navarro <ramses@sotosystems.com>, 04/13/2021

SYNTAX="Syntax: $0 <configfile.conf>"

if [ $# -lt 1 ]; then
  echo $SYNTAX
  exit 1
fi

. $1

if [ $? -gt 0 ]; then
  echo $SYNTAX
  exit 1
fi

sshfs -o ro $RHOST:/ $D2
cd $D2
logger myborg.sh "Starting Borg backup for $RHOST."
#borg create --exclude-from=$EXL --compression zlib --stats $D1::$JOB $DIRS
borg create --exclude-from=$EXL --compression zlib $D1::$JOB $DIRS
cd
umount $D2
rm $EXL
logger myborg.sh "End of Borg backup for $RHOST."

# vi ~/bin/myborgall.sh 
#!/bin/bash
#
# Run all the borg jobs one after the other

LIST="
node1
node2
"

for EACH in $LIST ; do
  /root/bin/myborg.sh /backup/$EACH.conf
  echo $EACH
done

# vi /backup/node1.conf 

#!/bin/bash
#
# borg backup config file.

RHOST=node1
DIRS="."

DATE=`date +%Y%m%d%H%M-$RANDOM`
JOB=$RHOST-$DATE
D1=/backup/$RHOST
D2=/mnt/$RHOST 
EX="dev lost+found media mnt proc run srv sys tmp opt/tmp"
EXL=/tmp/EXL-$JOB.txt

for a in $EX ; do echo $a ; done > $EXL

mkdir -p $D1
mkdir -p $D2

CHECK1=`ssh $RHOST uptime | grep -i "load average"`
if [ "$CHECK1" == "" ]; then
  echo "$RHOST not reachable."
  exit 1
fi

List Archives

$ borg list /backup
$ borg list /backup::Monday

Extract Restore

Extract (restore) files. Verify first to default cd to the directory where you want to restore.

$ borg extract /backup::Monday
$ borg extract /backup::Monday home/rasoto/Documents

Delete Archive

==== Delete Archive ====

Delete archive:

$ borg delete /backup::Monday

Mount Archive

# borg mount /backup/node1 /mnt/tmp/

# ls /mnt/tmp/
20210401a  20210401b  20210401c

# pwd
/mnt/tmp/20210401a

# find | head
.
./u1
./u1/clients
./u1/clients/broward.edu
./u1/clients/broward.edu/.ssh
./u1/clients/broward.edu/.ssh/testftp
./u1/clients/broward.edu/.ssh/id_rsa
./u1/clients/broward.edu/.ssh/id_rsa.ppk
./u1/clients/broward.edu/.ssh/id_rsa.pub
./u1/clients/broward.edu/.ssh/id_rsa.pub.ppk

Export Tarball

To export a backup to a tar ball:

# borg export-tar --tar-fiflter="gzip -9" /backup::Monday Monday.tar.gz

Install SSHFS

SSHFS mounts systems remotely via SSH. Install sshfs on backup host. If your distro does not have SSHFS then find the best available RPM via rpm.pbone.net.

rpm -ivh fuse-sshfs-2.8.5.el8.x86_64.rpm

Verify that SSH keys are set (beyond scope of this document).

Mount remote system read-only:

# sshfs -o ro server1:/ /mnt/server1

Refer to Backup Quick Start above to continue.

Remote Restore

Simply mount the backup, as described in sections above, then transfer them over to the restore node via rsync, SFTP or an SSHFS mount.

Prune

TODO: Create a cron script to prune old backup jobs that are more than 2 weeks old, via the “borg delete” function.

---

The End.

by Ramses Soto-Navarro, ramses@sotosystems.com

Overview
The Script
Cronjob
Remove IP

Overview

mydeny.sh script adds IP addresses to /etc/hosts.deny, which have too many bad SSH login attempts. It is a simple alternative to the older python denyhosts. It searches every night for IP addresses that failed to SSH more than 20 times, via cron. If so then it adds it to hosts.deny. Logging of each denied IP will be sent to /var/log/messages as mydeny.sh. Follow the parsing logic to automatically add more libwrap services to hosts.deny. This document is for experienced Linux administrators.

The Script

#!/bin/bash

MAX=20
DATE=`date +%Y-%m-%d`
MARK=$RANDOM
TMP1=~/tmp/$MARK-1.txt
TMP2=~/tmp/$MARK-2.txt

f_findbadssh () {
mkdir -p ~/tmp/
grep $DATE /var/log/messages | grep sshd | grep "error: PAM: User not known" | awk '{print $NF}' | sort | uniq > $TMP1
for a in `cat $TMP1` ; do echo -ne "$a: " && grep $a /var/log/messages | wc -l ; done > $TMP2
sed -i 's/://g' $TMP2
}

f_addtodh () {
cat $TMP2 | while read a ; do
	IP=`echo $a | awk '{print $1}'`
	COUNT=`echo $a | awk '{print $2}'`

	if [[ COUNT -gt MAX ]] ; then
		#echo "High Bad SSH Login Count = $COUNT for $IP. Adding to /etc/hosts.deny."
		CHECK1=`grep "$IP" /etc/hosts.deny`
		if [ "$CHECK1" == "" ]; then
			logger -t mydeny.sh "Adding $IP to /etc/hosts.deny."
			echo "sshd: $IP" >> /etc/hosts.deny
		fi
	fi
done
rm -f $TMP1 $TMP2
} 

f_findbadssh
f_addtodh

Cronjob

~ # crontab -l

# Add bad SSH login IPs to hosts.deny every 2 hours.
0 */2 * * * /root/bin/mydeny.sh

! # systemctl restart cron

Remove IP

To remove the IP from hosts.deny run:

# sed -i '/61.177.172.158/d' /etc/hosts.deny

by Ramses Soto-Navarro ramses@sotosystems.com

Overview
Register
Install RMT
Setup MySQL Password
RMT Setup
Setup Firewall Rules
Enable SLP Broadcast
RMT Server Status
Install Repos
Setup RMT Clients
Add Repo Install Directories
Create Mini ISOs

---

Overview

Brief notes about how to install RMT (Repository Mirror Tool) on SuSE 15.x. The audience is experienced Linux administrators.
NOTE: Do not install Apache; on SuSE it uses Nginx. Official Documentation:https://documentation.suse.com/sles/15-SP1/single-html/SLES-rmt/index.html

Register

Verify server is registered with SUSE via regular registration procedures: yast, Product Registration.
Verify online repos are populated:

# zypper refresh
# zypper repos

Install RMT

Install the repository mirror too. If not already setup, then MySQL will also be installed.

# zypper in rmt-server
# rcmysql start
# rcnginx start
# systemctl enable mariadb
# systemctl enable nginx

Setup MySQL Password

# set +o history
# mysqladmin -u root password 'password'
# set -o history
# mysql -u root -p
>show databases;
>quit

RMT Setup

# yast, 
	Network Services, RMT Configuration.

To get credentials: ssc.suse.com, Proxies, top right, click on eye:

Database username: rmt
Password: <select password>
CA Private Key Password: <select password>

Setup firewall rules

# firewall-cmd --get-active-zones
# firewall-cmd --list-all --zone=internal
# firewall-cmd --zone=internal --add-service=snmp
# firewall-cmd --zone=internal --add-service=http
# firewall-cmd --zone=internal --add-service=https
# firewall-cmd --zone=internal --add-service=ftp
# firewall-cmd --zone=internal --add-service=squid
# firewall-cmd --zone=internal --add-service=nfs
# firewall-cmd --zone=internal --add-service=nfs3
# firewall-cmd --zone=internal --add-service=syslog
# firewall-cmd --zone=internal --permanent --add-port=427/tcp
# firewall-cmd --zone=internal --permanent --add-port=427/udp
# firewall-cmd --runtime-to-permanent
# firewall-cmd --reload

Enable SLP Broadcast

# zypper install openslp-server
# systemctl enable slpd.service
# systemctl restart slpd.service

View RMT Server Status

Look at trigger section for the time it will be udpated.

# systemctl status rmt-server-sync.timer

View Products and Repos:

# rmt-cli products list --all
# rmt-cli repos list --all

View installed products and repos:

# rmt-cli products list
# rmt-cli repos list

Install Repos

Install the repos and packages for SuSE 12 SP4, 12 SP5, 15 SP1, syncronize, then mirror.

# rmt-cli products enable SLES/12.4/x86_64 SLES/12.5/x86_64 SLES/15.1/x86_64 

Alternative:

#rmt-cli products enable 1625 1878 1763
# rmt-cli sync 
# rmt-cli mirror

Setup RMT Clients

Setup the other servers to register and install packages via the repo servers:

# wget http://test-repo1/tools/rmt-client-setup
# sh rmt-client-setup https://test-repo1.example.com
	Do you accept this certificate? [y/n] y
	Start the registration now? [y/n] y

# zypper refresh
# zypper repos
# zypper list-updates
# zypper update

Install a test package mc (midnight commander):

# zypper install mc
# zypper info mc
# zypper packages
# mc

Add Repo Install Directories

Upload ISOs and mount them, ready for local over the LAN install:

http://test-repo1.example.com/pub/suse/suse-12-sp4/dvd1
http://test-repo1.example.com/pub/suse/suse-15-sp1/dvd1
http://test2-repo1.example.com/pub/suse/suse-12-sp4/dvd1
http://test2-repo1.example.com/pub/suse/suse-15-sp1/dvd1

Create Mini ISOs

The mini ISOs will be 90MB to 100MB; makes for faster and easier remote installs over the LAN while using the repo servers. Place them in the same directory as the original ISOs:

# mksusecd --create SLE-15-SP1-mini.iso --nano SLE-15-SP1-Installer-DVD-x86_64-GM-DVD1.iso
# mksusecd --create SLE-12-SP4-mini.iso --nano SLE-12-SP4-Server-DVD-x86_64-GM-DVD1.iso

---

The End.

by Ramses Soto-Navarro ramses@sotosystems.com 10/10/2020

---

Overview
Download
Disk Image
Boot Ministick
Manual Startup
Auto Startup
Remount Set Root
Configure SSHD
Remote Login
SSHD Problem

---

Overview

The FreeBSD 10 ministick does not have sshd enabled by default. It must be manually configured. The same goes for the FreeBSD 12.1 ministick (mini memory stick image). mfsBSD already offers it by default. Here is how to enable it on the FreeBSD ministick. There are no permanent settings yet, so it will have to be entered every time - good for disaster recovery practice. More on remastering later.

Download

$ DIR="https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/12.1"
$ wget $DIR/FreeBSD-12.1-RELEASE-amd64-mini-memstick.img.xz
$ xz -d FreeBSD-12.1-RELEASE-amd64-mini-memstick.img.xz
$ ln -s FreeBSD-12.1-RELEASE-amd64-mini-memstick.img mini.img

[Read more…]

Overview

Notes about configuring a wifi wireless network on FreeBSD 12.1, using an old laptop with an unsupported wireless card. Lots of forums said that the card is not supported and that it could not be done. But I refused to believe it, and this is a testimony to the resilience of FreeBSD.

OS: FreeBSD 12.1-RELEASE-p10
Laptop: Dell Inspiron 1525 (circa 2007)
Memory: 4GB
Wireless Card: 802.11g Broadcom BCM4328 SIBA bus BCM4312 rev 15

The history: while running FreeBSD 12.1 everything worked on my old Dell Inspiron 1525 laptop, except my wifi network. Going through blogs I discovered that the wifi card is not supported by generic FreeBSD kernel, so it does not work by default; and requires special tweaking. There’s no official guide for this wireless card. Tried many different recommendations from forum postings. Below is what what worked for me. There may be better ways; but if so then please comment.

[Read more…]