FreeBSD Full / Incremental Tape Backup Shell Script

This gave me a the motivation to finally get around to writing a backup script for my server. It’s basic framework is a combination of this script and another one I found on this site. It’s optimized for a freebsd/zfs system with a local tapedrive…it’ll even automatically eject tapes and resume on a new tape load. Hope it helps someone.

Main Part:

#!/bin/bash
## /root/dobackup.sh
## last update 09162009 by ahurt
#

# datasets to backup - use zfs paths not mount points
BACKUP_SETS="pool0/filebase pool0/vmware pool0/backuppc"

# number of snapshots to keep of each dataset
# snaps in excess of this number will be expired
# oldest snaps deleted first...this must be non zero
SNAP_KEEP="8"

# where you want your log files
LOGBASE=/root/logs

# where your tape drive is located
TAPE="/dev/nsa0"

# the length of the tape (in KBs)
TAPE_LENGTH="61440"

# tape swap script
TAPE_SWAP="/root/tapeswap.sh"

# the tape blocksize in bytes
BLOCKSIZE="32768"

# the tar blocksize in 512byte sectors
TAR_BLOCKS="64"

# any additional tar arguments
TAR_ARGS=""

# path to binaries
TAR=/usr/local/bin/gtar
MT=/usr/bin/mt
MKDIR=/bin/mkdir

# get the current date info
DOW=$(date +"%a")
MOY=$(date "+%m")
DOM=$(date "+%d")
YR=$(date "+%Y")

# backup log file
LOGFILE="${LOGBASE}/${DOW}-backup.log"

############################################
##### warning gremlins live below here #####
############################################

## init our backup paths
BACKUP_PATHS=""

## main backup function
full_backup(){
        ## do our snapshots..we don't want to tar a live fs
        zfs_snap
        ## initiate our tape
        $MT -f $TAPE comp on
        $MT -f $TAPE blocksize $BLOCKSIZE
        ## build our tar command
        if [ "${TAR_ARGS}x" != 'x' ]; then
                local tarcmd="$TAR $TAR_ARGS -F $TAPE_SWAP -H pax "
        else
                local tarcmd="$TAR -F $TAPE_SWAP -H pax "
        fi
        ## build the rest and run it
        tarcmd+="-L $TAPE_LENGTH -b $TAR_BLOCKS --transform=${XFROM} "
        tarcmd+="--show-transformed-names -clpMSvf $TAPE $BACKUP_PATHS"
        echo "RUNNING: $tarcmd"
        $tarcmd
        ## check the tar return
        if [ $? -eq 0 ]; then
                echo "FINISHED: Rewinding and ejecting tape"
                $MT -f $TAPE rewind
                $MT -f $TAPE offline
        else
                echo "ERROR: $TAR exited abnormally please check logs"
                exit 1
        fi
}

## partial/incremental fucntion
partial_backup(){
        ## we are doing an incremental...set our tar args
        ## only backup files newer than yesterday
        if [ "${TAR_ARGS}x" != 'x' ]; then
                TAR_ARGS+=" -N yesterday"
        else
                TAR_ARGS="-N yesterday"
        fi
        ## call our main backup function
        full_backup
}

## create our zfs snapshots
zfs_snap(){
        ## set our snap name
        local sname="${DOW}-${MOY}${DOM}${YR}"
        ## remove snapshot paths from filenames...this is passed to tar
        ## with --tramsform in the backup function above
        XFROM="s/\\/\\.zfs\\/snapshot\\/${sname}//g"
        ## generate snapshot list and cleanup old snapshots
        for dset in $BACKUP_SETS; do
                ## get current existing snapshots that look like
                ## they were made by this script
                local temps=`zfs list|grep -e "${dset}\@[Mon|Tue|Wed|Thu]"|\
                awk '{print $1}'`
                ## just a counter var
                local index=0
                ## our snapshot array
                declare -a snaps
                ## to the loop...
                for sn in $temps; do
                        ## while we are here...check for our current snap name
                        if [ $sn == $sname ]; then
                                ## looks like it's here...we better kill it
                                ## this shouldn't happen normally
                                echo "Destroying OLD snapshot ${dset}@${sname}"
                                zfs destroy ${dset}@${sname}
                        else
                                ## append this snap to an array
                                snaps[$index]=$sn
                                ## increase our index counter
                                let "index += 1"
                        fi
                done
                ## set our snap count and reset/reuse our index
                local scount=${#snaps[@]}; index=0
                ## how many snapshots did we end up with..
                if [ $scount -ge $SNAP_KEEP ]; then
                        ## oops...too many snapshots laying around
                        ## we need to destroy some of these
                        while [ $scount -ge $SNAP_KEEP ]; do
                                ## zfs list always shows newest last
                                ## we can use that to our advantage
                                echo "Destroying OLD snapshot ${snaps[$index]}"
                                zfs destroy ${snaps[$scount]}
                                ## decrease scount and increase index
                                let "scount -= 1"; let "index += 1"
                        done
                fi
                ## come on already...make that snapshot
                echo "Creating ZFS snapshot ${dset}@${sname}"
                zfs snapshot ${dset}@${sname}
                ## build our backup paths/snapshot paths
                local mnt=`zfs get mountpoint ${dset}|tail -1|awk '{print $3}'`
                ## add this to our global var
                if [ "${BACKUP_PATHS}x" != 'x' ]; then
                        BACKUP_PATHS+=" ${mnt}/.zfs/snapshot/${sname}"
                else
                        BACKUP_PATHS="${mnt}/.zfs/snapshot/${sname}"
                fi
        done
}

## make sure our log dir exits
[ ! -d $LOGBASE ] && $MKDIR -p $LOGBASE

## this is where it all starts - do either a full
## or partial depening on day
## office is closed friday-sunday
case $DOW in
        Mon)    full_backup;;
        Tue|Wed|Thu)    partial_backup;;
        *) ;;
esac > $LOGFILE 2>&1
[/code]

Second Part: (this is the tar info-script)
[code]
#! /bin/sh
##
### tar info script to automate tape changes
### below should match settings in backup script
##

# where your tape drive is located
TAPE="/dev/nsa0"

# path to binaries
MT=/usr/bin/mt

## script loop below ##

echo "Preparing volume $TAR_VOLUME of $TAR_ARCHIVE...."
sleep 15s
echo "Rewinding tape $TAPE ..."
$MT -f $TAPE rewind
sleep 5s
echo "Ejecting tape $TAPE ..."
$MT -f $TAPE offline
sleep 5s
echo "Waiting for next tape..."

while true; do
        ## wait 15 seconds between checks
        sleep 15s
        ## issue a status command and squelch output
        ## this will return an error if there is no tape
        $MT -f $TAPE status >/dev/null 2>&1
        ## check if status returned error or good
        if [ $? -eq 0 ];
        then
                ## if we are in here status worked
                echo "New tape detected...preparing..."
                ## make sure take is ready...and let's quick erase it
                $MT -f $TAPE rewind
                $MT -f $TAPE erase 0
                ## that's it...
                echo "Tape ready...resuming operation..."
                ## exit clean and tar will continue
                exit 0
        fi
done

Not sure if anyone else could use it...but I hope it helps someone get a start anyways.

Reply

I’ve updated the above to include lockfile management, enabling setting snapshot only days, tape full days and, tape incremental days or just to skip certain days. You can see the full script here: http://woodstock.anbcs.com/shell/dobackup.sh

Thanks again Vivek for giving me the motivation to get that setup…it’s been a real lifesaver for us here.

– Aaron

Reply