APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Auto Edge archive removal script

By Bruce Garlock

Ever since Microlite released BackupEdge 3.x, the way the purging of older backups changed drastically with D2D backups.  We use an HP "Storage Server NAS" (DL-380) with 9TB of space, and it's maxed out. We have six different Linux servers running Edge 3.x, and a handful of Windows servers being backed up to them via FTP.

The way I wanted Edge to work, was just like the days when we backed up to Tape.  We would pull Master backups each night, label the tapes M-F, and take a SAT tape off-site for an extra layer of protection. When Monday rolled around, the new backup would replace the old Monday backup, rinse, lather, repeat.  This changed when we upgraded to Edge 3.x, and started using the NAS.  Once we had multiple servers with multiple jobs backing up to the same resource, and not having enough free space on the NAS because backups were piling up, I had to resort to manually deleting things (through edgemenu) on the NAS to free up space.  This got old rather quickly, as I had to look down the long list of backup jobs, and determine which ones could be deleted.  If I missed one day, the NAS would fill up, and backups would fail.

Edge uses "Lazy Reclaimation" to remove older backups.  It's TTL must expire, and depending on the quota, edge would remove the backup and make room for the new one.  The theory was to keep as many old backups as possible, before removing a backup.  I get that.  Great idea in fact, but we are running out of space, and it's hard to set a quota of the resource when so many different servers are baking up to the same resource, using multiple different types of SW.

I sent an email to support, to find out the best way to remove older archives, via the command line, so I could script something up and stick it in cron to be run before the nightly backup fires up.  I wanted things to mimic the ways tapes work, so I scripted it to remove all backups prior to todays date - six, prior to the nightly backup starting.  That in effect, gave me the same way things would work with tapes.

I took an old ksh script Tony had written for me years ago (like 97/98, I think) that helped me organize log files.  See: Yesterday script

Here's my script that works great (I use two NAS devices as resources.  One in our main building, and one in another building connected via Fiber)  My next project is to get this all backed up online, but backing 9TB online isn't exactly trivial or cheap.


[[email protected] ~]# cat /usr/local/bin/edgeclean.sh
#!/bin/ksh
#  B. Garlock - Edge program to remove backups automatically.  Edge
needs things in the format of:
#
# Orginal script with help from A.P. Lawrence back in 1997/98.
#
#  See:  http://aplawrence.com/Unix/yesterday.html
#
#  /usr/lib/edge/bin/edge.segadm -f ftp-360 -f
-zDATEBEFORE="$MONTH/$LASTWEEK/$YEAR"
#  The Day of the month *MUST* have a leading ZERO, if the date is < 10
#
#
#
#

RESOURCE164="ftp-164"
RESOURCE360="ftp-360"
set -A DAYS Sat Sun Mon Tue Wed Thu Fri Sat
set -A MONTHS Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

YESTERDAY=$((`date +%d` -1))
LASTWEEK=$((`date +%d` -6))
MONTH=`date +%m`
YEAR=`date +%Y`
NDAY=`date +%u`
WEEKDAY=${DAYS[`date +%u`]}

if [ $YESTERDAY -eq "0" ]
then
       MONTH=$((MONTH-1))
       if [ $MONTH -eq "0" ]
       then
               MONTH=12
               YEAR=$((YEAR-1))
       fi
set `cal $MONTH $YEAR`
shift $(($# - 1))
YESTERDAY=$1
fi

# Add a ZERO in front of the Day, if there isn't one.  Edge needs this.
if [ ${#LASTWEEK} -eq 1 ]; then
  LASTWEEK="0$LASTWEEK"
fi

# Add a ZERO in front of the Day, if there isn't one.  Edge needs this.
if [ ${#YESTERDAY} -eq 1 ]; then
  YESTERDAY="0$YESTERDAY"
fi


TMONTH=${MONTHS[MONTH]}
# uncomment next line for debugging
echo $WEEKDAY $YESTERDAY $MONTH $TMONTH $YEAR

echo "Yesterday:  $MONTH/$YESTERDAY/$YEAR"
echo "Last Week:  $MONTH/$LASTWEEK/$YEAR"

# Run edge.segadm to remove older backups.  This should be done PRIOR
to the backup, so there is enough space for the new
# one.  We decrement the day by one, so that the previous weeks backup
is removed properly.

/usr/lib/edge/bin/edge.segadm -f $RESOURCE360
-zDATEBEFORE="$MONTH/$LASTWEEK/$YEAR" -d

/usr/lib/edge/bin/edge.segadm -f $RESOURCE164
-zDATEBEFORE="$MONTH/$LASTWEEK/$YEAR" -d

Yea, it's ugly as hell, but I think it takes most things to do with dates and handles about every situation.  I'd love to hear if this needs some improving, or if anyone has any ideas on how to make it bullet proof.



Got something to add? Send me email.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Auto Edge archive removal script




Increase ad revenue 50-250% with Ezoic


More Articles by © Bruce Garlock



Kerio Samepage


Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us





Technology is both a tool for helping humans and for destroying them. This is the paradox of our times which we're compelled to face (Frank Herbert).

There are only two hard problems in computer science: cache invalidation and naming things. (Phil Karlton)












This post tagged: