Repository Cleanup Script
Description
The purpose of this script is to be ran against a directory containing versioned files
that have the same base name. In instances where you are dumping packages such as RPMs
or DEB files to a repository server, and want to keep the last 5 or 10 builds, but clean
anything older than that, then this script will do just that. It can be ran against a
directory and given 2 arguments, the first being the file name, the second being the
number of previous versions that you would like to keep.
Usage
THIS SCRIPT REQUIRES 2 ARGUMENTS:
- Arg1 - File Name - The non unique base name of the file being cleaned.
- Arg2 - Keep Revs - The number of instances of the file to keep.
1. Copy the script:
Copy the script and place it either in the directory that is being cleaned or
in a separate directory if it will be used to clean multiple directories.
Lets clean this directory up using the script, feeding it the 2 required arguments.
- Arg1 - File Name - fuse
- Arg2 - Keep Revs - 5
2. Prepare to run the script:
In this example we will look at a directory containing 9 rpms that came from jenkins builds
and were deposited into the proper yum server directory.
-rwxr-xr-x. 1 root root 2.8K May 19 14:18 clean.sh
-rw-r--r--. 1 root root 77K May 19 14:12 fuse-2.9.4-29.x86_64.rpm
-rw-r--r--. 1 root root 77K May 19 14:12 fuse-2.9.4-30.x86_64.rpm
-rw-r--r--. 1 root root 69K May 19 14:12 fuse-2.9.5-1.x86_64.rpm
-rw-r--r--. 1 root root 335K May 19 14:12 fuse-2.9.5-2.x86_64.rpm
-rw-r--r--. 1 root root 335K May 19 14:12 fuse-2.9.5-3.x86_64.rpm
-rw-r--r--. 1 root root 335K May 19 14:12 fuse-2.9.5-4.x86_64.rpm
-rw-r--r--. 1 root root 31K May 19 14:12 fuse-2.9.5-5.x86_64.rpm
-rw-r--r--. 1 root root 31K May 19 14:12 fuse-2.9.5-6.x86_64.rpm
-rw-r--r--. 1 root root 32K May 19 14:12 fuse-2.9.5-7.x86_64.rpm
3. Run the script:
Running this statement should run through the directory and keep the latest 5 copies
of the RPMs in the directory, while removing anything more than the selected 5.
NOTICE:
This script will prompt you to ensure that the files that it has selected to remove are correct. You can avert this by running the script piping the answer into it
i.e echo "y" | ./clean.sh fuse 5
./clean.sh fuse 5
List of files that have been saved:
-----------------------------------
fuse-2.9.5-7.x86_64.rpm
fuse-2.9.5-6.x86_64.rpm
fuse-2.9.5-5.x86_64.rpm
fuse-2.9.5-4.x86_64.rpm
fuse-2.9.5-3.x86_64.rpm
List of files that have been removed:
-------------------------------------
fuse-2.9.5-2.x86_64.rpm
fuse-2.9.5-1.x86_64.rpm
fuse-2.9.4-30.x86_64.rpm
fuse-2.9.4-29.x86_64.rpm
Would you like to delete the files in the delete list? (Y/y|N/n)
y
removed ‘fuse-2.9.5-2.x86_64.rpm’
removed ‘fuse-2.9.5-1.x86_64.rpm’
removed ‘fuse-2.9.4-30.x86_64.rpm’
removed ‘fuse-2.9.4-29.x86_64.rpm’
4. Verify:
List the directory and ensure that the files that were removed were correctly removed.
-rwxr-xr-x. 1 root root 2.8K May 19 14:18 clean.sh
-rw-r--r--. 1 root root 335K May 19 14:12 fuse-2.9.5-3.x86_64.rpm
-rw-r--r--. 1 root root 335K May 19 14:12 fuse-2.9.5-4.x86_64.rpm
-rw-r--r--. 1 root root 31K May 19 14:12 fuse-2.9.5-5.x86_64.rpm
-rw-r--r--. 1 root root 31K May 19 14:12 fuse-2.9.5-6.x86_64.rpm
-rw-r--r--. 1 root root 32K May 19 14:12 fuse-2.9.5-7.x86_64.rpm
Script
NOTICE:
This script can be copied as is. There is no need to modify anything within the script contents.
#!/bin/bash
# Repo Cleanup Script
# Rich Nason
# Copyright 2016 appcontainers.com
# Grab argument from command line and assign to FileName variable
FILENAME=$1
REVS=$2
# If no arguement was supplied, then exit the script with indicator message
if [[ -z $1 ]];then
printf "\n"
printf "\n"
echo "You must supply an argument after the script to specify the rpm name to search for"
echo "The argument will consist only of the package name, not the extention."
echo "A second argument with the number of revisions to keep can also be supplied. (Default: 5)"
printf "\n"
echo "example 1: ./clean.sh nodejs"
echo "example 2: ./clean.sh nodejs 10"
printf "\n"printf "\n"
exit 0
fi
# If no number of revisions is set, auto set to 5
if [[ -z $2 ]];then
REVS=5
fi
# Set the
FILELIST=(`ls -vr $FILENAME*`)
SAVELIST=()
DELETELIST=()
# For each file in the directory with the name specified in the argument Reset the SAVE Bit to 0
# Then run a comparison test against latest 10 version numbers of the build. If there is a match,
# then set the SAVE bit to 1, if there is no match, then the SAVE bit will remain at 0.
# All files with the save bit set to 0 will be deleted.
for x in ${FILELIST[@]}; do
SAVE=0;
#for y in `ls -vr eros_ads_content* | head`; do
#for y in `ls -vr eros_ads_content* | head`; do
# Head will only list the first 5 entries of the LS operation, If one of those files, matches a file
# in the complete file list from the parent for loop, then the SAVE bit is set, which will save the file.
for y in `ls -vr $FILENAME* | head -$REVS`; do
if [[ $y == $x ]]; then
SAVE=1;
SAVELIST=(${SAVELIST[@]} $y)
fi
done
# If the SAVE bit was not marked, via a comparison match meaning it is not one of first 10 files,
# then it will be removed.
if [[ $SAVE == 0 ]]; then
DELETELIST=(${DELETELIST[@]} $x)
#echo $x
fi
done
# Print Report to show which files are to be deleted, and which are to be saved
echo "List of files that have been saved:"
echo "-----------------------------------"
#echo ${SAVELIST[@]}
for f in ${SAVELIST[@]};do
echo $f;
done
printf "\n"
printf "\n"
echo "List of files that have been removed:"
echo "-------------------------------------"
#echo ${DELETELIST[@]}
for f in ${DELETELIST[@]};do
echo $f;
done
printf "\n"
printf "\n"
# Ask if you would like the files deleted, and if y then delete the files
echo -e "Would you like to delete the files in the delete list? (Y/y|N/n)"
read CONFIRM
if [ "$CONFIRM" = "y" ] || [ "$CONFIRM" = "Y" ]; then
for f in ${DELETELIST[@]};do
rm -frv $f;
done
fi
printf "\n"
printf "\n"
Great article