BASH Hacks: The Comma Delimited List

BASH is by no means a complete language, and for all but a very few things it's not the right one to go with. But for scripts that make intense use of external programs it's really the best, and so this will be the first article in a series of many that deals with simple BASH hacks to make your scripting life easier.

One of the biggest problems with BASH is that the data structures are fairly primative, and so writing a script that expands with your infrastructure is often times very hard. You end up having to do massive hacks to the script every time you add a new server, or customize the script per server etc. I've developed a bit of code that I use in any script that deals with potential areas for growth using a comma delimited list to perform a set of actions on each member of that list. This makes the code much more portable and transparent, and as your infrastructure grows or shrinks all you have to do is edit a variable definition at the beginning of one file, and the script is ready to go again. Here's the basic template:


#!/bin/bash
#################################
## commadelimited.sh
##
## A template for implementing a comma
## delimited list in BASH
#################################

######VARIABLE DEFINITIONS#######
LIST=1,2,3,4,5
#################################

# Add a trailing comma to the list variable
LOOPVAR=${LIST},

# Loop as long as there is a comma in the variable
while echo $LOOPVAR | grep \, &> /dev/null
do

# Grab one item out of the list
LOOPTEMP=${LOOPVAR%%\,*}

# Remove the item we just grabbed from the list,
# as well as the trailing comma
LOOPVAR=${LOOPVAR#*\,}

# some action with your variable
#
# echo $LOOPTEMP
#
# for example

done

exit 0


So what exactly does this do . . . well, here we go in detail. After defining the comma delimited list in a variable:


LIST=1,2,3,4,5


We have to add a comma to the end of this list, because we grep for a comma in the loop, and the loop would therefore terminate before processing the last item in the list, as there would no longer be a comma, because we trim the trailing comma from the list with the item itself:


LOOPVAR=${LIST},


Now we're ready to rock. If you havn't had the pleasure of using an if grep or while grep statement, then I would suggest you start to use it, this is one of the more usefull features of BASH:


while echo $LOOPVAR | grep \, &> /dev/null
do



This will loop for as long as the loopvar variable contains a comma in it.


LOOPTEMP=${LOOPVAR%%\,*}


This grabs the first item from what's left of the LOOPVAR list. The way this works is the % in the ${} indicates that we want to trim the string from the end. The fact that there are two of them (%%) tells it to trim greedy, and match as much as possible. The regular expression that follows the %% says to chop from the furthest comma to the right, followed by anything.


LOOPVAR=${LOOPVAR#*\,}


This statement is similar to the last one, but this one takes out the first value and the trailing comma and shortens the list by one. The # tells it to trim the string from the beginning, and only one # tells it to be non-greedy and grab everything up through the first comma from the left, as defined by the regular expression that follows.

So as we loop, the list gets shorter by one item per iteration, and eventually cycles down to an empty string, where the grep \, fails and the loop breaks.

The following script is an application of the comma delimited list idea that I use in my everyday work:


#!/bin/bash
################################
## sendadminmail.sh
##
## This script is called by monitoring scripts
## to send mail to all of the administrators
## of our company should something go
## wrong. Which given my skills, should
## never happen.
################################

#######VARIABLE DEFINITIONS########
#
# Comma Delimited List of Administrative E-mail addresses
ADMINLIST=test@test.com,test@test2.com,test@test3.com
#
###################################

# initialize the loop variable, the trailing comma is
# so that the last entry will be honored
LOOPVAR=${ADMINLIST},

# this loop loops through each individual entry on the list
while echo ${LOOPVAR} | grep \, &> /dev/null
do

# Grab the first e-mail address out of the comma-delimited list
LOOPTEMP=${LOOPVAR%%\,*}

# Remove the first e-mail address from the
# LOOPVAR comma-delimited list
LOOPVAR=${LOOPVAR#*\,}

# send the e-mail
echo "$1" | mail -s "$2" "$LOOPTEMP"

done

exit 0