A Shell Tutorial: Loops & Subs

A Few Shell Script Examples

The shell, on a higher level, can be thought of as both a job control language and a programming language. Strictly, it is neither one, but it can perform the functions of both. It can express very articulate and sophisticated coding instructions and it can also "bark" out orders for the OS to follow. Most users have only seen the "barking" side...

Program Envelope Basics: Documentary Descriptions

Every shell script should be treated as a program. It should be thought of as an entity, an object unto itself. Something that may have a life span that might "out live" you! So it needs to contain an awareness of its own history, its own development. This translates into documentation! Take a look at the example below:

Sample Shell Script Template
#! /bin/sh # script name # date # your name or ID # ------------------------------------------------------------ # Comments... # # ------------------------------------------------------------ # Location.: # Inputs...: # Operation: # Calls....: # Output...: # Called-by: ... code ... #----- End of script name

Of course, you may not need to include all the items listed above. But at the very least, you need the first and last lines, and a descriptive comment block is highly recommended!

Below is an excerpt of a shell script that follows the guidelines listed above.

Sample Shell Script Documentation Header

#! /bin/sh # junkman # 02-26-03 # 03-30-03 # 04-01-03, 04-04-03, 05-06-03 # KA1FSB/kbn # # ---------------------------------------------------------------- # This script opens each file and reads the Julian date; it then # compares that to today's Julian date. If a certain day difference # exists, then delete the file, or age it off the system. # ---------------------------------------------------------------- # Location.: /root/mail # Input....: /root/mail/junkfile # Operation: deletes aged files in /root/mail/junkdir, tallies junkdom # Calls....: # Output...: /root/mail/junkdom # Called-by: /root/do_mheads ... some code here... #----- End of junkman

TASK : To document a script program by using a standarized template.

This preamble gives a quick overview of the script, highlighting the main features and functions along with key files that should help to jog your memory when going back to fix or enhance the code. Programmers are notoriously negligent when it comes to documentation. But, it really pays off in the long run! You simply cannot over-document! (I frequently believe that my code has enough documenting, only to wish that I had included more, much more!)

And, of course, the degree of detail could be even more descriptive. For example, the dates could be associated with some coding change, or the observation or fix of some bug. A good practice is to go back over your comment block and fill in the "missing pieces" as often as you have time for it.

Program Building Blocks

A program can be conceived of as a collection of subroutine blocks, or functions, where each sub-process is invoked by a call to one of these functions. And this, in turn, implies that there is a main or central process from which these calls can be made. Even the simplest of scripts can benefit from breaking the tasks into modular units. Here is a very simple example...

Subroutine or Function Code Fragment

... #----- Variable inits ERRORS="/root/ax25.errors" do_netrom () { #----- Netrom subroutine set ups /bin/echo " -> Setting up NET/ROM configs..." | tee -a $ERRORS #----- Order of ops is critical... /usr/sbin/nrattach nr0 | tee -s $ERRORS /sbin/ifconfig nr0 44.56.26.10 netmask 255.0.0.0 broadcast 44.255.255.255 #----- Start it up in the background... /usr/sbin/netromd -i -l -t 20 -d & sleep 1 } # #----- Main body of script... # #----- And now a call to do_netrom do_netrom ...

TASK : To start up the netrom server, netromd, and to configure the nr0 interface.

Here is the routine, do_netrom, that is called from the main body of the script. This routine doesn't do much processing on its own other than to call system functions or programs like: echo, nrattach, ifconfig, netromd, tee, and sleep. In fact the shell is the consumate "designator," rarely doing anything on its own without the help of the OS binary calls! So this fits neatly into the category of "barking out" orders or command lines, a very simple routine, many of which could also be entered manually at the user prompt.

Then why bother with creating a stand-alone routine? My answer is that sometimes you may not always want to run this, or perhaps you need to only for a special test. You may easily place a hash mark (#) in front of the call and that routine will not be run. Plus, this group of code is easily copied and inserted into another script. It is clearly identifiable as a subroutine or function. It has a name, followed by a space and parentheses, and two curly braces enclosing the code lines. But, the best reason is that it may be developed into a more versatile module that could handle a broader class of "requests" as we will see soon...

Let's look at a few other items while we are here. The word or term ERRORS appears at or near the top of the script. This is a variable, meaning it can be assigned a value using the equal (=) character. There can be no spaces to the left of the assignment character. There may be to the right if the phrase or string is quoted. To use the variable in the script, it is referred to as $ERRORS or as ${ERRORS}, using curly braces. The latter form is often used when the variable is concatenated into a long string, such as "We need to watch this error-"${ERRORS}.

An interesting symbol, and one you certainly know about, is the pipe (|) symbol. This effectively joins two programs together, their inputs and outputs, that is. In the echo statement, the effect is to print the message to the screen as echo usually does, but also to feed that message to the program tee which then appends it to the $ERRORS file, or the path and file name stored in that variable.

Another opportunity afforded by "functionalizing" your instruction sequences is that you may include extra documentation in the routine description lines near the top of the subroutine body. I have even listed page numbers from books or website addresses that I wanted to recall! These "working notes" can prove to be invaluable to you or some "following" programmer...

Repetitions with Loops

One activity that scripts and programs do very well is repetition, and the shell is no exception! In fact, this is where we can put the shell to work, which has a looping structure of for/do/done. Let's take a look at a typical "for loop":

Looping Subroutine Fragment

... #----- Variable inits... MODNAME="" OBJPATH="" MYLOG="/root/ax25.errors" #----- Get Linux version LINUX_VER=`uname -r` #----- Path of the modules to load... MOD_PATH="/lib/modules/${LINUX_VER}" #----- This is the list of modules that need to be loaded #----- for a full radio configuration... MODLIST="net/bsd_comp net/hdlcdrv net/baycom" ... #===== Load the module drivers... echo " -> Loading required modules..." | tee -a $MYLOG #----- Loop 'til you droop... for i in $MODLIST do MODNAME=`echo $i | cut -f2 -d"/"` echo -n "$MODNAME " | tee -a $MYLOG OBJPATH=${MOD_PATH}"/"${i}".o" insmod $OBJPATH sleep 1 done echo "" | tee -a $MYLOG ...

TASK : To load a sequence of modules, in this case, for a baycom driver.

Let's walk through the code beginning with the variable initializations. It's probably "old hat" until the LINUX_VER line. Note the back quotes that surround the command, uname -r. These special quotes tell the script to run this program with an argument of -r. The result is that the return value, 2.0.30 on my machine, is stored in the LINUX_VER variable, which in turn, is appended to the MOD_PATH string so we know where to find these objects.

The next key variable is the MODLIST, which contains a listing of the modules we want to load. Each item is separated by a space. There must be a separator or else the "for loop" will not know how to manage the list. You might also notice that there is a directory name, net, just before the module name. This allows just a bit more flexibility when specifying the paths to these modules.

OK, it's time for the loop. The "for i in MODLIST" will left-shift the first name in the list, net/bsd_comp, into the variable i, thereafter referred to as $i. It will send this variable through the loop, and thereafter each one in turn, until the MODLIST appears "empty." Then the loop will stop iterating and move on to the next sequence of code instructions outside the do/done block. It is critical that a loop know when to stop, otherwise it will keep on looping "forever," not a good situation... This one stops when it gets to the end of the list, MODLIST.

Now let's examine the body of the loop, line-by-line. The first line

• MODNAME=`echo $i | cut -f2 -d"/"`

uses back quotes to excute an echo command. This pushes the variable in $i into a pipe, not displaying it here, which then uses cut to isolate the second field, -f2, as defined by a separator "/" character, -d"/". We have extracted what is just beyond the "/" character from the entire string, net/bsd_comp, or the name of the module, bsd_comp. That name alone now resides in MODNAME.

The second line

• echo -n "$MODNAME " | tee -a $MYLOG

is for display and logging. It echoes the name to the screen with no new line, the -n, and also feeds and appends it to a logging file, the one stored in $MYLOG.

The third line

• OBJPATH=${MOD_PATH}"/"${i}".o"

builds the complete path to the module object, ending in a ".o". Notice that all the elements of the string are adjacent to each other with no intervening spaces, and notice the curly braces surrounding the variables. This is how the shell joins or concatenates items, whether they are literals, such as the "/" and ".o", or variables, such as $MOD_PATH.

And the fourth line

• insmod $OBJPATH

actually does the work of inserting the module in the mods list, since the variable $OBJPATH contains all the information that is required. In the last line, the "older" machines may need to take a rest :) So we sleep 1. And that completes the loop!

If you would like to see the context from which this code sample was extracted, please see the do_ax25 script which also contains extensive application of argument receiving subroutines.

Making Subroutines More Useful

So far, we have seen the basic structure of subroutines and how to call them. But we can extend their usefulness by passing arguments, data, into the routine to change their behavior or change what they will be working on. In this way, one routine can perform many "variations" on a task. Lets look at our first do_netrom routine with a slight change in coding...

Subroutine Being Passed Arguments

... #----- Variable inits, still global, i.e., "seen" by all ERRORS="/root/ax25.errors" do_netrom () { #----- Define and clear some local variables, not "seen" outside this function local IFACE="" local STARTNETROM="" #----- Place the first argument item into this variable... IFACE="$1" #----- Left-shift the argument string, getting the next data item into $1 shift #----- Place the next agrument item into this variable... STARTNETROM="$1" #----- Netrom subroutine set ups /bin/echo " -> Setting up NET/ROM configs for $IFACE... | tee -a $ERRORS #----- Order of ops is critical... /usr/sbin/nrattach $IFACE | tee -s $ERRORS /sbin/ifconfig $IFACE 44.56.26.10 netmask 255.0.0.0 broadcast 44.255.255.255 #----- Start it up in the background if flag is true or is not null ... if [ "$STARTNETROM" ] then /usr/sbin/netromd -i -l -t 20 -d & fi sleep 1 } # #----- Main body of script... # #----- Initialize arguments; These will not interfere with those in subroutine IFACE="nr0" STARTNETROM="" #----- Make the call and pass in data which is separated by one space do_netrom $IFACE $STARTNETROM #----- New data, next interface nr1 and a flag STARTNETROM to run the daemon IFACE="nr1" STARTNETROM=1 #----- Call this routine again with different data, last call so start daemon do_netrom $IFACE $STARTNETROM ...

TASK : To call the subroutine twice, first setting up the interface for the nr0 device and then for the nr1 device. On the last call, start up the netromd daemon since all the devices have been attached. So the task is not only to execute system commands, but to also make the shell do something above and beyond those commands, to be "smart" enough to handle more than one device.

Admittedly, this is a somewhat unusual use of this routine since most operators will probably be using only one netrom device. But, the point is to demonstrate how a subroutine may accept and process related arguments and also how it can start up a selected process only after all the interfaces have been configured and only if the process flag has been set.

As is evident, there is alot more new material in this code fragment than in its predecessor! But here is where we are really making the shell script do something useful. This code is beginning to approach a higher level of abstraction, internal re-useablility, since it can accept a broader "class" of arguments and knows how to handle them. Note, the first version was very hard-coded, only being able to handle nr0, but this version can handle as many netrom interfaces as we can "legally" configure...

Let's dive down into a few of the pertinent details. Perhaps the word "local" would seem to stand out as a bit unusual, but it tells the script that the variable that follows is not global, is not seen outside of this routine. Thus, you could have a variable of the same name in the main body of the script which would have an independent existence. These variables could have competely different values. The $IFACE in the routine and the $IFACE in the main body are treated as separate "objects."

The next difference is the appearance of the variable $1. This stands for the value in the first position after the subroutine name. The $2 would stand for what is in the second position. However, here we are using the word shift to left shift each value in turn into $1, very much like what the for loop does when it shifts its data list into its "i" variable.

Then in the set up and configuration command lines, we have substitued the variable $IFACE for a hard-coded device name, affording adaptibility. Finally, we see a new logic structure the if/then/else/fi. This tests a condition and if true, does what ever is in the "then" block. Otherwise, it will do what is in the "else" block. In this case, there is no other choice.

In the main body, we call the subroutine with two arguments each separated by a space. Therefore we must assign values to these variables. Then, when we call the routine again with these same variables, we must be sure to have assigned them the new values before we pass them into the routine. Note, too, the variable names are the same as in the routine, but these are global, meaning they can be referenced from anywhere. The ones in the routine are local, meaning their values will be unique to the confines of the routine in which they were declared local.

Can we improve this main body coding to make it even more well structured, making a clear distinction between data and process and then joining them in a logic structure? Yes, if we assume that we might want to set up all four netrom interfaces: nr0, nr1, nr2, and nr3. Although this is unikely, it is not unthinkable on a large system. We could define a data variable like this:

• NETNODES="nr0: nr1: nr2: nr3:1"

The first field is the name of the device and the second, separated by a colon, is the flag which begins the daemon used here only at the end. All the flags are null except the last one.

We can use a loop structure as we did in our module loading above.

Revised Main Body
... #----- Set up the data area NETNODES="nr0: nr1: nr2: nr3:1" #----- A Loop which calls a subroutine... for i in $NETNODES do IFACE="" IFACE=`echo $i | cut -f1 -d":"` STARTNETROM="" STARTNETROM=`echo $i | cut -f2 -d":"` do_netrom $IFACE $STARTNETROM sleep 1 done ...

TASK : To call a subroutine for as many times as we have device names in the $NETNODES list, passing in each name in turn until done. The last argument contains a special flag to trigger an event in the subroutine. A clear side-effect of this re-structuring is to "objectify" the code by function and "data type," or as much as we can possibly hope to do this in the shell environment...

And as noted, we have used cut extensively to tease apart the values we needed to assign to the variables. The IFACE variable is in the first field and the STARTNETROM is in the second field. The separator is the colon (:). Each variable is initialized to null, or cleared, before any given assignment, always a good idea when writing scripts or programs.

It should be noted that this is just one way to solve this problem. As you grow in your script writing, you will evolve a style and an approach that will change over time, and probably appear to you to be getting better with experience. Programming is both an art and a science, depending on who you talk to...

Passing Arguments to a Program

In the same way that you can pass arguments to a subroutine, so too you can pass arguments to a script/program from the OS prompt. Let's say you called your netrom startup script, do_nrparms. This script might contain the code listed here that steps through a sequence of netrom device attaches. You might want to run this from another master script or from the prompt with a varying number of devices. Here is what that would look like:

• do_nrparms nr0: nr1: nr2:1

if we follow our data structure convention from our former subroutine example. The script can be set up to read the data from this command line. A convenient way to do this is to use a line like this near the top of the main script body:

• NETNODES=$@

Everything past the name of the script and its trailing space will be read or loaded into this $NETNODES variable. This is a handy way to "capture" an entire line and store it in a variable, most likely a gobal variable. (This same technique can be used for passing arguments into a subroutine as well.)

If you need to know how many data items were passed in, then use this assignment near the top of the script main body:

• NUMNODES=$#

Sometimes, you need a minimum number of arguments passed in and this technique can tell you if it meets the test. If the number is OK, then you proceed, if not then you issue a warning:

Command Line Argument Check
... #----- Assign a script variable with a count value NUMNODES=$# #----- Now test and branch accordingly... if [ $NUMNODES -lt 3 ] then echo "Usage: Must enter at least three devices" exit else #----- If OK, then could run the remainder of the main script body here echo "Command Line Arguments OK" fi ...

TASK : To test the command line for the "correct" number of arguments being passed in. If not correct, then issue a usage instruction or warning.

There is yet another way to parse arguments passed into a shell script. The command line interpreter can identify up to nine (9) arguments by number. These are $1 through $9. And "What is $0," you may ask? Well that is the name of the program or script. This is a very handy feature if you like to write code that can identify itself. For example:

Passing and Identifying Arguments by Position
ka1fsb:~# do_test one two three ... #----- Now capture name of script PROGNM=$0 #----- Store data ARGONE=$1 ARGTWO=$2 ARGTHREE=$3 echo "<${PROGNM}>: Argument one is $ARGONE" echo "<${PROGNM}>: Argument two is $ARGTWO" echo "<${PROGNM}>: Argument three is $ARGTHREE" ...

TASK : To pass in three arguments, or pieces of data, from the OS prompt, that will be identified by position, and also to capture the name of the script itself and use it in the program.

There may be cases when you know you will always be passing a fixed number of arguments, so it may be simpler to use this method. Again the same technique may be applied when passing data into a subroutine. And the script here is prefacing every echo with its own name so that you know where this processing is occurring, sometimes helpful when troubleshooting code as in cases where one script calls another...

Conclusions

I hope this brief article has expanded your view of the shell and encouraged you to try writing some programs in it. The shell can not only prove very useful at the standard tasks of loading, installing, and configuring, but it can venture into the field of prototyping new or original code which you can put to work on your system.