Let's understand a couple of things about Linux. Linux doesn't care about extensions, it's not interested in what the extension of this particular file is. My advice however, is for your reference (i.e. for the sake of readability), to append an .sh on the end of every shell script file. That will immediately alert you to the fact that this file is a script without having to perform any operation on the file.^[13]

Of course if you don't do that, it doesn't make any difference, it will still be a script. But my convention, (there are as many conventions as there are system administrators and Linux distributions) encourages the .sh on the end of a script file name. This tells me in no certain terms that this is meant to be a script.

At this point we should be able to run that script. So type the following on the command line:

script.sh 
                

When you do that, you will notice the following error:

script.sh: Command not found. 
                

The reason that the command is not found is that it looks in your search PATH for this "new" command.

Of course your PATH hopefully (if you're a halfway decent system administrator) doesn't have a '.' in it. In other words your PATH doesn't include your current directory.

In order to run this script you need to precede the script by the PATH to the script. Thus:

./script.sh
                

When you do this, it still won't run! Why? You haven't changed the script to be executable. The way that you do this is:

chmod +x script.sh
                

From thereon, the script is interpreted as an executable file and you can rerun that script by using the following command:

./script.sh 
                

You have to make every script executable with the chmod command. If you don't change its mode, it won't run. Every time you run it, it will show you a list of unique shells that are being used by your system.

You could give other users access to this script, or you could place this script in relevant home directories so that it could be executed.

Or you could put it into a place on the system that everybody has access to (e.g. /usr/bin).^[14]

It's important, now that you're learning to write scripts (which will ultimately take you on to writing programs and ultimately to becoming a fully-fledged open source developer), that you document your scripts well.

Since we're all such good programmers we will definitely want to do this. How? We can put comments in our scripts using a hash (#) to show that a particular line is a comment.

Edit script.sh as follows:

vi script.sh
                

Insert a hash or two at the top of this file and write a comment about what this script does, who wrote it, when you wrote it and when it was last updated.

# Student name - written February 2004.
# A script to produce the unique shells using the /etc/passwd file

cut -d: -f7 /etc/passwd |sort -u 






: w script.sh

                

This is the bare minimum comment you should make in a script. Because even if you don't maintain your scripts, there's a good chance that somebody in the future will have to; and comments go a long way to proving that you're a capable coder.

	Note
	It's a vital part of open source - to provide documentation. Comments can appear anywhere in a file, even after a command, to provide further information about what that particular command does.

Variables are a way of storing information temporarily. For example I may create a variable called NAME and I assign it a value of "Hamish":^[15]

NAME="Hamish"
                

A couple of conventions that we need to follow: variables usually appear in uppercase, for example I have assigned to a variable called 'NAME' the value 'Hamish'. My variable name is in uppercase. There is no white space between the variable name ('NAME') and the equals sign.

Similarly, without any white space enclose the value in double quotes. This process allocates space (memory) within the shell calling the reserved memory 'NAME', and allocates the value 'Hamish' to it.

How do we use variables?

In this case, we will use the echo command to print the output to the screen.

echo "Hello $NAME"
                

which would print:

Hello Hamish
                

to the screen. We could create a file with:

touch $NAME
                

This would create a file called 'Hamish', or else type:

rm $NAME
                

which would remove a file called 'Hamish'. Similarly, we could say:

vi $NAME
                

which would open the file 'Hamish' for editing. In general, we assign a variable with:

NAME=value
                

And we can use the variable in a variety of ways.

Does the variable have to be a single string? No, we could've assigned a variable with:

HELLO="Hello World"
                

Please set this variable from the command line and then test the following :

touch $HELLO
                

List your directory to see what it has produced.

Remove the file using the variable name:

rm $HELLO
                

What happens? Why?

So setting a variable is a case of assigning it using an equals sign.

Using a variable is achieved by preceding the variable name with a dollar sign.

As I indicated, the convention is to keep the variable name uppercase, however we don't necessarily need to adhere to it. My advice is to stick with the convention and keep them uppercase.

So far we've written very simple scripts. Our scripts have entailed simply an echo statement and maybe one other command. In order to achieve a higher degree of complexity, we need to tell the script what shell it's going to run under.

One might find that a little strange because we're already running a shell, so why do we need to tell the script what shell to run as? Well perhaps, even though we're running the bash as our default shell, users of this script may not be running the bash as their default shell. There are a couple of ways of forcing this script to run under the bash shell. One means of running our script using the bash may be:

sh script.sh
                

This would execute the script using the bourne shell (sh). This looks like a lot of work to repeat every time - insisting on the shell at the prompt. So instead, we use a shebang.

A shebang is really just a sequence of two characters - a hash sign followed by an exclamation mark. It looks like this:

#!
                

This is known as the shebang. Comments also start with a hash, but because this particular comment starts at the top of your script, and is followed immediately by a bang (an exclamation mark), it's called the shebang. Directly after the shebang, we tell the script what interpreter it should use.

If we had the following line at the top of our script:

#!/bin/ksh
                

This would run the contents of script.sh using the korn shell. To run the script using the bash we would have:

#!/bin/bash
                

If this was a perl program, we would start the script off with:

#!/usr/local/bin/perl
                

A sed:

#!/bin/sed
                

All subsequent commands would then be treated as if they were sed commands. Or perhaps we want to use awk:

#!/bin/awk
                

This assumes awk lives in our /bin directory. It might live in /usr/bin in which case it would be:

#!/usr/bin/awk
                

So we can include the shebang at the top of every script, to indicate to the script what interpreter this script is intended for.

While we have not included the shebang at the top of scripts written thus far, I'd encourage you to do so for the sake of portability. Meaning that the script will run correctly, wherever it is run.

We've seen a standard way of starting a script (the shebang), now I need to tell you about the standard way of ending a script.

Before we do that, we must understand what exit values are. Every program in Linux that completes successfully will almost always exit with a value of 0 - to indicate that it's completed successfully. If the program exits with anything other than 0, in other words, a number between 1 - 255, this indicates that the program has not completed successfully.

Thus, on termination of every script, we should send an exit status to indicate whether the script has completed successfully or not. Now if your script gets to the end and it does all the commands that it's supposed to do correctly, the exit status should be zero (0). If it terminated abnormally, you should send an exit status of anything but zero. I will therefore end every script with the command:

exit 0 
                

Thus, if no error is encountered before the end of the shell, the exit value will be zero.

Exit statuses also come in useful when you're using one script to call another. In order to test whether the previous script completed successfully, we could test the exit status of the script.

This is discussed in more detail later the section called “Exit status of the previous command”

There are some variables that need special attention, namely NULL and unset variables.

For example, if a variable called NAME was assigned with the following:

NAME=""
                

then the variable is set, but has a NULL value. We could have said:

NAME=
                

which too would have meant a NULL value. These are distinctly different from:

NAME=" "
                

A space between quotes is no longer a NULL value. So if you assign:

NAME="hamish"
                

this has a non-NULL value, while if you assign nothing to the NAME variable it's a NULL value. This distinction can sometimes catch you out when you're programming in the shell especially when doing comparisons between values. If the variable NAME were never set, a comparison like:

if [ $NAME = "hamish" ]; then
....
                

would return an error, as the test command requires a variable = value comparison. In the case of the NULL/unset variable it would test:

[ = "hamish" ]
                

which would be an error.

One method of handling NULL values in scripts, is to enclose the value in quotation marks, or surround them with "other characters". To display a NULL value NAME,

echo $NAME
                

would return a blank line. Compare this to:

echo :$NAME:
                

which would return

::
                

since the value is NULL. This way we can clearly see that a NULL value was returned. Another method of checking for NULL values in expressions is as follows:

if [ "${NAME}x" = "x" ]; then
.....
                

Here, if NAME is unset (or NULL), then:

"${NAME}x" would be "x"
                

and the comparison would be TRUE, while if

NAME="hamish"
                

then

"${NAME}x" would be "hamishx"
                

and thus the comparison would be FALSE.

What happens is if the value is not set at all? For example, what occurs if you unset a variable:

unset NAME
                

A similar result to the NULL variable occurs, and we can treat it in the same was as a NULL variable.

In sum then, the unset/NULL variables are very different from a variable that has an empty string as in

VAR="      "
                

Similarly, another question is: When does the shell do the interpretation of a variable?

In the statement:

echo $NAME
                

it does the $NAME variable substitution first before invoking the echo command.

What happens if we typed:

file="*"
ls $file
                

The output is equivalent to saying:

ls *
                

What happened in our example above? The variable file is being interpreted first, it then gets an asterisk (splat) which matches all files in the current directory and lists those files on the command line.

This illustrates that substitution of the variable occurs first, before any further command is executed.

What happens if I want to echo the following string?

hamishW
                

and my name variable NAME is currently set to 'hamish'? Can I do this:

echo $NAMEW
                

What's going to happen here?

The shell attempts to look for a variable NAMEW which clearly does not exist, but there is a variable NAME.

How do we make a distinction between the variable name and anything we want to follow the variable name? The easiest way to do that, is to use the curly brackets:

{}
                

Trying that again, we could write:

echo ${NAME}W
                

and the shell will now interpret the {NAME} as the shell variable and understand that 'W' is not part of the variable name.

In essence:

$NAME
                

is equivalent to

${NAME}
                

They achieve the same purpose, the only distinction between them is if one added a 'W' to the second example, it would not be considered as part of the variable name.

Since we're covering the topic of variables, now is a good time to make a distinction between environment and shell variables. Environment variables are set for every shell, and are generally set at login time. Every subsequent shell that's started from this shell, get a copy of those variables. So in order to make:

NAME="Hamish" 
                

an environmental variable, we must export the variable:

export NAME
                

By exporting the variable, it changes it from a shell variable to an environment variable.

What that implies, is that every subsequent shell (from the shell in which we exported the variable) is going to have the variable NAME with a value 'Hamish'. Every time we start a new shell, we're going to have this variable set to this value. It should go on and on like that. By exporting it, that's what we call an environment variable.

If a variable is not exported, it's called a shell variable and shell variables are generally local to the current shell that we're working in.

In other words, if we set a variable:

SURNAME="Whittal"
                

and at the prompt we now say:

bash 
                

starting a new shell, then:

echo $SURNAME
                

It will return a blank line. Why is there a blank line? Primarily because that shell variable wasn't exported from the previous shell to the new shell and is thus not an environmental variable. Shell variables are only available in the original shell where we issue the assignment of the variable.

We now have an understanding of variables, how we can set them and, in the next chapter we will look at quoting, specifically how we can run commands and assign the output of those commands to variables.

We've done basic shell scripting, but it would be nice to be able to do some basic arithmetic in the shell. While the shell is able to do basic integer arithmetic, it cannot do floating-point arithmetic. However, there are some ways of getting around this limitation. If we wanted to do floating point arithmetic we can use a utility called:

bc
                

which is a calculator.

We will have a chance to look at this later in the course. If you need to do lots of floating point arithmetic - I think you need to take a step up from this course and do a perl, Java or C course.

Let's concentrate on integer arithmetic.

There are a number of ways of doing integer arithmetic in the shell. The first is to enclose your expression in double round brackets:

$(())
                

Assuming you set a shell variable i:

I=10
                

You could then say:

$((I=I+5))
echo $I
                

It would return:

15
                

Arithmetic operators are as follows:

Read the man pages (man 1 bash or info) to find out about others. Within these $(()), you could do:

$((I=(15*I)-26))
                

By enclosing stuff inside round brackets within the arithmetic operator, you can change the precedence. The precedence within the shell is the good, old BODMAS (Brackets, Order, Division, Multiplication, Addition, Subtraction - see http://www.easymaths.com/What_on_earth_is_Bodmas.htm ).

So, the shell does honour the BODMAS rules.

Changing the order of the expression requires brackets.

$((J=I+5))
J=$((I+5))
J=$(( I + 5 ))
$(( J = I + 5 ))
                

all mean the same thing.

However, the following will produce errors due to the spaces surrounding the '=':

J = $(( I + 5 ))
                

We could, for example say:

I=$((k<1000))
                

What would happen here? This function would result in a true(0) or false(1) value for I.

If k<1000 then i=0 (true), but if k>=1000 then i=1 (false). 
                

You can do your operations like that, assuming that you have calculated the value of k before this step.

Although we currently do not have sufficient knowledge to perform loops, (we'll see later on how we use loops Chapter 8), I've included a pseudo-code loop here to illustrate how shell arithmetic can be used practically:

COUNT=0
loop until COUNT=10
    COUNT=$((COUNT+1))
done
                

COUNT, the variable, starts at 0, and increments by 1 each time round the loop. On count reaching 10, the loop exits.

Practically let's use the df command to do some examples. We're going to create a script called mydisk.sh.

At the top of your script include the shebang relevant to your shell, and at the end include your exit status.

#!/bin/sh
# This script will squeeze all spaces from the df command.
#
# First set the date and time
DATE=20031127
TIME=09:52

# Now squeeze the spaces from df
df -h|tr -s ' '

# We're done - no errors - exit 0
exit 0
                

Let's work this in sizeable bite-chunks. If you remember:

df -h|tr -s ' '
                

will pipe the diskfree output (size, percentage free and mount points) in human readable form to the translate command which will then squeeze multiple sequential spaces into a single space, giving:

/dev/hda6 3.8G 2.9G 744M 80% /
/dev/hda9 12G 10G 1.1G 90% /mnt
/dev/hda5 3.8G 2.5G 1.0G 70% /debian

df -h | tr -s ' ' | tr ' ' ',' | sed '/^\/dev/!d'; \

/dev/hda6,3.8G,2.9G,744M,80%,/
/dev/hda9,12G,10G,1.1G,90%,/mnt
/dev/hda5,3.8G,2.5G,1.0G,70%,/debian
                

From the output of the df command we are only interested in the partition that the device is on (in this example, da0s1{aeh} - nice to know RE's hey!), the size, the percentage free and the mount point in that order:

mount point,part,size,%free

df -h|tr -s ' '|tr ' ' ','|sed '/^\/dev/!d; \

s%/dev/\(hda[1-9]\+\),\([0-9]\+\.\?[0-9]\?[GMK]\?\),.*%\1;\2%g
-^_____^----^_____^--^_^-----^_^-^_^----_^_____-^-		^^^^^__-^_^
-1-----2----3----4---5-6-----7-8-9-1-----1------1-11111---1-2
-----------------------------------0-----1------2-34567---9-0

1=Start of search using the % not the / since / are in the RE
2=(hda) Start of the group (to match the hdaX on your linux machine)
3=(hda[0-9]). Match the range hda1, hda4, hda12, etc. 
- Note hda10 will not be matched here. Why not?
4=Match 0 or more of them (i.e. match hda3 or hda11)
5=Follow the hdaX by a comma
6=Start of the group 12G or 3.8G. Match a 0-9
7=Match 0-9 one or more times
8=Followed by an optional full stop (\.\?)
9=Optional full stop. See 8 above.
10=Followed immediately by an optional number (e.g. the .8 in 3.8)
11=The optional number in 10 above.
12=Followed by a G, K or M for Gigabytes, Kilobytes of Megabytes,
Optionally...
13=End of group to match the 3.8G or the 12G
14=Followed by a comma
15=Followed by anything
16=Zero or more times (for 15 above)
17=End of pattern started in 1
18=First placeholder (the hdaX)
19=Second placeholder (the 3.8G, 12G, etc.)
20=End of RE.

--This command looks like the following when it is printed on one line:--
df -h |tr -s ' '|tr ' ' ','|sed '/^\/dev/!d; s%/dev/\hda/\(hda[1-9]\+\),\([0-9]\+.\?[GMK]\?\),.*%\1;2%g'
                

Phew. I'll leave you to modify the RE to encompass all other fields we need. It's really not that difficult, just a little tricky.

As you remember we must make this script executable before we can run it, so:

chmod +x mydisk.sh
                

Now, let's run the script:

./mydisk.sh
                

Have you got the result we are after?

Of course, we could have achieved the above RE with a cut command, but there are even better ways of skinning this cat. Stay tuned.

Ensure the following for you scripts:

None of these scripts should be longer than 10 lines (at the outside)