| Revising some Basic Commands | ||
|---|---|---|
|
Chapter 1. Tour de Shell Scripting |  |
| Revising some Basic Commands | ||
|---|---|---|
|
|
Chapter 1. Tour de Shell Scripting | |
There are some basic commands that we are going to look at. The idea is to get you into the process of understanding how commands are structured and build an understanding of what the commands do.
From hereon out, I'm going to assume that you can find out lots of things about commands primarily by looking at info and man pages.
Almost every Linux command can be run from the command line using various switches (or arguments / options) which allow one to change the output of this command in a number of different ways.
The who command is designed to tell you who's logged on to the system.
If we run the who command without any switches, the left hand column shows the user id. This the user currently logged on to the system. In your case, you might be logged on as root, or perhaps as your user. The second column indicates where you are logged in.
riaan@debian:~> who
riaan :0 Apr 30 11:13 (console)
riaan pts/0 Apr 30 11:13
riaan pts/3 Apr 30 11:14
riaan pts/4 Apr 30 11:30
riaan pts/5 Apr 30 13:19
riaan pts/6 Apr 30 12:07
riaan pts/7 Apr 30 12:09
riaan@debian:~>
|
So if you look at the who command output, my user riaan is logged in from :0 which is the X console. He's also logged on to
pts/0 and
pts/1
|
These are pseudo terminals, indicating he's also logged into two pseudo terminals.
The final, third column indicates what time the user logged on.
The who command tells us about users using our system. That's great!
What are the other switches that we can use with who.
who --help
|
This will show you the various switches that we can use with the who; command. So if we use a:
who -H
|
it prints a heading line for us. The output should look as follows:
$ who -H
NAME LINE TIME FROM
heidi ttyp1 Nov 27 17:29 (168.210.56.177:S)
mwest ttyp2 Nov 10 15:04 (apotheosis)
heidi ttyp4 Nov 11 13:18 (168.210.56.177:S)
|
To view a short listing which is the default listing for the who command:
who -s
|
Using the -u switch:
who -u
|
will show the users and their process id''s.
In scripts, one can use the same commands as on the command line, including all the switches those commands use. One can run any command and produce standard text output, which one can use. We'll talk about how you can use the output later.
Run the command
$ who -u
root tty2 Aug 4 10:41 . 2839
riaan :0 Aug 4 07:53 old 2836 (console)
|
to identify which users are logged into your system and from which processes they are logged on.
This will show how long a terminal has been idle. It will show not only which users are logged on and what process ids they are but also how long that user has been idle. Idle users might have gone out for lunch or they might have left for the day. In default mode, most of these systems don't log you out if you're idle for longer than 10 or 15 minutes. In the old days, most systems were configured to automatically log you out after 15 minutes.
![[Note]](../images/admon/note.png) |
Note |
|---|---|
|
On Debian, the -i switch does not add any extra output, it simply prints a message suggesting that you not use -i as it will be removed in future releases. Use -u as above. However the -i switch may work with other brands of Linux. |
|
Okay, so that's the who command. We're going to use these commands later to build a system to monitor our system automatically, because we want to be spending our time doing things we enjoy.
How many users are currently logged onto your system?
Is any user logged onto your terminals tty1 -> tty6?
Who (which user and or terminal) has been idle for the longest time?
Ensure that all output is displayed with a heading line.
What run-level are you currently running in and what was your previous run-level? What other command will show you this information?
What would you imagine a users message status would be? (Take a guess or read the man page for write)
What does the w command do? You could run:
riaan@debian:~> whatis w
w (1) - Show who is logged on and what they are doing.
riaan@debian:~>
|
riaan@linux:~> w
21:40:17 up 11:03, 6 users, load average: 0.30, 0.34, 0.30
USER TTY LOGIN@ IDLE JCPU PCPU WHAT
root tty2 21:40 8.00s 0.06s 0.06s -bash
riaan :0 10:38 ?xdm? 58:31 0.04s -:0
riaan pts/0 10:38 11:01m 0.00s 2.08s kdeinit: kwrited
riaan pts/3 11:18 10:22m 14:37 2.63s /usr/lib/java/bin/java -mx32m -jar /home/riaan/jedit/4.2pre9/jedit.j
riaan pts/4 11:28 1:07m 0.21s 0.21s /bin/bash
riaan pts/5 11:28 0.00s 0.17s 0.03s w
|
which should print some information about the w command.
The w command not only tells us who are logged in, but also what they're doing. Are these users running applications? What actual processes are they running at this time? Perhaps someone's running an application like OpenOffice. w will tell us this.
If you look at the output of this command, it's got a list of headings that are fairly similar to the format of the who command.
Later we'll have a look at modifying the report columns, to get the output into a different format that may be more useful.
One of the reasons for taking you through these commands is that we're going to start writing our first shell scripts using these commands, so it is as well that we understand them now.
The date command is a useful command that can do all sorts of nifty things for us (apart from printing out the date).
It can convert between Unix time, (which is the number of seconds since 1/1/1970 - commonly known as the epoch) and a human readable (normal) date and time.
Conversely, it can also convert back from date time today to the number of seconds that have elapsed since the 1/1/1970. It can format output in a whole variety of ways. Let's look at some examples of the date command.
For that I'm going to do:
info date
|
If you scroll down, you will see a section with examples. Looking at the example:
date +" "
|
We may now include a string describing the format inside these quotation marks.
In the shell there's a big distinction between double quotes, single quotes (which is another lesson altogether, see Chapter 5), and back quotes - let's not get them confused for now.
Within this double quoted string we can include any number of arguments. What arguments can we include? Each argument starts with a percentage sign.
To display the time, we could use:
%H - -will give us the hours in 24 hour format (0-23).
%M - -will give us the minutes (0-59) of the day
|
If we had the following string:
date +"%H:%M"
|
we will end up with hours and minutes of the day on our system. The result of the above command should be similar to:
15:04
|
But let's say that we want the hours in 12-hour format rather than 24-hour format. We could then replace the %H with a %l. The result would then be:
3:04
|
There's a host of other things that we could do. For example if we are in 12-hour format, 3:04 doesn't indicate whether it is morning or afternoon. Thus we could include %p:
date +"%l:%M %p"
|
This would show us that the time is actually:
3:04 PM
|
rather than 3:04 AM.
That's for time, but what about for the date? What happens if we want to show the date, which is:
24-Nov-2003
|
then, we should in theory be able to create a date string to reflect this format.
A way we can do is this is using the following:
date +"%d-%b-%Y"
|
where %b is a short format for month to produce Nov instead of November.
If we want to combine the date and time:
date +"%d-%b-%Y %l:%M %p"
|
This would give us the full date and time report:
24-Nov-2003 3:04 PM
|
There are a lot of other parameters that you can use within the date command. You can view these by looking at the relevant info page with :
info date
|
We're going to use this command in our script, because in almost every script that you will write you are going to want to know what time the script started, what time the script ended, when it did a particular job within the script, etc.
Using the info command for assistance, format the output of date, so that it resembles the following: Today's date is Tuesday, 27 January 2004. The time is currently 11h 32m and 49s.
Show the date in Julian format.
What day of the year is today?
Include in the output of the date, the time zone and the AM/PM indicator
Given the number 1075200287, determine what date, day and time this represents.
The final command I want to describe is a command used to send output to the screen: echo.
We've seen so far that we were able to run commands but, as yet, we don't know how to simply output some text to the screen. We may want to print a string to the screen, prior to printing the date.
Something such as:
Today's date is:
24-Nov-2003 3:04 PM
|
We would need some way of echoing that to the screen, wouldn't we?
In order to do this, there is the echo command. echo can be a bit of a nasty gremlin because there are at least two echo commands on your system. One is a shell built-in, and one is an external command and therefore it can be a bit tricky.
We're going to start off with a vanilla case. Later on we will see how we can choose which echo command we want to use.
So by way of an example, we'll use it to format the date command.
echo "Today's date is: "
date +"%d-%b-%Y %l:%M %p"
|
This would be a good time to show you how to create your first shell script. We're going to edit a file and for this you can use any editor[1]
Open your editor (whatever editor you prefer) and put the following commands in the first lines of the file:
echo "Today's date is: "
date +"%d-%b-%Y %l:%M %p"
|
Save that file as myfirstscript and exit your editor.
You've just created your first shell script. Great! How easy was that? How do you run it to make it actually do its job?
Linux has three sets of permissions that set the permission mode of the file . One for the owner, one for the group and one for everyone else (i.e. Everyone that is not the owner, and is not part of the group to which the file belongs). You would have covered this in an earlier course (Linux Fundamentals). The mode of the file will also determine whether the file is executable or not.
Thus, to make the file executable, the mode of the file must have the execute (x) permissions set.
Note that this differs from Microsoft Windows which looks at the extension to decide the type of the file. Windows assumes that .com and .exe are executable files. Notice too, that myfirstscript does not have an extension. Is that valid? Sure it is. This is Linux after all!!
In sum, to make our script executable we must change the mode of the file. How? Using chmod (change mode) command as follows:
chmod +x myfirstscript
|
This script is now executable. Does this mean that we've executed it? No, not yet. We execute it with:
./myfirstscript
|
and that should output:
Today's date is:
24-Nov-2003 3:04 PM
|
Finally, you will notice that in order to execute this command, I preceded it with a "./".
Try running the script without the "./". What happens?
What the "./" does is that the shell FIRST looks in they current directory (.) for the script before trying to look in the PATH. We will cover this is a little more detail later.
Of course, you could add the script to a new directory ~/bin (in my case /home/hamish/bin). Since this directory is in my PATH environment, the script will be "found" and will execute even without using "./"
Write a simple script to print "Hello <YOUR USER NAME>" on the screen.
Modify your scripts to additionally output the directory you are currently in.
Write a simple menu system, with the menu options as follows(note: this menu does not have to respond to the user's input at this stage):
0. Exit
1. Output a text version
2. Output an HTML version
3. Print help
4. Provide a shell prompt
Select an option [1-4] or 0 to exit:
Include the current date and time in the top right hand corner of your menu
Ensure that your menu system (I'm assuming you called it menu.sh) can execute. Execute it.
Consult the appendixAppendix B and teach yourself the dialog package.
There are many examples in /usr/share/doc/dialog-xxx.yyy.
Once you understand this package, modify your menu system in 3 above to use the ncurses library (dialog) to spruce it up.
This section is going to cover file commands. File commands are commands such as ls (list).
Notice again, how the laziness of the Unix people comes to the fore. They could have had a command called list, but that would have required two more characters (and two more carpals - fingers!) and clearly that was a lot more effort, so we just have the ls command. The ls command shows us a listing of files in a particular directory.
This is an appropriate place to take a detour on our tour de scripting and have a look at file matching and wildcard matching. It may be something that you're familiar with, but let's have a look at it and come back to ls in a moment.
Wildcard matching allows us to match a number of files using a combination of characters. What are those characters?
Table 1.1. Wildcards
| Symbol | Name | Purpose |
| * | Splat | matches 0 or more of any character |
| ? | question mark | matches 0 or 1 character |
| [ ] | square brackets | matches one from a range of characters |
| ! | bang | invert the pattern |
So if we typed
ls *
|
we end up listing 0 or more of any characters. So it would match any filename of any length because any filename would have a minimum of a single character. The splat matches 0 or more characters following each other.
The question mark will match a single instance of any character. Later, when we study regular expressions, the full stop ( . ) matches any single character. Given the expression:
ls -la ab?a
|
this would match the files:
abba
ab1a
ab_a
ab9a
abca
...
|
What range of characters do [] include? Well we may say something like:
[abc]
|
which would match any (single character) within the following range: a or b or c but it does not match abc since this is 3 characters long. Similarly
[ab]cde
|
would match acde or bcde but it would not match abcde. Square brackets indicate a range of alternative characters. How would we match both:
ham01
|
and
ham10
|
Simple, with:
ham[0-9][0-9]
|
as [0-9] implies that any character in the range 0 to 9 will match.
Would it also match ham9?
No, for the reason that our pattern search is expecting 2 numeric characters after the pattern ham. (two sets of square brackets are displayed)
We could also combine letters and numbers, for example:
ls [abc123]
|
which would match the files:
a
b
c
1
2
3
|
(Yes Linux and UNIX can have files with all of the above names!)
Notice when we used the square brackets [0-9], we use the range specifier, which was the dash character in the middle. This dash has nothing to do with the minus sign and means match anything in that range from 0 through to 9.
Thus, typing:
[a-z]
|
matches any character in the range from a to z.
The final pattern matching wildcard is the bang command. The exclamation mark is the inverse of what you're trying to match. If we were to look at our pattern matching, we could say:-
ls [!a-c]*
|
which, would match everything NOT starting with an "a" or a "b" or a "c", followed by anything else.
So would it match abc ?
No, because it starts with an a.
Would it match bcde?
No, because it starts with a b.
Finally would it match erst?
Yes, because q is not in the range a to c. So it would match q followed by any set of zero or more characters.
Prior to doing the following command ensure that you create a new temporary directory, and change directory to this new directory [ by doing so, it will be easier to clean up after the exercise is over. ]
I'd like you run the following command (I'm not going to explain it now, it will be covered later).
touch {planes,trains,boats,bikes}_{10,1,101,1001}.{bak,bat,zip,tar}
|
This command creates a number of files for us in one go. Typing:
ls p*
|
will show us all files that start with a 'p', followed by 0 or more characters (planes_ in our case). Similarly
ls b*
|
will show us the files boats_ and bikes_ since only these files begin with a 'b'. If we typed:
ls bik*
|
it will produce all the bikes files, while
ls b*_??.*
|
will indicate all the files that start with a 'b', followed by 0 or more characters, followed by an underscore followed by exactly 2 characters, followed by a full stop, followed by 0 or more characters. Thus only
boats_10.bak
boats_10.bat
boats_10.zip
boats_10.tar
bikes_10.bak
bikes_10.bat
bikes_10.zip
bikes_10.tar
|
I've given you some additional examples that you can try out in your own time. It would be worth your time running through these examples and ensuring that you understand everything.
Run the following command in a new subdirectory
touch {fred,mary,joe,frank,penny}_{williams,wells,ferreira,gammon}.{1,2,3,4,5}
|
Look at the following tu understand the different fields
touch {fred,mary,joe,frank,penny}_{williams,wells,ferreira,gammon}.{1,2,3,4,5}
------Name---------- ----------Surname------- -Category-
|
list all the people who fall into category 5
List all the people whose surnames begin with a w.
List only people in category 2, whom's surnames begin with a w, and whom's first name begins with an "f"
Move all "fred's" into a directory on their own, but exclude the "gammons"
Ensure that joe and penny wells in category 3 and 5 are deleted.
List only those people whose names have 5 characters in them.
Returning from our detour, there's a whole bunch of other file commands that we can look at. We've touched on the ls command, which gives us a listing of files and from the previous set of examples on pattern matching and wildcarding, you will have gotten an idea of how ls works.
ls , like every other Linux command can take switches. Below is a quick summary to some of the switch options.
-l show a long listing (include file name, file size, date last modified, the permissions, the mode of the file, the owner, the group of the file)
-a shows all files including {hidden} files (. and ..)
Two special hidden files are:
. this file is our current directory
.. this file is our previous or parent directory.
|
Often in your home directory, you will have full control over your . directory (i.e. your current directory) but you will most probably have absolutely no control of your .. directory (i.e. your parent directory).
We have other commands like:
cp copy files or directories
mv move files or directories
wc shows the number of lines, number of words and number of characters in a file.
wc -l show us the number of lines in our file.
nl numbers the lines in our file.
There are many system commands that we can use. We're going to start using these in our shell scripts.
Remember, a shell script is nothing more than a group of Linux commands working together to produce a new command.
In order to build our system that is going to manage our Linux machine, we're going to need to know a little bit about system commands. System commands such as:
df shows the disk free space
du shows the disk usage
fdisk shows the partitioning on our disk
iostat shows the input output status
vmstat shows the virtual memory status
free shows the amount of free memory
We will use these commands, but they are a subset of the commands available to us for monitoring our system.
For example, mostly we want to run a command to check whether our file systems are filling up. It makes no sense to have a full filesystem! For that we might use the df command
df would produce a listing of our partitions on our Linux machine and will show us which partitions are 100% full, which partitions are almost empty, which partitions are mounted, etc..
What we're going to be doing, is working our way towards building a system that will automatically show us when a partition becomes 100% full.
Or perhaps we want to build a system that shows us when it's greater than a particular threshold. So we might set a threshold of 95% full, it's no good telling us when the system has crashed that it's crashed; we want to know before the system crashes.
Several switches can be used with df such as:
df -h
|
This produces output in human readable format. If you run this command now, you should see at least one partition and that's the root partition. It should show you how much space is being used, how much is available, what the size of the partition is and what particular filesystem it's mounted on.
The df command is what we'll start working on for our practical, because it most certainly is one of the most useful commands that you're going to need to run to make sure your system is not becoming overfull.
Alternately, you could use the du (disk usage) command to show you which files are consuming the largest volume of disk space:
du - s
|
will show a summary of our current filesystems' capacity. Again, how do you get information about these commands? Using the info or man command will inform you about switches pertinent to these commands.
For example a further switch for the df command might be:
df - hT
|
which will tell us what type of filesystem we're running. It might be an extended 2, 3, or a vfat filesystem.
The "du" command, like the "df" command, has a complete set of switches unique to it and shortly we'll start looking at those in a bit more detail.
The fdisk command is used primarily to show us what partitions we have on a particular disk.
BlackBeauty:/install # fdisk -l
Disk /dev/hda: 10.0 GB, 10056130560 bytes
240 heads, 63 sectors/track, 1299 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 760 5745568+ 83 Linux
/dev/hda2 761 1299 4074840 5 Extended
/dev/hda5 761 827 506488+ 82 Linux swap
/dev/hda6 828 1299 3568288+ 83 Linux
|
It should show you what your partitions are.
If you execute the above command as a user (in other words, your prompt is a dollar) you're not going to be able to see what your partitions are set to. Why? Because only the superuser (root) has permission to look at the disk. In order to run this command, you need to be root.
baloo:/home/riaan# free
total used free shared buffers cached
Mem: 498572 493308 5264 0 48700 230492
-/+ buffers/cache: 214116 284456
Swap: 706852 8196 698656
|
This command shows the total memory, used memory, free memory, swap space, how much of our swap space has been used and how much of our swap space is still available for use.
baloo:/home/riaan# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 3792 3508 14208 59348 0 0 32 72 1065 818 11 4 84 2
|
The vmstat command shows us how busy our system is, how many processes are running and how many are blocked. It also shows memory information: how much memory is being used by the swap daemon and what our buffers and caches are. Additionally, it shows us how many processes are being swapped into and out of memory. Finally, it shows users, system, idle and waiting time. We're going to use it later to monitor our system
Finally, the iostat command.
iostat
Linux 2.6.4-52-default (debian) 09/02/04
avg-cpu: %user %nice %sys %iowait %idle
2.51 0.03 1.99 0.82 94.64
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
fd0 0.00 0.00 0.00 4 0
hda 3.93 304.75 23.83 3868650 302512
hdd 0.01 0.59 0.00 7524 0
|
This command gives you information about input and output on your system, and how well (or otherwise) it is performing.
We'll take a closer look at the system performance commands in more detail later.
|
Note |
|---|---|
|
In order to provide you with further information on the performance of your system, you should install the sysstat package. rpm -ivh sysstat.x.y.z-r1.rpm (RedHat system) (see the appendix for other distributions) The iostat command is part of the sysstat package, so if you don't install sysstat, then skip the iostat stuff) |
|
Write a simple script to display the free/used disk space on your machine
Additionally, show the status of the memory and allow the vmstat commands to produce 3 iterations.
Using the iostat command, write a script that will illustrate how busy you machine is
[1] If you haven't been taught vi, an excellent place to start is using vimtutor. Type vimtutor on the command line to begin the lessons. It only takes 45 minutes, but is well worth the effort. vi is the best editor in the world, in my humble opinion Almost everyone in the UNIX/Linux world has some relationship with it - a love-hate relationship: some people love it, some people hate it. But if you're going to learn any editor, learn vi. Since it is the de-facto editor on almost any UNIX/Linux variant in the market today, learning the basics at least should stand you in good stead in you years as a system administrator.
|
|
 |
|
| Introduction to info, man and the whatis database |  |
stdin, stdout, stderr |