| The Shell Command Interpreter | ||
|---|---|---|
|
Chapter 4. Essentials |  |
| The Shell Command Interpreter | ||
|---|---|---|
|
|
Chapter 4. Essentials | |
The shell command interpreter is the command line interface between the user and the operating system. It is what you will be presented with once you have successfully logged into the system.
The shell allows you to enter commands that you would like to run, and also allows you to manage the jobs once they are running. The shell also enables you to make modifications to your requested commands.
The Bourne-Again shell is not the only shell command interpreter available. Indeed, it is descended from the Bourne Shell (sh), written by Steve Bourne of Bell Labs. This shell is available on all Unix variants, and is the most suitable for writing portable shell scripts. This is discussed in depth in the Shell Scripting Course.
The default shell, which is provided with most Linux based systems is the Bourne-Again shell ("bash").
Other popular shells include the C Shell (csh), written at UCB, and so called because its Syntax is similar to that of the C language.
The TC Shell (tcsh) is an extension of the C shell.
A very popular shell on most commercial variants of Unix is the Korn Shell. Written by David Korn of Bell Labs, it includes features from both the Bourne shell and C shell.
Finally one of the most powerful and interesting shells although one that hasn't been standardised on any distribution that I've seen, is the Z shell. The zsh combines the best of what is available from the csh line of shell utilities as well as the best that is available from the bourne or bash line of shell utilities.
One relatively easy way to determine what shell it is that you are currently running is by looking at the prompt. If you are using a bourne shell or derivative, you will see a dollar sign. This is more than likely what you will see at your bash prompt at the moment. If however you are using csh, or tcsh you will see a percentage sign. If you are logged in as root or the superuser, irrespective of your shell, your prompt will normally always be a hash.
The bash prompt:
student@debian:~$ _
|
The tcsh prompt:
student@debian:~% _
|
The root prompt:
root@debian:~# _
|
More modern shells allow you to access your history of commands, and automatically complete filenames and even correct mistyped commands.
The up and down arrow keys will allow you to traverse the history of commands that you have typed in.
The left and right arrow keys will allow you to navigate the cursor on the command which you are currently typing.
The backspace and delete keys will delete the character behind the cursor, and underneath the cursor, respectively.
The default mode at your bash prompt is the insert mode, which means that if you type a character somewhere on the line it will insert it at that position.
If you then press the insert key it will then toggle the mode so that it will go into overwrite mode, which means that it will overwrite whatever character is directly underneath your cursor.
Another useful feature is tab completion. What this means is that you can use the tab key in order to be able to complete a command, or be given a list of options of commands which match what you have typed so far.
Tab Completion:
student@debian:~$ pas <tab> passwd paste student@debian:~$ pas_ |
The shell is telling you that you need to make a choice between those two options. You can let it know which one by filling in one more letter ("s" or "t" in this case), and then pressing <tab> again.
student@debian:~$ pas <tab> passwd paste student@debian:~$ pass<tab> student@debian:~$ passwd_ |
Now pressing <enter> will finally result in the command being executed.
Another very useful feature of the bash shell, and one that I would recommend that you use often, is called "reverse case-insensitive history search". It will search through your history in reverse and case-insensitively (so it won't worry whether it was in upper or lowercase) for a command or even part of a command that you typed.
In order to access this, you can use the shortcut key combination CTRL-R, followed by a command or a subsection of a command that you have typed and you will notice that it will go back in history to locate this command
student@debian:~$ <ctrl-r> reverse-i-search)`p': passwd |
The shell has now searched back into your command history for the letter "p", and the first command that has matched is "passwd", which you typed earlier. Pressing <enter> now will result in the command being executed. Alternatively, you can use the arrow keys to scroll through other options, or continue typing letters until you have a better match.
There are several files which will affect the behaviour of your bash shell:
/etc/profile
/etc/bash.bashrc
$HOME/.bashrc
$HOME/.bash_profile
The file where your history of commands is kept is called:
$HOME/.bash_history
In this section, we will explain how the shell interprets the commands, which you give it to execute.
It is important to understand that the shell does actually interpret what you type to it.
What this means is that certain special characters will have to be interpreted or dealt with prior to the execution of the command.
An example is the asterisk or wildcard character. When a shell sees this character it will attempt to substitute any matching filenames in place of this wildcard character. It is important to note that this happens before the command is executed.
There are other special characters that will also be interpreted. Different shells interpret different characters in different ways. The most commonly interpreted characters are the asterisk, question mark, the various brackets, forward slashes and quotation marks.
We will learn the significance of each of these characters and the effect that they will have on the way that the shell executes your commands.
Once the shell has interpreted your command and run replacements where you have requested it, it will then check and see if the command is perhaps something that needs to be executed by the shell itself, in other words it is an internal command.
An internal command is a routine that is part of the shell itself and does not require the shell to open up an external file in order to execute it. Examples of internal commands are clear and history.
As mentioned before, the bash shell allows you to have shortcuts, or aliases. These aliases are interpreted before executing the command as an internal or external one. For example, it is possible to configure bash to treat 'll' (two lowercase letter L's) as a shortcut to the ls (directory listing) command. This is made into a shortcut by using the alias command, which is a shell built-in command.
You can use the "type" shell built-in to determine if a command is an internal, external or an alias.
student@debian:~$ <indexterm><primary>type</primary>
</indexterm>type type
type is a shell builtin
student@debian:~$ <indexterm><primary>type</primary>
</indexterm>type passwd
passwd is /usr/bin/passwd
student@debian:~$
|
If the command is not an internal command, then it will be an external command. An external command is an executable file that exists somewhere on the system and that you are able to run. An example of an external command is passwd and the name of the shell itself, in our example: "/bin/bash".
If the command is not a shortcut, or an alias, and is not an internal command but is an external command, the file that is executed must be readable and executable by you.
You need to have the appropriate permissions in order to be able to run it. It also needs to exist in a directory that exists inside a directory that is inside your search path.
The search path lists all the directories in which the shell can find commands that you would like to be able to run.
On a usual Linux system, the /usr/bin and the /usr/local/bin directories will all be inside your path environment variable, as this is where the system stores your common executable files. These files are also known as binary files, as opposed to source files, hence the "bin" directory name.
So what the shell will do is that it will check in each of those directories for the existence of the command that you have typed. If it finds a file that matches the name and you are allowed to execute it, then the shell will request that the kernel load that programme into memory, and execute it; in other words, instantiate it as a running process.
Once the command execution is completed, it will return control back to the shell.
If the shell is not able to find a matching command to execute, in other words it not a shortcut, it's not an internal command and it cannot find an external programme in the path that you have provided, then it will give you an error message.
The advantage of internal commands is that they are less expensive for the kernel to execute, and so consume fewer system resources (CPU time and memory). The advantage of external commands is that they are able to be far more flexible.
For this reason, internal commands are usually often used, simple tasks; whereas external commands are often large applications.
There are several environment variables that influence the way that the shell operates and that can be used by commands that the shell executes.
These variables are set up for you during the boot-up of the system and during your log in process, mostly determined by your choice of shell. We discussed the files used by the bash shell earlier.
One variable that was mentioned before was the search path, this information is held in a environment variable called PATH.
Under the bash shell you can run the "set" command to list the current shell environment variables and their associated values. "set" is a shell built-in command.
student@debian:~$
<indexterm><primary>set</primary>
</indexterm>set
BASH=/bin/bash BASH_VERSINFO=([0]="2" [1]="05a" [2]="0" [3]="1"
[4]="release"[5]="i386-pc-Linux-gnu")
BASH_VERSION='2.05a.0(1)-release'
COLUMNS=80
DIRSTACK=()
EUID=1000
GROUP=student
GROUPS=()
HISTFILE=/home/student/.bash_history
HISTFILESIZE=500
HISTSIZE=500
HOME=/home/student
HOST=debian
HOSTNAME=debian
HOSTTYPE=i386-Linux
HUSHLOGIN=FALSE
HZ=100 IFS=$' \t\n'
LINES=30
LOGNAME=student
MACHTYPE=i386
MAIL=/var/mail/student
MAILCHECK=60
OPTERR=1
OPTIND=1
OSTYPE=Linux
PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
PIPESTATUS=([0]="0")
PPID=225
PS1='\u@\h:\w\$ '
PS2='> ' PS4='+ '
PWD=/home/student
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:histexpand:
monitor:history:interactive-comments:emacs
SHLVL=5
TERM=Linux
UID=1000
USER=student
VENDOR=intel
_=passwd
student@debian:~$ _
|
Remember that these variables control your entire environment and as you will see they are pretty easy to change so if you are going to change one or part of your environment then please think it through to avoid problematic consequences.
If you wish to assign a value to an environment variable, the Syntax is:
student@debian:~$ VAR=value
|
Here, "VAR" is the variable name, and "value" is what we are storing inside of it.
Note that there is no space before or after the equal sign. The space would be assumed to be either part of the variable name or to be part of the value that you are assigning to the variable.
Setting a variable using the Syntax above will cause the variable to only be available to the current shell instance. Usually, you want the variable to be propagated to any commands that you execute from the shell. To do this, you need to "export" the variable.
student@debian:~$ VAR=value student@debian:~$ export VAR |
You can also combine this into a single command:
student@debian:~$ export VAR=value |
Let's have a look at an example of re-setting a variable and then exporting the value:
For this example we are going to use our login prompt variable called PS1, see the "set" command above.
student@debian:~$ PS1="newprompt $ " newprompt $ _ |
![[Note]](../images/admon/note.png) |
Note |
|---|---|
|
Enclose this in quotation marks to protect the spaces. |
|
Now open a new shell and check for yourself that the prompt returns to the original prompt:
newprompt $ bash
student@debian:~$
|
Exit back to the original shell and export your variable PS1.
newprompt $ bash
student@debian:~$
student@debian:~$ exit
newprompt $ export PS1
newprompt $ bash
newprompt $
|
Now your new prompt has been exported to all subsequent shells.
Please note however that once you logout of this session your prompt will return to the default in your next log in. If you want to change this permanently change PS1 and export the value into one of your startup files. ($HOME/.bashrc or $HOME/.bash_profile)
A useful command to query the contents of a single variable is the "echo" command.
This command simply displays any arguments that to pass to it on the screen. For example:
student@debian:~$ echo hello hello student@debian:~$ |
If you don't provide any arguments, then it will simply display a blank line. (This is useful for providing spacing inside shell scripts.)
Do you remember that we mentioned that the shell actually interprets the commands that you give it before it executes them?
For example, in order to display the contents of a variable field as opposed to the variable name we could precede the variable name with a special character, in this case a dollar sign ($), and this will display the contents of that variable.
student@debian:~$ VAR=avalue --Set the variable--
student@debian:~$ echo VAR --To test our theory--
VAR --Displays the word not the value stored.--
student@debian:~$ echo $VAR
avalue --Display the contents of the variable--
student@debian:~$ _
|
Remembering to include the dollar sign ("$") before the variable name is very important, as illustrated below:
student@debian:~$<indexterm><primary>echo</primary> </indexterm>echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games student@debian:~$<indexterm><primary>echo</primary> </indexterm> echo PATH PATH |
Two other important bash shell environment variables: 'TERM' specifies the terminal type.
Depending how you log in to the system, the terminal type will either be "xterm" or "vt100".
The terminal type determines what the terminal capabilities are, such as can it do colour, does it have a speaker attached so that it can generate a beep or do you want the beeps to be visual flashes on the screen.
The terminal type also lets applications know if the terminal can handle certain types of characters. Very old terminals used to be only able to handle uppercase letters, so if you set the terminal type to one of those you would only be able to see uppercase characters on the screen.
Sometimes, being able to set the term environment variable is useful if you are connecting to a system that doesn't know where you are connecting from and you wish to be able to specify the correct terminal type so that your display is correct.
A good terminal type to try if you're having problems with your terminal is vt100 as this works on almost all types of terminals.
Another important environment variable is PS1. This is the current prompt for the shell. The convention is that a C-style shell has a "%" prompt, where a Bourne-style shell will have a "$" prompt. The root user will have a "#" prompt; this allows you to easily see when you are a normal user or a user who has much more potentially destructive power!
|
Note |
|---|---|
|
Any changes that you make to the shell environment will be lost when you exit the current login session. If you wish to make the changes more permanent, you need to add the commands that you wish to run to either the system-wide /etc/bashrc file (the change would affect all bash users on the system), or to your own $HOME/.bashrc file (the changes would affect you only). |
|
Under Unix, and Linux, most commands are abbreviated using 2 to 4 characters; with the more often used commands being shorter and the less-often used ones being longer.
|
"There are so many options to each command in Linux - get an overview of the command, that it exists at all as a tool for you to use. Working out how to use each and every nuance can be very confusing - you will get bogged down in detail instead of gaining a comprehensive overview. We are aiming for a lot of knowledge to be given and gained in this course and getting stuck on command details is not the main goal." |
||
| -- | ||
All Unix and Unix-like systems come with online documentation. The most common form are man pages; man being short for "manual".
These pages are, however, not very good for teaching someone who is new to the system, but are very good as reference material.
They will document all the useful, and sometimes even obscure, switches and features of the command line tools that you have at your disposal.
The man pages are divided into several numbered sections:
| 1 - General Commands |
| 2 - System Calls |
| 3 - Subroutines |
| 4 - Special Files |
| 5 - File Formats |
| 6 - Games |
| 7 - Macros and Conventions |
| 8 - Maintenance Commands |
| 9 - Kernel Interface |
You will often see references such as "ls(1)"; this is referring to the page on "ls" in section 1 of the man pages.
You can use the " man" command to look up a page:
student@debian:~$ <indexterm><primary>man</primary> </indexterm>man man Reformatting man(1), please wait... |
This will display the man page on the man command.
man(1) man(1)
NAME
man - format and display the on-line manual pages
SYNOPSIS
man [-acdfFhkKtwW] [--path] [-m system] [-p string] [-C config_file]
[-M pathlist] [-P pager] [-S section_list] [section] name ...
DESCRIPTION
man formats and displays the on-line manual pages. If you
specify sec- tion, man only looks in that section of the manual. name is
normally the name of the manual page, which is typically the name of a
command, function, or file. However, if name contains a slash (/) then man
interprets it as a file specification, so that you can do man ./foo.5 or
even man /cd/foo/bar.1.gz. See below for a description of where man looks
for the manual page files. [ ... ] September 2, 1995 man(1)
|
The "man" command will look for the first page it can find which matches what you've asked for. If you want to see a page in a specific section, you can specify it thus:
student@debian:~$<indexterm><primary>man</primary> </indexterm> man 7 man |
I recommend looking through the man page of each of the commands that follow, just to get a feel for what the commands can do, and what their various switches are.
You may have noticed that the man pages are displayed a page at a time - that is due to the influence of the "less" command that we will do in the next few pages of this course. This is a good example of the Unix paradigm where small tools are used together to make more complicated ones.
|
Note |
|---|---|
|
You can change which "pager" application is used with man(1) and other utilities by setting the PAGER environment variable in your shell. |
|
You can use the "-k" switch to tell man to search for pages which contain specific keywords:
student@debian:~$ man -k bash bash (1) - GNU Bourne-Again SHell bashbug (1) - report a bug in bash builtins (1) - bash built-in commands, see bash(1) rbash (1) - restricted bash, see bash(1) |
The "apropos" command is the functional equivalent of "man -k".
Syntax: man [chapter] <page> man -k keyword apropos keyword |
.The GNU Project distributes most of its software together with documentation in GNU Texinfo format.
This is another place where you should look for manuals and reference material for software on a Linux system.
You can access these pages by using the " info" command.
You may optionally specify which "info" page you wish to look at as a parameter. Unlike Linux man page, GNU Info pages allow you to use hyperlinks, much like you are used to using in your web browser.
Inside the GNU info reader, you can use the arrow keys to move the cursor, and can use the <enter> key to select which hyperlink you wish to follow. Other useful keys are:
q - quit
n - next page
p - previous page
Syntax: info [page] |
What's the current date and time on the system?
student@debian:~$<indexterm><primary>date</primary> </indexterm> date Thu Jan 15 16:05:07 SAST 2004 |
Syntax: date |
Want to see a pretty calendar of the current month?
student@debian:~$ cal January 2004
January 2004
Su Mo Tu We Th Fr Sa
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
student@debian:~$ _
|
Syntax: cal [[month] year] |
The "ls" command, short for "list directory contents", displays a list of files and directories within your current directory.
student@debian:~$ ls dataset student@debian:~$ one.txt. two.txt |
The "ls" command has several switches which modify its behaviour.
The "-l" (long) switch will display additional information about each of the items that it lists; these include the file permissions, owner, group, size and date of last modification.
student@debian:~$ ls -l total 2 drwxr-xr-x 2 student student 4096 Feb 19 03:10 dataset drwxr-xr-x 2 student student 4096 Feb 19 03:10 dataset2 |
The "-a" (all) switch causes "ls" to display even hidden files. Under Linux, any file that begins with a period (.) is considered to be a hidden file. These files are not displayed in a "ls" listing unless the "-a" switch has been specified
student@debian:~$ ls -a . .. .bash_history .bash_profile .bashrc dataset dataset 2 |
These files are often referred to as "dot files", and usually contain application configuration information particular to the user whose home directory they reside in.
The reason for hiding them is to free up your workspace from the clutter it creates, thus allowing you to more easily access your data files.
Syntax:
ls [-la]
|
The "pwd" command, short for "print working directory" will print out the name of your current directory.
student@debian:~$ pwd
/home/student
|
|
Note |
|---|---|
|
if a command is a shell built in command, you may have to look at the man pages for the shell to find a write-up on the built-in command. (man bash) |
|
Syntax:
pwd
|
You can use the "cd" (change directory) command to navigate your way around the filesystem.
There are two special types of directories.
The "." directory is an alias for your current directory, and the ".." is an alias for the parent to your current directory.
The "cd" command without any arguments will return you to your home directory from wherever you are.
student@debian:~$ pwd
/home/student
student@debian:~$ cd ..
student@debian:/home$ pwd
/home
student@debian:/home$ cd
student@debian:~$ pwd
/home/student student@debian:~$ _
|
You can specify the path for "cd" to change into as being either an "absolute" one, or a "relative" one.
An absolute (full) path begins with a slash ("/"), and indicates the absolute location of something on the filesystem, by specifying it from the root directory up.
A relative (partial) path does not begin with a slash, and merely indicates a location off the current branch of the filesystem. In other words, the new directory is being specified relative to the current one.
Relative path example:
student@debian:~$ cd ..
student@debian:/home$ ls -l
drwxrwxr-x 1 student group 16 student
student@debian:/home$ cd student
student@debian:~$ pwd
/home/student
|
Absolute path example:
student@debian:~$ pwd /home/student student@debian:~$ cd /tmp student@debian:~$ pwd /tmp student@debian:~$ cd /home student@debian:/home$ cd /home/student student@debian:~$ pwd /home/student |
As you can see, both perform equivalent operations. However, relative paths are usually shorter to type
Question to make you think: Is the special ".." path a relative or an absolute one? What about the "." path?
A useful "cd" shortcut to learn is to use the dash (-) parameter.
This allows you to quickly change back to your most recent directory.
student@debian:~$ pwd
/home/student
student@debian:~$ cd /usr/local/src --Using an absolute pathname --
student@debian:/usr/local.src$ pwd
/usr/local/src
student@debian:/usr/local.src$ cd -
/home/student
student@debian:~$ pwd
/home/student
student@debian:~$ _
|
Syntax:
cd [path]
|
The "cat" command, short for concatenate, is most often used to display the contents of short text files.
student@debian:~$
<indexterm><primary/>
</indexterm>cat /etc/passwd
root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:100:sync:/bin:/bin/sync games:x:5:100:games:/usr/games:/bin/sh
man:x:6:100:man:/var/cache/man:/bin/sh lp:x:7:7:lp:/var/spool/lpd:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh news:x:9:9:news:/var/spool/news:/bin/sh
uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh proxy:x:13:13:proxy:/bin:/bin/sh
postgres:x:31:32:postgres:/var/lib/postgres:/bin/sh
www-data:x:33:33:www-data:/var/www:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh
operator:x:37:37:Operator:/var:/bin/sh
list:x:38:38:SmartList:/var/list:/bin/sh irc:x:39:39:ircd:/var:/bin/sh
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh
nobody:x:65534:65534:nobody:/home:/bin/sh guest:x:1000:1000:Debian
student:x:1000:1000:Student User,,,:/home/guest:/bin/bash
identd:x:100:65534::/var/run/identd:/bin/false
sshd:x:101:65534::/var/run/sshd:/bin/false
|
So why is it short for "concatenate"? Because it can output several files all at once:
student@debian:~$ pwd /home/student student@debian:~$ ls dataset one.txt. two.txt student@debian:~$ cd dataset student@debian:~/dataset$ ls one.txt two.txt student@debian:~/dataset$ cat one.txt The coldsleep itself was dreamless. Three days ago they had been getting ready to leave, and now they were here. Little Jefri complained about missing all the action, but Johanna Olsndot was glad she'd been asleep; she had known some of the grownups on the other ship. -- A Fire Upon the Deep, Vernor Vinge (pg 11) student@debian:~/dataset$ cat two.txt An hour's difference either way and Peregrine Wickwrack- rum's life would have been very different. -- A Fire Upon the Deep, Vernor Vinge (pg 17) student@debian:~/dataset$ cat one.txt two.txt The coldsleep itself was dreamless. Three days ago they had been getting ready to leave, and now they were here. Little Jefri complained about missing all the action, but Johanna Olsndot was glad she'd been asleep; she had known some of the grownups on the other ship. -- A Fire Upon the Deep, Vernor Vinge (pg 11) An hour's difference either way and Peregrine Wickwrack- rum&apos;s life would have been very different. -- A Fire Upon the Deep, Vernor Vinge (pg 17) student@debian:~/dataset$ _ |
Syntax: cat [file1 file2 ... ] |
What if you want to view a longer text file? You can use a utility called a pager to do this. Two of the most common ones available on Linux based operating systems are "more" and "less".
The "more" command is very simple, and only allows you to page down a line at a time using the <enter> key, or 24 lines at a time using the <spacebar> key. Hitting "q" will return you to the shell prompt.
The "less" command is the successor to "more", and has more features. You can use the arrow keys to navigate your way around the document.
You can also perform a search my pressing the "/" key, followed by the string which you wish to search for, and finally followed by the "<enter>" key.
To search for the next occurrence of the same string, you can simply press "/" followed by "<enter>". To search backwards from your current position, you can use the "?" instead of a "/".
Pressing the "h" key will display the help page for "less", which contains several more keystrokes which you may find useful.
Syntax: less [file1 ...] more [file1 ... ] |
Now you have almost all the tools at your disposal to read all the documentation that comes with your Linux distribution!
The "ps" command will give your a listing detailing your current "process status", listing your processes which are currently running on the system on your current terminal.
student@debian:~$ <indexterm><primary>ps</primary>
</indexterm>ps
PID TTY TIME CMD
2700 pts/1 00:00:00 bash
3034 pts/1 00:00:00 ps
|
Syntax: ps |
|
|
 |
|
| Logging into a Linux System |  |
Files and Directories |