Richlab UNIX-based Computing Resources Tutorial

Contents
1.0. Introduction
2.0. High-level UNIX Overview
2.1. UNIX Today
2.2. Modular Architecture
2.3. UNIX is a Multiuser System
2.4. UNIX Supports Remote Connectivity

3.0. Accessing UNIX Computing Resources
3.1. Basic Understanding of UNIX Required
3.2. Accessing UNIX Resources from Windows: Cygwin and Cygwin's X Server
3.3. SSH Secure Shell from Cygwin
3.4. SSH Secure Shell from Linux or UNIX
3.5. SSH Secure Shell from Apple OS X
3.6. Running Software on the Remote Server
3.7. Performance

4.0. Secure UNIX File Transfer
4.1. Secure Copy (scp)
4.2. Secure File Transfer Protocol (sftp)

5.0. Using the vi Editor

Part 2: The KDE Environment


1.0. Introduction

Many people today are sufficiently familiar with Windows or Macintosh computers to be able to use them at work or at home without special instruction. Most people are not, at least initially, familiar with the powerful and secure UNIX computing resources used in academic research laboratories such as this one.

The first aim of this tutorial is to provide a very high-level summary of UNIX today and some architectural aspects of UNIX-based operating systems which are key to understanding the flexibility and power of these operating systems.

The second aim of this tutorial is to help you understand how to access the secure UNIX-based computing resources available in this lab. While the examples shown are specific to this lab, you will find similar protocols and programs in use in any modern, security-conscious UNIX computing environment.

The third aim of this tutorial is to provide a brief introduction to a modern Linux graphical desktop and demonstrate how KDE, Konqueror, Open Office, and other Linux software can meet many common office computing requirements that, until recently, required Microsoft software on a Windows or Macintosh workstation.

2.0. High-level UNIX Overview

2.1. UNIX Today

There are many UNIX-based operating systems in existance today. Of these, two of the currently most significant, and the two which you will encounter in this lab, are Linux and Apple's OS X.

Linux is an entirely free, Open Source operating system, created by hundreds of programmers collaborating across the Internet. Using "free" in the context of describing Linux has two important connotations. First, "free" means that Linux doesn't cost money. People with a lot of time and knowledge of how to put all the peices together download Linux from the internet for free everyday. However, most people find it much more convenient to purchase a packaged distribution from a vendor such as Mandrake, Redhat or SuSE. The second meaning of "free" in this context connotes "freedom" -- both freedom of speech as guaranteed by the First Amendment of the Constitution in the United States, and freedom from restrictive usage licenses.

"Open Source" refers to the fact that the source code (i.e., instructions) used to create the software is openly available on the internet for anybody to review, criticize, or improve. This has important ramifications on the security and stability of software. Because so many eyes are looking at the source code, bugs in Open Source software are repaired much more quickly than in proprietary systems such as Windows or Solaris. If the software is well-written to begin with, this tends to lead to greater security and stability.

"Open Source" is an English-language term to describe this kind of software. In Latin America and other parts of the world you may hear the term software libre.

Apple's OS X represents a combination of Open Source and proprietary software. The core of the system, called the kernel, is an Open Source project named Darwin. On top of Darwin, Apple has placed a number of their proprietary technologies.

2.2. Modular Architecture

UNIX-based operating systems consist of modular layers of software, like an onion. Every layer has a clean, well-defined interface separating it from other layers. This means that one can peel off layers that are not needed, create enhanced replacements for existing layers, or add additional layers as needed. This architecture can be contrasted with that of Windows and certain other operating systems where numerous dependencies between the various "layers" make it impractical or impossible to separate the various components of the system.

The core of a UNIX-based operating system is called the kernel (Fig. 1). In non-graphical console mode, a user interacts with the system via a shell, which you can think of as a layer over the kernel. In light of the onion analogy, it should not surprise you that this software is called a "shell"! If you are familiar with DOS, you may have the impression that a shell session looks like a DOS session on a PC.

The original UNIX shell was called the Bourne Shell, or just sh. The modular architecture of UNIX provided many opportunities for programmers to write improvements and replacements to the original shell, with the result that today there are numerous shell variants, including C Shell csh, Korn Shell ksh, ash, tcsh, and others ... all the way out to zsh!

Compared to something like Windows, this plurality of options, as exemplified by shell software, makes UNIX appear very intimidating. And, it doesn't stop at shells! There are also numerous options for graphical window managers, desktops, and every other kind of software you can imagine. This kind of flexibility is also at the heart of why UNIX is so powerful.

In addition to shells, modern UNIX also has a graphics system called X Windows. As illustrated in Fig. 1, you can think of the X windowing system as a layer over the kernel which provides basic graphics services to the user.

Fig. 1. Modular architecture of UNIX. UNIX-based operating systems are built from modular layers of software with clearly-defined interfaces between each layer. This figure shows just two layers that may typically be found running on top of the core, or kernel: X Windows and the KDE Desktop.

X Windows without a window manager is, as shown in Fig. 1, a pretty drab affair that exposes little of its inherint functionality to the end user. To achieve the kind of functionality that an end user expects, a window manager like Openbox or a combination of a window manager and desktop manager like KDE or Gnome usually runs on top of X. Fig. 1 shows the KDE desktop which is arguably the most advanced among those available on UNIX, and is what you will find on the Linux workstations in the lab.

2.3. UNIX is a Multiuser System

Another key concept about UNIX is that it is a multiuser operating system. It is not unusual to have hundreds of users logged onto a single machine at the same time (Fig. 2). This contrasts with Windows where only one user can be logged on to the system at a time. Even Windows servers only support a single logged-in user.

tetris% who | wc -l
205
tetris%
Fig. 2. Multiple Users. A UNIX machine is typically used by many people simultaneously. The command shown above displays the number of users logged into this machine: 205.

2.4. UNIX Supports Remote Connectivity.

To support hundreds of simultaneous users, UNIX systems provide advanced facilities for remote connectivity. Not only is it possible to log into a remote UNIX machine from another machine over a network connection and conduct a remote shell session, X Windows features a client-server architecture which provides the ability to export the graphical display of a remotely-executing program to your local display manager (Fig. 3).

Fig. 3. Remote X Display. X Windows features a client-server architecture which provides the ability to export the graphical display of a remotely-executing program to your local display.

3.0. Accessing UNIX Computing Resources

3.1. Basic understanding of UNIX required

In order to use the UNIX-based computational resources of this lab efficiently, you need to have a basic comfort level with UNIX and the UNIX command-line environment. You need to understand how to login and logout, change your password, navigate directories, and start and terminate programs from a UNIX shell. For coverage of these basic topics, please refer to the following excellent tutorial from the University of Surrey, UNIX Tutorial for Beginners.

The rest of this tutorial assumes that you have a basic understanding of UNIX, are familiar with terms like home directory, shell, telnet, redirecting output, and file permissions, and know what ls, cp, cd, pwd, and less do. If any of these terms or utility names are unfamiliar, please study this first.

3.2. Accessing UNIX Resources from Windows: Cygwin and Cygwin's X Server

If you need to access UNIX resources from a Windows environment, Cygwin is arguably the best, and certainly the lowest-cost, solution.

Cygwin is a suite of GNU UNIX tools that have been ported to the Microsoft Windows environment. Cygwin is now owned and distributed by Redhat. These freely-downloadable tools are indispensible for providing connectivity between Windows and UNIX or Linux hosts.

All Richlab Windows workstations now have icons available on the desktop for accessing Cygwin and Cygwin's port of XFree86's X Windows software (Fig. 4).

Fig. 4. Desktop Icons. All Richlab Windows workstations now have icons on the desktop to start Cygwin's command-line client and Cygwin's port of XFree86's X Windows.

Clicking on the X Windows icon will start X Windows immediately, while clicking on the Cygwin icon will bring up Cygwin's command-line client (Fig. 5). The command-line client may remind you of DOS, but provides a UNIX Bourne-Again Shell (bash) and access to a substantial set of UNIX command-line utilities.

Fig. 5. Cygwin Command-line Client. The Cygwin client, looking a bit like a DOS window, provides an indispensible subset of UNIX utilities in a Windows environment. Type startx to start the X windows server.

Cygwin's command-line client is sufficient for accessing console-based UNIX programs such as the email client, Pine. However, you will need to start the X server in order to access any graphical UNIX programs. As shown, you can start X Windows by simply typing startx and pressing return.

The X server has been set up with a window manager called Openbox. Compared to the KDE or Gnome desktop on Linux, Openbox appears minimalistic, but it is functional.

On most of the Richlab Windows workstations, the initial Openbox desktop will appear empty. Click with the right mouse button to display Openbox's root menu. From this menu, choose xterm to start a terminal session (Fig. 6).

Fig. 6. XFree86 with Openbox on Windows. The Richlab Windows workstations have been setup to run a X11 window manager called Openbox. Compared to the Linux desktops KDE and Gnome, Openbox is certainly minimalistic, but functional. Use the right mouse button to display Openbox's root menu and choose xterm.

(Note that Openbox's root menu shows entries for Star Office, Mozilla, and AcroRead (Adobe's Acrobat Reader for UNIX): these programs are not actually available. Of course, they would be available if Openbox were running in the Linux environment for which it was designed. But since this is actually Cygwin -- and hence Windows-- these programs are not actually there. However, this is of no concern to us).

3.3. SSH Secure Shell from Cygwin

The Richlab Linux workstations and servers have been set up to refuse telnet connection requests because telnet is an insecure protocol. In telnet, user passwords are sent over the network in an unencrypted format. That is an unacceptable security risk.

SSH, or Secure Shell, is a modern replacement for telnet which encrypts all communications between a client and server. Cygwin and all Richlab UNIX/Linux workstations use OpenSSH for secure communications. OpenSSH is started from an xterm shell by typing ssh.

As with most UNIX utilities, many command line options exist. An example of the syntax that you will need to connect to a UNIX server using OpenSSH from within Cygwin is shown in Fig. 7:

Fig. 7. Using Secure Shell. Secure Shell, ssh, replaces the insecure telnet protocol in modern, secured UNIX environments. The details of using OpenSSH are described in the text.

The -2 signifies that you want to use the newer SSH protocol version 2 which is considered more secure than the older version 1 protocol. The -X is required to forward the X11 connections. Without this, you cannot run remote graphical programs. The full URL name of the machine we wish to connect to is provided next. Finally, we need to tell ssh the username to use to login to the server. This is done with the -l (letter el for login) option followed by the user's login name.

3.4. SSH Secure Shell from Linux or UNIX

Things would be simpler if we were accessing the Eyegene server from a Richlab Linux or other UNIX workstation instead of from a Windows workstation. First, the Richlab Linux workstations are set up to use SSH protocol version 2 by default, so we would not need to type "-2". Secondly, the Richlab Linux workstations are setup to forward X11 connections by default, so we could also dispense with "-X". Finally, a user's login name on a Linux workstation is normally identical to his or her login name on the remote server, so the "-l <username>" could, in most cases, also be dispensed with. The command line from Fig. 4 above would now look like that in Fig. 8:

edtrager@nix_workstation:~> ssh eyegene.ophthy.med.umich.edu
Fig. 8. Example Linux SSH command line. In the right UNIX environment, very few or no additional command-line
parameters may be necessary when invoking ssh. See text for discussion.

For a fuller discussion of ssh's command line options, refer to this manual page. For common usage, the options explained from Fig. 7 and Fig. 8 above should be sufficient.

3.5 SSH Secure Shell from Apple OS X

Apple's OS X operating system is UNIX-based, but the graphical user interface uses Apple's proprietary technology which is not based on X. As a result, it is necessary to manually start an X server on OS X before connecting to a remote UNIX server in order to be able to run remote graphical programs. This procedure is akin to that required when using Cygwin on Windows.

The Richlab Apple computers running OS X have a port of XFree86 called XDarwin installed. All you need to do is start the X server in rootless mode (Fig. 9).

Fig. 9. Starting XDarwin under OS X. Click on the XDarwin icon on the program menu or on your taskbar and choose "rootless".

By default, this will bring up xclock and several xterm windows (Fig. 10). X on the Apple computers is currently set up to use another minimalistic window manager called Oroborus:

Fig. 10. XTerm and XClock under Oroborus. Once you have an xterm window under XDarwin, you can ssh to a remote server just as you would under Linux.

Once an xterm terminal appears, you can ssh to a remote machine using the same procedure as you would for Linux (see Fig. 8). Since the configuration of ssh has been set identically to that on the Linux workstations, there should be no need to specify X11 forwarding or other command-line parameters.

3.6 Running Software on the Remote Server

If you have followed the directions shown in this document to log into a remote server using ssh from a Cygwin, Linux, or Apple OS X xterm, you can now simply type the name of the software you want to run on bash command line.

Depending on the speed of your network connection, you may need to wait a few moments for the software to appear on your local X display. Once it comes up, however, you should be able to interact with the software exactly as if you were sitting at the console of the remote server (Fig. 11).

Fig. 11. Running Remote UNIX Clients. Once you have logged into a remote UNIX machine using SSH with X11 forwarding, you can run any program on the remote machine just as if you were sitting at the machine's console. In this figure, the graphical sequence editor Consed is shown running from a remote connection to the Eyegene server.

3.7. Performance

If you are accessing a server from a workstation anywhere within the University network, you will probably not even notice the network latency associated with a remote X session. If you are using a remote DSL connection, expect to wait a long time before even seeing a window appear on your computer screen. If you are using a dial up connection, stick to command-line utilities and avoid graphical programs altogether if possible.

4.0. Secure UNIX File Transfer

People employ various means to copy files between networked computers. Most methods, like ftp and email, were not designed with security in mind. Two methods are available for secure file transfer between networked UNIX machines: scp and sftp. Both of these programs are usually distributed as part of the Secure Shell package (e.g., OpenSSH on Linux, Cygwin, and Apple OSX). Usage of these are described below.

4.1. Secure Copy (scp)

The secure copy program, scp, is modeled on the original UNIX cp program. The syntax of UNIX cp and scp are shown in Fig. 12:

cp <SOURCE> <DESTINATION>
scp <SOURCE> <DESTINATION>
Fig. 12. cp and scp both copy files from a source to a destination.

In the case of scp, either the source or the destination, or both can refer to remote files.

Because scp operates in a networked environment, one must specify a username and machine URL in addition to the path of a remote file using the syntax shown in Fig. 13. Note how the username is separated from the machine's URL using the "@" character. A colon ":" is used to demarcate the file path from the machine URL. You can specify a file path relative to the default home directory of the user, or provide an absolute file path:

Syntax: <USERNAME>@<MACHINE_URL>:<FILE_PATH>
Example: edtrager@myhost.mysubnet.org:/usr/local/share/data.tar.gz
Fig. 13. Using scp requires specifying a username and machine URL in addition to the path of the remote file(s).

Some examples are shown in Fig. 14.

Copy /etc/printcap from the local machine to a new machine, "newhost", in the med.umich subdomain. The destination requires superuser privileges. localhost> scp /etc/printcap root@newhost.med.umich.edu:/etc/printcap
root@newhost.med.umich.edu's password: *******
localhost>
Copy a data file from one's home directory on a remote server to the home directory on the local machine. Maintain the file's original name. localhost> scp edtrager@eyegene.ophthy.med.umich.edu:mydata.txt .
edtrager@eyegene.ophthy.med.umich.edu's password: *******
localhost>
Copy a file "research.draft" in John's "genes" subdirectory on remote server "alpha" to a new name, "research.final" in Smith's "markers" subdirectory on another remote server, "beta". This copy will require supplying two passwords: first for the user "john" on alpha, and secondly for the user "smith" on beta. (Sharing passwords is not a good idea. For the sake of example, we assume that "john" and "smith" are one and the same person, but have different usernames on the two machines). localhost> scp john@alpha.med.umich.edu:genes/research.draft smith@beta.med.umich.edu:markers/research.final
john@alpha.med.umich.edu's password: *******
smith@beta.med.umich.edu's password: *******
localhost>
Fig. 14. Some examples of using scp to transfer files between machines.

Scp is probably the fastest way to transfer an individual file between machines when you are sure of the source and destination file names. If you are not sure about the file names or plan to transfer more than one file, the more interactive sftp command described below may be preferred.

4.2. Secure File Transfer Protocol (sftp)

Sftp works almost identically to an interactive UNIX command-line ftp program. Commands like "pwd" (present working directory), "cd" (change directory), and "ls" basically work like their UNIX shell counterparts. To grab a file from a remote machine, use the "get" command. To place a file on a remote machine, use the "put" command. An example session is shown in Fig. 15.

edtrager@richlab:~> cd images
edtrager@richlab:~/images> sftp eyegene.ophthy.med.umich.edu:
Connecting to eyegene.ophthy.med.umich.edu...
edtrager@eyegene.ophthy.med.umich.edu's password:
Changing to: /home/edtrager/
sftp> cd /images
sftp> ls
drwxrwxr-t    3 root     users        1608 Oct  8 12:44 .
drwxr-xr-x   21 root     root          466 Sep  6 14:28 ..
-rw-r--r--    1 edtrager users       79872 Apr  4  2002 kellogg.jpeg
-rw-r--r--    1 edtrager users      154782 Jan  4  2002 dance1.jpg
-rw-r--r--    1 edtrager users       11707 Nov 26  2001 umseal1.png
-rw-r--r--    1 edtrager users      196304 Jan  4  2002 cattleya_in_chicago_1.jpg
sftp> get umseal1.png
Fetching /images/umseal1.png to umseal1.png
sftp> put holiday.jpeg
Uploading holiday.jpeg to /images/holiday.jpeg
sftp> quit
edtrager@richlab:~/images>           
           
Fig. 15. An example sftp session. Commands like pwd, cd, and ls are similar to their UNIX command-shell counterparts. Use get to download, put to upload files.

5.0. Using the vi Editor

It is true that there are other editors for UNIX, and you may well choose to use those other editors for much of the work that you end up doing on UNIX machines (see this, for example). However, there are many times when you may just need to edit a few lines in an existing file or write a simple script to accomplish a task more quickly. Or you might need to work on somebody else's machine where your favorite non-vi editor is not available. In these cases, the veritable vi editor is the right tool for the job!

For the uninitiated newbie, vi seems unintuitive and a general pain-in-the-arse to use. However, as is true with many things in the UNIX world, once you have crossed the initial usability hurdle and have begun to see its hidden virtues, vi will begin to grow on you. You might even get to the point where you don't want to use anything else but vi!

This has happened to many other people. In fact, Tim O'Reilly provides some evidence that about twice as many UNIX aficionados choose vi as do emacs. So, there must be something in there, but the only way you'll find out is by investing some time in learning vi!

Besides the fact that vi is a powerful editor, it also happens to be the one editor that is guaranteed to be present, in one version or another, on virtually all UNIX machines. Our tutorial below is meant only to provide an introduction to the rudimentary features of the program. Our hope is to provide enough of an introduction so that you won't get "stuck" using vi. A definitive resource on all things related to vi, including various tutorials, may be found on the VI Lovers Home Page.

Vi is invoked from the shell command line by typing: vi name_of_file_to_edit.

Vi operates in two modes: a command mode, and an edit or insert mode. When you first start the program, vi is in command mode. In command mode, vi uses one or two-key combinations to navigate and execute commands on the file being edited. For example, you can move the position of the cursor in vi using the keys h,j,k, and l (Fig. 16).

Fig. 16. The keys h,j,k, and l allow you to move around in vi without ever taking your fingers off the keyboard.

Most modern versions of vi also allow you to navigate using the arrow keys, but many vi users find that never having to take one's fingers off of the letter keys is very convenient.

Once you have navigated to a location you wish to edit, the command keys i (insert), a (append), or o (open) can be used to enter edit mode. i initiates editing at the position preceding the cursor, a at the position following the cursor, and o opens up a new line for inserting text.

Once you have completed inserting text, press the escape key, ESC, to return to command mode. The u (undo) command key can be used to undo the most recent insertion or edit.

A large number of command options begin by typing the colon key, :. For example, to save the file, type :w (write). To save the current edits to a new file called "my_new_file", type :w my_new_file. To quit vi, type :q.

A table of some common vi command key combinations is shown below (Fig. 17).

Navigation
h Move left one character.
j Move down one line.
k Move up one line.
l Move right one character.
b B b: Move back one word, counting various punctuation as words.
B: Move back one word, not counting punctuation as words.
w W w: Move forward one word, counting various punctuation as words.
W: Move forward one word, not counting punctuation as words.
Deleting Text
x Delete one character.
r Replace one character.
d Delete command. See examples below ...
dd Delete the entire current line.
dw Delete a word.
3dw Delete three words, starting with the word at the current cursor position.
d$ Delete from the current cursor position out to the end of the line.
Inserting Text
i Enter insertion mode starting at the current cursor location. i is useful for inserting text into the middle of a line.
a Append. This is nearly identical to i, except that editing begins one character to the right of the current cursor location. a is useful for adding text to the end of a line.
o Open a new line for editing. This is similar to a, except that a new line is inserted below the current line for inserting new text.
Copying Text
y "Yank," (i.e., copy) text. See examples below.
yw Copy one word.
3yw Copy the next three words.
yy Copy a complete line of text.
3yy Copy three lines of text (the current and next two lines of text).
p Put (insert) previously yanked (y) or deleted (d) text after the current cursor location.
P Put (insert) previously yanked (y) or deleted (d) text before the current cursor location.
"byy Yank one line of text and place it into the buffer named "b". 26 named buffers are accessible using the quote mark (") plus a lower-case letter of the Roman alphabet.
"bp Place the text in the named buffer "b" into the document at the current cursor position.
Joining Lines
J Join (i.e. merge) two lines into one.
Searching
/ Search for a pattern. Enter the pattern after the "/" slash key and press return. For example "/gene" will search for the first occurence of the word "gene" in your document.
? Search for a pattern in the reverse direction. Enter the pattern after the "?" slash key and press return.
n N n: Continue searching for the next instance of a pattern.
N: Continue searching for the next instance of a pattern, but reverse the current direction of the search.
Saving and Quitting
:w Write. Save the file.
:q Quit. Quits vi if the file has already been saved.
:q! Quit without saving. The exclamation point overrides vi's directive that prevents quitting without saving.

Vi has many powerful features. One particularly powerful feature is the ability to precede a command with a numeral indicating the number of times you want to repeat the command. For example, the y (yank) command is used to "yank", or copy text. So, for example, yw will copy the one word following the current cursor position, while 3yw will copy the next three words. The p (put) command can then be used to paste the "yanked" text in a new position. Similarly, the command yy will yank a complete line of text, while 3yy will yank the next three lines of text.

Once you become proficient in the use of some of vi's shortcut commands, you will begin to realize just how powerful vi truly is. You should now know enough about vi to be able to start playing around with the program. For more information, refer to one of the online tutorials referenced at The Vi Lovers Home Page, or purchase the O'Reilly book, Learning the vi Editor by Linda Lamb and Arnold Robbins.

Continue on to Part 2, KDE Environment -- >

<-- Back to Eyegene home page

2002.10.18.ET