Installing Software on the Cluster

  1. Goals
  2. Overview
  3. Wielding Powers
  4. Nitty Gritty Details
    1. Scripts
    2. Simple Binaries
    3. Complicated Applications
  5. Final Steps

Goals

We would all like to have easy access to every piece of software ever written.  Wide access and easy access are somewhat conflicting goals.  For the greatest amount of software to be installed we must all share the tasks of locating, (writing!), and installing software.  You will be more likely to actually install software if it is easy to install, but to make the software easy to use its installation must be a little complicated.

The complication in the lab is that we have two different kinds of computers, but we want to be able to switch between the two types without having to remember the details of which programs run on what system.  When you install software you must ensure that you have installed it to run on both Silicon Graphics (SGI) and Alpha computers.  Most software can be run on either system.

The greatest temptation is to just do enough work so that you can run the program.  If you do not install the software on both hardware types you will just push that work onto someone else, and finishing the installation will be much harder for them because they will not be familiar with the software.

You cannot install software for general use without being granted special permission by the system manager.  If you want to install some software ask me to grant your account the additional powers.

Overview

I have tried to design the file systems on the cluster to make the installation of software easy.  The configuration should be able to accommodate many different styles of program packages.  Your major task will be to look over the software to be installed and figure out where it fits on our system.

Some files can be shared between the SGI's and the Alphas while others must be customized to the exact hardware of the computer system.  Scripts are examples of the former and program binaries examples of the latter.  The file systems on the Cluster have been designed in reflection of this difference.

Layout of /usr/local and /usr/common
Files which cannot be shared by the two types of computer are placed in the /usr/local file system.  Files which can be shared are stored in the /usr/common file system.  The "path" includes references to "/usr/local/bin:/usr/common/bin".  You will note that there are two, different, /usr/local file systems.  Even though the names are the same, the version of /usr/local on the SGI's is a different set of files than the /usr/local on the Alphas.

One point to remember is that "bin" directories are where executable scripts or binaries are to be placed while "etc" directories contain configuration files, usually data files of some sort.

Some programs need to be installed as an entire directory tree.  In such a case you will have to add that tree to either /usr/common or both /usr/local's.  If all else is the same it is better to install something once than to install it twice.  Try to put software in /usr/common unless there is no way to make it fit.

Wielding Powers

The power you must be granted before you can change files in these areas is to use the "newgrp" command to change your group to "install".  All of the customizable files in /usr/common and the /usr/local's should be available to read, write, and execute access by members of group "install".  Files you create will belong to you but others in the "install" group will be able to modify them.

Here is the standard procedure

  1. Execute "newgrp install"
  2. Do what is required to install your software.
  3. Ensure that all files have the access mode "g+rxw".  This can be changed with the "chmod" command.
  4. Ensure that all files which have to be accessed by a general user have the access mode "o+rx".  This will include scripts, program binaries, data files, and directories.
  5. Execute "exit" to restore your normal group.

Nitty Gritty Details

Scripts

If you want to make some nifty script available to everyone copy it to /usr/common/bin.  A script is something written in sh, csh, tcsh, bash, perl, and a bunch of other languages.  A script can be distinguished because they can be written to the screen with "cat" or "more".  You may not understand anything more than the comments but you will be able to see that it is composed of the normal character set.

Some scripts that you may find have not been written portably.  It is odd but true that many Unix commands behave a little different on the two types of computers we have.  When you install a script you should test it on both kinds of computers.  A problem can often be fixed with only a little work.  It you are having difficulties ask one of the people in the lab with more experience in programming.

In some cases a script cannot be written that will work on both SGI's and Alphas because of small variation.  Instead of making two copies of the script, which only differ by this small amount, we have created some environment variables that you can use in the script to switch between the two modes.  The symbol is HOSTTYPE and it will be equal to either "alpha" or "iris4d".  Someday there may be additional possibilities.  You should not assume, "if is is not an alpha it must be an iris4d".  If the value of HOSTTYPE is unrecognized you should either be very sure that the code will work, or flag an error.
 

Simple Binaries

A simple binary is a program which was written in some computer language, like Fortran or C, which must be compiled before it is used.  It is considered simple because the entire application can be contained in a single file.  Binary programs are always specific to the computer hardware and must be stored in /usr/local/bin.  You must, however compile the program twice, once on an SGI and once on an Alpha.

If you are putting code in a system area we would like to have access to the source code.  This will likely be a collection of files and needs to be stored in a separate subdirectory or directory tree.  The convention in the lab is to place the source code in a subdirectory of the "bin" directory and give it the name of the binary, but with all upper-case letters.  I like to keep the number of copies of files to a minimum, just to avoid duplication and confusion.  If it makes sense for your software, I would prefer that the source code be placed in a subdirectory of /usr/common/bin.  There will then be a single copy accessible on all computers.
 

Complicated Applications

Some applications require a host of files.  They are made up of a number of scripts, binaries, and data files.  These will usually be placed in a single directory tree which cannot be modified easily. Where you put it will depend on how the application itself handles different architectures.  Some packages are designed to embody several hardware types in a single directory while other do not.

If your package can accommodate both the SGIs and Alphas in a single directory, install it in /usr/common.  If your package cannot, you will have to make two copies and place each in the proper /usr/local.

These options may be clearer when specific examples are considered.  The TNT refinement package must be present in a coherent directory tree, however it cannot accommodate different hardware types.  It is installed on our computers in /usr/local/tnt and there are two, full copies.  This configuration works but it cumbersome to maintain, because when a bug is found the correction must be entered twice.

An example of a program package which can live in two worlds is the software for printing on the color Techtronics printer.  This software consists of scripts, binaries, and data, however the scripts and data are not dependent upon computer type.  This software is installed in /usr/common/phaser, although some of the scripts are installed in /usr/common/bin.  (The scripts were placed in /usr/common/bin in order for the existing path to cover them.  Everything you type is slowed down more and more as the path gets longer and longer.  Please try to keep the path short.)
 

Final Steps

If all you have done is to place some files in /usr/common/bin or /usr/local/bin you are done.  If you were required to create a new directory you must ensure that people can run the program.  You may have to add your directory to every user's path or make other changes to their environment.  These customizations are done in the file /usr/local/etc/csh.login.  When you look in this file (and remember, there are two of them) you will see that it has been created by many hands.  The installation instructions for your software should indicate what needs to be done.  Certainly I cannot predict.