Build A Computer Grid Using Globus

For the past five months, we have been discussing various types of clustering technologies and how they can be implemented on various platforms. These range from OpenMosix for high-performance clustering to Condor-based high-throughput clustering. It covered Network Load Balancing on Windows to Virtual Servers in Linux. This time, we’ll look at something that goes beyond clustering. It’s a middleware for a clustering infrastructure. Sounds interesting? Let’s try to understand what it is.

This middleware ‘Globus’ provides better and transparent communication between your cluster and jobs.

A middleware, as the name suggests, is a layer that provides improved communication between two other layers, which are your cluster and your jobs. The jobs can be transferred and submitted to geographically dispersed locations in a secure and robust way using any ordinary Web/Application server. The software is from the Globus Alliance and is basically called the Globus Toolkit. It has all the necessary utilities to submit, analyze and manage your jobs from a pool. The beauty of the software is that you can also use any other clustering software’s interface for submitting jobs to Globus, a case in point being Condor. As this is a very large project and beyond the scope of a two-page article, we’ll talk about what it’s all about and how it can be implemented. You can expect more on it in the coming issues. 

The list of sample Globus jobs being hosted by Tomcat. Computers can access these jobs, process them locally and submit their results back

What is Globus?



The philosophy of the Globus project is very interesting. Its main goal is to use the World Wide Web as an infrastructure for sharing computational power and resources, just as you would share information over the net, which is something we’re all comfortable doing. Below is a line taken from the Globas site (http://www.grid-support.ac.uk/ starterkits/whatisglobus.htm), which reflects the exact goal of the project

‘We can hardly imagine the types of applications we might construct if access to supercomputers, live satellite imagery and mass storage were as straightforward as access to the Web. The Globus Project is developing the technology that can make this vision a reality.’

Installing the Globus tool-kit



To install the Globus tool-kit, first copy the gt3.0.1-linux-installation.tar.gz file from this month’s PCQXtream CD to your computer and run the following commands to unzip and untar it.

#tar —zxvf gt3.0.1-linux-installation.tar.gz

Globus is a middleware that provides better and transparent communication between your cluster and jobs

This will create a new folder called gt3.0.1-linux-installation. Keep this folder aside and first sort out the dependencies. The dependencies for Globus are: ANT, J2SDK and optionally Tomcat. Globus has its own container, but we preferred to use Tomcat for our testing, since it’s easier to install and configure. First check whether all the dependencies are present and if not, you’ll need to get them either from the Net or from an earlier PCQuest CD. You’ll find J2SDK 1.4 in last month’s PCQuest DVD and also in the July 2003 CD. You can get the latest version of ANT from www.ant. apache.org, and Tomcat from http://www.jakarta. apache.org/tomcat. To install them just unzip and untar them after downloading as follows:

#tar —zxvf apache-ant-1.5.4-bin.tar.gz

This will create a directory called apache-ant-1.5.4. Untar tomcat: 






#tar —zxvf tomcat-4.1.27.tar.gz

and install the J2SDK 1.4.1 by running first the bin file and then installing the newly created rpm like this:






#./j2sdk-1_4_1_01-linux-i586-rpm.bin 


#rpm —ivh j2sdk-1_4_1_01-fsc-linux-i586.rpm

Now export the paths of ANT and J2SDK by entering the following lines in your user’s .bash_profile file and finally relogin. 






export JAVA_HOME=/usr/java/j2sdk1.4.1_01


export ANT_HOME=/apache-ant-1.5.4 #replace wit

#where you untared ANT 



Also append /usr/java/j2sdk1. 4.1_01/bin at the end of the PATH variable.

Now create a new directory called globroot or any other name you want, and export its path as GLOBUS_HOME and OGSA_HOME as follows:

export GLOBUS_HOME=/globroot 



export OGSA_HOME=/globroot

Go to the directory where you have untarred gt3 and run the installation like this:

#cd gt3.0.1-linux-installer 



#./install-gt3 /gobroot

This will take some time to unpack and install all the tools. Your Globus toolkit is now installed. 

Configuring Globus



You first need one Application server to work as a Globus container. To do so first run tomcat like this:

#TOMCATHOME/bin/startup

and then go to globroot directory and run the following command:






#cd $GLOBUS_HOME


#ant -Dtomcat.dir=/TOCATHOME/ deployTomcat

Now restart your Tomcat and you will find that the container is running. To check whether it’s running or not, open your favorite browser and go to the address http://127.0.0.1:8080/ogsa. Here, you’ll find the AXIS main page of ogsa (Open Grid Services Architecture). Click on the visit link and you will find the sample services here.

These are the Globus jobs, which are hosted by Tomcat or any other application server. They can be accessed by any machine on your network or the Internet through a Web browser. What this means is that if you create compute intensive application, then you can post it to the server running Globus. It will be hosted by the application server (Tomcat in our case) running on that server. Other machines can then access this application from a Web browser and do the necessary processing locally. Finally, they can submit the results back to the Globus server. The application has to be created into multiple parts, so that different computers take up different jobs. Globus will then aggregate all the results from the different machines. We will demonstrate how Globus handles these various jobs in the next issue.