Clustering Linux Distros 

After the rise in the popularity of Live Linux distributions, a new specialized class of distros came into existence-the clustering/parallel processing distros. These distros are of two major types. One is the standard PVM3 supporting HPC clusters and the other are the load balancing clusters that use OpenMosix at its core.

The biggest reason for the popularity of these distros is the fact that you can turn any heterogeneous network into a cluster within 15 minutes and without even touching whatever else is set up on that system. That is, you boot off the distro's live CD, perform your number crunching and simply reboot off the hard disk to get the machine back into its normal state. 

So to test them out, we downloaded three most popular clustering distributions and tested them against each other. You can get all these distros in our last month's DVD. This article is not a complete shootout in the traditional sense. It is more or less a how-to on the three distros. In the beginning, we wanted to do our regular shootout with them, but then we realized that it would not be possible to compare a load balancing cluster against an HPC cluster. 

So we will first see the difference between HPC and load-balancing clustering distros, and then try to understand how and where each would be used. 

Load Balancing Clusters (LBC)



The biggest benefit of an LBC is that you can share any day-to-day application on different machines. This means that you do not need to write any applications specifically for the cluster. And in an LBC the whole network will be available to you as a single machine with the added processing power and memory of each machine on that network. So in this case you don't need to do any PVM coding to access the extra processing power. The clustering application which is used for this is OpenMosix and is basically a kernel patch. To know more about installing and using OpenMosix, check our May 2005 issue or go online to

https://www.pcquest.com/content/linux/103050601.asp.

High Performance Computing (HPC)



On the other side of the spectrum are PVM (Parallel Virtual Machine) or HPC (High performance computing) clusters. These clusters need special programs using PVM and MPI standards. Such programs can use the power of machines with multiple processors. These multiple processors can be in a single hardware box or can be distributed among multiple machines in a network. But the problem with this is that other than harnessing much more flops per

machine, it cannot be used to run common day-to-day applications.

You can adjust the sliding buttons to set the amount of processing power to be shared with the cluster

So if you have a network where some machines are always overloaded and others have extra resources, you should go for an LBC. On the other hand, if you need a massive number crunching and you can do the required customization in your software or your software is MPI/PVM compliant then you should go for the HPC

distros.

PVM standards



PVM supports software that runs on each machine in a user-configurable pool, and provides a unified, general and powerful computational environment of real-time applications. Users can write programs in C or FORTRAN and can access PVM using calls to PVM library routines for functions such as process initiation, message transmission and reception, and synchronization via barriers or rendezvous. Users can also optionally control the execution location of specific application components. The PVM system can transparently handle message routing, data conversion for incompatible architectures, and other tasks necessary for operation in a heterogeneous network environment. 

PVM is best suited for heterogeneous applications that use specific power of individual machines on a network. PVM is widely used for making parallel supercomputer environments because it can do scientific computing, and are used for applications such as molecular dynamics simulations, superconductivity studies, distributed fractal computations and matrix algorithms. 

What we did



We created a small network with just three nodes and booted each node with the distro. We made one machine the master node and the other two as the client or slave node. Finally we ran linpack (a benchmark which overloads the machine and reads the flop counts) on all the machines to test the number of teraflops on a standard node. The machines were all standard PCs with a single P4 2.8GHz CPU and 512 MB of RAM.

ClusterKnoppix



This was the only distribution of the three that was based on OpenMosix (LBC). This is also the easiest one to configure. PVM/MPI are not supported on ClusterKnoppix and thus, we were not able to run Linpack on it. Still let's see how to configure and use it.

To create the master node, burn a CD of ClusterKnoppix and boot a machine with it. Press Enter at the boot prompt. Within a few minutes, the OS will detect your hardware and boot the machine into a nice KDE desktop. Now open a terminal window and run:

# ifconfig eth0 192.168.3.10 netmask 255.255.255.0 up



# add —net 0.0.0.0 gw 192.168.3.1

On the cluster configuration window, set master or slave according to the set up you have defined

The following command initiates the OpenMosix server.

# tyd —f init 

Adding the slave nodes into the cluster is very easy. Just follow the same steps as mentioned above but give a fresh IP address to each one and finally run the following to add the node. The IP given below is the same as that of the master node.

# tyd —m 192.168.3.10

ClusterKnoppix in dynamic environment



You can also run OpenMosix in a dynamic environment where systems get random IPs from a DHCP server. While this will eat up some more network bandwidth, it saves a lot of work when creating a huge cluster. To do so just run the following command in all the nodes

# omdiscd —n eth0

To monitor your cluster, go to the KDE terminal on any cluster node and run the following command.

# openmosixview

This will pop up a window that shows a list of all the servers in the cluster, the amount of RAM and processor utilization in each. From here you can configure the load-balancing efficiency of the cluster. Each node has a slider bar for load balancing. When you move this bar, OpenMosix automatically adjusts the processes running on each machine. It will automatically migrate them to

another node if the existing node is overloaded. You can also migrate the processes manually from one node (processor) to

another by selecting the Processes option from the toolbar. It will show you a list of the processes that are running. By double clicking on any process, you'll get another window in which you can choose the node you want to migrate the process to.

Tablix 



Based on morphix 0.0.2, this HPC clustering distro is very easy to set up compared to the other HPC distro in the shootout. It

creates clusters on PVM3, which enables a collection of heterogeneous computers to be used as a logical and flexible parallel computational resource. With Tablix, you can share individual computers. Local memory multiprocessors of vector supercomputers, specialized graphics engines or scalar workstations can also be interconnected by a variety of networks, such as Ethernet or

FDDI. 

To setup Tablix, you would need ateast two machines with a sufficient amount of RAM. Then connect these machines on a switched 100 Mbps Ethernet network. Now choose any one of the machines that you would like to use as a control cluster.

To create the master node, boot off the Tablix Live CD. You will get a GUI Cluster Configuration window. Here select 'Master node' as the role of this computer. Then on the same window, check on the 'Start DHCP server on master node' option. By default Tablix sets up 172.16.0.1 as the IP for the master node, but this can be changed. Once you click on OK on this window, a box will pop up saying the master node is waiting for nodes to connect. 

Here Parallel Knoppix is running its own benchmark and plots the performance on a graph

Now come to the slave node and boot off the CD as before. This time, select 'Slave node' instead from the configuration screen. Tablix will automatically add the node to the HPC cluster. Back on the master node, you should see this machine listed in the 'Waiting for connect' box. To test whether the cluster is actually working or not, open the Tablix folder on one of the desktops and start Tablix with 'tablix

file.xml'. 

When we ran Linpack on a Tablix node we got 4.21 teraflops. With 24 such nodes, you can take this upto a 100

teraflops.

Parallel Knoppix 



Parallel Knoppix is another HPC clustering distribution based on Knoppix Live CD. This allows you to create a parallel processing using the LAM-MPI and/or MPICH implementations of MPI. Its implementation is quite easy and doesn't take more than 15 minutes, if the machines have PXE network cards. Here one machine will act as master node and rest as slaves. The master node needs a hard disk.

To implement Parallel Knoppix, boot the machine selected to be the master (with a hard disk) node using the Parallel Knoppix CD. Open KDE>Parallel Knoppix and select 'Setup Parallel Knoppix'. This will launch a script in a terminal window. 

You will get a message 'Terminal Server is started to boot the slave computers' click on OK. On the window to configure the terminal server, click on OK to proceed. Now, you will be asked how many nodes you need in the cluster. Give a number between 2 and 200. Then you will be required to select the network drivers for your cluster nodes. Make sure to select all the network card drivers that you have in the cluster. 

You need to export the CD to a mounted NFS disk space. To do so, select the Disk Option and click on OK. With this, your master node is ready to serve your slave nodes. Boot all the slave nodes using their network cards and they should remotely boot from the terminal server, running on the master node. All slave nodes should be mounted on master's NFS partition. If nodes are not mounted properly, then you won't be able to use that node's processing power. So make sure the configuration you have done above is correct. 

Using Parallel Knoppix 



Here, we will use one of the example files on the Parallel Knoppix CD to check the cluster. Go to the master node and copy the

/home/knoppix/Desktop/ParallelKnoppix/Octave directory to the NFS share. Then open a terminal window and change to this NFS share. Now go to

/parallel_knoppix_working/Octave/kernel directory and issue:

# octave 

It will open another console where you where you have to type-in 'kernel_example1'. It will start drawing an output of kernel regression, both in serial and in parallel.

It achieved 3.17 teraflops by running Linpack on the nodes. So, this distro will require 32 such nodes to achieve 100

teraflops.

Anindya Roy and Sanjay Majumder