Running Linpack on Win Compute Cluster

Continued from page: 2

Friday, November 02, 2007

Porting Linpack for Windows Compute Cluster
Here, we will see how one can port (re-compile) Linpack source in Windows and then run it on Windows Compute Cluster Environment. We tried it and used it for benchmarking our created Microsoft Compute Cluster. But we faced a problem. Basically, Linpack is an application used majorly for testing Linux based clusters and trust me, porting it to run on MSCCS was not at all a child's play.

In this article, we will see how with the help of some tools and libraries, you can recompile the HPL source files in your Windows architecture and run it on the top of MSCCS.

Prerequisites
The list of prerequisite SDKs and libraries is too long, but the first thing that you need is MS Visual Studio 2005. Install it on any of the nodes of your MS Compute Cluster. The compiler is to be installed on one of the nodes because it ensures that you are compiling your application on the right hardware architecture and as a result you'll get better performance.

After this, download both the AMD and Intel's Math Kernel libraries. Download and install the file called 'acml3.5.0-64-win64' from http://tinyurl.com/ 2k6tny. Also download and install the Intel's Math Kernel library use the following link: http://tinyurl.com/2p9m8f.

Now install the MS Compute Cluster SDK from http://tinyurl.com/3yjyg9. Just make sure that you download and install the 64-bit version. Now the installation is done but for Linpack to work properly you'll have to perform some nasty tricks. This is because the makefile that we are going to use for compiling Linpack had a lots of path names hardcoded.

To begin with, first create a folder called “scratch” at C:\ of the node where you have installed all the above mentioned components. Then go to the folders where you have installed ACML and MKL.

By default they will be in the Program Files folder if you did not give any other path. Go to the AMD folder first and rename the ACML3.5.X file as ACML3.0.0. Similarly, go to the Intel's folder and rename 9.1.x as 8.0.1. So, the hacking part is done and we are ready to work on the actual file.

Applications built using Visual Studio with Manifest option enabled, can't be run using MS CCS. Therefore, disable that option before you compile Linpack

Compiling Linpack
Now download the latest version of HPL from http://tinyurl.com/2mopw8. Unzip it in a way that the HPL folder comes under the C:\scratch folder.

In Linux, Linpack uses the make command for compilation. But the makefiles are generally created for different Linux distros and not for the Windows. So, now you have to grab a makefile for Windows. To make our task easier, if we also get a .vcproj file for Linpack then we can use it to compile Linpack directly on VS 2005. You can download all the required components from our forum. The link for the same is http://forums.pcquest.com/forum/viewtopic.php?t=6154&highlight=. Go to this link and download the xphl_port.zip file. Unzip it under the C:\scratch\hpl folder and copy the HPL_timer_walltime.c to the C:\scratch\hpl\testing\timer folder. There will be a file with the same name already sitting in that folder, so while copying replace the old one with the new one.

Double click on the xhpl.vcproj file and open it as a VC++ project in VS 2005. You have to build the project but before that one more thing is required. The VS 2005 while compiling an exe embeds the manifest file inside the exe, which is not recognized by our mpiexec command that finally you have to use for running Linpack. So, you have to tell VS2005 not to embed the manifest file while compiling. To do so, go to the Property page of the xhpl project and click on Manifest Tool> input output and change the value of 'Embed Manifest' from Yes to No. Now close this window and go to the Build menu, and click on the Build Project option to compile Linpack. The exe will be created in the C:\scratch\hpl\bin\ 64\xhpl.exe.

Once you have submitted the job, you can then view the status of the job under the Job Monitor window

Running XHPL
To run XHPL you have to use the Compute Cluster Job Manager. For this go to Program Files> Microsoft Compute Cluster Pack. Then, go to the File> Submit job Menu. This will open up a window. Here provide a descriptive job name and go to the Processors tab. Then select the number of processors that you want to use from your cluster to process your job. Remember, the number you provide should be equal to the number of cores and not the number of physical processors.

Now go to the Tasks tab and in the Command line field, type in the command you want to run. If it's an MPI process that you are going to run (which Linpack is) then the command will be something like 'mpiexec xhpl.exe'. To add tasks click on the Add button. Tasks that have been added will get listed under the task list.

Select the task and click on the Edit button. Here, provide the working directory, and input and output file name. The working directory is essentially the shared location where the xhpl.exe sits and it should look something like Error! Hyperlink reference not valid.

The output file can be any file where you want to get the output of Linpack. By default it is hpl.out. The input file is of course the HPL.dat file. Provide these values and submit the task to get executed.

This will start the xhpl process on all the nodes. But if it fails then you have to modify the hpl.dat file in the bin folder. This is the file where you set all runtime settings for xhpl and from here you can also tune XHPL for performance. Tuning XHPL is a tedious job and it is not possible for me to cover it in these two pages.

While writing this article, I am still trying to figure out how to get the best performance out of our cluster by tuning XHPL. So far, I have achieved some 46 GFlops, but there is still a long way to go. So, when I am done with this tuning, next month I will talk about how to tune XHPL in detail. Till then you can refer to the article hosted at http://tinyurl.com/23q98y.

Page(s)   1  2  3