We have talked about cloud computing on more than one occasion last year. It
has since developed into a buzz word today and lot of big companies are talking
about and deploying it in their enterprises. The main benefit of Cloud Computing
that we have focused on till now is that it gives a more customizable and
granular control over the hardware when we go for a hosted application approach.
We have also talked about some of the pioneers who have deployed Cloud Computing
for providing such on-demand hosted services to their customers. The more well
known ones are Amazon's EC2 and Google's AppEngine. Probably Amazon was the
first one to release their Cloud offerings to the public and hence has become
quite popular in this domain.
If you have read our old issues, all of this would not be news for you. This
time we talk about something completely different. We thought, if Cloud is such
a great technology and gives such a great granular and detailed control over the
Datacenter resources, then why enterprises don't deploy their own cloud setups
and reap those benefits instead of going and buying them from some third party
service provider? And while doing our first level of research we figured out
that, yes, it is actually very much possible to deploy your own cloud in a data
center and by just using commodity hardware and open source middleware. The
benefit you get out of such an approach is immense. We shall also talk about
some of those benefits in this article. But before we go into details, let's
refresh ourselves with what Cloud computing is all about.
Cloud for Enterprises
There are thousands of documents which explain what Cloud Computing is, and it
is such a vast topic that understanding it in great detail requires a lot of
effort. But for those IT Managers and CIOs who don't have that much of time to
invest for learning, we can explain it in a nutshell as 'a cluster of
Virtualization Servers.'
The easiest way of understanding cloud is by understanding why is a cloud
required in the first place. The emergence of cloud computing has happened in
data centers because of the inherent drawback in virtualization infrastructure.
There's a need for true on the fly, on-demand scalability which no
virtualization platform can provide today. You might disagree with me on this
statement because generally virtualization is supposed to be a great enabler for
on the fly resource allocation. But the point to note over here is that a
Virtual machine (VM) cannot be scaled beyond the resources available on the Host
machine. In case the resource requirement of a VM increases beyond the resources
available on the host machine, one needs to migrate it on another virtual
machine with requisite resources. And to top it of, if the resource requirement
of the VM was temporary, then there is no proactive mechanism which can throw
back the VM to the old Host machine, thus saving the resources of the new Host
machine.
And this is where Cloud Computing comes to light. It is also sometimes
referred to as Eclectic Computing because it gives the ability to acquire and
release resources from a unified pool of hardware depending on requirement. To
understand it in detail, let's imagine a scenario where you need to build a web
server which will receive 1000 hits per day for 25 days in a month and 1,00,000
hits for the other five days.
Now if you want to get this server, you have to make sure that the server can
take the maximum possible load. So if you want to host this server elsewhere,
you have to sign an service level agreement (SLA) for 1,00,000 hits per day,
whereas your actual average requirement is far lesser than this. So you end up
paying for something which you don't actually need.
Even if you want to run this server in house, then also you have to buy a server
which can take the load of 1,00,000 hits per day and not the one which can only
take a load of 1000 hits per day, which would be a waste of money.
Just imagine a scenario where each and every server in your data center is
connected via a single middleware which converts it into a huge pool of unified
resources in terms of processors and RAM, etc and you run your servers as
virtual machines on top of it. And depending upon load at a given point of time,
VMs can just acquire the available resources in the pool (to a certain defined
max limit), use it and release it when the job is done. That's what Elastic
Computing is all about. And let's say the complete cloud is utilized to its max,
then all you have to do is to plug in one or more new free servers depending on
the resource requirement and it will automatically add up to the pool of
existing resources.
So, with this thought, we are going to deploy our own Cloud Computing
Infrastructure and run some VMs on top of it. We will be using Amazon's EC2
client for connecting and using the Cloud we create. In other words, we can even
say that we are going to build an EC2 compatible cloud.
Here we go...
To begin with, let's first identify what all we need to build a Cloud. Of
course we need some servers, which will be the nodes of the Cloud, and then we
need a Controller Server, which will manage the complete cloud. To connect all
of these, we will require a network, preferably a Gbps network, and we will need
a clustering and a virtualization middleware.
Here all the software we are going to use is open source or free and is
available for download. And if you are going to do a test setup of cloud for
understanding and research, only two servers will be more than enough for you.
You don't need to worry about spending a huge amount of money in this
deployment. You can just do a test run in your backyard.
Installing the controller
The software which we are going to use as the Cloud middleware is called
Eucalyptus and is an Open Source project managed by UCSB. Eucalyptus stands for
“Elastic Utility Computing Architecture for Linking Your Programs To Useful
Systems”. A long name that!
There are two ways by which you can deploy Eucalyptus. One is a lengthy
process where you install multiple Linux machines, install Java and Xen on them,
and then Install the Client components of Eucalyptus. Then install a separate
Linux machine and install the controller components of Eucalyptus. Next you
configure and use it (if you still have energy left for doing that!).
Else, there is a simpler approach. Install a ROCKS clustering front end with
Java and Xen roll, then add Eucalyptus Roll to it manually. Then let the ROCKS'
TFTP server take over the node deployments and you are done. Once the nodes are
installed, you can just download Amoazon's EC2 client and start using it.
After booting the central management machine with Rocks 5.0 DVD, type frontend to start installation |
I am sure, you must be preferring the second option more and that's what I
also did. Now let's look at deploying Eucalyptus using ROCKS. For those who
don't know what ROCKS is, here's a small intro: it is a Clustering/Grid
middleware something similar to OSCAR and comes with a brilliant node monitoring
tool called Ganglia (which we have talked about before) and TFTP based remote
deployment support for installing nodes.
Installing ROCKS front end
Download ROCKS 5.0 Boot, Core and OS Roll DVD from http://www.rocksclusters.org/wordpress/?page_id=82.
Select the right architecture before downloading. We went with a 64-bit model,
which is most likely the desired architecture for a Cloud. But a word of caution
here! Don't get carried away with the instinct of downloading the latest and the
greatest because on the same site, ROCKS version 5.1 DVD is also available which
doesn't work with Eucalyptus. And no document anywhere says that. I first did
that and wasted a complete day in trying to deploy Eucalyptus for Rocks 5.1.
Once you have download the DVD ISO and burnt it, get a machine (can be any
machine with preferably one GB of RAM, P-IV and above with two network cards).
Now, connect one network card with your network switch which preferably has a
DHCP server and an Internet connection. We will call this the “production
network.” And connect the other network card to an isolated network (preferably
a Gbps network); where you are going to connect all your Cloud node machines. We
shall call this is Cloud network. Make sure there is no DHCP server running on
the Cloud network because Rocks frontend is going to run its own DHCP server,
pre-configured for node's remote installation.
Make sure to select Rolls shown in the image while installing Rocks for your Cloud |
Now put in the DVD and fire up the installation, once the boot screen comes,
type “frontend” and hit enter. This will start the installation for ROCKS'
management server. After a couple of booting minutes, you will see a screen
which asks you to select the Rolls you want to install on the Rocks server.
There are two ways of installing the rolls: one is through the DVD and the other
is through the Internet. Select the CD/DVD option and you will see a page with a
list of available roles. Select the rolls shown in the image in the previous
page. Don't forget to specifically include the base, Xen, Java, OS and Ganglia
roles.
Once you have selected the rolls, click on the Submit button and you will be
redirected to the previous page. Now hit next and you will see a screen which
will ask you to fill in details for creating a self certificate for the HTTPS
web access of the Rocks front end. Enter your location information and proceed.
Running 'rocks list roll' will show you the list of Rolls installed. Here, check if Eucalyptus Roll is installed or not. |
Once this is done, the installer will ask you standard information such as
password, timezone, etc. and it will ask you how to partition the server hard
drive. It is good to do a automatic partitioning to avoid any confusion. Just
make sure you don't have any data on the hard disk. After gathering this
information, the system will start copying the file and depending on your system
resources it will take from 10 to 20 minutes to finish the install.
Adding the Eucalyptus Roll
Once the ROCKS installation is over, it is time to install the Eucalyptus Roll
on the Rocks front end. For this you have to download the eucalyptus roll ISO
from http://eucalyptus.cs.ucsb.edu/downloads/31 and copy it to the front end
system's '/' folder. Now add the eucalyptus role by running the following
command:
# rocks add roll clean=1 /
$eucalyptus-1.3.x86_64.disk1.iso
Now enable the roll by running the following command:
#rocks enable roll eucalyptus
Once this is done, you can check if the eucalyptus roll is installed and
enabled or not, by running the following command:
# rocks list roll
This should show something like the visual above.
Now you have to modify the TFTP based remote installation of the node in such
a way that when rocks does the remote installation, the eucalyptus node
components also gets installed on the nodes. To do so run the following command:
# cd /home/install && rocks-dist dist
This command will take some time to run and will build the new package list
for remote installation of nodes.
In the inter-ether command screen select the VM Container option to start the installation of Eucalyptus nodes. |
Installing nodes
Now this is the time to install nodes over the network. For this, run the Rocks
command “insert-ether” from a terminal. It will open up a window with a list of
device types. Select the last option which says “vm-container” and hit enter. A
blank window will open. Now boot up all the nodes and make sure they are all
connect to the cloud network and their boot preference is set to Network/PXE
boot. Once you do so you will start seeing entries being added to the blank
window of “insert-ether”. You will see one entry each for the number of nodes
you have booted.
On the nodes, the remote installation will start automatically and the
complete installation will happen with any manual intervention. You can even
have your nodes headless and see the installation screen remotely through a VNC
client with the host name of the nodes shown in the insert-ether window.
Once the nodes have been installed with Eucalyptus client, you have to now
install and configure the Eucalyptus web management interface and manager on the
front end/management machine. For this run the following commands. These
commands will install and configure all eucalyptus server side rpms:
#kroll eucalyptus > build.sh
#sh ./build.sh
Once this is done, reboot all your node machines. Don't know why, but I faced
a lot of trouble getting all nodes attached to the cloud till the time I
rebooted all of them once. Once the reboot process has been done, you can say
your EC2 enabled Eucalyptus cloud is up and running. Now this is the time to
start using it.
Adding images
Now your cluster is ready but you need some virtual machines to run on top of
it. For doing this, you have to upload some Linux OS images to it. For getting
your hands dirty, you can use a very tiny Linux image which is available from
Eucalyptus' website. You can download it from http://eucalyptus.cs.ucsb.edu/releases/euca-ttylinux.tgz.
once downloaded, untar and unzip it and run the following command:
#/opt/eucalyptus/usr/sbin/euca add_image --disk-image
ttylinux.img --kernel-image vmlinuz-2.6.16.33-xen --image-name ttylinux
And the image will be updated to the cloud.
Once your nodes start booting up you will see them getting registered in this window one after the other. |
Using the Cloud with Amazon's EC2 client
If you are comfortable with Amazon's EC2 client then you can just skip this
section. But if you are not, then read on. To start with, first get a client
machine, any Windows or Linux machine will do. But the process will vary a bit.
We did it on a Linux machine.
First download Amazon's EC2 toolset and APIs from
http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-24159.zip. Now unzip it.
But before using the commands first generate and download a certificate for your
cloud's account, so that you can authenticate the machine from the system that
is running the EC2 client. To do so, open the following link:
https://ip.of.the.cloud.front.end:8443
Change “ip.of.the.cloud. front. end” with the IP or FQDN of the Cloud's
frontend. A login page will appear. Login through it using admin as username and
password. Once you login for the first time to the Eucalyptus page, you will be
asked to change the Admin password. Change it and proceed and you will see the
admn page. Now from here, you can add more users and then login through them to
get the certificate or you can just download the Admin's certificate to get
started. To download the Admin's certificate, click on “Generate Certificate”
button on the page. It will download a zipped file. Once downloaded, unzip it to
~/.euca folder.
Now move EC2 Tools folder to opt and run the following commands to register
the EC2 and certificate paths, and the cloud's website:
export EC2_HOME=/opt/ec2-api-tools-1.3-24159
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=~/.euca/euca2-*-pk.pem
export EC2_CERT=~/.euca/euca2-*-cert.pem
export EC2_URL=http:// ip.of.the.cloud.front.end:8773/services/Eucalyptus
Now, you will be able to run the following command to see how many cores you
have in your cloud and how many are free for you:
#ec2-describe-availability-zones
Now you have to create a private key, so that you can use it to connect to an
instance when you run them, through SSH. To do so, run:
ec2-add-keypair key >> key.private
Now run the following command to get the EMI identifier for the image/s you
have added:
#ec2-describe-images
It will return a value like emi-xxxxxxx. Now run the following command to
start instances of the image on the cloud:
#ec2-run-instances emi-xxxxxxx -k key
And the linux image will boot up on the cloud. You can access it through SSH.
But for that you will need its IP which you can get by running the following
command:
#ec2-describe-instances
To connect to the image using SSH and the private key you just created, run
the following command:
ssh -i key.private root@ip-addresses-of-the-image
This screen shows that our 'Cloud' is Up with all 24 cores out of the 24 available free. |
And you are done. In case you still have any doubts or want to provide your
feedback on this article, please visit http://forums. pcquest.com, a 3 lakh
strong community of techies, constantly discussing new tech and services.