Create Your Own Private Cloud

Most discussions around Cloud Computing nowadays are either focused on the

security issues associated with it, or providing basic definitions of the three

different types of Cloud based services i.e SaaS, PaaS and IaaS. A third thing

is of course the loud noise created by all the vendors around their own cloud

based offerings. There's hardly anybody talking about actual usage of cloud

computing. Which public cloud service should you choose? What's a private cloud

and how to get started on it? What are the tools that would let me create my own

private cloud? There are very few discussions on these aspects.

So while the public cloud based services have been covered elsewhere in our

cover story, in this story we'll focus on how to build your own private IaaS

(Infrastructure as a Service) based private cloud. For this, we've used Ubuntu's

10.04 Server Edition, which is completely free. We've given it on this month's

DVD, else you can download it from ubuntu.com/cloud/private.

The benefits of setting this up are many. It has the elasticity to grow or

shrink your compute capacity based on your application's needs. You can rapidly

deploy new applications whenever required on your existing IT infrastructure.

This way, you don't end up investing in additional hardware resources. The

Ubuntu Enterprise Cloud provides the same APIs as the popular EC2 cloud from

Amazon, so if you've worked with that, this should be a piece of cake. What you

create and run on Ubuntu, you can also run on EC2.

Let's Implement



You need at least two servers for deploying the Ubuntu Cloud. One (Machine

A) would act as the cloud, cluster, warehouse, and storage controller; while the

other (Machine B) would be the node controller. One thing to keep in mind is

that the node controller should support virtualization as virtual machines would

be running on it. By default a single virtual machine would run per core,

therefore having a multi-core node controller is highly recommended. We

configured Machine A on a Core2duo X6800 processor based machine with 2 GB DDR 2

RAM and 80 GB HDD. Machine B was running on an AMD Phenome II X4 965 processor

with 4 GB DDR 3 RAM and 250 GB HDD.

To install the Ubuntu based Cloud, select 'Install Ubuntu Enterprise Cloud' from the first screen for installation on both Machines A and B.

Choose 'Select cloud installation mode', and then select cloud, cluster, warehouse, and storage controller for Machine A and choose node controller for Machine B.

While installing on Machine A, give a range of available Elastic IPs in the window named 'configuring eucalyptus-cc'.

You'll need to burn the Ubuntu server ISO on a CD/DVD and boot machine A from

it. From the first screen that pops up, choose 'Install Ubuntu Enterprise

Cloud'. The installer would automatically check for Eucalyptus components on the

network and recommend that you install the required components. From the screen

with 'Select Cloud installation mode' heading, select Cloud, cluster, warehouse,

and storage controller. Other Cloud specific inputs required during installation

are cluster name (cluster1 in our case) and range of public IPs that can be

assigned to instances. Once Machine A is up and running, you'll need to setup

the Node Controller. For this, boot Machine B with the Ubuntu Enterprise Server

CD/DVD and  select 'Install Ubuntu Enterprise Cloud'. This would automatically

detect the cluster1 and pre-select 'node controller' in 'Select Cloud

installation mode'. Once the installation is done, reboot machine B and we are

done.

Configuring your Private Cloud



Before you can start using the nodes, they have to be registered. In our

case, registration is done automatically as we are installing Ubuntu Enterprise

Cloud using the ISO. To verify this registration, run following command on

Machine A:

cat /var/log/eucalyptus/registration.log

Next you need to  get credentials of the users. This can be done either

through a web browser or by using the command line. Just visit the IP address of

your Cloud controller (in our case, 192.168.2.10), login using 'admin' as both

user name and password. This action would prompt you to change your password.

Once done with these changes, click on 'Credentials' tab on the top, and then

click on 'Download Credentials' button on the top. Save this zipped file to '~/.euca',

then unzip this file using:

unzip -d ~/.euca mycreds.zip

To check if nodes are properly registered, run the following command on Machine A: 'cat /var/log/eucalyptus/registration.log'.

You can install sample images plus create users using the web interface of the cloud controller. For this, login as using 'admin' as both username and password.

Credentials of users can be taken either from a web browser or from the command line. Just visit the IP address of your cloud controller and go to 'Credentials' tab.

Click on the 'Region'. This would pop up a window named 'Manage EC2 Regions'. Enter region name and the IP address 'http://192.168.2.10:8773/services/Eucalyptus'.

Click on 'Credentials' tab on the top; this would pop up 'Manage EC2 Credentials' window. Get value of 'AWS Access Key' and 'AWS Secret Access Key' from 'http://192.168.2.10'.

The other way of achieving credentials is by running the

following commands on the cloud controller: 

mkdir -p ~/.euca



chmod 700 ~/.euca


cd ~/.euca


sudo euca_conf --get-credentials mycreds.zip


unzip mycreds.zip


ln -s ~/.euca/eucarc ~/.eucarc


cd —

To use your credentials, install the required cloud user

tools as follows:

sudo apt-get install euca2ools



And to check if everything is running properly


. ~/.euca/eucarc


euca-describe-availability-zones verbose

 Installing and running an image



We have created our infrastructure. Now we show you how to run applications

on top of this. You can either create your own image or get the sample images

from the image store on web interface. Simply access Machine A from a web

browser by typing its IP address, and login as admin. From this interface, click

on 'Store' and then click on 'Search'. Now simply click on install button

presented in front of the available images. Once installed, the image would show

'How to run?' link in front of it. This link displays commands to run the

pre-installed image. All installed images can be seen under the Images tab.

Get all the installed images under 'Images' tab. Select the listed instance and click on 'Start Instances (s)' on the top. This would imitate instance and assign Elastic IP to it.

In Hybridfox, go to 'Volume and Snapshots' tab. Here you can create volumes; click on green plus sign and fill in amount of storage required and put available zone (Cluster1 in our case).

In 'Attach Volume' pop up window, select Instance and add device name '/dev/sdb'. You have successfully created and attached volume to your running instance.

Using Command line to run image



To run images go to Machine A and run following commands:

if < ! -e ~/.euca/mykey.priv >; then



mkdir -p -m 700 ~/.euca


touch ~/.euca/mykey.priv


chmod 0600 ~/.euca/mykey.priv


euca-add-keypair mykey > ~/.euca/mykey.priv


fi

These commands are to create a keypair (ssh key) that you

can use to log into your instance as root, once it boots. As the key is stored,

you only need to create them once. You also need to allow access to port 22 of

instance. This can be done using:

euca-authorize default -P tcp -p 22

-s 0.0.0.0/0

Then create instances of your registered images as follows:

euca-run-instances $EMI -k mykey

emi-E1461079 -t m1.small —addressing private

You can get an image id from the web interface of the Cloud

controller by clicking on 'How to run?' under the 'Store' tab. You can monitor

state of instance using the following:

watch -n5 euca-describe-instances

Output of this command shows information about the

instance, including its state. While first-time caching is being performed, the

instance's state will be 'pending'. When the instance is fully started, the

above state will become 'running'. Look at the IP address assigned to your

instance in the output and then connect to it:

IPADDR=$(euca-describe-instances | grep $EMI | grep running

| tail -n1 | awk '{print $4}')



ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR

To terminate instance run

INSTANCEID=$(euca-describe-instances | grep $EMI | grep

running | tail -n1 | awk '{print $2}')



euca-terminate-instances $INSTANCEID

Using Hybridfox extension to run Images



Besides using command line, one easy and efficient way to

manage your Ubuntu private cloud is via Firefox extension known as Hybridfox.

Download this extension from http://code.google.com/p/hybridfox/. Once

installed, you need to connect it to Ubuntu private Cloud. Click on  'Region' on

the top. This would pop up a window named 'Manage EC2 Regions' asking for region

name and endpoint URL. Name region (India, in our case) and put

'http://192.168.2.10:8773/services/Eucalyptus' where '192.168.2.10' is the IP

address of Machine A. Second important step to be carried out is to add

credentials to hybridfox. Click on 'Credentials' tab on the top; this would pop

up 'Manage EC2 Credentials' window. Get value of 'AWS Access Key' and 'AWS

Secret Access Key' from '192.168.2.10', click on 'Credentials' tab and then

click on 'Show keys' button under 'Query interface credentials'. Here 'Query ID'

is 'AWS Access Key' and 'Secret Key' is 'AWS Secret Access Key'. Now refresh

your page and you should get all the installed images under 'Images' tab. Simply

select the listed instance and click on 'Start Instances(s)' button on the top

(green start icon). This would imitate instance and assign Elastic IP to it. You

can see the different states of imitated Instance; initially it starts as

'pending' then changes to 'running' and when terminated changes to

'terminated'.  To get list of available Elastic IP's, simply click on 'Elastic

IP's' tab on the top. Elastic IP's are nothing but public IP's that were defined

during installation on Machine A. These are assigned to instances for access by

users.

Allocate Resource on the fly



To summarize IaaS in plain English, it gives the capability

of adding/ subtracting resources to your application on the fly. Suppose you had

anticipated that x number of CPU's and y amount of memory would be enough to run

your application, but later you found  that extra memory/storage/processing

power was needed, you might be in a fix. However, if you are using Cloud setup,

it is just a matter of few clicks that would add those extra resources needed by

your application and that too on the fly. Here, I show you how to create a

virtual storage unit and then attach this storage unit to running instance on

the fly. In Hybridfox, go to 'Volume and Snapshots' tab on the top. Here you can

create volumes, click on green plus sign and fill in amount of storage required

plus put available zone (Cluster1, in our case). Once volume is created, it

would be assigned a volume ID, now select this volume and attach it to running

instance by clicking on green button with tick mark. In 'Attach Volume' pop up

window, select Instance and add device name '/dev/sdb'. With this, you have

successfully created and attached volume to your running instance. You can also

detach volumes by clicking on red button with cross image.

All these steps show how easy it is to take the first step

in creation of your own private Cloud and that too with minimal investment.