by October 9, 2006 0 comments

Data is the lifeblood of any organization. It is, therefore, important that
this blood is kept clean and free of all impurities, so that it has a smooth
flow and is available as and when required. Sound like something that a healthy
human body should do? Well….isn’t your IT infrastructure similar and
complex? It has many different applications that are constantly generating so
much data and working on it. It has a network that must ensure smooth flow of
data across all applications, no matter where they are located. It has various
storage devices to store data, so that data is always available whenever you
need it. If this storage infrastructure is not managed properly, it could have
serious implications for your business. What you need, therefore, is a good
storage management strategy not only to manage the data properly, but also to
ensure business continuity. Storage management should, therefore, be top
priority for any IT manager today. We arrived at this conclusion based on views
of 58% of the respondents of our
survey, who said that storage management was far more important than any other
IT infrastructure management area. Another 25% said that it was equally or more
important than other IT infrastructure management streams.

Key challenges
How difficult or challenging is the task of managing storage? For more than 50%
of our respondents, it was manageable, but another 38% found it to be either
difficult or very difficult.

We can divide the specific challenges themselves faced by the IT managers
them into issues related to data backup, user related problems, data
consolidation, availability and performance. Plus, issues related to managing
data spread across a large geography.

Issues related to backup include reliability of storage media, insufficient
backup windows and speed of restoration should the need arise to do so. There
were also worries related to rapid changes in storage technologies. As tape has
been around for a long time, the associated challenges of backing up data to
tape are fairly well known. There’s no single solution to the challenges, but
there are a few guidelines that need to be kept in mind. For instance, when
going for a tape solution, you need to ensure that there’s a pre-defined
roadmap for it so that five years down the line, your existing solution will
still work with whatever is available at that time. 



While researching this story, we interacted with 25
users and CIOs across large enterprises to understand their strategies for
managing storage. We had futher in-depth sessions with a few others who
have been .

There are some very clear strategic trends that emerge
from this survey, which have been presented in this story. The
interpretations of what the CIOs have told us are ours and they are in no
way responsible for the same.

How do you manage the ever
growing volume of data in your organization?

Likewise, you need to ensure that the vendor will be able to support your
existing backup solution in future as well. It shouldn’t happen that there are
no drives available to read the extremely critical data that you’ve backed up
to tapes today. On the data archival front, there were worries about retaining
data for compliance, identifying and distinguishing the right data that needs to
be archived, and finally retrieving it in case of an investigation. Moreover,
the challenge is not in data retrieval per say, but the speed and accuracy at
which it’s done. Here again, policy plays a more important role here than
technology. You need to ensure that whatever you’re backing up is checked for
accuracy. You need to perhaps test your backups at regular intervals to ensure
that you don’t get a heart attack when you really need to restore the data.

What’s your biggest hurdle in ensuring data compliance? 

Another key challenge was managing data that’s spread across a large
geography. There can be many issues to tackle in such cases. The data could be
residing on multiple platforms and being used by different appls. You might be
adding additional branch offices at a very rapid pace, for which additional
storage capacity needs to be provisioned for, etc. The amount of storage usage
and application performance due to growing data is the other issue faced by CIOs.
As data increases continuously, it’s a challenge to ensure that the
applications that use them are able to do so with good response time.

One of the key reasons for so many storage challenges is that the there’s
no end to the growing amount of data. It just keeps growing and growing. The
opinion on how easy was it to manage this growth was almost equally split
between being manageable to difficult to manage. 55% of the respondents felt
that this was manageable while another 41% felt that it was difficult. But by
and large, there were two ways in which a majority of the respondents were
managing the growing volumes. One was through storage consolidation, and the
other was to optimize storage utilization by following data management policies.
That is perhaps the best combination to follow today. There might be instances
where adding more storage capacity can help, but that is not the best method
beyond a certain point.

Major and minor pain points
Owing to its vastness, there are lots of management tasks that need to be
prioritized and taken care of. The task that seemed to keep the CIOs busiest was
adhering to statutory requirements for data retention and archival. This was the
issue that CIOs spend most of their time on. This is not surprising as more than
60% of the respondents said that their organization is required to store data
for legal requirements. 

The biggest hurdle to staying compliant appears to be consolidating multiple
islands of data. Some 52% respondents stated this as their biggest challenge. As
30% of the respondents to our survey were from BFSI, they had to follow various
guidelines like the banking regulation act, prevention of Money Laundering Act
as well as RBL and SEBI guidelines. Some CIOs also said that their companies
followed US based and international guidelines such as Basel II, Serbanes Oxeley,
etc. More than 50% of the respondents felt that the Indian IT laws on data
storage for compliance were actually sufficient for their industry. However, an
alarming 36% said that the Indian laws were insufficient. The key issue that
emerged here was a lack of clarity on the IT laws for data compliance. We were
also told that there were contradictions of data requirement under various laws.
There weren’t any specific laws defined for data storage, and whatever
landmarks were set were insufficient. Therefore a consolidation of data laws
themselves is required, along with a clearly laid out policy. The next most time
consuming task was managing data security, with 40% of the respondents saying
that most of their time is spent taking care of this. Another 32% of the
respondents said that they spent moderate time on this issue. On the other hand,
email archival, regardless of its high importance in storage management, isn’t
really a pain area for CIOs. A majority of the respondents said that they spend
the least time handling email archival woes. The other area that our respondents
didn’t spend much time was migrating applications and data from one server to
another. While migration is definitely not to be taken lightly, it also isn’t
something that’s done frequently.

Existing vs future deployments
More than 60% of the respondents already had centralized their storage
management, and 23% wanted to do it in the near future. Email archival is also
being taken seriously with more than 50% respondents already using it, and
another 23% of the respondents wanting to deploy it in near future. Same
situation prevails for disaster recovery and business continuity planning
solutions, with more than 50% current ownership and another 23% wanting to
deploy it in the near future.

There were more fiber based SAN implementations than IPSAN based, but the
future holds more promise for IP based SANs. Our survey revealed that 27% of the
respondents already had IP based SANs, while another 23% said that they plan to
deploy it in the near future. On the other side, 36% of the respondents already
had fiber based SANs, but only 9% more displayed interest in deploying the same
in the near future.
The area of maximum interest amongst the CIO community was ILM (Information
Lifecycle Management). Though only 9% of the respondents had already deployed
it, another 32% wanted to deploy it in the near future. The next major area of
interest was disaster recovery and business continuity planning implementation.
Interestingly, 52% respondents had already deployed disaster recovery and BCP,
while another 28% said they plan to do so in near future.

Content management, document management, continuous data protection and
e-mail archival were the other three promising areas for the future. There were
24% respondents each for these four areas, who plan to deploy them in the near
future. Content addressable storage or CAS and snapshotting for all their hype
aren’t very popular yet. There weren’t too many takers for these two
technologies. In fact, there weren’t too many current ownerships for CAS.
Storage virtualization for all its hype is still not a priority for most. In
fact, the number who already had it was higher than the number of CIOs who
planned to deploy it in the near future.

Manpower and vendor management
A good 44% of the respondents to our survey said they had dedicated manpower for
managing storage. There were another 36% who didn’t have any dedicated staff
for the job, and relatively few had outsourced the same.

Managing such manpower is not easy and the key reason for this seems to be
the rapid advances in storage technology. Every other day, something new and
exciting comes up in storage, promising great benefits to your business. If you
deploy the same, your manpower must have the skills to manage it. Providing
training on the latest technologies therefore emerged as the key concern. The
next biggest concern is finding and retaining dedicated storage professionals.
As technologies change rapidly, storage professionals would want to learn the
same and if the organization isn’t able to provide the necessary training,
they would look for better opportunities. The results on vendor related issues
were fairly interesting. Around 36% of the respondents were more than satisfied
by their primary storage vendors. However, another 44% who were moderately
satisfied. That is, they didn’t have any major issues with their vendors, but
weren’t exactly overwhelmed by their support. So when we asked for specific
issues related to vendor management, hardly any responses came out. There were
stray cases of inadequate onsite support or the technology of data storage being
provided. There’s no end to challenges in storage management, as this is one
stream that’s under constant development with new technologies evolving every
day. This is understandable considering that the growth in volumes of data in an
organization isn’t going to stop or slow down. Organizations will therefore
continuously require new and better technologies to manage them. Therefore, you
need to constantly evolve your storage management strategies to manage this data
and the infrastructure that holds it in place.

Anand Bhatnagar, Vice President, The Bank
of Rajasthan Limited
Ashish Dandekar, AVP Enterprise Management, NSE.IT Limited
Q: You’d
mentioned that you plan to deploy ILM in the near future. What are the
benefits you are likely to get by deploying it?

If we can’t define the life cycle of data, then it will be a
never-ending process and data pruning will have to wait for dead end.
While presently what we and mostly everyone in the banking industry has
adopted is needs based management. While data is mostly backed up to
magnetic media, sometimes either due to an application upgrade or
non-availability of the specific platform to access that data, the
archival (historical data) remains a thing of mandatory requirement only.
Hence while accessing the desired data when really needed and also at the
same time pruning and destroying the “not so useful” data will
save the space and management effort to a great extent. Another dimension
is falling of stored data in wrong hands will also be safeguarded, when
there are policies and systems in place for the same.

Q: You’d mentioned that your biggest
challenge in storage management is keeping a balance between the
application response time due to an increase data in the application and
data availability in the application for
making the analysis more meaningful. Can you please elaborate on the same?
Basically, we would like to know how you are ensuring good application
response time.

There is an ever-increasing need of historical data in
any organization for analytical purpose. But as the volume of the data
increases, efficiency moves in reverse order. So on one side, there is
always pressure to retain data that’s at least eight to 10 years old, on
the other side whatever indexes you have, backup time, data extraction
becomes more time consuming That was the reason I mentioned that the
biggest challenge is keeping the balance between response time and data
availability. Currently we are managing the same by maintaining a parallel
system for keeping an archival reporting system, which is accessed by
users more frequently. This way we reduce access to such data by users and
data also remains in the system for exigencies and analytical uses.

Q: How do you ensure high data availability for your

High availability of data is ensured by making it available on common
storage with servers that are attached to the same in clusters. The
implementation of RAID 0+1, a redundant power supply, redundant capacity
of hard disks and hot swappable hard disks are some of the measures we use
to ensure high data availability in the application. Data replication to
DR and also the “archival in making—redo logs” to near line
site are also steps in the same

Q: What are some of the challenges you face in doing
storage consolidation? How are you addressing them?

Storage consolidation was considered necessary for ensuring backup at
regular intervals and for reducing downtime. This is more complicated when
applications are on different platforms and common storage has to
integrate with them. We achieved this by making separate virtual
volumes for each application on the central storage and while deciding for
the backup, the same was taken from the snapshot of the database. Thus
while data backup from a single storage could take a backup of multiple
applications, application downtime was also reduced considerably.

Another dimension to doing data consolidation is for
making a common database for multiple islands of databases in the
organization. Though the ideal solution for the same is data warehousing
and implementing business intelligence tools on the same, but we did not
have enough budgets for the same to implement it immediately. Therefore we
found a middle path and skipped the data-warehousing step for the time
being. We developed separate ETL (Extraction Transformation and Loading)
scripts for extracting data from different databases and got the cube
developed using SQL 2005 through one of our technology partners. Then by
using reporting tools, we generated analysis from this cube and thus the
data was utilized from
multiple resources.


Q: What are
some of the best practices to follow for doing storage consolidation?
You need to examine the growth of your data by checking all your
historical data. You also need to look at your current storage
requirements and projections/analysis of requirements for next 3-4 years.
Plus, you need to identify the real business need for doing consolidation.
Check whether your organization is subject to Indian or foreign compliance
norms. For storage consolidation, you need to do forward thinking and
visualize your organization’s IT strategy for next 3-4 years, keeping in
mind the cost of technology obsolescence. You need to plan &
strategize for business continuity, and don’t mix it with the
implementation time. That needs to be accounted for separately.

Q: Suggest a few guidelines for optimizing storage

Verify your business need and firm up on your data requirements. Often,
server and storage consolidation become a joint activity. So look for
alternates with a competitive techno commercial analysis. Ensure that the
device is benchmarked before procurement whenever possible for ensuring
smooth compatibility. Consolidate your data requirements and use business
requirements as predictions for data growth.

Q: What are some of the best practices to follow in
storage management?

Centralize data and keep the infrastructure in a data center under
supervision of a storage custodian (an employee who is responsible for
storage in the organization). Follow back-up policy and media redundancy.
If tapes are used, ensure testing on a yearly basis as tape is prone to
getting infected by fungus. Store the media in fireproof cabinets. If
possible, take 2 copies of back up, and store one of them at an off-site

Sensitive data can be encrypted. The media should be
stored in non-condensing but controlled humidity conditions under constant
temperature (OEM stipulates temperatures between 16 to 32 degrees C) for
better life expectancy. When placing purchase orders, ensure better
services by a formal Service Level arrangement/ agreement for ensuring
better uptime and problem resolution.

Q: What should you look for when choosing a disaster
recovery partner?

For disaster recovery, it’s always preferred to examine what’s best
for an organization through independent organizations that will help you
devise a proper approach and strategy. These organizations follow best
practices and have certified professionals to help strategize on DR/BCP.
Post strategizing the DR plan & strategy, implementation can be done
by an OEM/vendor with the best matching solution.


No Comments so far

Jump into a conversation

No Comments Yet!

You can be the one to start a conversation.