A 10-Point Guide to Big Data

There's a lot of hype that surrounds Big Data, and it has reached a level where people have heard of the technology, but don't really know how to leverage it (PCQ Big Data Survey, pg. 12). Here are ten points that I feel capture the whole essence of how organizations can benefit from Big Data. They may not make much sense if read in isolation, but when read together, I think they help to join all the missing dots for understanding this new technology.

1: All organizations have structured data: It resides in databases, Excel sheets, etc. and is easy to work with because it has been around for ages, and there are both tools and talent available to manage it.

2: All organizations also have unstructured data: This comes from a lot of sources-social media (Facebook posts, Twitter, etc), email, sensors in machinery, HVAC equipment, text from mobiles, and so on. A significant amount of this unstructured data also comes from outside the organization, like customer feedback on social media, weather data, etc.

3: There's a lot more unstructured data than structured data: Some studies indicate that 15% of the data in organizations is structured, while the rest is unstructured.

4: Unstructured data is exploding and gaining importance. The sources of unstructured data have increased as highlighted in point 2. Moreover, this data is growing exponentially-millions of posts on Facebook, Twitter, WhatsApp, Nimbuzz, etc. Oodles of raw data is generated by sensors in machinery, automobiles, HVAC equipment, and so on.

5: Till now, it was extremely difficult to capture unstructured data. Organizations could stand to gain a lot if they could capture this raw data, but it has always been a challenge due to lack of good and cost-effective tools.

6. Big Data technologies provide the necessary tools to capture, store, refine, and analyze unstructured data...even in real time. What was not possible till now has become possible with Big Data tools. They let you capture and import raw data, structure it, establish relations with structured data, refine results and export them to analytical tools to draw meaningful inferences and conclusions.

7. The key is to identify the right data to capture. This will vary for different organizations. It could be customer feedback on social media for retailers, production output from sensors in machinery, truck location and idling time of trucks for a logistics company, AC temperature at different points of time from the HVAC equipment sensors for data center providers or real-estate players. Once you've identified this data, you can even combine it with existing structured data and do wonders.

8. Invest in Big Data tools. Every new technology requires both time and financial investments, and Big Data is no different. So play around with some of the Big Data tools that are out there before putting them to serious business use.

9. Train people on Big Data Tools. You need people who can use the Big Data frameworks, and create the required scripts to import raw data, structure it, run queries on it, and export it to analytics tools that even non-technical business manager

could use.

10. Learn how Big Data can lead to "Big" benefits for your business. All technology investments are meaningless unless they help create business value for the organization, and Big Data is no different. CIOs therefore need to find out the real business benefits that can be derived from Big Data for their industry.

There are enough examples already out there to start with. You can read more about them here:

Should You be Hiring a Big Data Expert in Your Company?