by September 23, 2013 0 comments

What’s big data in the pool of information and data? Just because a data set is large that does not make it ‘big’ data! Right?
There are several attributes that define big data. Size is one. The complexity of different types of data also makes makes it big. Such data include everything from streaming video to Twitter feeds to camera footages. Almost 90% of this is unstructured data. However, before you act upon them, you have to understand the data. The other thing that defines big data is speed at which it is generated from sources including camera feeds from lots of surveillance cameras. Twitter feeds is another example of real time data streaming. When you talk about big data, it is also about types of applications people are developing to solve complex problems. So big data is also about new ways to be able to do complex analysis of data and make sense of them with actionable insights.

What’s the equation of big data? How much data should an organization save? Is there a clear path to go ahead with this?
The cost of compute and storage is going down and hence saving data is no longer a terribly expensive proposition. One of the unique things of the large amount of data which is unstructured is that you don’t really know which of them will help you in future. One of the live examples is the log files that systems generate at our Intel lab. We have been storing those files generated by our machines for ages that give us information about they are performing, etc. We have never thought of making use of this log files.

Now we’re using those files to understand various parameters including when our machines go down, what all issues they face before going down, etc because downtime in Intel factory costs millions of dollars per minute. So to the extent that we should intervene proactively to better manage those machines. Had we thrown this historical information, we would not have been in a position to arrive at the insights that we have today. The bottomline is, it’s not really expensive storing data today.

Any compelling use case for big data across industry verticals?
Healthcare holds a lot of potentials. Genomics, for instance, is one area where big data can do wonders. Analyzing billion genes is very complicated. Each person generates 5 terabytes of data for one sequence. So if we can have the entire gene sequence of a person or millions of gene sequences for that matter, we will be able to correlate them which may save lives of many people. We can better understand the problems that we face, and how to recover them by analyzing huge pool of data. The University of North Carolina for instance,is using big data to find better treatments for patients through genomic sequencing technologies.

How much of the overall big data activities in the market are actually translating into business benefits?
I think lots of people are at data gathering and collecting phase today. We have done some research at Intel which establish that of all the people who are engaging in big data projects, only a meagre 6% are deriving real business benefits. Rest are learning different technologies while collecting data simultaneously. So basically people are building the stage for big data. People don’t really know how to put Hadoop to use, how to correlate data, and where they can find a data scientist, etc.



What’s the biggest issue with big data today?
‘Missed benefits,’ for people don’t really understand the benefits of this new trend. We’re at a stage when people don’t really know how much insight the data can provide and to what use. However, big data is much more than we really think it can do. It can solve big medical problems as explained earlier. Having said, soon we will see people investing on it heavily.

How do you see big data five years down the line?
There are some privacy issues and govt regulations coming on the way of leveraging big data for businesses. However, it’s just the beginning and it will take some time before businesses and government invest heavily on it to gain insights. Big data has humongous potentials that businesses can leverage to add to the operational efficiency. Let’s see how it pans out in coming days, however, we at Intel bet big on it.

To know more about how to leverage this new trend, read the following…

Volume, Variety & Velocity: Taking Big Data to Next Level

Should You be Hiring a Big Data Expert in Your Company?


No Comments so far

Jump into a conversation

No Comments Yet!

You can be the one to start a conversation.