Making Sense of Big Data for SMEs

With the advent of newer, more powerful technologies like Apache Spark, NoSQL databases, and powerful storage & compute infrastructure, Big Data is no longer the hegemony of expensive solution providers

- Karthik Sridhar, Founder & CEO, DataCulture

As with any new technology introduced into a market, big data, coupled with cloud computing, has seen some risk taking early adopters but the large majority has been that of skeptical users who lament “yet another new technology” phenomenon and refuse adopting or even experimenting with it. The reality today however is far from it – big data has been surrounded by newer analytical technologies that are delivering quantum value to its users

What is the new paradigm?

Advent of newer, more powerful technologies like Apache Spark, NoSQL databases, powerful storage & compute infrastructure from cloud service providers like AWS, Microsoft Azure which run platform agnostic, statistics-oriented programming environments and languages like R & Python, have dramatically reduced the time for computing and delivery of data critical applications, and with that the cost of affording and running such applications. Just like cloud applications, big data analytics is no longer the hegemony of expensive and bloated licensed statistical software.

Insights rule the roost

The age of predictive analytics has been made real, not only because technology has become affordable, but the talent and ability to build specific big data solutions has been made possible. For SMEs, it becomes that much more critical to evaluate and adopt big data to get a leg up in their ability to compete in the market. Let me illustrate that with some key application areas:

Marketing has been a leading function in adopting any new technology that brings with it the power of greater sales. By conducting deeper analysis of external and internal data, marketers today are actively adopting marketing automation applications that provide pointed insights into customers, their buying patterns, their sentiments for or against a brand. These insights lead to more personalization, improve customer’s conversions rates. Such applications are no longer the domain of larger enterprises. These applications cost 1/10th of what they used to cost two years ago.

Moving forward, Supply Chain & Logistics are the next big horizontal frontier, where big data analytics extends its abilities. Huge value for cost conscious SMEs can now be realized through simple application areas like reducing revenue leakages, by predicting weight differences in shipment. More complex application areas like predicting shipment delays and predictive capacity planning are now a plug and play applications. While such technologies have previously been mainstay at large multinational companies, the data generated by medium and small sized companies are meaningful enough to leverage predictive analysis. The plug and play nature of such analysis ensures that there is little expenditure done on the IT side, and more time is invested in investigating areas where analysis can truly make an impact.

Algorithms are the key

In the end, no matter what the technology or the cost or the manpower – the true answer to unlocking value from big data comes from applying the right set of algorithms on the right sets of data to address key decisions for an organization. Algorithms help manage data complexity and draw links and inferences from data for whatever problem that needs solving, and further extend its utility through APIs and application integration.

The choices, that an SME needs to make while evaluating and approaching big data are three in important areas: Is the technology cloud oriented and therefore fast, reliable, secure and versatile in handling different data types? Does the technology give you the ability to change parameters to evaluate outcomes from data? And lastly, is the technology extensible through APIs and therefore allows you to integrate with multiple applications and achieve true automation.

Newer and more pertinent applications built from predictive analysis of small sized data are the biggest contributors to big data technology. The next biggest contribution is the increased affordability of such technology. Considering how much has changed in the past two to three years in the domain, it is just the right time that small and medium enterprises arise from the trough of disillusionment and adopt big data techniques to achieve market enlightenment.