How to Avoid Costly Failures With Your Big Data & Analytics Investments
Author:Chenwei Liu & Mark Meloon
The past few years have seen an explosion of new technologies for storing, analyzing and displaying the enormous amount of data available to businesses today. Feature articles on big data, predictive analytics and data science, rarely heard about a few years ago, regularly appear in the mainstream business press. Studies from major think tanks and management consulting companies have shown again and again that companies can leverage technologies such as Hadoop, RDBMS, Hive, Pig, YARN, Tez, Spark, Sqoop and so on, to generate new, significant benefits from their existing data assets.
The pressure to “do something” related to big data is growing. But do what, exactly? Businesses are eager to acquire the latest and greatest technology, both to gain a competitive advantage and to demonstrate to their stockholders that they operate on the cutting edge of their industry. However, hidden below the excitement of publicized successes lies the reality that achieving these intended goals is neither automatic nor assured.
RELYING SOLELY ON TECHNOLOGY IS THE ROAD TO RUIN
Technology dominates the public big data conversation. Technology is easy to grasp, and there is no ambiguity about it. Either you have a Hadoop cluster running Spark or you do not. What has gotten lost in the hype is that these technologies are simply tools — powerful tools, to be sure — but the ultimate success of your big data efforts lies in the hands of the people using the tools. For the projects to be successful, specialized expertise is needed to correctly analyze big data problems, along with a willingness among decision makers to act on the insights gleaned.
“Big data” does not simply mean “lots of data.” It means that companies have the ability to ask more questions, as well as ask fundamentally new questions. Those questions are laced with nuances that can cause the naïve application of traditional business intelligence (BI) techniques to fail.
To realize the promise of big data analytics, one must have a) relevant, high-quality data; b) the correct technology to use this data to solve the business problem at hand, given the appropriate constraints; and c) highly trained individuals who specialize in extracting the true value from the ocean of data and communicating it clearly and compellingly to those who are in a position to take action on it. It’s worth noting that the analysts referred to in “c” are the ones with the knowledge of how to judge “a” and “b.”
Evaluating the staff in your company who use the technology, however, is more difficult to appreciate and is challenging to assess. One can try to ground this in fact, such as hiring only those who have 3-plus years of writing complex SQL queries; however, this is no guarantee that they are good at it! You almost need to be an expert in big data analytics yourself to be able to judge the skill, knowledge and expert judgment of these all-important analysts. This paper will give you the solid foundation and understanding needed to make more informed decisions. And that foundation rests on the new field of data science.
Read the full report here.