Making Big Data Work for Government
“Big data” comes with many promises, but the data alone is not a silver bullet. True, it holds the potential for extracting business or mission intelligence and improving decision-making, but without the application of expert domain knowledge to give data contextual meaning, big data is nothing but a whole lot of dark figures.
This industry brief explores how analysts who are thoroughly grounded in an agency's mission can become data scientists without acquiring the advanced skills required to interact with the data's underlying technologies. Our industry must develop an abstraction layer that facilitates turning data into insight. Creating such a layer will require two elements: (1) improved data querying technology and (2) intuitive, more manageable interfaces to control querying tools.
Subduing Big Data
The technological challenges of capturing, securing and managing the worldwide explosion of data are not insignificant (the amount of worldwide data currently measures about 2.7 zettabytes and is projected to double every two years3). Yet, corralling big data is as much an organizational challenge as a technological one. Industry development of big data technologies has largely bifurcated into two main functions: data discovery and the more traditional business intelligence.4 Companies have been investing much time and money in the former.5 Gartner estimates that big data initiatives in 2013 will total $34 billion,6 and big data will drive $232 billion in spending over the next five years.7
Many organizations are trying to address the technology challenge of analyzing big data. The TechAmerica Foundation’s Big Data Commission, for example, recently released a report that gives organizations guidelines for an effective big data program. Examination of these steps shows a decided focus not just on technical capabilities, but on organizational policies as well. The Commission recommends that CIOs take a “holistic approach to help guide the agency from an information management perspective and follow these key steps:
- Identify data and content that are vital to its mission
- Identify how, when, where and to whom information should be made available
- Determine appropriate data management, governance and security practices
- Identify and prioritize the information projects that deliver the most value.
Download the entire Big Data brief (PDF, 267 kb) to continue reading.
