Personally I think that the increase in availability of information will do to people what the increase in the number of TV channels has done since the early 90’s: more TV to watch by fewer people and less to be interested in. I don’t think that our future will be such that more people will be involved in it—quite the opposite: fewer people will make more of the decisions because more of us will abdicate those choices to them.

Is the explosion in the amount of information a useful tool? Yes and No. Yes in that we can now make decisions based on a wide, and previously unavailable knowledge base, and No, because we now spend hours and hours researching even the most minor of decisions. Where you used to go to the garage to buy a set of tires and relied on the recommendation of the so-called expert, we now spend hours researching every recommendation of every so-called expert on which tires to buy. Do we end up with better decisions?

Frankly, the dawn of the Information Age is just starting.  While I’m excited by the prospects of being able to learn and understand and see the world around me, the sheer amount of information and its overwhelming effects on daily life are frightening. Will be we become more individualized and isolated by it? Time will tell.

Very nice write-up. I would like to thank you for the effort. It also ignited a few thoughts in me about digital disruptions from more than one angle.

I would like to share my bit about digital disruptions and open source software.

The crisis could be avoided if we had looked at how open source works.

As Alex wrote about his experience designing C-Trac, several large banks use very sophisticated financial simulations to second guess as to how the market will change. These simulations are done via software that is proprietary and closed.

That is what I believe is a problem for customers of simulation software (banks and financial corps), which does not give the full picture about which algorithms it uses or parameters it takes to come to a conclusion, or about issues such as licensing, feature supports, bug fixes, policy changes, etc. It’s a pity that banks have to succumb to these mud-coded abstract classes.

If the software was created as open source, then customers could adjust it so that it fits the bank’s position and customer base.

However, some modules of the financial software might not be realistic candidates for open source since not all banks are generic, and it is difficult to imagine who would spend the money to develop highly customized modules (functionality) as open source modules.

Finally, in my view, we should also talk about an over-simplistic glorification of open source—for example, that the next hurricane or tsunami could well be averted if only meteorologists use open source software. Open source software is a big step forward in providing clarity to business processes and pooling the best minds to solve problems.  Open source is a powerful tool but it is not a silver bullet for all situations.

Your comparison of IT in the political campaigns should evaluate their social networking, which is more innovative than posting video or accepting donations online. For example, if you go to http://my.barackobama.com/ you can network, find campaign events, read what party activists are saying, and participate in a community.  To participate, you give the party contact points; it’s more than a virtual world. Both parties are using social networking to some degree, and it adds a new meaning to the political ground campaign.

To C. Viscount’s comment on 10/23/08:

I agree with you that the increase in availability of information gives us so many choices that making a decision in a timely manner could become more difficult. Herbert Simon, “among the founding fathers of several of today’s important scientific domains, including artificial Intelligence, information processing, decision-making, problem-solving, attention economics, organization theory, complex systems, and computer simulation of scientific discovery,” [1] pointed this out over 35 years ago:

“...in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it” (Simon 1971, http://en.wikipedia.org/wiki/Attention_economy#CITEREFSimon1971).

Nassim Taleb, however, in his recent book “The Black Swan,“ points out that information overload is an addressable problem when he states that “information wants to be reduced.“ [2] It is a problem, in fact, that intelligent beings, he says, have been addressing in various ways since the beginning of time.

As is typical, though, there are lags between problem and solution; disruptive technologies are helping resolve information issues others created. To take your TV example, while at first it was confusing to navigate hundreds of cable channels, and now would be impossible to navigate thousands of Internet channels, cable introduced the TV Guide Channel and the Internet introduced broadcast programming search engines (e.g., VeohTV), both of which make it easier to find content of interest than in the days of seven TV channels and a hardcopy TV Guide.

We will increasingly harness smarter technology—e.g., semantic agents that can be said to understand the meaning of content and can on our behalf tirelessly sort through a myriad of information sources to present to us only those that match our stated interests. Specialized agents, which understand, for example, geography (Metcarta) or music (Pandora), are making great strides. While todays’ technology is far from perfect, and in fact will never become perfect, it is approaching “good enough,” which is what’s needed to make timely decisions.

On the other hand, social networking technology is helping us help each other sort through information, allowing us to tag – meaning classify – content, rate content, recommend content, forward content to those we deem interested, and rate providers of content. The combination of technology and human intervention is extremely powerful (e.g., Twine).

Ask yourself this:  Are you able to answer a random question more quickly today than a decade ago?  (Hint:  You can use Google, ask your social network for a lifeline, or, as you did, post a question or comment on a blog).

[1] http://en.wikipedia.org/wiki/Herbert_Simon
[2] Taleb, Nassim Nicholas, “The Black Swan” (New York: Random House, 2007), p. 64.

Great summary.

“Goldcorp, Wikinomics and Changing the World” is also a superb sample of “Open Source in New Domains” that was elaborated on in CSC’s LEF Report “Open Source: Open for Business” (2004).  See report at /aboutus/leadingedgeforum/knowledgelibrary/uploads/1142_1.pdf (pp. 65-69).

To Venky Namilakonda’s comment on 10/25/08:

Thank you for joining the conversation. Let me note here that although it may take me awhile to respond to posts, I will respond to as many as I can. Of course, this blog is a multi-way conversation, so others may respond as well.

Yes, C-Trac was proprietary software, but I think that the problem of lack of transparency in financial services, and its solution, lies not in open source software but in open, shared data. Financial firms seek competitive advantage by building proprietary models that guide their investment strategies and policies, to help them maximize return on investment and minimize risk. I don’t think these firms would want to share their models, which is rightfully their intellectual property, nor do I think for our purposes it is required. But I do think what should be shared is the data—the transactions that firms enter into based on whatever models they are running. Given the transaction data, those in the world most interested and best equipped to understand it can build their own models to analyze the actual risks being taken and the risk-adjusted return on investment, and can warn investors if anything seems amiss. Likewise, hurricane and tsunami data can be published for review by the best meteorological minds in the world, serving as a check and balance for those directly involved in making disaster prevention and recovery decisions.

As you say, open source is not a silver bullet for all situations, but I think open data might very well be for most.