If 2012 is the year of Big Data, it will likely be the year vendors and consultants start to over-promise, under-deliver, and put processes in motion that will generate insights and potential risks for years to come. […]

As Big Data becomes the next great savior of business and humanity, we need to remain skeptical of its promises as well as its applications and aspirations.

Existential Issues With Big Data
Determinism teaches that what will be, will be. Existentialism deals with a humanity in the throes of chaos. Big Data can be seen as either a lens through which determinism is revealed, or a tool for navigating an existential world. As a scenario planner, I take the existential position and see a number of existential threats to the success of Big Data and its applications.

To summarise and paraphrase the threats Rasmus perceives:

1. Overconfidence: most managers are overconfident and miscalibrated. In other words, they don’t recognize their own inability to forecast the future, nor do they recognize the inherent volatility of markets. Both of these traits portend big problems for Big Data as humans code their assumptions about the world into algorithms
2. When learning happens: “If organizations rely on Big Data to connect far-ranging databases [..] -who, it must be asked, will understand enough of the model to challenge its underlying assumptions, and re-craft those assumptions when the world, and the data that reflects it, changes?”
3. Complexity: Complex models reflect the worldviews & personal understandings of the people who’ve built them, and may not be easily transferable to new staff - or combinable with other models.
4. Feedback loops: Creating connections between datasets can amplify the impact of errors or incorrect assumptions
5. Algorithms & a lack of theory: for some fields of endeavour, there is no consensus theory about how they work. Consequently “data scientists can’t create a model because no reliable underlying logic exists that can be encoded into a model”
6. Confirmation bias: “Every model is based on historical assumptions and perceptual biases. Regardless of the sophistication of the science, we often create models that help us see what we want to see”
7. The world changes: “We must remember that all data is historical. There is no data from or about the future. Future context changes cannot be built into a model because they cannot be anticipated.”

The company that keeps getting mentioned as the one best able to handle the “fuzziness” of big data (i.e. different datasets having similar but not isometric categories) is Palantir, the slightly Orwellian firm focusing on post-9/11 security and government data. But here I find Rasmus’s comment particularly interesting:

“They have to be cautious about applying their ideas to different domains where underlying rules might not be so clear or data so well-defined.”

Indeed. Fuzziness is not trivial!