Big Data is fun, everyone is talking about it and although it fell off the hype cycle last year it’s bigger than ever. Big Data tends to be joining other trends like decision management, machine learning, autonomous vehicles, prediction markets and in-memory realtime analytics.

Within Rubix we have a group of people motivated to find out what the implications are on the information technology landscape and how smart data, process and system integration can help the business in solving these challenging Big Data problems. But what are these Big Data problems? What do we want to achieve by solving these problems? And maybe more important, what is the real purpose of Big Data Science and how will this have impact on society? But before jumping into technical solutions à la Hadoop, machine learning or real-time streaming analytics this blog post is all about the ultimate pursuit of Big Data. Let’s try to answer the why-question before the how-question.

Real World Situational Awareness

Situational awareness of situation awareness is the perception of environmental elements with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event. (source: Wikipedia).

Real World Situational Awareness is basically the understanding of everything that happens around us as individuals, perhaps also the understanding of everything that gives us happiness. Wouldn’t it be awesome if somehow we would be fully in control of this and already had all the information to predict the outcome of every single decision just by matching patterns or correlating events that ever happened? This could revolutionize life and impact  happiness by influencing success, productivity or health. This will change the way we look at life, this will determine our experience or life journey (another marketing buzzword).

The other day I saw a documentary about study performed by a MT scientist (Deb Roy) that captured 90.000 hours of video of his son’s first words and graphed it (link). The purpose was to understand how we learn language, in context, through the words we hear but also the precise mapping of tight feedback loops between the child and caregivers. For example the study tracked the length of every sentence spoken to the child in which a particular word was included. Right around the time the child started to say the word, what Roy calls the “word birth,” the caregivers unconsciously stress it by repeating it back to him all by itself or in very short sentences. The infant actually shaped the caregivers’ behavior, the better to learn. This study somehow gives us insight and understanding how infants learn language and makes us conscious about this process thus increasing our awareness on this subject.

Real World Situational Awareness can be extremely important in certain work domains or industries where the information flow can be quite high and poor decisions may lead to serious consequences, for example piloting an airplane, performing surgery on a patient or monitoring the electricity grid. In these industries it is critical to make a correct decision within a strict time-limit where real world situational Awareness would most definitely help in the decision and outcome.

If we look at the definition of Big Data Science being a field “dedicated to the processing, storage and analysis of large collections of data that frequently originate from disparate sources” and the purpose “to turn data into the right information and put it in the right context to gain actionable insight and prediction” we see quite some similarity with the purpose of Real World Situational Awareness. It’s all about having the actionable insight and making the right decisions!

1. “having the perception of environmental elements with respect to time or space” (by spatial data or time series analysis)
2. “the comprehension of their meaning” (by pattern matching algorithms, machine learning)
3. “the projection of their status after some variable has changed” (by predictive and prescriptive analytics)

I like to see Real World Situational Awareness as the ultimate pursuit of Big Data Science, but at the same time Big Data Science seems to be a precondition set for Real World Situational Awareness. In all cases if we want to meet the expectations set by trends like decision management, machine learning, autonomous vehicles, prediction markets and in-memory realtime analytics we will be doing “Big Data Science”, whether we like it or not. “Big Data Science” joined the party and isn’t leaving anywhere!