OK this is going to be one of my soapbox posts, so consider yourself warned.  I have been doing a fair amount of reading on big data, visualizations, and analytics in general and I have come to a clear conclusion. 

“The value of data should not be measured in bytes, it should be measured based on its usefulness.”

I know the focus right now is how can a database or a server crunch all kinds of data quickly, and this is a software and hardware problem that is being solved by a host of startups and large companies.  This is great news, but as an end user I really don’t care.  I just want the answer.

You see, the data crunch issue or the ability to display a bunch of stuff in a visualization can be solved by writing code and optimizing software and hardware, but will my decisions be better as a result?  Will the data (or answer) arrive when I need it?  Will I be able to use that result to solve a problem in my supply chain, increase my revenue, or reduce my DSO?  

In thinking about this I keep going back to the weather example.  Weather data is massive; and the prediction algorithms being applied and models being created keep growing.  Though what I care about is if I should wake up for a run in the morning.  Will it be snowing, raining or too cold to run?

My weather apps have solved this by focused and timely data that is actionable by the end user.  Lets explore the key things that make this work well:

Just the Basics

There can be too much information and it is up us not to cross that line.  These days I am seeing a bunch of visualizations like this one:

I am not saying this is bad and I love some of these big data visualizations, but consider the problem and the user before you present something.  In my weather use case a visualization like this one might not have high value to the end user and in fact, they just don’t need all that data for the decision they are attempting to make.

Consider this simple hourly line chart in the Yahoo Weather app on an iPad.

This is a high value result that is very useful!

User Context

Having the users context is key for high value data; knowing the users location, if they are traveling, how they are traveling, and more can change how and what data an app might present.  Weather is too simple to make this point because, of course you want to know what the weather is like where you are at, but what about he supply chain example I gave you earlier.  If I have a distribution issue I need to solve, knowing that the user is at a specific distribution center where there is an inventory shortage can be critical in providing valuable data. 

Having context is critical but hard to solve.  The first step is for analytic companies to focus on users context and at least attempt to understand how that applies to the data set.  The great news is that all of our new devices are filled with great information on user context and consumer applications are successfully leveraging this information.

Timely Information

If I am outside stuck in the rain, and I get an alert that it is going to rain, then my app may have failed me!  

The key is giving the user that value data at the right time and letting them decide how or if it will impact their decisions.  

For example getting a visualization about a marking campaign and how it is trending on Twitter prior to my meeting with my CFO can be extremely valuable.  Seeing reach data while in the meeting can help determine if the campaign will be extended and if it is going to impact the bottom line.  That has real impact and value.

These are broad stroke concepts and will not happen over night but directionally I would love to see idea of #valuedata trending more than #bigdata.  These concepts are the ones they keep us up at night and over time you will see more and more of this take hold in the analytics market.