Big data? Big deal.

Ever heard the saying “it’s not the size that counts, but what you do with it“?

No doubt you’ve heard the latest buzz words in the world of technology – the biggest of which is “big data”.  With exponential data growth, organisations are scrambling to understand how they will cope with the rapidly changing structured and unstructured data that supposedly now defines their business.

So what’s the big deal with big data?

Essentially big data is defined by the 3 V’s – Volume, Velocity and Variety.

Volume is the forefather of big data characteristics. A decade ago we defined large volumes of data in terms of Terabytes.  Now, we talk Petabytes.  And whilst technologies have existed in the market for years focused specifically on capturing, managing and understanding large volumes of data, it’s important to note that big data is not just a big database.

Big data is also concerned with the frequency at which data is generated, captured and stored – that is, the Velocity.  No longer are we content with loading historical data warehouses long after the trade has been executed or the fraudulent claim processed – there are growing business needs for real-time and near real-time processing of information.

The third V as it relates to big data is Variety.  Data no longer fits into neatly structured tables, but instead incorporates geo-spatial, machine logs, sentiment, physical data points, social, text and web to name a few.  Big data includes all data.

There is a wealth of articles and blogs written about big data on the Web if you need to know more.  As you continue your research into this hot topic, just remember, it’s not the size of the data that counts, but what you do with it.

Solving the issue of big data is no small feat.  But even then, it’s just data.  As previously discussed in The Yin Yang of BI, capturing and managing data by itself does not deliver any value to the business.  It’s how you use and interrogate that data to derive new insight and make more informed decisions that can truly transform organisational performance.

So once you’ve considered the 3 V’s of big data, consider the 3 W’s of analytics – Who, What and When.

What business decisions need to be made?  What insight can we derive from the information that we’ve captured to help support those business decisions?  That is, we need to turn data into insight.  Big data makes this process infinitely complex.  For example, imagine capturing information about what people are saying about your brand in social media.  In this case, a simple report would be near impossible to produce and likely to wield no real value to the business because of how much data it would contain.  Using analytical methods that are designed to cope with the volume, velocity and variety of big data, we can derive insight into the sentiment and buzz words of the market as a whole – providing the business with insight that it can both understand and act upon.

Who could benefit from the information that we’re capturing?  Who needs to know what’s really happening or what might happen in order to make better business decisions?  To be of value, information, or more importantly insight, must be made easily available to the people that need it in order to support their business decisions.  In today’s pervasive world, that may be in the form of traditional reports and dashboards, or mobile notifications and disconnected analytical tools.  In the sea of big data, there are countless amounts and types of information available – now more than ever it’s important to ensure only what is most relevant to an individual decision maker is available as- and when- needed.  This step in the process is how we turn insight into action.

When do they need to know in order to make a more informed decision?  Information that comes in with greater velocity is processed in real-time for a reason – decisions need to be made in real-time.  This makes it imperative to ensure insight is being derived at the time new information is captured and being incorporated into analytical decision management systems.  Even when real-time is not a requirement, it is still an imperative to make sure the right information is made available to the decision maker in time to take action.

IBM is leading the way in addressing both the 3 V’s of big data with InfoSphere BigInsights and Netezza Data Warehouse Appliances, as well as the 3 W’s of analytics with SPSS and Cognos.

This is the first of many posts looking at how big data is impacting specific industries and how to architect solutions to derive big insights from big data.  To be notified of follow up posts make sure you subscribe for email notifications at the bottom of this page.