The Difference Between Big Data and a Lot of Data

The Difference Between Big Data and a Lot of Data

The term “big data” has been around for a while now, but I still come across people who make the same basic mistake when someone asks them to explain what exactly it is.

The problem, as I have pointed out in the past, is due to the name. Big data was never meant to be purely about the size of the data. Right from the start, when the first attempts were made to codify the “rules” of big data, this was the case.

Gartner’s famous “3 V’s” of big data were, in fact, minted to make this very point. In addition to data volume, data velocity and variety were identified as essential to understanding how and why information could be captured, analyzed, and learned from.

So, from the beginning, big data should have more accurately been labelled “big, fast and varied data” – although of course that doesn’t sound so catchy!

So, the problem is this: When clients approach me to work with them, they often say, “We already do big data.” What they mean is, they have big – often huge – datasets. However, they often will have it stored in traditional structured databases and will be used to interrogating it using SQL.

What they have is a lot of data. But that does not mean, by any stretch, that they are “doing big data.”

Related Stories

5 Signs You Are a Big Data Hoarder.
Read the story »

Big Data and Market Research Myths and Missteps.
Read the story »

The Big Data Landscape Requires Community, Collaboration.
Read the story »

Redefine Big Data for Your Business.
Read the story »

“Variety” in particular is a very important element of big data. Increasingly, much more data is becoming available to us in the form of messy, “unstructured” data. This includes the millions of photographs and videos uploaded to social media and the wider Internet, or captured on cameras and closed-circuit television in commercial or industrial settings. This data contains tremendous amounts of value to marketers or anyone who wants to understand the behaviour of people in a particular environment. After all, a picture paints 1,000 words – but only if we know how to read them.

It is combining this sort of new, messy, and exciting data with the traditional business analytics we have always carried out that makes “big data.” Not simply analyzing terabytes of structured financial data to answer simple questions such as, “What are our best-selling products and services?” While it is useful to know the answer to those kinds of questions, wouldn’t it be better to be asking, “Why are these our best selling products and services?”

A lot of data, on their own, are worthless. In fact, it’s worse than that – such data can be positively dangerous, as time and resources have to be spent storing it and keeping it safe from inappropriate eyes. And that’s even before you add in the time and resources that will be wasted if you try to do something with it without understanding what big data is all about.

When big data was emerging as a fashionable buzzword, a lot of people in business really did see it as simply a catch-all term for “a lot of data.” As a result, a lot of businesses spent a lot of time and money measuring, recording, and storing as much data as possible in the hope that, at some point, they’d work out how to glean some actionable insights from it.

These earnest but wrong-headed endeavors were so common that the phrase “data rich but insight poor” became ubiquitous among critics of the “big data revolution.” And it was absolutely a fair comment.

But in the years that have passed, those who truly have grasped the meaning beyond the unfortunate label of big data have shown that it absolutely, unquestionably is possible to generate tremendous value and growth from it, in every industry from banking, finance, and insurance to disaster relief and fighting cancer.

What all of the companies and organizations that have excelled in this field have realized right from the start is that, when it comes to data, it isn’t the size that’s important, it’s what you do with it.

The key point I want to make here is that there is a vast difference between “having a lot of data” and “doing big data.” When you have a large data set that is fast moving, ever changing, and includes unstructured data, and when you are using distributed storage and in-memory analytics, then we are talking big data!

This is why I prefer the term “smart data,” which emphasizes that thinking intelligently about what to do with your data, and how you can use it to achieve your aims, is far and away a more important element of the big data equation than the simple size.

There’s nothing at all wrong with collecting a lot of data. After all, one of the key principles of big data is that the more you record, the more accurately your sample will reflect reality when it comes to the simulations and modelling where the real value is found.

But if you are considering setting off on a big data adventure yourself, it’s important to remember that there’s far more to big data than size.

Bernard Marr is a bestselling author, keynote speaker, strategic performance consultant, and analytics, KPI, and big data guru. He helps companies to better manage, measure, report, and analyze performance. His leading-edge work with major companies, organizations, and governments across the globe makes him an acclaimed and award-winning keynote speaker, researcher, consultant, and teacher.

- See more at: http://data-informed.com/the-difference-between-big-data-and-a-lot-of-data/#sthash.4edoYckX.dpuf