The Difference Between Big Data and a Lot of Data

by   |   September 21, 2015 5:30 am   |   8 Comments

Bernard Marr

Bernard Marr

The term “big data” has been around for a while now, but I still come across people who make the same basic mistake when someone asks them to explain what exactly it is.

The problem, as I have pointed out in the past, is due to the name. Big data was never meant to be purely about the size of the data. Right from the start, when the first attempts were made to codify the “rules” of big data, this was the case.

Gartner’s famous “3 V’s” of big data were, in fact, minted to make this very point. In addition to data volume, data velocity and variety were identified as essential to understanding how and why information could be captured, analyzed, and learned from.

So, from the beginning, big data should have more accurately been labelled “big, fast and varied data” – although of course that doesn’t sound so catchy!

So, the problem is this: When clients approach me to work with them, they often say, “We already do big data.” What they mean is, they have big – often huge – datasets. However, they often will have it stored in traditional structured databases and will be used to interrogating it using SQL.

What they have is a lot of data. But that does not mean, by any stretch, that they are “doing big data.”

Related Stories

5 Signs You Are a Big Data Hoarder.
Read the story »

Big Data and Market Research Myths and Missteps.
Read the story »

The Big Data Landscape Requires Community, Collaboration.
Read the story »

Redefine Big Data for Your Business.
Read the story »

“Variety” in particular is a very important element of big data. Increasingly, much more data is becoming available to us in the form of messy, “unstructured” data. This includes the millions of photographs and videos uploaded to social media and the wider Internet, or captured on cameras and closed-circuit television in commercial or industrial settings. This data contains tremendous amounts of value to marketers or anyone who wants to understand the behaviour of people in a particular environment. After all, a picture paints 1,000 words – but only if we know how to read them.

It is combining this sort of new, messy, and exciting data with the traditional business analytics we have always carried out that makes “big data.” Not simply analyzing terabytes of structured financial data to answer simple questions such as, “What are our best-selling products and services?” While it is useful to know the answer to those kinds of questions, wouldn’t it be better to be asking, “Why are these our best selling products and services?”

A lot of data, on their own, are worthless. In fact, it’s worse than that – such data can be positively dangerous, as time and resources have to be spent storing it and keeping it safe from inappropriate eyes. And that’s even before you add in the time and resources that will be wasted if you try to do something with it without understanding what big data is all about.

eBook: Big Data Analytics: Advice from Big Data Guru Bernard Marr

When big data was emerging as a fashionable buzzword, a lot of people in business really did see it as simply a catch-all term for “a lot of data.” As a result, a lot of businesses spent a lot of time and money measuring, recording, and storing as much data as possible in the hope that, at some point, they’d work out how to glean some actionable insights from it.

These earnest but wrong-headed endeavors were so common that the phrase “data rich but insight poor” became ubiquitous among critics of the “big data revolution.” And it was absolutely a fair comment.

But in the years that have passed, those who truly have grasped the meaning beyond the unfortunate label of big data have shown that it absolutely, unquestionably is possible to generate tremendous value and growth from it, in every industry from banking, finance, and insurance to disaster relief and fighting cancer.

What all of the companies and organizations that have excelled in this field have realized right from the start is that, when it comes to data, it isn’t the size that’s important, it’s what you do with it.

The key point I want to make here is that there is a vast difference between “having a lot of data” and “doing big data.” When you have a large data set that is fast moving, ever changing, and includes unstructured data, and when you are using distributed storage and in-memory analytics, then we are talking big data!

This is why I prefer the term “smart data,” which emphasizes that thinking intelligently about what to do with your data, and how you can use it to achieve your aims, is far and away a more important element of the big data equation than the simple size.

There’s nothing at all wrong with collecting a lot of data. After all, one of the key principles of big data is that the more you record, the more accurately your sample will reflect reality when it comes to the simulations and modelling where the real value is found.

But if you are considering setting off on a big data adventure yourself, it’s important to remember that there’s far more to big data than size.

Bernard Marr is a bestselling author, keynote speaker, strategic performance consultant, and analytics, KPI, and big data guru. He helps companies to better manage, measure, report, and analyze performance. His leading-edge work with major companies, organizations, and governments across the globe makes him an acclaimed and award-winning keynote speaker, researcher, consultant, and teacher.

Subscribe to Data Informed
for the latest information and news on big data and analytics for the enterprise.

Tags: , , ,


  1. Lee Wood
    Posted September 22, 2015 at 1:16 pm | Permalink

    Agree. Far better to know what data needs to be collected and why AND to ask “why” throughout the analysis process to identify compelling and relevant results.

  2. Posted September 23, 2015 at 12:42 pm | Permalink

    I agree, “smart” data is a better term. Data is used to support group intelligence. Data has to be flowing to be of value. If you don’t add value too data, you have nothing. It isn’t the data we want to be protect, but the correlation among data items.

  3. Maria Barros Weiss
    Posted September 25, 2015 at 9:19 am | Permalink

    Should have been called key data. To be smart means more than anlysis, needs actuation. Big data is more only on the side of monitoring not actuating, so I’d call it Key Data.

  4. Marilize Tapsell
    Posted September 27, 2015 at 1:34 pm | Permalink

    Agree. Also no point having smart data if you are not smart about how you go about doing the analysis & analytics of the data you hold, from which to create meaningful and value added insights.

  5. Posted September 27, 2015 at 5:40 pm | Permalink

    That’s a common misconception Tom. Data is no different than a can of soup. It has value even before someone’s eating it. However, soup is a balance sheet asset whereas antiquated accounting standards do not allow data to be recognized. This shouldn’t stop orgs from behaving as if it is. See for more on infonomics. –Doug Laney, VP Research, Gartner, @doug_laney

  6. Posted September 27, 2015 at 5:41 pm | Permalink

    Thanks for the shout-out Bernard. Good to see Gartner’s/my 15 year old “3Vs” definition of Big Data still useful.

  7. Neftaly Dominguez
    Posted April 20, 2016 at 12:56 pm | Permalink

    Excellent! A very good explanation, deep and clear.

  8. Posted September 17, 2017 at 9:16 am | Permalink

    Thank you very much for this clear definition. Still very true in 2017!

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>