Big Data is dead, long live Smart Data

I’m not a data scientist. But I wish I were. I’ve always had the greatest of respect for them and I love working with them. If I were starting out in planning again today, I’d definitely ensure I acquired those skills.

Of course, there have been some pretty big claims being made concerning Big Data’s potential. But whilst there’s been demonstrable progress thus far in terms of harnessing this potential to reduce crime rates or identify the harmful side effects of mixing certain medications for example, it’s important to recognise that there’s still a fair way to go.

Consider for a moment the fact that less than one per cent of the more than 2.8 trillion gigabytes of data generated annually is being used for analysis. Moreover, only about three per cent of it is tagged and ready for manipulation. And as the old adage goes, it’s not the size of your data; it’s what you do with it that counts.

Beware the power of prediction

From a marketing perspective, the first major shift came with the use of addressable data. This was the data that powered the direct marketing revolution of the 1980s. Next came the addition of transactional and demographic data to help us target, profile and segment consumers. More recently, we have begun to use detailed behavioural, attitudinal, cultural and emotional data to enrich our understanding of consumers.

Arguably one of the most exciting aspects of harnessing Big Data is our ability to predict a consumer’s future behaviour. The marketers at a well-known US retailer for example, identified that a key customer segment for them was mothers to be. So their analyst colleagues developed a pregnancy prediction model based on a customer’s purchasing history of 25 specific products.

The retailer’s model was so accurate that it was able to estimate a woman’s due date to ‘within a small window’, enabling its marketers to send tailored advertising materials to these women promoting products that could help them at various stages of their pregnancy. Of course, like the discipline of meteorology, it was by no means an exact science, and there were some unintended outcomes.

In the case of this retailer, there was an incident where an angry father brandishing coupons for baby clothes that had been sent to his teenage daughter visited one of its stores. Having accused the store manager of encouraging his daughter to get pregnant and then storming out, it subsequently transpired that the young women in question was indeed expecting.

Although the power of prediction should always be wielded with caution, the fact remains that if we can use Big Data to increase our predictive ability by just a few percentage points, there is a compelling opportunity to drive significant improvement in ROI. More importantly, this story exemplifies why we need to stop talking about Big Data and instead focus on Smart Data.

The retailer in question didn’t achieve this degree of accuracy by aimlessly collecting and analysing all of the audience data it had at its disposal. Rather, it set out with a clear objective and gathered data to help achieve it.

In other words, it had a data plan. I believe that all brands should have their own data plan. And while no two organisations are alike, in my opinion there are three key factors that they should all consider when it comes to data – volume, speed and scope.


Let’s start with volume. I used to work for one of the supermarket loyalty programmes with up to 65,000 stock keeping units (SKUs) and millions of customers. One of my colleagues there described capturing data as being like trying to drink from a fire hose. Today, I imagine these figures are much higher and the challenge is far greater.

If you are a large brand, the volume of data flooding into your organisation on a daily basis is almost imperceptible and it’s impossible for you to analyse all of it, at least with current software. You therefore need to be selective and focused in your collection and analysis. Determining what to focus on is a challenge in itself, but a good starting point is choosing data that is easily actionable.


The next key consideration is the speed with which data flows into your brand. Although keeping pace with the torrent of data pouring into your organisation can be difficult, if you do manage to do so, you will reap the rewards. Being fleet of foot means you can react to statistics in real-time to engender more natural interactions with customers and seize time-sensitive opportunities.

One way to do achieve this level of agility is to implement triggers that are initiated when a consumer behaves in a certain manner. One of my financial services clients employed this approach to sell pet insurance. He had access to the supermarket spending data of his audience and when a customer purchased pet food they would receive an automated email from his company carrying a sales message within 24 hours. If you contrast this with his previous approach, which was based around regular CRM campaigns, this strategy is much more efficient, targeted, timely and relevant.


The third important consideration is scope. The vast amount of data I mentioned earlier is generated at numerous sources and determining which to focus on is a crucial decision for brands. As digital data is plentiful and undoubtedly valuable, many brands tend to prioritise it, but it’s seldom an isolated driver of behaviour. To build an accurate picture of customers, I argue that you need to integrate digital data into multi-platform marketing analysis.

Bringing analysis to life

The next challenge is how all of this data should be analysed. At my organisation, Karmarama, we take an approach called ‘data collision’. Essentially, we compare and contrast an eclectic mix of data about a consumer to generate a rounded snapshot of this individual. This data can be any information, from any source – for example, user-generated images, vox pops or tweets.

In my opinion, the more variety there is in the mix, the better. We can cross-reference qualitative and quantitative data, as well as emotional and behavioural data. In addition to exposing fresh insights, this approach capitalises on the rich nature of this data to bring analysis to life.

The aforementioned steps represent a skeleton strategy to effective data management. Nevertheless, by building a plan around them you can extract valuable insights from consumer information that can be actioned easily to deliver impressive results. Not only will you be able to identify and seize opportunities on the fly, you’ll be able to drive revenue and delight customers. Unless you’re unwittingly revealing a teenage pregnancy to unsuspecting parents of course…

Leave a Reply

%d bloggers like this: