Hurray for Accurate Depictions of Big Data

I have long been a fan of the incredible, eclectic blog BoingBoing.net. Cory Doctorow today has this excellent review of a book on Big Data. In the review, he describes big data as:

"a computational approach to business, regulation, science and entertainment that uses data-mining applied to massive, Internet-connected data-sets to learn things that previous generations weren't able to see because their data was too thin and diffuse."

Awesome definition. Notice that "big data" means massive, Internet-connected data sets. Analyzing your CRM data is not big data. It's just data. Applying weather corrections to sales (which some companies have been doing for 25 years) is not Big Data. It's just data. Figuring out your customers' various warehouse sizes in order to tailor solutions to them is not Big Data. It's just data.

If you have been following my blog, you know that I believe passionately about the value of small data. Most companies do not use the data they have. Therefore, I would assert that these companies are ill-advised to investigate Big Data. Rather, they would be better off figuring out what customers want and how to aggregate the information they already have to serve those needs.

In a consulting engagement I had when I first moved to Pittsburgh, I met the CEO of a large regional grocery. He said these exact words to me: "Our problem is that we have all this data, but we don't know what to do with it." Unfortunately, the next moment he was pulled away, and I never got to say to him what I wanted to say:

• Figure out how customers could help themselves and provide the data to them. For example, let customers opt in to a system that links pharmacy information with shopping data and then let customers scan foods to ensure that they don't run afoul of prescription or health restrictions such as salt content. The grocery would consolidate pharmacy sales with them and provide a great service.
• Figure out what products sell well together. For example, determine how sales of core items such as spices or core canned vegetables such as kidney beans affect the sales of other items that might be in a recipe and then adjust inventory levels to ensure the critical items are always in stock.
• Attack low-profit brands with house brands. Purposefully stock out of the national brand on occasion and see who switches to the store brand and what type of person doesn't switch. Target incentives to the non-switchers and align pricing and shelf displays to maximize house brand sales.
• Provide a way to scan products on the grocery cart itself. Use this data to negotiate with suppliers and optimize brand mix by seeing what products customer consider before they decide on a brand.
• Capture location-based information on the grocery cart. Use this information in conjunction with sales to reorganize higher-value items in locations where the grocery carts pass more frequently.
• Analyze sales at a particular time of day to see what high-profit items might fall in popularity. Time screen-based in-store advertising to promote those products at the "off times."

There are so many ways to capture the value of "small data" that many companies just do not consider. Why invest millions of dollars in "Big Data" when you aren't using the data you have? And why not combine "Big Data" approaches with existing "small data" to amaze and please your customers? You don't need a genius to get started on either project.