In Praise of Small Data

I can't read anything these days without hearing about "big data." Just popped over to Google News today and learned that Cloudera, a company basically distributing an easier-to-use version of open-source Hadoop as I understand it, raised $65 million in a valuation pegging them as a $700 million company. Holy mackerel!

These crazy valuations put me in mind of what I call "small data." If big data means synthesizing meaning from a million different pieces of disparate information coming from a variety of sources, little data means synthesizing meaning from several thousand pieces of information. In the former case, think of my company Vocollect's wearable computers collecting thousands of bits of information about thousands of distribution center picks per day from hundreds of thousands of workers. In the latter case, think of my company's less than five thousand customers.

Of course there are exciting things to be discovered from the millions of interactions we see from the wearable computers. But there are even more valuable things we could learn from our existing customer base, and I have found that most companies--even gigantic, multi-billion dollar ones--are sorely lacking in the ability to aggregate, clean, and take meaning from these existing customers.

Back at one of my last jobs, we found after six months of aggregating and cleaning that 25% of our sales were coming from 300 customers out of 40,000. You might hear people talk about the "80/20" rule, but that's the "25/1" rule for those of you keeping track. As in, "25% of our revenue comes from 1% of our customer base"! You better bet that the sales leadership, marketing department, customer service team, and even the VP now know the names of every single one of those 300 customers and that the company treats them a lot better than they used to.

Little data is about making small investments in technology, process, and people power to get better information that you should already have access to today. The focus requires all three:

1. Technology: This is the area everyone always thinks about when data analysis discussions bubble to the surface. Here, I advocate both investments in technology to store the data like Salesforce.com, but also technology to clean the data so it's not completely worthless. How useful is it to sell your brand new freezer-rated wireless headset to current customers if you don't know which ones have freezers? Acquiring the information that's missing requires the second investment...
2. Process: Great "little data" companies fix the problems of who is responsible for information-gathering, how the information gets into the system in the first place, how you compare it against other systems to ensure links and accuracy, and how it gets cleaned and updated over time. Each of these process fixes ensures that when marketing or sales or finance go to use the information, it gives an accurate and up-to-date picture of the business. That's not possible without...
3. People power: Great companies assign responsibilities and ownership for the information and, yes, pay for it when necessary. The CEB, my first company, was better at this than any company for which I have ever worked. The way they ensured information was retained was to withhold sales commissions unless the information made its way into ELvIS, our Enterprise-Level Information System (precursor to a real CRM). ELvIS was, by the way, built on MS Access but worked just fine for a long time because of the people and process controls in place. Proving that you don't need a top-flight CRM until the body of data gets too large to manage.

Don't get me wrong. I am generally a huge fan of big data. That's one of the reasons I continue to be bullish on Google, the company with more data than possibly any other company in the world (and a company that understands its value). I'm just saying that small- to medium-sized companies can do amazing things with little data if they pay attention to it and manage it well. That's why you need to hire somebody with experience in this kind of "little data" program and then put serious management attention and focus around it.

A little self-promotion here: I have a lot of experience with "little data." If you ever want to get serious about selling to your existing customers and finding more customers that look like your existing ones, give me a call.