Find Your Co-Data

"Co-data" is my term for data that goes well and augments your core data set. I particularly like that The Weather Channel has found consumer behavior data predicting what you will buy depending on the weather. You don't need The Weather Channel's giant data set to find this data set. It could be as easy as looking at your fellow local businesses' websites.

Let's say you're a cab driver. You want to minimize wait times and maximize distance driven. How about finding out when colleges in your area start up again? Or checking out when a particular bar closes? Or finding out the time a particular show (preferably one with drunken attendees interesting in safe-cabbing it home) gets out?

I tried this simple method when I worked at PPG Industries. Of course, our sales of exterior paint increased when the weather got pleasant. Pulling free data off the NOAA Climate Data Center enabled me to do some rudimentary comparisons between our past sales by region and temperature. I found that people start painting more at about 50 degrees F, and that over about 84 degrees F the amount they paint starts to drop off (too hot out).

Using such basic data and simple correlation, I was able to optimize the load-in for our largest retailer's stores so that we had enough exterior paint early in the season... but not too early. I also found that using last year's sales to predict when we should ship this year was a lousy measure; better to use the average over the past three years and then build back two weeks for safety.

At Vocollect, we're discovering lots of cool ways to use the information we have to make our workers' lives easier. We're helping by giving simple suggestions such as prompting the user to access a feature when we notice the feature could be used to solve a problem we deduce the worker is having. The next phase will be to combine this user data with simple information we have from other sources to help suggest, for example, how two coworkers can avoid each other in a distribution center aisle to ease congestion delays.

All this work goes back to my feeling about big data: you don't need it if you have plain old "data" that you're not using in the first place.

A/B Testing For Everyone

The folks over at the phenomenal Marketing Experiments Blog had yet another post about A/B testing that reminded me of some consulting work I did in the past. Often, I have found that organizations think you have to be a gigantic company to do A/B testing. The reality is that a company of any size can A/B test just about anything, sometimes to dramatic effect. And a small company can apply very sophisticated marketing analysis very inexpensively in this age of free, high-powered statistical languages.

When I worked for Strategic Energy, management believed we couldn't just send our customers a contract and re-sign them for three years of electricity usage. I said, "What's the harm in trying?" We sent a hundred customers a thank-you for letting us serve them along with a new contract for service. About 35 of them sent us back a signed contract. How much did that test cost? About $300 and half a day of work. After that experience, Strategic Energy started sending every customer under a certain size a renewal contract, saving tens of thousands in sales costs per year for those that responded.

We then sent out postcards to the remaining customers plus about 200 more asking them to contact us about their contract renewal. On one postcard, we put an existing customer photo and an inspirational message about saving their business money. On the other postcard, we put a funny beach photo and a message to the effect of, "Wouldn't you rather be spending your time on the beach than renewing an electricity contract?" We assigned customers randomly to one or the other. To our surprise, the beach one got a statistically significantly better response. Simple A/B test done. Learning learned.

I applied this kind analysis to the funding solicitation work of the Jewish Federation of Greater Pittsburgh to equally powerful effect. In this case, some simple linear regression showed that of the greatest factors influencing the size of the gift was whether the gift was given online (even when holding donor age constant). Pushing customers to the website to donate increased the size of the gifts, and some tweaking to the website itself increased gift sizes even further. All that we needed to complete this analysis was a history of donations and some basic information about the donors and when they responded.

The barrier to basic A/B testing usually lies in company culture, not in cost or capabilities. Companies need to get wired for a "learning culture" that emphasizes marketing science over gut feel. This change must emanate from the senior executive team, and they have to understand how powerful data management and analysis can be to improve marketing response rates, revenues and profits.

As analysis professionals, we need to bring these smarts to the executive team so that they can bring culture change to the rest of the company. I try to remind myself of this goal periodically when I find myself tiring of yet another explanatory meeting with the VPs. Although sometimes repetitive and tiresome, the meetings to explain what we are planning to do after we test result in the executive support necessary to internalize the learning from the testing over the long term.

In Praise of Small Data

I can't read anything these days without hearing about "big data." Just popped over to Google News today and learned that Cloudera, a company basically distributing an easier-to-use version of open-source Hadoop as I understand it, raised $65 million in a valuation pegging them as a $700 million company. Holy mackerel!

These crazy valuations put me in mind of what I call "small data." If big data means synthesizing meaning from a million different pieces of disparate information coming from a variety of sources, little data means synthesizing meaning from several thousand pieces of information. In the former case, think of my company Vocollect's wearable computers collecting thousands of bits of information about thousands of distribution center picks per day from hundreds of thousands of workers. In the latter case, think of my company's less than five thousand customers.

Of course there are exciting things to be discovered from the millions of interactions we see from the wearable computers. But there are even more valuable things we could learn from our existing customer base, and I have found that most companies--even gigantic, multi-billion dollar ones--are sorely lacking in the ability to aggregate, clean, and take meaning from these existing customers.

Back at one of my last jobs, we found after six months of aggregating and cleaning that 25% of our sales were coming from 300 customers out of 40,000. You might hear people talk about the "80/20" rule, but that's the "25/1" rule for those of you keeping track. As in, "25% of our revenue comes from 1% of our customer base"! You better bet that the sales leadership, marketing department, customer service team, and even the VP now know the names of every single one of those 300 customers and that the company treats them a lot better than they used to.

Little data is about making small investments in technology, process, and people power to get better information that you should already have access to today. The focus requires all three:

1. Technology: This is the area everyone always thinks about when data analysis discussions bubble to the surface. Here, I advocate both investments in technology to store the data like Salesforce.com, but also technology to clean the data so it's not completely worthless. How useful is it to sell your brand new freezer-rated wireless headset to current customers if you don't know which ones have freezers? Acquiring the information that's missing requires the second investment...
2. Process: Great "little data" companies fix the problems of who is responsible for information-gathering, how the information gets into the system in the first place, how you compare it against other systems to ensure links and accuracy, and how it gets cleaned and updated over time. Each of these process fixes ensures that when marketing or sales or finance go to use the information, it gives an accurate and up-to-date picture of the business. That's not possible without...
3. People power: Great companies assign responsibilities and ownership for the information and, yes, pay for it when necessary. The CEB, my first company, was better at this than any company for which I have ever worked. The way they ensured information was retained was to withhold sales commissions unless the information made its way into ELvIS, our Enterprise-Level Information System (precursor to a real CRM). ELvIS was, by the way, built on MS Access but worked just fine for a long time because of the people and process controls in place. Proving that you don't need a top-flight CRM until the body of data gets too large to manage.

Don't get me wrong. I am generally a huge fan of big data. That's one of the reasons I continue to be bullish on Google, the company with more data than possibly any other company in the world (and a company that understands its value). I'm just saying that small- to medium-sized companies can do amazing things with little data if they pay attention to it and manage it well. That's why you need to hire somebody with experience in this kind of "little data" program and then put serious management attention and focus around it.

A little self-promotion here: I have a lot of experience with "little data." If you ever want to get serious about selling to your existing customers and finding more customers that look like your existing ones, give me a call.