What to ask your data analysts

by CXOtoday News Desk    Jul 16, 2013

Data analysts

We live in an era of big data. Whether you work in financial services, consumer goods, travel and transportation, or industrial products, analytics are becoming a competitive necessity for your organization. But as the banking example shows, having big data—and even people who can manipulate it successfully—is not enough. Companies need general managers who can partner effectively with “quants” to ensure that their work yields better strategic and tactical decisions.

Thomas H Davenport in his blog in The Harvard Business Review writes that you might be adept at using spreadsheets and know your way around a bar graph or a pie chart, but when it comes to analytics, we often feel quantitatively challenged.

So what does the shift toward data-driven decision making mean? Davenport lists out how to get around it in this article—a primer for non-quants that is based on extensive interviews with executives, including some with whom he has worked as a teacher or a consultant.

You, the Consumer

Start by thinking of yourself as a consumer of analytics. The producers are the quants whose analyses and models you’ll integrate with your business experience and intuition as you make decisions. Producers are, of course, good at gathering the available data and making predictions about the future. But most lack sufficient knowledge to identify hypotheses and relevant variables and to know when the ground beneath an organization is shifting. Your job as a data consumer—to generate hypotheses and determine whether results and recommendations make sense in a changing business environment—is therefore critically important. That means accepting a few key responsibilities. Some require only changes in attitude and perspective; others demand a bit of study.

Learn a little about analytics: You need to understand the process for making analytical decisions, including when you should step in as a consumer, and you must recognize that every analytical model is built on assumptions that producers ought to explain and defend.

When using big data to make big decisions, non-quants should focus on the first and the last steps of the process. The numbers people typically handle the details in the middle, but wise non-quants ask lots of questions along the way.

1. Recognize the problem or question: Frame the decision or business problem, and identify possible alternatives to the framing.

2. Review previous findings: Identify people who have tried to solve this problem or similar ones—and the approaches they used.

3. Model the solution and select the variables: Formulate a detailed hypothesis about how particular variables affect the outcome.

4. Collect the data: Gather primary and secondary data on the hypothesized variables.

5. Analyze the data: Run a statistical model, assess its appropriateness for the data, and repeat the process until a good fit is found.

6. Present and act on the results: Use the data to tell a story to decision makers and stakeholders so that they will take action.

What Big Data consumers must do

Align yourself with the right kind of quant: Karl Kempf, a leader in Intel’s decision-engineering group, is known at the company as the “überquant” or “chief mathematician.” He often says that effective quantitative decisions “are not about the math; they’re about the relationships.” What he means is that quants and the consumers of their data get much better results if they form deep, trusting ties that allow them to exchange information and ideas freely.

Focus on the beginning and the end: Framing a problem—identifying it and understanding how others might have solved it in the past—is the most important stage of the analytical process for a consumer of big data. It’s where your business experience and intuition matter most. After all, a hypothesis is simply a hunch about how the world works. The difference with analytical thinking, of course, is that you use rigorous methods to test the hypothesis.

Ask lots of questions along the way: No matter how much you trust your quants, don’t stop asking them tough questions. If you don’t understand a reply, ask for one that uses simpler language.

1. What was the source of your data?

2. How well do the sample data represent the population?

3. Does your data distribution include outliers? How did they affect the results?

4. What assumptions are behind your analysis? Might certain conditions render your assumptions and your model invalid?

5. Why did you decide on that particular analytical approach? What alternatives did you consider?

6. How likely is it that the independent variables are actually causing the changes in the dependent variable? Might other analyses establish causality more clearly?

Establish a culture of inquiry, not advocacy

We all know how easily “figures lie and liars figure.” Analytics consumers should never pressure their producers with comments like “See if you can find some evidence in the data to support my idea.” Instead, your explicit goal should be to find the truth.

When It All Adds Up

Warren Buffett once said, “Beware of geeks…bearing formulas.” But in today’s data-driven world, you can’t afford to do that. Instead you need to combine the science of analytics with the art of intuition. Be a manager who knows the geeks, understands their formulas, helps improve their analytic processes, effectively interprets and communicates the findings to others, and makes better decisions as a result.