NPS Weaknesses

In the December 2003 Harvard Business Review article, “The One Number You Need to Grow”, Frederick Reichheld claims a single summary number from one customer survey question is a sufficient basis for measuring and managing customer loyalty.

Customers answer the question on a 0-to-10 scale: “How likely is it that you would recommend [company X] to a friend or colleague?” Anyone rating 0 to 6 is labeled “detractor”, 7 or 8 “passively satisfied”, and 9 or 10 “promoter”. The percent promoters minus the percent detractors is the one number Reichheld advocates ‘“ the so-called “net promoter score.” According to Reichheld, “This number is the one number you need to grow. It’s that simple and that profound.” p. 54.

Actually, there are several critical logical, conceptual and statistical problems with Reichheld’s proposition.

Testing the logic
Business leaders should challenge this strategy with the following types of questions:

• Is increasing recommendation really the best way to drive business success?
• Will it have more impact than reducing customer loss?
• If I lose 35 percent of my customer base per year, but most of those who stay would recommend, am I really in good shape?
• Will recommendation be more powerful than increasing current customers’ volume, cross-sales, or share of purchase?
• Will it be more powerful than more controllable marketing actions aimed at targeted acquisition of profitable new customers?

Obviously recommendations are important, particularly in certain sectors and markets. But are they the main thing, the one thing that companies need to grow and manage business success? Reichheld’s case seems implausible.

Interestingly, Reichheld contradicts his own past writing. He previously argued that recommendation is an outcome of loyalty, not an indicator of the construct itself. He focused more on customer loss, noting that reducing loss by even 5 percent radically multiplies profitability. Are increased recommendations the key to that kind of reduced loss? Not likely. Reichheld’s past writing also emphasizes sustaining base profit across time through retention and increasing customers’ volume, share of purchase, new product purchases, services cross-sold, and so on. Somehow, those powerful outcomes of loyalty are now supplanted by emphasis on recommendations alone.

Confusion about cause and effect
Perhaps Reichheld’s motivation was good but his methodology was flawed. Perhaps he chose recommendation through a classic statistical mistake ‘“ interpreting correlation as if it implies causation. In a statistical search through survey items to see which correlated most with certain business measures, he concluded recommendation usually was best. “… A strong correlation existed between net-promoter figures and a company’s average growth rate … Remarkably, this one simple statistic seemed to explain the relative growth rates …” p. 51.

Clearly, he is inferring causation, advocating management of the net promoter score as a means to realize enhanced business performance. So just raise referral rates to drive business success, right? Why not then just incentivize customer referrals? That would make them go up. However if the business model is bad, don’t look for sustainable success. Reichheld’s former work is more plausible: manage the underlying cause of both referrals and business success ‘“ namely loyalty. Those trained in scientific methods will instantly recognize the significant difference between the following two “path models”:

Recommendations > Business Success

Loyalty > Recommendations > Business Success

“What causes what” really matters if you care about managerial control, not just statistical relationship. And measuring and managing recommendations is not the same as measuring and managing loyalty ‘“ not conceptually, managerially, or from a scientific measurement standpoint (psychometrically). Best scientific practice requires clear distinctions between antecedents, outcomes, and indicators (i.e., causes, effects, and manifestations respectively). We may want to monitor the effect (recommendations), but what we want to manage is the cause (loyalty). After all, if measuring the effect were enough, why wouldn’t we merely track financials? Couldn’t we say revenue, or market share, or profitability is the one number we need?

Recommendation is an effect. It is fine to monitor it as a sign of business health. However, to control upward movement, we’ve got to manage the underlying cause. And that leads to another major flaw in Reichheld’s approach.

No specific guidance for enterprise actions
Reichheld’s one number may be a sign of business health, but it provides no direction about what to do. If you have a 104-degree fever it calls for improvement. But merely having the number doesn’t diagnose the specific health problem or tell how to treat the situation. Reichheld misses this by implying one number is sufficient to drive improvement action. He writes, “The managerial charge, ‘We need more promoters and fewer detractors in order to grow.’ The goal is clear-cut, actionable, and motivating.” p. 54. It may be motivating and clear-cut, but it is far from actionable. What specifically should a company improve, among all possible things it could improve, to move the score upward? Good luck to the manager left to speculate about what specific actions need to be taken.

Reichheld even seems to acknowledge this at one point, contradicting his premise and explicitly conceding the point in a momentary lapse from his one-question mantra. “Follow-up questions can help unearth the reasons for customers’ feelings and point to profitable remedies. But such questions should be tailored to the three categories of customers. Learning how to turn a passively satisfied customer into a promoter requires a very different line of questioning from learning how to resolve the problems of a detractor.” p. 53.
Follow up questions? What happened to “one number?” Apparently, he understands more than one question ultimately will be needed. Thus his case is self-defeating.

Other problems
There are several other problems, details of which space will not allow. To name some in laundry list fashion, these problems include:

• Reichheld’s definition of loyalty is out of step with the best loyalty theorists.
• He takes shots at research suppliers when he himself sits on the board of a company selling in that space.
• He implies real-time customer data requires short surveys.
• Reichheld makes conclusions about cause and effect which violate scientific principles (e.g., “effect” happening prior to “cause”).
• His net promoter statistic can produce the same score under very different scenarios.

Taken in total, logical, conceptual, and statistical problems converge to one conclusion ‘“ one number is not all you need to grow.

By: Doug Grisaffe, Ph.D., Professor of Marketing, University of Texas, Arlington