Why supply chain reorganization now tops the CEO’s agenda
Amid mounting geopolitical tensions, CEOs must take direct control of supply chain resilience, says IMD’s Carlos Cordon...
Innovation
NPS: A (somewhat) useful metric, not a magic wand
Net promoter score has been heralded as one of the most useful metrics available to businesses today – but companies need to understand its limitations and avoid misusing it, warns Frédéric Dalsace, Professor of Marketing and Strategy at IMD.
Net promoter score (NPS®) has become a ubiquitous metric in the business world. NPS was developed in 2003 by Fred Reichheld, a partner at US management consultancy, Bain & Company, as a predictor of how firms would grow. Over the past 20 years, it has seen astonishingly widespread adoption: as many as two-thirds of Fortune 1000 businesses reportedly use NPS or one of its variants. Moreover, the metric has become hugely influential on business-strategy development: NPS is “more than a metric,” enthused one IBM employee in a 2021 article: “One could use the word religion.”
However, there are reasons for dissenting from this new “religion.” Challenges have been raised to the core methodology for calculating NPS, and to how organizations use – or misuse – it in mapping consumer responses. Understanding the inherent limitations of the metric, and the misguided thinking that leads businesses to lean too heavily on a single metric, could be the key to using it effectively. In short, there are three aspects to NPS: the good, the bad, and the ugly. Given the seriousness of these issues, we propose a series of actions that firms using NPS should take to counterbalance its weight.
The good
The first – and perhaps most powerful – reason NPS is so attractive to leaders is its simplicity. It is remarkably easy to understand and use. For those who are unfamiliar with it, NPS generates a single number through a simple technique of asking customers how likely they are to recommend the company to a friend or colleague. Answers are numeric, on a scale from 0 to 10, and responses are divided into three camps: promoters (scoring 9 or 10), neutrals (7 or 8), and detractors (0‒6). Subtract the number of detractors from promoters and, voilà, you have your NPS. (Reicheld has shared the methodology, enabling anyone to apply it.)
In businesses where word of mouth is a key promotional avenue, NPS is particularly important. It is scalable and easily adapted for use at every level, from promoting a business’s overall standing down to the performance of an individual customer service agent. Equally, it can be used across country or regional levels. There is also value in the fact that it has been so successful: as a widely accepted benchmark, it is a common currency that can be employed to make useful comparisons.
These are powerful advantages. What, then, are the drawbacks?
The bad
There are four fundamental criticisms to be made of NPS – or, at least, of how it is often used:
1. Validity
The most fundamental challenge comes from academic statisticians who analyzed NPS and failed to establish statistical validity. One early study found that “NPS is not significantly associated with either a firm’s sales growth or gross margin performance in our regressions.” Another, that “NPS has performed poorly as a predictor of brand/company performance.” Another concluded, simply, that: “NPS is neither superior nor inferior to other metrics.” Recent academic research has confirmed these results (“the methodological concerns raised by academics are valid”) to the point that the strong divergence between academics and practitioners on this topic has become itself an area of research. So, while NPS is commonly regarded as a useful metric, it cannot be perceived as a growth predictor or as a holy grail for business strategy.
“The most fundamental challenge comes from academic statisticians who analyzed NPS and failed to establish statistical validity.”
2. Wasted information
The method of calculating NPS – subtracting the number of detractors from promoters – wastes a lot of information. Neutrals scoring 7 or 8 are disregarded and variation within groups is ignored. For example, scores of 0 or 1 are counted the same as 5 or 6, yet with one group covering such a wide range of scores, they represent very different sentiments. To produce one simple metric, a huge amount of potentially valuable information is ignored. In fact, a simple average would be more diagnostic about the true situation.
3. Time-sensitivity
A further set of issues which are rarely well understood by NPS users relates to drifts in customer demographics that can occur over time. For one thing, there is a tendency to survey only “stayers.” Truly unhappy customers leave, and consequently cannot be subsequently surveyed. This could give an inflationary bias to NPS as negative opinions are naturally filtered out. Conversely, customer enthusiasm is prone to fading over time. When NPS is used in the early life stages of a new company or product, the customer base is, by definition, made up of early adopters – they have identified the product as something for which they have an immediate need and do not want to wait for the modificatory process of market opinion. Over time, that enthusiasm tends to wane. More broadly, customer expectations inevitably evolve as time passes; what they once perceived to be impressively innovative begins to seem mundane or commonplace, and scores may dip accordingly.
4. Cultural sensitivity
NPS is highly sensitive to cultural differences. Do North Americans, Brits, and the French express satisfaction or delight in the same way? The US grading system is widely perceived as being more generous than, say, the French one. Against that backdrop, the NPS scoring system assumes that these cultural differences are not meaningful. That is a particular issue for multinational businesses, yet many still use NPS to make comparisons across markets.
The ugly
Secondly, there is a problem in relation to NPS as a tool for influencing remuneration levels. This gives employees a strong incentive to introduce bias into the system for personal benefit. This could be direct, for instance when a salesperson or service provider asks a customer for feedback and pleads, “Please score 9 or 10, or I won’t get my bonus.” It may also be indirect, for example when communications soliciting feedback nudge participants more subtly toward positive scores.
Over the years, I have collected creative and terrifying examples that I show in my classes, to the stupefaction of the participants. In larger samples, there is even the potential for mechanized generation of false returns – gaming the system on an industrial scale and generating data that is completely unconnected to customer opinion and, therefore, of no use at all to analysts. In any of these cases, the bias seriously undermines the value of NPS as an accurate gauge of customer sentiment.
Those issues may not have been evident in the design process of NPS, but they have become significant as its use cases have developed.
Getting real value from NPS
Companies will want to retain NPS as a tool in the struggle to understand their markets; however, they will need to work on the “bad” and “ugly” elements in the following areas to ensure that they are using the tool in the most effective way possible.
1. Experiment with variations
Start with a simple average and follow this average overtime. Using a different range of scores for the categories may yield a more accurate picture – for example, 8-10 for promoters, not just 9 and 10 , or put the 6 in the passive category.
2. Add additional questions
Companies should avoid using NPS as their one key indicator of where to focus resources. NPS may be useful as one metric among many, giving a more balanced picture that considers a range of factors. Add additional questions relating to areas such as direct satisfaction, intent to repurchase, and include more open questions (“Tell us why…”), to allow customers to express their opinions more fully and in their own words. Consider wave-based research on usage and attitudes, and in-depth satisfaction surveys.
3. Draw on other sources
Given the limitations of NPS, companies should also look at objective performance measures (such as waiting time, critical incidents, etc.). For performance management, seek out other metrics, including behavior-based assessments by supervisors.
4. Control for possible drifts
To put NPS in context, organizations should seek to analyze data to test for drifts in composition of customer demographics over time and to survey changes in customer expectations. Comparing your NPS to a competitor’s can be helpful, as can benchmarking drifts against those of non-competing firms.
5. Use trends, rather than absolute numbers
Given the various cultural factors we touch on above, a trend-based approach is vital when comparing across regions or countries. Again, differences may be benchmarked with non-competing firms to provide context.
6. Closely monitor NPS field use
To tackle the issue of gaming, organizations should closely monitor the integrity with which NPS surveys are administered. Unusual favorable jumps in results should be checked thoroughly. Make use of “mystery shoppers” to vet the process from a user’s perspective.
To a certain extent, NPS has been a victim of its own success; what was developed as a useful but limited tool has been applied in ways for which it was never intended. Simply put, NPS should not be the number that businesses consider using to guide growth.
Rather than developing a quasi-religious obsession with NPS, organizations would be better advised to try to develop a more balanced dashboard, of which the NPS could remain one element. Not a magic wand.
Authors
Related
Tesla’s crossroads: Can Elon Musk’s stock-market darling retain an edge over rivals?
The EV pioneer’s once-unrivaled dominance is being tested by declining sales, fierce competition, and questions about its commitment to sustainability and innovation. IMD’s Michael Wade examines Tesla’s strategic direction and leadership. ...
Six things that help pick winning startups
After 25 years of selecting startups to work with IMD’s MBA and EMBA participants, Jim Pulcrano examines the factors that may have helped identify promising young ventures that go on to prosper....
AI-as-a-Service: be prepared for the next wave
15 April 2024 • by Annabelle Gawer in Innovation
AI-as-a-Service will impact executives and their strategic decisions. It’s crucial to assess the benefits and repercussions it will bring....
Audio available
Learn Brain Circuits
Join us for daily exercises focusing on issues from team building to developing an actionable sustainability plan to personal development. Go on - they only take five minutes.
Explore Leadership
What makes a great leader? Do you need charisma? How do you inspire your team? Our experts offer actionable insights through first-person narratives, behind-the-scenes interviews and The Help Desk.
Join Membership
Log in here to join in the conversation with the I by IMD community. Your subscription grants you access to the quarterly magazine plus daily articles, videos, podcasts and learning exercises.