Net Promoter - Criticism of NPS

Criticism of NPS

Despite its popularity among business executives, the Net Promoter concept has attracted some controversy from academic and market research circles. Research by Keiningham, Cooil, Andreassen and Aksoy disputes that the Net Promoter metric is the best predictor of company growth. Furthermore, Hayes (2008) claimed there was no scientific evidence that the "likelihood to recommend" question is a better predictor of business growth compared to other customer-loyalty questions (e.g., overall satisfaction, likelihood to purchase again). Specifically, Hayes stated that the "likelihood to recommend" question, does not measure anything different from other conventional loyalty-related questions.

Environmental factors may exert an influence on customers' response to the "recommend" question—making comparisons across business units or industries difficult in certain cases. Examples include comparing businesses with an associated social stigma (e.g., cigarettes or online dating) and businesses with different levels of service fulfillment (e.g., delivery services as compared to gyms). Moreover, determining when the survey should be delivered may be more obvious in some cases than in others (such as in the case of a gym), where customer attitudes may be likely to change over time.

Daniel Schneider, Jon Krosnick, et al. found that out of four scales tested, the 11-point scale advocated by Reichheld had the lowest predictive validity of the scales tested.

Others have taken issue with the calculation methodology, claiming that by collapsing an 11-point scale to three components (e.g., Promoters, Passives, Detractors), significant information is lost and statistical variability of the result increases. The validity of NPS scale cut-off points across industries and cultures has also been questioned.

Proponents of the Net Promoter approach point out that the statistical analyses presented prove only that the "recommend" question is similar in predictive power to other metrics, but fail to address the practical benefits of the approach, which are at the heart of the argument Reichheld put forth. Proponents of the approach also counter that analyses based on third-party data are inferior to analyses conducted by companies on their own customer sets, and that the practical benefits of the approach (short survey, simple concept to communicate) outweigh any statistical inferiority of the approach.