Sunday, November 25, 2007

Relationships, Transactions and Heuristics

Background

There are two general types of customer satisfaction surveys: 1) Customer Transaction Surveys and 2) Customer Relationship Surveys. Customer Transactional Surveys allow you to track satisfaction for specific events. The transactional surveys are typically administered soon after the customer has a specific interaction with the company. The survey asks the customers to rate that specific interaction. Customer Relationship Surveys allow you to measure your customer’s attitudes across different customer touchpoints (e.g., marketing, sales, product, service/support) at a given point in time. The administration of the relationship surveys is not linked to any specific customer interaction with the company. Relationship surveys are typically administered at periodic times throughout the year (e.g., every other quarter, annually). Consequently, the relationship survey asks the customers to rate the company based on their past experience. While the surveys differ with respect to what is being rated (a transaction vs. a relationship), these surveys can share identical customer touchpoint questions (e.g., technical support, sales).

A high-tech company was conducting both a transactional survey and a relationship survey. The surveys shared identical items. Given that the ratings were coming from the same company and shared identical touchpoint questions, we would have expected the ratings to be the same for both the relationship survey and the transactional survey. The general finding, however, was that ratings on the transactional survey were typically higher than ratings for the same question on the relationship survey. What score is correct about the customer relationship?

So, why don’t ratings of identical items on relationship surveys and transactional surveys result in the same score? Humans are fallible.

Availability Heuristic

There is a line of research that examines the process by which people make judgments. This research shows that people use heuristics, or rules of thumb, when asked to make decisions or judgments about frequencies and probabilities of events. There is a heuristic called the “availability heuristic” that applies here quite well and might help us explain the difference between transactional ratings and relationship ratings of identical items. Before continuing, please click the hyperlink below and complete the exercise.

http://www.uwp.edu/academic/psychology/demos/UTICdemo/UTICdemo.html

People are said to employ the availability heuristic whenever their estimate of the frequency or probability of some event is based on the ease with which instances of that event can be brought to mind. Basically, the things you can recall more easily are estimated by you to be more frequent in the world than things you can’t recall easily. The example in the hyperlink demonstrates this nicely. When presented with a list containing an equal number of male and female names, people were more likely to think that the list contained more male names than female names due to the fact that more males’ names were famous names. Because these famous names were more easily recalled, the people think that there must be more male names than female names.

Customers, when rating companies as a whole (relationship surveys), are recalling their prior interactions with the company (e.g., their call into phone support, receipt of marketing material). Their “relationship rating” is a mental composite of these past interactions, negative, positive and mundane. Negative customer experiences, unlike positive or mundane ones, tend to be more vivid, visceral, and, consequently, are more easily recalled compared to pleasant experiences. When I think of my past interactions with companies, it is much easier for me to recall negative experiences than positive experiences. When thinking about a particular company, due to the availability heuristic, customers might overestimate the number of negative experiences, relative to positive experiences, that actually occurred with the company. Thus, their relationship ratings would be adversely affected by the use of the availability heuristic.

Ratings from transactional surveys, however, are less vulnerable to the effect of the availability heuristic. Because the customers are providing ratings for one recent, specific interaction, the customers’ ratings would not be impacted by the availability heuristic.

Summary and Implications

Customer satisfaction ratings in relationship surveys are based on customers’ judgment of past experiences with the company, and, consequently, are susceptible to the effects of the availability heuristic. Customers may more easily recall negative experiences, and, consequently, these negative experiences negatively impact their ratings of the company overall. While it would appear that a transactional survey could be a more accurate measure than a relationship survey, you shouldn’t throw out the use of relationship surveys just yet.

While average scores on items in relationship surveys might be decreased due to the availability heuristic, the correlation among items should not be impacted by the availability heuristic because correlations are independent of scale values; decreasing ratings by a constant across all customers does not have any effect on the correlation coefficients among the items being rated. Consequently, the same drivers of satisfaction/loyalty would be found irrespective of survey type.

I’d like to hear your thoughts.

Customer Loyalty 2.0, Part 1: Measurement and Meaning of Customer Loyalty

Background

The use of customer loyalty survey data to help manage customer relationships has received much technological innovation over the past decade. Web-based surveys provide an easy vehicle for customers to provide feedback. For example, individual customer concerns are addressed through the use of automated prompts (typically in the form of emails) to Account team members who are responsible for quickly resolving specific causes of customer loyalty. Additionally, organization-wide customer loyalty issues are identified through automated analyses (e.g., driver analysis) which highlight common causes of customer loyalty/disloyalty. Furthermore, customer survey results are accessible 24x7 by all employees through Web-based reporting tools. Finally, companies even link customer survey data to their CRM systems to enhance day-to-day account management with both attitudinal data and operational data. It is clear that efforts in the field of customer loyalty have simplified the process of data collection, analysis, reporting, and integration with existing business systems.

While the quality of the customer loyalty survey process has seen a great deal of improvement in business settings, the quality of the measurement and meaning of customer loyalty has not kept pace. Our latest research on customer loyalty, however, tries to narrow this gap. Customer Loyalty 2.0 represents this advancement in the measurement and meaning of customer loyalty. The purpose of the discussion is to demonstrate the limitation of current customer loyalty measures, highlight our latest research findings, introduce the idea that customer loyalty measured through surveys is best conceptualized as a multidimensional entity. That is, customer loyalty is not a single entity. More on that idea later.

Customer Relationship Management and Customer Loyalty

While many objective measures of customer loyalty exist (e.g., defection rate, number of referrals), customer surveys remain a frequently used way to assess customer loyalty. There are a few reasons for the popularity of customer survey use in customer experience management. First, customer surveys allow companies to quickly and easily gauge levels of customer loyalty. Companies may not have easy access to objective customer loyalty data or may simply not even gather such data. Second, results from customer surveys can be more easily used to change organizational business process. Customer surveys commonly include questions about customer loyalty as well as the customer experience (e.g., product, service, support). Used jointly, both business attribute items and loyalty indices can be used (e.g., driver analysis, segmentation analysis) to identify reasons why customers are loyal or disloyal. Finally, objective measures of customer loyalty provide a backwards look into customer loyalty levels (e.g., defection rates, repurchase rates). Customer surveys, however, allow companies to examine customer loyalty in real-time. Surveys ask about expected levels of loyalty-related behavior and lets companies “look into the future” regarding customer loyalty.

While there has been a change in business nomenclature around the application of customer surveys from "customer relationship management” to “customer experience management," the analytical techniques used to understand the survey data (e.g., segmentation analysis, driver analysis) remain exactly the same. The ultimate goal of customer loyalty survey analyses, no matter what business nomenclature you use, is to identify the reasons why customers are loyal or disloyal. You might think of customer loyalty as the ultimate criterion in customer relationship/experience management.

Customer Loyalty and Financial Performance

There are several objective measures of customer loyalty:

  • Number of referrals: Word of mouth/Word of mouse
  • Purchase again
  • Purchase different products
  • Increase purchase size
  • Customer retention/defection rates

Based on the objective measures of customer loyalty, we can see how company financial growth can occur through the increase in customer loyalty. Through the referral process, companies can grow through the acquisition of new customers. The idea is that the customer acquisition process relies on existing customers to promote/recommend the company to their friends, who, in turn, become customers. Another way of strengthening the financial growth of a company is through increased purchasing behavior (e.g., increase amount of purchases, purchase different products/services) of existing customers. Finally, company growth is dependent on its ability to not lose existing customers at a faster rate than they acquire them. For example, customer defection rate is an important metric in the wireless service industry where customer defections are common.

Measurement and Meaning of Customer Loyalty

Customer loyalty, when measured through surveys, is typically assessed through the use of standard questions or items, mirroring the objective measures listed earlier. For each item, customers are asked to rate their level of affinity for, endorsement of, and approval of a company. The items usually ask for a rating that reflects the likelihood that the customer will exhibit positive behaviors toward a company. Commonly used customer loyalty survey questions include the following items:

  • Overall satisfaction
  • Likelihood to choose again for the first time
  • Likelihood to recommend
  • Likelihood to continue purchasing same products/services
  • Likelihood to purchase different products/services
  • Likelihood to increase frequency of purchasing
  • Likelihood to switch to a different provider

The first question is rated on a scale (e.g., 0 = Extremely dissatisfied to 10 = Extremely satisfied. The remaining questions allow respondents to indicate their likelihood of behaving in different ways toward the company (e.g., 0 = Not at all likely to 10 = Extremely likely. Higher ratings reflect higher levels of customer loyalty.

Attitudinal Measures of Psychological Constructs

Constructs are unobservable entities we use to describe a set of observable indicators. In the survey world, these observable indicators are responses to questions. We use constructs in everyday life when we describe the state of people. We say Mary is "happy" because she laughs, smiles, and jokes. We say that John is "depressed" because he is frowning, slouching, and is looking downward. In this case of attitudinal measures, we use constructs to describe a set of responses to standard questions. Tests/Surveys are developed that measure "anxiety," "job satisfaction," "supervisor support," "introversion/extroversion," and "customer loyalty." The questions on these tests/surveys bring the constructs into the observable world. Questions in inventories measure personality traits; questions in surveys measure customer loyalty; questions in employee surveys measure supervisor support. Figure one illustrates the relationship between the construct and the observable indicators (questions).

When measuring a particular psychological construct, researchers develop many items in order to calculate an overall score as a measure of that particular construct. Everything being equal, we know that scores based on many questions are more reliable than any one of the single scores. Consider measuring your child's math ability in college. In general, you would have more confidence that a score based on a 50-item math test would be a more reliable indicator of your child's math ability than a score based on any single item from that test.

Individual Loyalty Items vs. Composite Loyalty Scores

Customer surveys, oftentimes, include multiple loyalty questions. There are different approaches in how these loyalty questions are used. One approach is to use single loyalty questions as the loyalty measure. For example, Reichheld (2006) recommends the use of the "likelihood to recommend" as the single best question to use as a measure of customer loyalty. Still other researchers use "overall satisfaction" as their key measure of customer loyalty (Fornell, et al., 2006). Another approach is to use a composite score (typically averaging across items) based on several loyalty questions. The

question now becomes, "When multiple loyalty items are used in a customer survey, should we use a composite score as our ultimate loyalty criterion or use each item as unique measures of customer loyalty?"

The Net Promoter Debate: A Summary

The Net Promoter Score (NPS) is used by many of today’s top businesses to monitor and manage customer relationships. Fred Reichheld and his co-developers of the NPS say that a single survey question, “How likely are you to recommend Company Name to a friend or colleague?”, on which the NPS is based, is the only loyalty metric companies need to grow their company. Despite its widespread adoption by such companies as General Electric, Intuit, T-Mobile, Charles Schwab, and Enterprise, the NPS is now at the center of a debate regarding its merits.

NPS Methodology

The NPS is calculated from a single loyalty question, “How likely are you to recommend us to your friends/colleagues?” Based on their rating of this question using a 0 to 10 likelihood scale where 0 means “Not at all Likely” and 10 means “Extremely Likely,” customers are segmented into three groups: 1) Detractors (ratings of 0 to 6), 2) Passives (ratings of 7 and 8) and 3) Promoters (ratings of 9 and 10). A company can calculate its Net Promoter Score by simply subtracting the proportion of Detractors from the proportion of Promoters.

NPS = prop(Promoters) – prop(Detractors)

NPS Claims

Fred Reichheld, the co-developer of the NPS (along with Satmetrix and Bain & Company) has made very strong claims about the advantage of the NPS over other loyalty metrics. Specifically, they have said:

  1. The NPS is “the best predictor of growth,” (Reichheld, 2003)
  2. The NPS is “the single most reliable indicator of a company’s ability to grow” (Netpromoter.com, 2007)
  3. “Satisfaction lacks a consistently demonstrable connection to… growth” (Reichheld, 2003)

Reichheld support these claims with research displaying the relationship of NPS to revenue growth. In compelling graphs, Reichheld (2006) illustrates that companies with higher Net Promoter Scores show better revenue growth compared to companies with lower Net Promoter Scores. Reichheld sites only one study conducted by Bain & Associates (co-developers of the NPS) showing the relationship between satisfaction and growth to be 0.00. [1]

Recent Scientific Challenges to NPS Claims

Researchers, pointing out the NPS claims are only supported by Reichheld and his co-developers, have conducted rigorous scientific research on the NPS with startling results. For example, Keiningham et al. (2007), using the same technique employed by Reichheld to show the relationship between NPS and growth, used survey results from the American Customer Satisfaction Index (ACSI) to create scatterplots to show the relationship between satisfaction and growth. Looking at the personal computer industry, they found that satisfaction is just as good as the NPS at predicting growth. Keiningham et al. (2007) found the same pattern of results in other industries (e.g., insurance, airline, ISP). In all cases, satisfaction and NPS were comparable in predicting growth.

Still, other researchers (Morgan & Rego, 2006) have shown that other conventional loyalty measures (e.g., overall satisfaction, likelihood to repurchase) are comparable to NPS in predicting business performance measures like market share and cash flow.

Contrary to Reichhheld, other researchers, in fact, have found that customer satisfaction is consistently correlated with growth (Anderson, et al., 2004; Fornell, et al., 2006; Gruca & Rego, 2005).

Problems with NPS Research

The recent scientific, peer-reviewed studies cast a shadow on the claims put forth by Reichheld and his cohorts. In fact, there is no published empirical supporting the superiority of the NPS over other conventional loyalty metrics.

Keiningham et al. (2007) aptly point out that there may be research bias by the NPS developers. There seems be a lack of full disclosure from the Net Promoter camp with regard to their research. The Net Promoter developers, like any research scientists, need to present their analysis to back up their claims and refute the current scientific research that brings their methodological rigor into question. To date, they have not done so. Instead, the Net Promoter camp only points to the simplicity of this single metric which allows companies to become more customer-centric. That is not a scientific rebuttal. That is marketing.

References

Anderson, E. W., Fornell, C., & Mazvancheryl, S. K. (2004). Customer satisfaction and shareholder value. Journal of Marketing, 68 (October), 72-185.

Fornell, C., Mithas, S., Morgensen, F. V., Krishan, M. S. (2006). Customer satisfaction and stock prices: High returns, low risk. Journal of Marketing, 70 (January), 1-14.

Gruca, T. S., & Rego, L. L. (2005). Customer satisfaction, cash flow, and shareholder value. Journal of Marketing, 69 (July), 115-130.

Hayes, B. E. (1997). Measuring Customer Satisfaction (2nd Ed.). Quaility Press. Milwaukee, WI.

Ironson, G.H., Smith, P.C., Brannick, M.T., Gibson W.M. & Paul, K.B. (1989). Construction of a "Job in General" scale: A comparison of global, composite, and specific measures. Journal of Applied Psychology, 74, 193-200.

Keiningham, T. L., Cooil, B., Andreassen, T.W., & Aksoy, L. (2007). A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71 (July), 39-51.

Morgan, N.A. & Rego, L.L. (2006). The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25(5), 426-439.

Netpromoter.com (2007). Homepage.

Reichheld, F. F. (2003). The One Number You Need to Grow. Harvard Business Review, 81 (December), 46-54.

Reichheld, F. F. (2006). The ultimate question: driving good profits and true growth. Harvard Business School Press. Boston.



[1] http://resultsbrief.bain.com/videos/0402/main.html