The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 5 of the discussion. If you missed them, read Part 1, Part 2 and Part 3 and Part 4. I calculated the derived importance and performance for one of the PC Manufacturers in the study. Table 4 contains the results for this PC Manufacturer. The column labeled "Mean" reflects the performance for each of the business attributes; the columns labeled "Derived Importance on ALI" and "Derived Importance on PLI" reflect the importance for each of the business attributes on advocacy loyalty and purchasing loyalty, respectively.
Customer Loyalty 2.0 represents this advancement in the measurement and meaning of customer loyalty. The purpose of the following analyses is to provide additional validity evidence regarding the measures of loyalty, Advocacy Loyalty Index (ALI) and Purchasing Loyalty Index (PLI). A common approach to establishing the validity is to show how the ALI and PLI are related to attributes of service and product quality. Below, I employ a common method, loyalty driver analysis, used by companies to identify the key drivers of customer loyalty. This method includes analysis that shows the relationship between business attributes and customer loyalty. In the process of establishing validity evidence, I will provide real-life illustrations regarding the merits of conceptualizing customer loyalty in this multidimensional framework that can help companies increase growth through new and existing customers.
Loyalty Driver Analysis
Loyalty driver analysis enables companies to identify the business attributes that are important to ensuring customer loyalty. Companies can allocate resources to important business areas that have the greatest impact on increasing customer loyalty. While each response from a survey can be (and should be) examined to deal with specific causes of customer loyalty/disloyalty at an individual, customer level, driver analysis can be thought of as a macro look at the customer base (or customer segment). By analyzing data from a large segment of customers, driver analysis helps companies to identify the common causes of customer loyalty across these customers. Consequently, based on the results of the driver analysis, companies can make organization-wide improvements to their business processes that will have an impact on the customer segment of interest.
Two pieces of information are examined in driver analysis: 1) derived importance: the degree of impact of each business attribute on customer loyalty and 2) performance: the level of performance of each business attribute.
Derived Importance
The degree of impact each business attribute has on customer loyalty is determined. This degree of impact is indexed by a correlation coefficient (sometimes, referred to as “derived importance”) between ratings of a business attribute and a customer loyalty index (either ALI or PLI). Each business attribute has a corresponding “derived importance” that indicates the impact that the business attribute has on customer loyalty. The higher the correlation, the greater the impact that business attribute has on customer loyalty. The degree of impact (e.g., correlation coefficient) can vary from 0.0 (no impact) to 1.0 (perfect impact).
Using the Wireless Service Provider and PC Manufacturer studies, we can identify the derived importance of each of the business attributes. Given we have multiple measures of customer loyalty, each of the business attributes will have multiple derived importance, one for advocacy loyalty, one for purchasing loyalty and one for retention loyalty (Wireless study only). In both studies, respondents were asked to indicate the degree to which they agree or disagree with statements regarding their customer experience on a scale from 1 (strongly disagree) to 5 (strongly agree). Statements reflected business attributes that ranged from product quality (e.g., reliable service, PC reliability) to service quality (e.g., customer service reps, technical support reps). These ratings were correlated with each of the measures of customer loyalty, ALI, PLI and RLI.
Performance
Next, company’s performance on each business attribute (customer ratings) is calculated. Performance of a given business attribute is simply the average rating of agreement. Higher scores reflected better performance (better customer experience). Possible performance scores could range from 1 (worst customer experience) to 5 (best customer experience). Business attributes that have low performance ratings have ample room for improvement. Business attributes that have high performance ratings have little room for improvement.
Wireless Service Provider Driver Analysis
I calculated the derived importance and performance for one of the Wireless Service Providers in the study. Table 3 contains the results for this particular Wireless Service Provider. The column labeled "Mean" reflects the performance for each of the business attributes; the columns labeled "Derived Importance on ALI," "Derived Importance on PLI," and "Derived Importance on RLI" reflect the importance for each of the business attributes on advocacy loyalty, purchasing loyalty and retention loyalty, respectively.
Table 3. Descriptive Statistics and Derived Importance of each Business Attribute for a Wireless Service Provider
In driver analysis, both the performance and derived importance are examined simultaneously to understand where improvements would have the greatest chance to improve customer loyalty. If business attributes that have a large impact on customer loyalty (high derived importance) and have low performance ratings, companies might consider allocating resources to these business attributes in order to improve customer loyalty. If ratings of business attributes are high, however, companies can promote these business attributes as strengths and best practices. Using both the derived importance of each business attribute and the performance (e.g., rating) of each business attribute, we can create a Loyalty Matrix (see figures below) that allows us to visually examine all business attributes at one time.
The abscissa (x-axis) of the Loyalty Matrix is the performance rating (agreement, performance, satisfaction) of the business attributes. The ordinate (y-axis) of the Loyalty Matrix is the impact (derived importance) of the business attribute on customer loyalty. The Loyalty Matrix is divided into quadrants using the average score for each of the axes. Key drivers appear in the upper left quadrant and are often referred to as Key Drivers. Key drivers reflect business attributes that have both a large impact on customer loyalty and have low performance ratings relative to the other business attributes (these key drivers appear in red). Because we have three customer loyalty indices, we can calculate three separate Loyalty Matrices, each for the specific customer loyalty index. Below are the Loyalty Matrices for a particular Wireless Service Provider.
Figure 4. Advocacy Loyalty Driver Analysis - Wireless Service
Provider
Figure 5. Purchasing Loyalty Driver Analysis - Wireless Service
Provider
Figure 6. Retention Loyalty Driver Analysis - Wireless Service
Provider
The results each of the driver analyses seem fairly consistent. We see that Customer Service Representatives (CSRs) have a relatively large impact on customer loyalty compared to the primary product offering attributes (Reliable service, Good coverage). Whether customers will be loyal to this Wireless Service Provider depend to a greater degree on the competency of the CSRs than on the product offering attributes.
Using the three Loyalty Matrices, we can draw some conclusions regarding how this particular Wireless Service Provider can increase advocacy, purchasing and retention loyalty. When employing the use of driver analysis, we typically focus on the the Key Drivers (those attributes in the upper left hand quadrant) as areas to focus if we want to improve customer loyalty. We do so because these are the attributes that have a large impact on customer loyalty and have much room for improvement. To improve loyalty, no matter how it is measured, the results of the driver analysis indicate that the company should focus on improving CSR attributes as these attributes have a relatively large impact on advocacy, purchasing and retention loyalty and have much room for improvement.
PC Manufacturer Driver Analysis
Table 4. Descriptive Statistics and Derived Importance of each Business Attribute for a PC Manufacturer
Below are the Loyalty Matrices for a particular PC manufacturer.
Figure 7. Advocacy Loyalty Driver Analysis - PC Manufacturer
Figure 8. Purchasing Loyalty Driver Analysis - PC Manufacturer
We see that there are differences in what drives advocacy loyalty and purchasing loyalty. With regard to advocacy loyalty, we see that both of the PC attributes (PC reliability, and PC features) have a big impact on advocacy loyalty, more so than the technical support attributes. Whether customers will be advocates of this PC manufacturer depend highly on the computer itself and less so on the quality of technical support. With regard to purchasing loyalty, however, we see that many of the technical support attributes (excellence, timeliness, understands needs, availability) have a relatively big impact on purchasing loyalty. Interestingly, PC attributes do not have a big impact on purchasing loyalty.
Using the two Loyalty Matrices, we can draw some conclusions regarding how this particular PC Manufacturer can increase advocacy loyalty and purchasing loyalty. To improve advocacy loyalty, the driver analysis seems inconclusive. While PC features are big determinants of advocacy, they are rated as relatively good. Consequently, there is not much room for improvement in these attributes. The technical support attributes, while rated as relatively low, do not have a large impact on advocacy loyalty. If this PC Manufacturer wants to improve purchasing loyalty, however, the results of the driver analysis indicate that they should focus on improving technical support attributes as these attributes have a relatively large impact on purchasing loyalty and have much room for improvement.
Missed Opportunities to Improve Customer Loyalty
When practitioners talk about "customer loyalty," they are usually referring to advocacy loyalty; many loyalty programs are, in fact, based solely on advocacy-related content. For example, the Net Promoter Score is based on the "likelihood to recommend" question. Additionally, the American Customer Satisfaction Index (ACSI) is based on the "satisfaction" question. If the PC manufacturer relied solely on this question, the results of the driver analysis suggests that they should focus ensuring that their PCs are reliable and has features their customers want. This driver analysis found that many of the technical support items were not important in improving advocacy loyalty (they appeared in the lower left quadrant). The use of advocacy-related loyalty questions as a way of measuring and defining customer loyalty limits improvements in acquiring new customers.
The driver analysis using the PLI painted an entirely different picture. We saw that many technical support items were now key drivers of customer loyalty. Expanding the definition of customer loyalty to include increased purchasing intentions clearly shows that loyalty can be improved beyond mere referrals of new customers. Companies can now identify business attributes that, when improved, would increase purchasing loyalty of existing customers. Improving technical support for the PC Manufacturer would increase revenue from existing customers through increasing their purchasing behavior (buy different products and increasing their purchasing frequency). Looking at both advocacy loyalty and purchasing loyalty, this PC manufacturer can maximize revenue through both new and existing customers.
Summary
The evidence from two separate studies shows that the Advocacy Loyalty Index (ALI) and the Purchasing Loyalty Index (PLI) measure two different types of loyalty. Even though the two types of loyalty are correlated (advocates tend to be purchasers), the relationship between the ALI and PLI is not perfect, suggesting that these loyalty indices measure unique constructs. We have good evidence that the loyalty indices are each measuring some unique aspect of customer loyalty.
The results of the present analysis show that, as expected, the measures of customer loyalty are logically related to the customer experience. Customers who have a better customer experience tend to have higher levels of customer loyalty. Furthermore, the impact that the business attributes have on customer loyalty depends on the customer loyalty index that is used.
For more information about the Advocacy Loyalty Index and the Purchasing Loyalty Index and more detailed information about the driver analyses in the studies reported here, you can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.
Sunday, December 30, 2007
Customer Loyalty 2.0, Part 5: Measurement and Meaning of Customer Loyalty: Drivers of Advocacy Loyalty and Purchasing Loyalty
Sunday, December 23, 2007
Customer Loyalty 2.0, Part 4: Measurement and Meaning of Customer Loyalty: Advocacy Loyalty and Purchasing Loyalty
The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 4. If you missed them, read Part 1, Part 2 and Part 3.
Validity of Loyalty Indices
As we saw, one form of reliability, internal consistency reliability, can be indexed by Cronbach's alpha estimate. This estimate tells us the degree to which the items in the scale are correlated with each other. Scales that have items that are highly correlated with each other have higher reliability compared to scales that have items that are not highly related to each other. Validity, on the other hand, involves a process of understanding what the scale is actually measuring. Now that we have scales that are highly reliable, the next step is to gather evidence that helps us identify what the scales are actually measuring. Establishing the validity of scales is a more complex process than establishing the reliability of scales.
Content-related Evidence
Content-related evidence is concerned with the degree to which the items in the scale are representative of a “defined universe” or “domain of content.” The domain of content typically refers to all possible items that could have been used in the scale. The goal of content-related validity is to have a set of items that best represent the defined universe.
Our initial set of loyalty questions appears to be a good representation of our "defined universe" of possible customer loyalty items. We set out to measure customer's behaviors/attitudes that help companies grow. Company growth occurred in two general ways: 1) through acquiring new customer and 2) increasing purchasing behaviors of existing customers. While our set of loyalty items might not be exhaustive of all possible loyalty items that could have been asked in the survey, they do appear to be a representative sample of all loyalty questions.
Criterion-related Evidence
Criterion-related validity involves demonstrating the statistical relationship between the loyalty indices and other variables. Based on customer loyalty theory and research, we expect that our measures of customer loyalty would be related to certain types of variables (e.g., customer satisfaction). Previous loyalty models support the notion that customers who perceive the customer experience as good (e.g., good product and service quality) would report higher levels of customer loyalty compared to customers who perceive the customer experience as poor.
In this section, I will explore criterion-related evidence of the loyalty indices. First, I will present the descriptive statistics of and correlations among the loyalty indices. Second, I will examine the relationships of the customer loyalty indices with other variables (e.g., customer tenure, number of prior recommendations). In addition to helping us determine the quality of these loyalty indices, examining these relationships provides a practical vehicle by which we can better understand how companies can better manage the customer relationship and improve customer loyalty.
Descriptive Statistics of and Correlations among Loyalty Indices
The descriptive statistics of and correlations among the loyalty indices are presented for both the Wireless Service Provider study and the PC Manufacturer study below.
Table 1. Descriptive Statistics and Correlations for Wireless Service Provider Study
Table 2. Descriptive Statistics and Correlations for PC Manufacturer Study
We see that Advocacy Loyalty was substantially higher than Purchasing Loyalty for the Wireless study (7.29 vs. 5.63) and PC study (7.34 vs. 5.79). Customers generally report higher levels of advocacy loyalty compared to purchasing loyalty.
Next, let us examine the relationship among the customer loyalty indices. Although the loyalty indices are distinct constructs (as determined by the factor analysis), they are correlated with each other. Specifically, the correlation between the ALI and PLI is .64 and .67 for the Wireless study and PC study, respectively. This correlation shows us that customers who are advocates are also somewhat more likely to increase their purchase behavior compared to customers who are not advocates. Additionally, for the Wireless study, Retention Loyalty is more strongly related to the ALI (r = .61) than the PLI (r = .31).
In the next set of analyses, I wanted to understand how the ALI and PLI were related to three customer variables: 1) the number of years they have been a customer, 2) the number of people to which customers recommended in the past 12 months, and 3) age of the customer. The results appear below for the wireless service provider study.
Customer Tenure and Customer Loyalty
There is no statistically significant relationship between customer tenure and advocacy loyalty or purchasing loyalty (see Figure 1). Both advocacy and purchasing loyalty remain flat over customer tenure. Retention loyalty, however, is significantly related to customer tenure; Customers who have been with their providers for three or fewer years are less likely to defect compared to customers who have been with their providers for five or more years. Conversely, long-tenured customers are more likely to remain.
The extent to which customers will advocate or increase their purchasing behavior seems to be independent of the length of the customer relationship (a longitudinal study would be a better test of this hypothesis, though). The extent to which customers will switch to another provider, however, is related to the length of the customer relationship. This result might be indicative of the impact of the wireless service contracts on retention loyalty; Short-term customers who were likely to switch providers likely did so after their contract expired (two-year contract lengths are typical). Consequently, the remaining customers (those who become long-term customers), by the very definition of their segment, are made up of customers who are less likely to defect.
Figure 1. Relationship between Customer Tenure and Customer Loyalty
*Retention Loyalty is the reverse coded response to likelihood to switch question; so higher scores mean a lower likelihood of switching wireless service providers.
Number of Recommendations and Customer Loyalty
The next set of analyses looked at the relationship between each of the loyalty indices and the number of people customers recommend in the previous 12 months. In the survey, customers were asked to indicate the number of friends/colleagues they recommended the provider to in the past 12 months. As you can see in Figure 2, the number of previous recommendations is strongly related to both advocacy loyalty and purchasing loyalty and is weakly related to retention loyalty. Customers who did not recommend their wireless service provider to anybody showed lower levels of advocacy and purchasing loyalty compared to customers who did recommend their wireless service provider. Retention loyalty, however, showed less of a relationship to the number of recommendations; while there is a slight increase in customer retention for customers who recommend their provider to 1 to 5 friends, retention drops back down for those customers who recommended their provider to 6 or more friends.
Figure 2. Relationship between Number of Prior Recommendations and Customer Loyalty
*Retention Loyalty is the reverse coded response to likelihood to switch question; so higher scores mean a lower likelihood of switching wireless service providers.
Surprisingly, the extent to which customers recommended their service provider has a substantial impact on their future purchase behaviors. It appears that, to increase purchasing behavior, a company should not only improve the customer service delivery system, but should also find ways to simply encourage customers to recommend them to their friends. Humans tend to behave in ways that are congruent with their beliefs. The mere act of recommending a given service provider may have the ultimate impact on purchasing behavior (“If I am recommending them, I should be purchasing from them.”).
Wireless service providers should continue to find ways to encourage customers to recommend them to their friends. The current findings strongly suggest that customers who recommend their service provider to friends may not only improve financial performance through helping grow the customer base, but also through their own increased purchasing behavior.
Customer Age and Customer Loyalty
There was no significant relationship of age with advocacy loyalty. However, customers’ age was significantly related to purchasing loyalty and retention loyalty. Younger customers have higher levels of purchasing loyalty and retention loyalty compared to older customers. Purchasing loyalty is at its highest for customers between the ages of 26-30 years. The lowest is for customers who are 51+ years. Conversely, retention loyalty is at its highest for customers who are 51+ years and lowest for customers who are 31-40 years (see Figure 3).
The relatively greater likelihood that younger customers will increase their purchase behavior may be driven by characteristics of the younger customers (e.g., keeping up with friends) or marketing that is targeted at younger customers to purchase additional features from their service provider (e.g., text messaging, ring tones, Web access). These same same reasons may encourage customers to switch to providers who offer seemingly more attractive service packages.
Figure 3. Relationship between Age and Customer Loyalty
*Retention Loyalty is the reverse coded response to likelihood to switch question; so higher scores mean a lower likelihood of switching wireless service providers.
Summary
The evidence from two separate studies show that the Advocacy Loyalty Index (ALI) and the Purchasing Loyalty Index (PLI) measure two different types of loyalty. Even though the two types of loyalty are correlated (advocates tend to be purchasers), the relationship between the ALI and PLI is not perfect, suggesting that these loyalty indices measure unique constructs. We have good evidence that the loyalty indices are each measuring some unique aspect of customer loyalty.
Customer loyalty is not a unidimensional construct, but rather a multidimensional construct that can help reliably measured. When we say a customer group has high vs. low loyalty, we need to clarify to which loyalty we are referring. It is possible that a given customer group can have different levels of loyalty (e.g., high advocacy, low purchasing). It is clear that a blanket statement about levels of "customer loyalty" can be ambiguous.
Customers are more willing to sell their friends on the merits of their wireless service provider than they are to spend more of their own money on their wireless service provider. Perhaps purchasing loyalty is lower than advocacy loyalty due to other constraints that might limit the degree to which customers are able to purchase more (e.g., limited need, limited financial resources) yet not impact the degree to which customers could be advocates for a company/product/brand.
For more information about the Advocacy Loyalty Index and the Purchasing Loyalty Index, you can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.
a>.
Monday, December 17, 2007
Customer Loyalty 2.0, Part 3: Reliability of Loyalty Indices
The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 3 of the discussion. Part 1 can be found here ( http://businessoverbroadway.blogspot.com/2007/11/customer-loyalty-20.html). Part 2 can be found here ( http://businessoverbroadway.blogspot.com/2007/12/customer-loyalty-20-part-2-advocacy.html).
Loyalty Indices
The results of the factor analyses support the use of composite scores, each representing one of the loyalty dimensions. These composite scores are referred to as scales/indices/metrics. This process of calculating these scales/indices/metrics is done by averaging the items that load on the same factors. Based on the results of the present analyses, we can calculate three indices:
When using scales from surveys to measure constructs, we need to be concerned about the quality of the scales. The quality of these surveys are typically discussed with respect to reliability and validity. Reliability refers to the degree to which scores are free from measurement error. Validity refers to the degree to which the scale measures what is was designed to measure. Before we use these new scales, I will briefly discuss these measurement principles in the context of classical test theory.
Classical Test Theory
Classical test theory is based on the idea that an observed score (X) from a survey can be decomposed into two different scores, a true score (T) and an error score (E) where:
X = T + E
As the equation implies, as error decreases, the observed score (X) matches the underlying true score (T). Classical test theory is concerned with the relationships among the three variables, X, T and E. The relationships among these three components are used to understand the quality of the scores that result from the scales. The first quality of measurement, reliability, is concerned with the relationship between the observed score (X) and the true score (T).
Reliability
Reliability is the degree to which measurements are free from random errors. Reliability deals with precision or consistency of measurement. Scales or indices with high reliability are better at distinguishing people on the continuum of customer loyalty. Our goal in developing customer loyalty indices is to have a measurement instrument that delivers reliable results. Reliability can be thought of as the relationship between the true underlying score and the observable score we get from our survey. Random error decreases the measurement’s reliability; that is, as random error is introduced into measurement, the observed score is not a good reflection of the true underlying score. For one to feel confident that a questionnaire’s scores accurately reflect the underlying dimension, the questionnaires must have high reliability. Although many types of reliability exist, internal consistency reliability is vital to surveys.
Internal consistency indicates the extent to which the items in the measurement are related to each other. The higher the interrelationship among the items, the higher the internal consistency. If a questionnaire is designed to measure one underlying construct, the items are expected to be related to each other – that is, people who respond in one way to an item are likely to respond the same way to the other items in the measure.
There are several statistical indices used to estimate the degree of internal consistency. The most commonly used index is Cronbach’s coefficient alpha (Cronbach, 1951). Basically, this alpha coefficient indicates the degree to which items are related to each other. Cronbach's alpha increases when the correlations among the items increase. Cronbach's alpha can range from 0 to 1.0. A reliability of 0 indicates that the observed score is not related to the underlying true score; a reliability of 1 indicates that the observed score is a perfect indicator of the underlying true score. Generally, a reliability of .8 or greater is an acceptable level of reliability.
There are a couple of key benefits to using customer loyalty indices that have high reliability. First, customer loyalty scales with high reliability are better able to distinguish between varying levels of customer loyalty than loyalty scales with low reliability. Because scales with higher reliability have excellent precision, they are able to distinguish small differences in loyalty. Second, using a loyalty scale with high reliability, you are more likely to find significant relationships with other variables when loyalty is truly related to them.
Although reliability is an important ingredient in the evaluation of a questionnaire, it cannot solely determine the quality of the questionnaire. The questionnaire’s validity must also be addressed.
Validity
Validity refers to the degree to which evidence supports the inferences made from scores derived from measurements, or the degree to which the scale measures what it is designed to measure. Unlike reliability, there is no single statistic that provides an overall index of the validity of inferences about the scores.
The methods for gathering evidence of validity can be grouped into three categories: content-related evidence, criterion-related evidence, and construct-related evidence. These labels simply enable people to discuss the types of information that might be considered when determining the validity of the inferences.
Content-related evidence is concerned with the degree to which the items in the questionnaire are representative of a “defined universe” or “domain of content.” The domain of content typically refers to all possible items that could have been used in the questionnaire. The goal of content-related validity is to have a set of items that best represent the defined universe.
Criterion-related evidence is concerned with examining the systematic relationship (usually in the form of a correlation coefficient) between the loyalty scale and another measure, or criterion. In this case, what the criterion is and how it is measured are of central importance. The main question to be addressed in criterion-related validity is how well the scale can predict the criterion.
Construct-related evidence is concerned with the questionnaire as a measurement of an underlying construct. Unlike criterion-related validity, the primary focus is on the scale itself rather than on what the scale predicts. Construct-related evidence is derived from both previous validity strategies. A high degree of correlation between the scale and other scales that purportedly measure the same construct is evidence of construct-related validity. Construct-related validity can also be evidenced by a low correlation between the scale and other scales that measure a different construct.
The figure below illustrates the distinction between reliability and validity. Recall that reliability deals with precision/consistency while validity refers to meaning behind the scores. The diagram consists of four targets, each with four shots. In the upper left hand target, we see that the there is high reliability in the shots that were fired yet the bull’s-eye has not been hit. This is akin to having a scale with high reliability but is not measuring what the scale was designed to measure (not valid). In the lower right target, the pattern indicates that there is little consistency in the shots but that the shots are all around the bull’s-eye of the target (valid). The pattern of shots in the lower left target illustrates low consistency/precision (no reliability) and an inability to hit the target (not valid). The upper right pattern of shots at the target represents our goal to have precision/consistency in our shots (reliability) as well as hitting the bull’s-eye of the target (validity).
Reliability of Loyalty Indices
Reliability estimates were calculated for each of the loyalty indices. For the Wireless Service Provider sample, the reliability (Cronbach’s alpha) of the Advocacy Loyalty Index (ALI) was .92. The reliability estimate (Cronbach's alpha) for the Purchasing Loyalty Index (PLI) was .82. For the Personal Computer Manufacturer sample, the reliability of the ALI was .94. The reliability of the PLI was .87. These levels of reliability are considered very good for attitude research. The high reliability of each of the scales suggests that there is minimal measurement error associated with each composite score. Thus, we can feel confident that the observed scores (X) we get from the survey results are a very good reflection of the underlying true scores (T).
Validity of Loyalty Indices
Establishing the validity of the loyalty scales is a more complex process and will be discussed in the next blog.
You can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.
References
Allen, M.J., & Yen, W. M. (2002). Introduction to Measurement Theory. Long Grove, IL: Waveland Press.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
Tuesday, December 11, 2007
Customer Loyalty 2.0, Part 2: Advocacy, Purchasing and Defection Loyalty
The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 2 of the discussion. Part 1 can be found here.
Individual Loyalty Items vs. Composite Loyalty Scores
Customer surveys, oftentimes, include multiple loyalty questions. There are different approaches in how these loyalty questions are used. One approach is to use single loyalty questions as the loyalty measure. For example, Reichheld (2006) recommends the use of the "likelihood to recommend" as the single best question to use as a measure of customer loyalty. Still other researchers use "overall satisfaction" as their key measure of customer loyalty (Fornell, et al., 2006). Another approach is to use a composite score (typically averaging across items) based on several loyalty questions. The question now becomes, "When multiple loyalty items are used in a customer survey, should we use a composite score as our ultimate loyalty criterion or use each item as unique measures of customer loyalty?"
To answer that question, we need to understand exactly what each loyalty question is measuring. Specifically, we want to know if each of the loyalty questions is an observable indicator of the construct of "customer loyalty." If we can provide evidence that the seemingly distinct loyalty questions are really measuring the same construct, it would be appropriate to create an overall index of customer loyalty. If we can provide evidence that each loyalty question, however, measures something different from the other loyalty questions, it would be appropriate to use each loyalty question as a unique measure of customer loyalty. A statistical analysis technique that is used to understand the meaning of items is called factor analysis. Factor analysis is a data reduction technique that explains the statistical relationships among a given set of variables using fewer unobserved variables (factors). The output of a factor analysis will tell us two things: 1) The number of factors that can explain the relationships among the set of observed variables and 2) Which variables are related to which factors. Specifically, for our problem, a factor analysis will help us identify if the relationship among the set of many loyalty items can be explained by fewer factors (constructs). It is important to note that an exploratory factor analysis involves some form of judgment when determining the number of factors as well as which variables are related to the smaller set of factors. A full discussion is beyond the scope of this discussion but the interested reader can read more about this topic (Hayes, 1997). In the next section, I apply factor analysis to seven loyalty questions.
Wireless Service Providers Study (2007)
The survey was fielded in June 2007, asking a sample of 994 general consumers in the United States ages 18 and older about their attitudes toward their current wireless cell phone provider. The survey data for this study was collected by GMI (Global Market Insite, Inc., www.gmi-mr.com), who provided online data collection and consumer panels.
This particular study on wireless service providers included seven (7) loyalty questions and additional quality-related questions. About 44% of the respondents were male. Sixty-one percent of the respondents were 40 years old or younger.
To examine the dimensionality of the loyalty items, a factor analysis was conducted on seven loyalty items (see Part 1 for items). The results of the factor analysis suggested that seven items measure three (3) constructs. The figure below represents the factor pattern matrix. The elements in the factor pattern matrix are called factor loading and essentially reflect the correlation between each item and the three factors. Higher factor loadings indicate a stronger relationship between the item and the underlying factor.
The items loading on the first factor are the standard loyalty questions typically used in customer loyalty research (e.g., overall satisfaction, choose again for first time, recommend, continue purchasing). The items loading on the second factor are the specific purchasing behavior questions. The remaining loyalty question appears to form its own factor. The factor pattern matrix suggest that, instead of thinking of each item as representing seven distinct variables, the first four items measure one underlying construct, the next two items measure a different underlying construct and the last item measures yet a different construct.
As a general rule, the naming of the factors should encompass the entire set of items that load on the factor; the items that load on the first factor appear to have a strong emotional component to them, reflecting the extent to which customers advocate the company. Consequently, this factor is labeled Advocacy Loyalty. The items that load on the second factor reflect specific purchasing behaviors. Consequently, this second factor is labeled Purchasing Loyalty. The item that represents the third factor reflects defection and is, therefore, labeled Defection Loyalty. The naming of factors in a factor analysis involves some level of creativity and subjectivity. Other researchers might label the factors with different words (but probably similar words); the underlying construct being measured, however, remains the same.
PC Manufacturers Study (2007)
This survey was fielded in July 2007, asking a sample of 1058 general consumers in the United States ages 18 and older about their attitudes toward their personal computer manufacturer. All respondents were interviewed to ensure they meet correct profiling criteria, and were rewarded with an incentive for filling out the survey. The survey data for this study was collected by GMI (Global Market Insite, Inc., www.gmi-mr.com), who provided online data collection and consumer panels.
This study was slightly different than the prior study. An additional purchasing loyalty question was created to tap another element of the purchasing loyalty construct (Likelihood to increase the frequency of purchasing) and the defection question was removed. A factor analysis was conducted on all seven (7) items. The results of this factor analysis suggested that the seven questions measure two underlying dimensions. The figure below reflects the factor pattern matrix of this factor analysis (after rotation).
The results of the factor analysis of these items are similar to the results using the Wireless Provider sample. That is, it appears that the seven seemingly disparate loyalty items actually measure two underlying constructs, Advocacy Loyalty and Purchasing Loyalty.
Summary
Based on the results of the two separate factor analyses, we see that the apparently disparate loyalty questions actually only reflect fewer distinct loyalty constructs. Rather than thinking of each loyalty item as measuring some unique dimension of customer loyalty, the results indicate that there is much overlap in loyalty questions (at least how customers respond to these questions). Customers tend to respond to loyalty questions in similar ways and do not make distinctions among general loyalty-related questions. Specifically, given the Advocacy Loyalty questions, if customers rate one loyalty question high, they will likely rate the other loyalty questions high. Conversely, if customers rate a question low, they will likely rate the other questions. The same can be said for responses to the Purchasing Loyalty questions.
Satisfaction, Recommend and Purchase Same
Of particular interest are three specific loyalty items that load on Factor 1: 1) satisfaction, 2) recommend, and 3) purchase same. The Net Promoter Score (NPS) developers state that the "recommend" question is the best predictor of business growth (Reichheld, 2003, 2006). This conclusion has come under recent attack from other researchers who have found that the "satisfaction" and "purchase same" questions are just as good as the "recommend" question in predicting business growth (Fornell, et al., 2006; Keiningham, et al., 2007; Morgan & Rego, 2006). The current results cast additional doubt on the conclusions by the NPS camp. The recommend question appears to measure the same underlying construct as these other two loyalty questions. Given these loyalty questions measure the same thing, we should not expect the "recommend" question to be a better predictor of business metrics compared to the "satisfaction" and "purchase same" loyalty questions.
Objective vs. Subjective Measures of Loyalty
It is important that we make the distinction between objective measures of loyalty and subjective measures of loyalty. These objective metrics of customer loyalty have minimal measurement error associated with them. Because these metrics are not subject to interpretation, these objective loyalty metrics have unambiguous meaning. The number of recommendations a customer makes is clearly distinct from the number of repeat purchases that customer makes. I’m not saying that these measures of customer loyalty are unrelated, but that they are measurably different constructs (similar to the fact that height and weight are different constructs but are related to each other – taller people tend to weigh more than shorter people).
Measuring customer loyalty via questions on surveys is an entirely different process; customers’ ratings of each loyalty question (e.g., likelihood to recommend, satisfaction, likelihood to repurchase) become the metric of customer loyalty. Even though we are able calculate separate loyalty scores from each loyalty question (e.g., NPS, Overall Satisfaction, Likelihood to Repurchase), the distinction among the loyalty questions may not be as clear as we think. Because of the way customers interpret survey questions and the inherent error associated with measuring psychological constructs, ratings need to be critically evaluated to ensure we understand the meaning behind the ratings.
When using questionnaires to measure constructs, we need to be mindful of how the customers interpret and respond to the questions. Questions might appear to contain very different content (e.g., recommend, satisfaction, purchase same) yet the customers apparently do not make those same distinctions when ratings these questions.
Conclusions
Customers' ratings of a set of loyalty questions suggest that there are two, very general, loyalty constructs, Advocacy and Purchasing; and a third construct, Defection loyalty. The present findings suggest that we can create composite loyalty scores. These composite scores and their definitions are:
The next part of Customer Loyalty 2.0, I will explore the differences among the measures of customer loyalty. Toward this end, I will try to identify the different antecedents and consequences of customer loyalty. Rather than thinking of customer loyalty as a one-dimensional construct, this multi-dimensional approach to and measurement of customer loyalty can help companies better understand how to improve growth through both new and existing customers.
You can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.
References
Fornell, C., Mithas, S., Morgensen, F. V., Krishan, M. S. (2006). Customer satisfaction and stock prices: High returns, low risk. Journal of Marketing, 70 (January), 1-14.
Hayes, B. E. (1997). Measuring Customer Satisfaction (2nd Ed.). Quaility Press. Milwaukee, WI.
Keiningham, T. L., Cooil, B., Andreassen, T.W., & Aksoy, L. (2007). A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71 (July), 39-51.
Morgan, N.A. & Rego, L.L. (2006). The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25(5), 426-439.
Reichheld, F. F. (2003). The One Number You Need to Grow. Harvard Business Review, 81 (December), 46-54.
Reichheld, F. F. (2006). The ultimate question: driving good profits and true growth. Harvard Business School Press. Boston.
