Customer Loyalty 2.0, Part 5: Measurement and Meaning of Customer Loyalty: Drivers of Advocacy Loyalty and Purchasing Loyalty

The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 5 of the discussion. If you missed them, read Part 1, Part 2 and Part 3 and Part 4.

Customer Loyalty 2.0 represents this advancement in the measurement and meaning of customer loyalty. The purpose of the following analyses is to provide additional validity evidence regarding the measures of loyalty, Advocacy Loyalty Index (ALI) and Purchasing Loyalty Index (PLI). A common approach to establishing the validity is to show how the ALI and PLI are related to attributes of service and product quality. Below, I employ a common method, loyalty driver analysis, used by companies to identify the key drivers of customer loyalty. This method includes analysis that shows the relationship between business attributes and customer loyalty. In the process of establishing validity evidence, I will provide real-life illustrations regarding the merits of conceptualizing customer loyalty in this multidimensional framework that can help companies increase growth through new and existing customers.

Loyalty Driver Analysis

Loyalty driver analysis enables companies to identify the business attributes that are important to ensuring customer loyalty. Companies can allocate resources to important business areas that have the greatest impact on increasing customer loyalty. While each response from a survey can be (and should be) examined to deal with specific causes of customer loyalty/disloyalty at an individual, customer level, driver analysis can be thought of as a macro look at the customer base (or customer segment). By analyzing data from a large segment of customers, driver analysis helps companies to identify the common causes of customer loyalty across these customers. Consequently, based on the results of the driver analysis, companies can make organization-wide improvements to their business processes that will have an impact on the customer segment of interest.

Two pieces of information are examined in driver analysis: 1) derived importance: the degree of impact of each business attribute on customer loyalty and 2) performance: the level of performance of each business attribute.

Derived Importance

The degree of impact each business attribute has on customer loyalty is determined. This degree of impact is indexed by a correlation coefficient (sometimes, referred to as “derived importance”) between ratings of a business attribute and a customer loyalty index (either ALI or PLI). Each business attribute has a corresponding “derived importance” that indicates the impact that the business attribute has on customer loyalty. The higher the correlation, the greater the impact that business attribute has on customer loyalty. The degree of impact (e.g., correlation coefficient) can vary from 0.0 (no impact) to 1.0 (perfect impact).

Using the Wireless Service Provider and PC Manufacturer studies, we can identify the derived importance of each of the business attributes. Given we have multiple measures of customer loyalty, each of the business attributes will have multiple derived importance, one for advocacy loyalty, one for purchasing loyalty and one for retention loyalty (Wireless study only). In both studies, respondents were asked to indicate the degree to which they agree or disagree with statements regarding their customer experience on a scale from 1 (strongly disagree) to 5 (strongly agree). Statements reflected business attributes that ranged from product quality (e.g., reliable service, PC reliability) to service quality (e.g., customer service reps, technical support reps). These ratings were correlated with each of the measures of customer loyalty, ALI, PLI and RLI.

Performance

Next, company’s performance on each business attribute (customer ratings) is calculated. Performance of a given business attribute is simply the average rating of agreement. Higher scores reflected better performance (better customer experience).  Possible performance scores could range from 1 (worst customer experience) to 5 (best customer experience). Business attributes that have low performance ratings have ample room for improvement. Business attributes that have high performance ratings have little room for improvement.

Wireless Service Provider Driver Analysis

I calculated the derived importance and performance for one of the Wireless Service Providers in the study. Table 3 contains the results for this particular Wireless Service Provider. The column labeled "Mean" reflects the performance for each of the business attributes; the columns labeled "Derived Importance on ALI," "Derived Importance on PLI," and "Derived Importance on RLI" reflect the importance for each of the business attributes on advocacy loyalty, purchasing loyalty and retention loyalty, respectively.

Table 3. Descriptive Statistics and Derived Importance of each Business Attribute for a Wireless Service Provider

In driver analysis, both the performance and derived importance are examined simultaneously to understand where improvements would have the greatest chance to improve customer loyalty. If business attributes that have a large impact on customer loyalty (high derived importance) and  have low performance ratings, companies might consider allocating resources to these business attributes in order to improve customer loyalty.  If ratings of business attributes are high, however, companies can promote these business attributes as strengths and best practices. Using both the derived importance of each business attribute and the performance (e.g., rating) of each business attribute, we can create a Loyalty Matrix (see figures below) that allows us to visually examine all business attributes at one time.

The abscissa (x-axis) of the Loyalty Matrix is the performance rating (agreement, performance, satisfaction) of the business attributes. The ordinate (y-axis) of the Loyalty Matrix is the impact (derived importance) of the business attribute on customer loyalty. The Loyalty Matrix is divided into quadrants using the average score for each of the axes. Key drivers appear in the upper left quadrant and are often referred to as Key Drivers. Key drivers reflect business attributes that have both a large impact on customer loyalty and have low performance ratings relative to the other business attributes (these key drivers appear in red). Because we have three customer loyalty indices, we can calculate three separate Loyalty Matrices, each for the specific customer loyalty index. Below are the Loyalty Matrices for a particular Wireless Service Provider.

Figure 4. Advocacy Loyalty Driver Analysis - Wireless Service
Provider

Figure 5. Purchasing Loyalty Driver Analysis - Wireless Service
Provider

Figure 6. Retention Loyalty Driver Analysis - Wireless Service
Provider

The results each of the driver analyses seem fairly consistent. We see that Customer Service Representatives (CSRs) have a relatively large impact on customer loyalty compared to the primary product offering attributes (Reliable service, Good coverage). Whether customers will be loyal to this Wireless Service Provider depend to a greater degree on the competency of the CSRs than on the product offering attributes.

Using the three Loyalty Matrices, we can draw some conclusions regarding how this particular Wireless Service Provider can increase advocacy, purchasing and retention loyalty. When employing the use of driver analysis, we typically focus on the the Key Drivers (those attributes in the upper left hand quadrant) as areas to focus if we want to improve customer loyalty. We do so because these are the attributes that have a large impact on customer loyalty and have much room for improvement. To improve loyalty, no matter how it is measured, the results of the driver analysis indicate that the company should focus on improving CSR attributes as these attributes have a relatively large impact on advocacy, purchasing and retention loyalty and have much room for improvement.

PC Manufacturer Driver Analysis

I calculated the derived importance and performance for one of the PC Manufacturers in the study. Table 4 contains the results for this PC Manufacturer. The column labeled "Mean" reflects the performance for each of the business attributes; the columns labeled "Derived Importance on ALI" and "Derived Importance on PLI" reflect the importance for each of the business attributes on advocacy loyalty and purchasing loyalty, respectively.

Table 4. Descriptive Statistics and Derived Importance of each Business Attribute for a PC Manufacturer

 

Below are the Loyalty Matrices for a particular PC manufacturer.


Figure 7. Advocacy Loyalty Driver Analysis - PC Manufacturer



Figure 8. Purchasing Loyalty Driver Analysis - PC Manufacturer

We see that there are differences in what drives advocacy loyalty and purchasing loyalty. With regard to advocacy loyalty, we see that both of the PC attributes (PC reliability, and PC features) have a big impact on advocacy loyalty, more so than the technical support attributes. Whether customers will be advocates of this PC manufacturer depend highly on the computer itself and less so on the quality of technical support. With regard to purchasing loyalty, however, we see that many of the technical support attributes (excellence, timeliness, understands needs, availability) have a relatively big impact on purchasing loyalty. Interestingly, PC attributes do not have a big impact on purchasing loyalty.

Using the two Loyalty Matrices, we can draw some conclusions regarding how this particular PC Manufacturer can increase advocacy loyalty and purchasing loyalty. To improve advocacy loyalty, the driver analysis seems inconclusive. While PC features are big determinants of advocacy, they are rated as relatively good. Consequently, there is not much room for improvement in these attributes. The technical support attributes, while rated as relatively low, do not have a large impact on advocacy loyalty. If this PC Manufacturer wants to improve purchasing loyalty, however, the results of the driver analysis indicate that they should focus on improving technical support attributes as these attributes have a relatively large impact on purchasing loyalty and have much room for improvement.

Missed Opportunities to Improve Customer Loyalty

When practitioners talk about "customer loyalty," they are usually referring to advocacy loyalty; many loyalty programs are, in fact, based solely on advocacy-related content. For example, the Net Promoter Score is based on the "likelihood to recommend" question. Additionally, the American Customer Satisfaction Index (ACSI) is based on the "satisfaction" question. If the PC manufacturer relied solely on this question, the results of the driver analysis suggests that they should focus ensuring that their PCs are reliable and has features their customers want. This driver analysis found that many of the technical support items were not important in improving advocacy loyalty (they appeared in the lower left quadrant). The use of advocacy-related loyalty questions as a way of measuring and defining customer loyalty limits improvements in acquiring new customers.

The driver analysis using the PLI painted an entirely different picture. We saw that many technical support items were now key drivers of customer loyalty. Expanding the definition of customer loyalty to include increased purchasing intentions clearly shows that loyalty can be improved beyond mere referrals of new customers. Companies can now identify business attributes that, when improved, would increase purchasing loyalty of existing customers. Improving technical support for the PC Manufacturer would increase revenue from existing customers through increasing their purchasing behavior (buy different products and increasing their purchasing frequency). Looking at both advocacy loyalty and purchasing loyalty, this PC manufacturer can maximize revenue through both new and existing customers.

Summary

The evidence from two separate studies shows that the Advocacy Loyalty Index (ALI) and the Purchasing Loyalty Index (PLI) measure two different types of loyalty. Even though the two types of loyalty are correlated (advocates tend to be purchasers), the relationship between the ALI and PLI is not perfect, suggesting that these loyalty indices measure unique constructs. We have good evidence that the loyalty indices are each measuring some unique aspect of customer loyalty.

The results of the present analysis show that, as expected, the measures of customer loyalty are logically related to the customer experience. Customers who have a better customer experience tend to have higher levels of customer loyalty. Furthermore, the impact that the business attributes have on customer loyalty depends on the customer loyalty index that is used.

For more information about the Advocacy Loyalty Index and the Purchasing Loyalty Index and more detailed information about the driver analyses in the studies reported here, you can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.

Customer Loyalty 2.0, Part 3: Reliability of Loyalty Indices

The measurement of customer loyalty has been a hot topic lately. With the latest critiques of the Net Promoter Score coming in from the both practitioners and academic researchers, there is much debate on how companies should measure customer loyalty. I wanted to formally write my thoughts on this topic to get feedback from this community of users. Much of what I will present here will be included in the third edition of my book, Measuring Customer Satisfaction. I welcome your thoughts and critiques. Due to the length of the present discussion, I have broken down the entire discussion into several parts. I will post each of them weekly. Below is Part 3 of the discussion. Part 1 can be found here ( http://businessoverbroadway.blogspot.com/2007/11/customer-loyalty-20.html). Part 2 can be found here ( http://businessoverbroadway.blogspot.com/2007/12/customer-loyalty-20-part-2-advocacy.html).

Loyalty Indices

The results of the factor analyses support the use of composite scores, each representing one of the loyalty dimensions. These composite scores are referred to as scales/indices/metrics. This process of calculating these scales/indices/metrics is done by averaging the items that load on the same factors. Based on the results of the present analyses, we can calculate three indices:

Advocacy Loyalty Index (ALI): The ALI measures the extent to which customers are advocates of the company. This index is calculated by averaging the responses to the following questions: Overall Satisfaction, Choose Again, Recommend, Purchase Same.

Purchasing Loyalty Index (PLI): The PLI measures the extent to which customers are likely to increase their purchasing behavior. This index is calculated by averaging the responses to the following questions: Purchase Different, Purchase Increase, Purchase Frequency (only for PC Study).

Defection Loyalty Index: The DLI measures the degree to which customers are likely to defect. This index is the score from the single item: Likelihood to switch to another provider.

When using scales from surveys to measure constructs, we need to be concerned about the quality of the scales. The quality of these surveys are typically discussed with respect to reliability and validity. Reliability refers to the degree to which scores are free from measurement error. Validity refers to the degree to which the scale measures what is was designed to measure. Before we use these new scales, I will briefly discuss these measurement principles in the context of classical test theory.

Classical Test Theory

Classical test theory is based on the idea that an observed score (X) from a survey can be decomposed into two different scores, a true score (T) and an error score (E) where:

X = T + E

As the equation implies, as error decreases, the observed score (X) matches the underlying true score (T). Classical test theory is concerned with the relationships among the three variables, X, T and E. The relationships among these three components are used to understand the quality of the scores that result from the scales. The first quality of measurement, reliability, is concerned with the relationship between the observed score (X) and the true score (T).

Reliability

Reliability is the degree to which measurements are free from random errors. Reliability deals with precision or consistency of measurement. Scales or indices with high reliability are better at distinguishing people on the continuum of customer loyalty. Our goal in developing customer loyalty indices is to have a measurement instrument that delivers reliable results. Reliability can be thought of as the relationship between the true underlying score and the observable score we get from our survey. Random error decreases the measurement’s reliability; that is, as random error is introduced into measurement, the observed score is not a good reflection of the true underlying score. For one to feel confident that a questionnaire’s scores accurately reflect the underlying dimension, the questionnaires must have high reliability. Although many types of reliability exist, internal consistency reliability is vital to surveys.

Internal consistency indicates the extent to which the items in the measurement are related to each other. The higher the interrelationship among the items, the higher the internal consistency. If a questionnaire is designed to measure one underlying construct, the items are expected to be related to each other – that is, people who respond in one way to an item are likely to respond the same way to the other items in the measure.

There are several statistical indices used to estimate the degree of internal consistency. The most commonly used index is Cronbach’s coefficient alpha (Cronbach, 1951). Basically, this alpha coefficient indicates the degree to which items are related to each other. Cronbach's alpha increases when the correlations among the items increase. Cronbach's alpha can range from 0 to 1.0. A reliability of 0 indicates that the observed score is not related to the underlying true score; a reliability of 1 indicates that the observed score is a perfect indicator of the underlying true score. Generally, a reliability of .8 or greater is an acceptable level of reliability.

There are a couple of key benefits to using customer loyalty indices that have high reliability. First, customer loyalty scales with high reliability are better able to distinguish between varying levels of customer loyalty than loyalty scales with low reliability. Because scales with higher reliability have excellent precision, they are able to distinguish small differences in loyalty. Second, using a loyalty scale with high reliability, you are more likely to find significant relationships with other variables when loyalty is truly related to them.

Although reliability is an important ingredient in the evaluation of a questionnaire, it cannot solely determine the quality of the questionnaire. The questionnaire’s validity must also be addressed.

Validity

Validity refers to the degree to which evidence supports the inferences made from scores derived from measurements, or the degree to which the scale measures what it is designed to measure. Unlike reliability, there is no single statistic that provides an overall index of the validity of inferences about the scores.

The methods for gathering evidence of validity can be grouped into three categories: content-related evidence, criterion-related evidence, and construct-related evidence. These labels simply enable people to discuss the types of information that might be considered when determining the validity of the inferences.

Content-related evidence is concerned with the degree to which the items in the questionnaire are representative of a “defined universe” or “domain of content.” The domain of content typically refers to all possible items that could have been used in the questionnaire. The goal of content-related validity is to have a set of items that best represent the defined universe.

Criterion-related evidence is concerned with examining the systematic relationship (usually in the form of a correlation coefficient) between the loyalty scale and another measure, or criterion. In this case, what the criterion is and how it is measured are of central importance. The main question to be addressed in criterion-related validity is how well the scale can predict the criterion.

Construct-related evidence is concerned with the questionnaire as a measurement of an underlying construct. Unlike criterion-related validity, the primary focus is on the scale itself rather than on what the scale predicts. Construct-related evidence is derived from both previous validity strategies. A high degree of correlation between the scale and other scales that purportedly measure the same construct is evidence of construct-related validity. Construct-related validity can also be evidenced by a low correlation between the scale and other scales that measure a different construct.

The figure below illustrates the distinction between reliability and validity. Recall that reliability deals with precision/consistency while validity refers to meaning behind the scores. The diagram consists of four targets, each with four shots. In the upper left hand target, we see that the there is high reliability in the shots that were fired yet the bull’s-eye has not been hit. This is akin to having a scale with high reliability but is not measuring what the scale was designed to measure (not valid). In the lower right target, the pattern indicates that there is little consistency in the shots but that the shots are all around the bull’s-eye of the target (valid). The pattern of shots in the lower left target illustrates low consistency/precision (no reliability) and an inability to hit the target (not valid). The upper right pattern of shots at the target represents our goal to have precision/consistency in our shots (reliability) as well as hitting the bull’s-eye of the target (validity).



Reliability of Loyalty Indices

Reliability estimates were calculated for each of the loyalty indices. For the Wireless Service Provider sample, the reliability (Cronbach’s alpha) of the Advocacy Loyalty Index (ALI) was .92. The reliability estimate (Cronbach's alpha) for the Purchasing Loyalty Index (PLI) was .82. For the Personal Computer Manufacturer sample, the reliability of the ALI was .94. The reliability of the PLI was .87. These levels of reliability are considered very good for attitude research. The high reliability of each of the scales suggests that there is minimal measurement error associated with each composite score. Thus, we can feel confident that the observed scores (X) we get from the survey results are a very good reflection of the underlying true scores (T).

Validity of Loyalty Indices

Establishing the validity of the loyalty scales is a more complex process and will be discussed in the next blog.

You can download a free copy of executive reports on the two studies (Wireless Service Providers and PC Manufacturers) at Business Over Broadway.

References

Allen, M.J., & Yen, W. M. (2002). Introduction to Measurement Theory. Long Grove, IL: Waveland Press.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.