|
Rating Data From a strictly statistical perspective, there is no reason why rating data cannot be used for a conjoint analysis. All that is required is that a derived ranking be created from the ratings data for each of the hypothetical products that would be explicitly ranked in a typical conjoint analysis. Once done, the statistical algorithms of a conjoint analysis can be run on these data. How can ratings data be used to create the derived rankings in the transposed matrix for input into a conjoint analysis? To discuss this, we introduce a hypothetical project involving tea where the three underlying attributes of the tested products are binary factors and where, because the fully-crossed design is only a 2 × 2 × 2 with eight unique combinations, all possible combinations are used in the study. The three factors and the associated levels include:
If each respondent is asked to rate the bipolar factors on some preference scale, say 1-8, where 1 indicates a preference for one end of the continuum (e.g., Hot) and 8 indicates a preference for the other end of the continuum (Cold), then developing the derived ranking for that subject for each combination simply requires a comparison function f(X) that evaluates each subject's score on each of the three underlying attributes in comparison to the ideal point for that combination, and sums across the three attributes to develop an overall derived ranking. A simple form of the comparison function f(X) is the absolute value of the differences between observed and ideal, summed across the attributes. To illustrate, assume that a respondent presents the following pattern of ratings:  The blue number, i.e., response 5 for all three ratings, indicates the selected response for this subject. The eight profiles yield the following derived rankings. Note that this example precisely illustrates one of the most significant problems with using rating scales in that a substantial number of ties occur in the data.
Note that low scores indicate a higher preference with this scoring system. The lowest score logically matches to the most preferred combination, and the highest score is logically the least preferred. Obviously, however, no single combination is ranked 1 because this subject does not have extreme opinions. This is not a crucial problem with estimating a conjoint solution. What is important, however, is the use of relatively few ranks, with three 10 scores and three 11 scores capturing the sole variation between the most and least preferred options. With some justification, we could argue that any person who holds such mushy opinions should appropriately be treated as having only very weak preferences for one product configuration versus another. For the statistical properties of the conjoint, however, the presence of the large number of ties is disconcerting in that degenerate solutions may well occur. A preferred pattern with middle-of-the-road responses such as these would be a pattern with scores of, let's say, 9.0, 9.2, 9.7, 10.2, 10.7, 11.1, 11.7, and 12. The presence of ties is the prior example is exacerbated by the fact that identical scores occur on all three ratings. Ties can well occur with other, more diverse and extreme ratings. Consider the following pattern of responses.  Using the absolute value of the simple differences model, this score profile yields derived rankings as follows:
While the problem with ties is diminished in this example because of the presence of more diverse ratings, it is not eliminated. The derived preference scores are logically appropriate at the extremes, i.e., the two most and least preferred options, but the middle four while separated into two logical groups of two continue to show ties. The continuing problem with ties can be further addressed by adding a more limited ranking task to the data collection effort. Rather than ranking all eight product combinations as in the typical conjoint problem, subjects can be asked to rank the intensity of their preference for their stated rating. Assume that the respondent in the immediately prior set of data stated that the most important issue in this set of ratings was a preference for a weak taste, followed by their preference for no sugar, followed by the preference for hot tea. Using rankings of 1-3 for these scores, a new set of derived rankings using a weighted simple difference model yields the following scores:
No ties exist, and a pure 1-8 ranking of the different combinations could be developed by simply rank ordering the derived preference scores. Simply using weights for the ranks in conjunction with the ratings does not completely eliminate any problems with ties, however. Using the same weighting system, but applying these weights to the first sample ratings profile that included the subject who responded with a score of 5 to each question (see page 4, above) yields the following distribution of derived preference scores:
Note that the range of the derived preference scores is, under this contrived example, very restricted. This is expected given the ties in the ratings. Using the weights derived from the attribute rankings, however, almost eliminates the problems with ties in the derived preference ranking score. Only in the exact middle of the distribution at a derived score of 21 does a tie exist. These examples illustrate the general logic of the alternative conjoint method. By virtue of having only three underlying attributes, however, these examples mask the fact that continued problems with ties will occur as the number of binary attributes increases, even with the weighting function. This problem can be minimized by using a more complex comparison function f(X) and a more complex weighting function. We consider the exact comparison function and weighting function that we have developed to be proprietary intellectual property, and we will not disclose the exact nature of this information. We will, however, add the following pieces of information to help further describe the transformations that are used:
These modifications to the simplistic example using absolute value differences as the comparison function and simple integer weights for the weighting function serve to minimize the problem with ties. At this point, we intend to conduct analyses of the data where the derived preference scores are converted into true ranks and also analyzed directly to determine the correspondence of the utilities that are obtained from using two different approaches. |