A Multiple-Item Model of Paired Comparisons: Separating Chance from Latent Preference

Abstract

The authors develop a flexible model to analyze relative preference scores: the binomial/Dirichlet model. This model assumes that (1) individual respondents make independent draws from binomial distributions when stating their preferences and (2) the latent (unobserved) preference parameters vary across respondents according to a Dirichlet distribution. Through the analysis of 44 tests that include from two to five products each, the authors show that the model fits the data relatively well. A multinomial/Dirichlet extension of the model that applies to repeat preference tests of two items provides a better fit than an alternative mixture model despite fewer parameters. To test two items and obtain an accuracy of +.05 with a 95% confidence interval for the mean preference intensities, a multinomial/Dirichlet model requires two paired comparisons (made at two points in time) per respondent and a sample size of 400; these requirements represent half the required number of preference measurements per respondent and half the required sample size of alternative methods. Although the illustrative examples refer to the comparison of known brands and unidentified products, the proposed methodology can be applied to many contexts, including the evaluation of product profiles in conjoint analysis.