Testing a Mixture of Rank Preference Models on Judges' Scores in Paris and Princeton

Abstract

AbstractRank preference and mixture models have been employed to evaluate the ranks assigned by consumers in taste tests of beans, cheese, crackers, salad dressings, soft drinks, sushi, animal feed, and wine. In many wine tastings, including the famous 1976 Judgment of Paris and the 2012 Judgment of Princeton, judges assign scores rather than ranks, and those scores often include ties. This article advances the application of ranking and mixture models to wine-tasting results by modifying the established use of a Plackett-Luce rank preference model to accommodate scores and ties. The modified model is tested and then employed to evaluate the Paris and Princeton wine-tasting results. Test results show that the mixture model is an accurate predictor of observed rank densities. Results for Paris and Princeton show that the group preference orders implied by the mixture model are highly correlated with the orders implied by widely employed rank-sum methods. However, the mixture model satisfies choice axioms that rank-sum methods do not, it yields an estimate of the proportion of scores that appear to be assigned randomly, and it also yields a preference order based on nonrandom preferences that tasters appear to hold in common. (JEL Classifications: A10, C00, C10, C12, D12)