The Importance of Utility Balance in Efficient Choice Designs

Abstract

Choice designs traditionally have been built under the assumption that all coefficients are zero. The authors show that if there are reasonable nonzero priors for expected coefficients, then these can be used to generate more statistically efficient choice designs, because the alternatives in their choice sets are balanced in utility—they have more similar choice probabilities. The authors demonstrate that the appropriate measure of choice design efficiency requires probability centering and weighting of the rows of the design matrix, and they illustrate how this criterion enables the analyst to appropriately trade off utility balance against three other principles: orthogonality, level balance, and minimal overlap. Two methods, swapping and relabeling attribute levels, provide complementary ways to increase the utility balance of choice designs. The authors apply a process for generating utility-balanced designs to five different choice designs and show that it reduces by 10–50% the number of respondents needed to achieve a specific error level around the parameters. A sensitivity analysis reveals that these gains are diminished, but still substantial, despite strong misspecifications of prior parameter estimates.