Comparison of Prior Setting Methods for Multilevel Model Effect Estimation Based on Small Sample Imbalanced Nested Data in Bayesian Framework

Abstract

In the fields of education and psychology, nested data with small samples and imbalances are very common. Bauer et al. (2008) first proposed adjusting the traditional multilevel model to analyze the small sample imbalanced nested data (SSIND). In terms of parameter estimation, the Bayesian method shows the possibility of providing unbiased estimation when the sample size is small. This study proposes that the Bayesian method should be used to analyze the SSIND. This study explores the performance of different treatment effects and nesting effects estimation methods in the multilevel model based on the Bayesian method that performs well in the case of small samples, to provide an appropriate and scientific method reference for the subsequent analysis of the model. Two prior setting methods are compared for multilevel model effect estimation based on a small sample of imbalanced nested data in the Bayesian framework. Two prior setting methods are gamma prior setting method and uniform prior setting method. The research results show that when the treatment condition ICC is small (0.05), the bias and RMSE values of the parameter estimation by the gamma prior setting method are larger and the performance is unstable, while the bias and RMSE values of the parameter estimation by the uniform prior setting method are smaller and the performance is relatively stable, so the uniform prior setting method is recommended; when the treatment condition ICC is large (0.15), the bias and RMSE values of the parameter estimation by the uniform prior setting method are larger and the performance is unstable, while the bias and RMSE values of the parameter estimation by the gamma prior setting method are smaller and the performance is relatively stable, so the gamma prior setting method is recommended; when the treatment condition ICC is between 0.05 and 0.15, both prior setting methods have similar effects. Furthermore, when the number of treatment groups is small (8), the gamma prior setting method is recommended; when the number of treatment groups is large (16), the uniform prior setting method is recommended; when the number of treatment groups is between 8 and 16, both prior setting methods have similar effects. Summarily, when we choose which prior setting method to use for the SSIND, we must consider the interaction between the ICC and the number of treatment groups.