Subgroup analysis using Bernoulli‐gated hierarchical mixtures of experts models

Abstract

SummaryWhen it is suspected that the treatment effect may only be strong for certain subpopulations, identifying the baseline covariate profiles of subgroups who benefit from such a treatment is of key importance. In this paper, we propose an approach for subgroup analysis by firstly introducing Bernoulli‐gated hierarchical mixtures of experts (BHME), a binary‐tree structured model to explore heterogeneity of the underlying distribution. We show identifiability of the BHME model and develop an EM‐based maximum likelihood method for optimization. The algorithm automatically determines a partition structure with optimal prediction but possibly suboptimal in identifying treatment effect heterogeneity. We then suggest a testing‐based postscreening step to further capture effect heterogeneity. Simulation results show that our approach outperforms competing methods on discovery of differential treatment effects and other related metrics. We finally apply the proposed approach to a real dataset from the Tennessee's Student/Teacher Achievement Ratio project.