Optimization‐Based Stable Balancing Weights Versus Propensity Score Weighting for Samples With High Covariate Imbalance

Abstract

ABSTRACTPurposeTo compare the performance (covariate balance, effective sample size [ESS]) of stable balancing weights (SBW) versus propensity score weighting (PSW). Two applied cases were used to compare performance: (Case 1) extreme imbalance in baseline covariates between groups and (Case 2) substantial discrepancy in sample size between groups.MethodsUsing the Premier Healthcare Database, we selected patients who (Case 1) underwent a surgical procedure with one of two different bipolar forceps between January 2000 and June 2020, or (Case 2) a neurological procedure using one of two different nonabsorbable surgical sutures between January 2000 and March 2020. Average treatment effects on the treated (ATT) weights were generated based on selected covariates. SBW was implemented using two techniques: (1) “grid search” to find weights of minimum variance at the lowest target absolute standardized mean difference (SMD); (2) finding weights of minimum variance at prespecified SMD tolerance. PSW and SBW methods were compared on postweighting SMDs, the number of imbalanced covariates, and ESS of the ATT‐weighted control group.ResultsIn both studies, improved covariate balance was achieved with both SBW techniques. All methods suffered from postweighting ESS that was lower than the unweighted control group's original sample size; however, SBW methods achieved higher ESS for the control groups. Sensitivity analyses using SBW to apply variable‐specific SMD thresholds increased ESS, outperforming PSW.ConclusionsIn this applied example, the optimization‐based SBW method provided ample flexibility with respect to prespecification of covariate balance goals and resulted in better postweighting covariate balance and larger ESS as compared with PSW.