A Meta-Analytic Approach to Growth Curve Analysis

Abstract

A meta-analytic approach to growth curve analysis is described and illustrated by applying it to the evaluation of the Arizona Pilot Project, an experimental project for financing the treatment of the severely mentally ill. In this approach to longitudinal data analysis, each individual subject for which repeated measures are obtained is initially treated as a separate case study for analysis. This approach has at least two distinct advantages. First, it does not assume a balanced design (equal numbers of repeated observations) across all subjects; to accommodate a variable number of observations for each subject, individual growth curve parameters are differentially weighted by the number of repeated measures on which they are based. Second, it does not assume homogeneity of treatment effects (equal slopes) across all subjects. Individual differences in growth curve parameters representing potentially unequal developmental rates through time are explicitly modeled. A meta-analytic approach to growth curve analysis may be the optimal analytical strategy for longitudinal studies where either (1) a balanced design is not feasible or (2) an assumption of homogeneity of treatment effects across all individuals is theoretically indefensible. In our evaluation of the Arizona Pilot Project, individual growth curve parameters were obtained for each of the 13 rationally derived subscales of the New York Functional Assessment Survey, over time, by linear regression analysis. The slopes, intercepts, and residuals obtained for each individual were then subjected to meta-analytic causal modeling. Using factor analytic models and then general linear models for the latent constructs, the growth curve parameters of all individuals were systematically related to each other via common factors and predicted based on hypothesized exogenous causal factors. The same two highly correlated common factors were found for all three growth curve parameters analyzed, a general psychological factor and a general functional factor. The factor patterns were found to be nearly identical across the separate analyses of individual intercepts, slopes, and residuals. Direct effects on the unique factors of each subscale of the New York Functional Assessment Survey were tested for each growth curve parameter by including the common factors as hierarchically prior predictors in the structural model for each of the indicator variables, thus statistically controlling for any indirect effect produced on the indicator through the common factors. The exogenous predictors modeled were theoretically specified orthogonal contrasts for Method of Payment (comparing Arizona Pilot Project treatment or “capitation” to traditional or “fee-for-service” care as a control), Treatment Administration Site (comparing various locations within treatment or control groups), Pretreatment Assessment (comparing general functional level at intake as assigned by an Outside Assessment Team), and various interactions among these main effects. The intercepts, representing the initial status of individual subjects on both the two common factors and the 13 unique factors of the subscales of the New York Functional Assessment Survey, were found to vary significantly across many of the various different treatment conditions, treatment administration sites, and pretreatment functional levels. This indicated a severe threat to the validity of the originally intended design of the Arizona Pilot Project as a randomized experiment. When the systematic variations were statistically controlled by including intercepts as hierarchically prior predictors in the structural models for slopes, recasting the experiment as a nonequivalent groups design, the effects of the intercepts on the slopes were found to be both statistically significant and substantial in magnitude. Furthermore, the contrasts for Pretreatment Assessment scores also predicted statistically significant proportions of variance in both the two common factors and the 13 unique factors of the subscales of the New York Functional Assessment Survey for all three growth curve parameters, confirming an influence of the initial status of individual subjects on treatment effect. This empirical example illustrates both the mechanics and the many practical benefits of a meta-analytic approach to growth curve analysis in program evaluation.