BACKGROUND
Executive functions (EFs) predict positive life outcomes and educational attainment. As such, it is imperative that our measures of EF constructs are both reliable and valid, with advantages for research tools that offer efficiency and remote capabilities.
OBJECTIVE
The objective of this study was to evaluate initial reliability and validity evidence for a mobile, adaptive measure of EFs called Adaptive Cognitive Evaluation - Explorer (ACE-X).
METHODS
We collected data from two cohorts of participants: a test-retest sample (N=246, age: mean=35.75, SD=11.74) to assess consistency of ACE-X task performance over repeated administrations, and a validation sample involving child/adolescent (n=5,436, age: mean=12.78, SD=1.60) and adult participants (n=484, age: mean=38.11, SD=14.96) to examine consistency of metrics, internal structures, and invariance of ACE-X task performance. A subset of participants (n=132, age: mean=37.04, SD=13.23) also completed a similar set of cognitive tasks using the Inquisit™ platform to assess the concurrent validity of ACE-X.
RESULTS
Intraclass correlation coefficients revealed most ACE-X tasks were moderately to very reliable across repeated assessments (ICC=.45-.79, P<.001). Mean comparisons with the literature indicated that patterns of responses were consistent with those reported in previous studies. Moreover, in comparisons of internal structures of ACE-X task performance, model fit indices suggested that a network model based on partial correlations was the best fit to the data (χ2(28)=40.13, P=.064, CFI=.99, RMSEA=.03[.00;.05], BIC=5075.87, AIC=4917.71), and that network edge weights are invariant across both younger and older adult participants. A Spinglass community detection algorithm suggested ACE-X task performance can be described by 3 communities (selected in 85% of replications): set reconfiguration, attentional control, and interference resolution. On the other hand, Pearson correlation coefficients indicated mixed results for the concurrent validity comparisons between ACE-X and Inquisit™ (r=-.05-.62, P<.001-.761).
CONCLUSIONS
These preliminary findings suggest that ACE-X is a reliable and valid research tool for understanding EFs and their relations to outcome measures.