Evaluating the Effectiveness of Website Content Features Using Retrospective Pretest Methodology

Abstract

Background and Objectives: In order to assess website content effectiveness (WCE), investigations have to be made into whether the reception of website contents leads to a change in the characteristics of website visitors or not. Because randomized controlled trials (RCTs) are not always the method of choice, researchers may have to follow other strategies such as using retrospective pretest methodology (RPM), a straightforward and easy-to-implement tool for estimating intervention effects. This article aims to introduce RPM in the context of website evaluation and test its viability under experimental conditions. Method: Building on the idea that RCTs deliver unbiased estimates of the true causal effects of website content reception, I compared the performance of RPM with that of an RCT within the same study. Hence, if RPM provides effect estimates similar to those of the RCT, it can be considered a viable tool for assessing the effectiveness of the website content features under study. Results and Conclusions: RPM was capable of delivering comparatively resilient estimates of the effects of a YouTube video and a text feature on knowledge and attitudes. With regard to all of the outcome variables considered, the differences between the sizes of the effects estimated by the RCT and RPM were not significant. Additionally, RPM delivered relatively accurate effect size estimates in most of the cases. Therefore, I conclude that RPM could be a viable alternative for assessing WCE in cases where RCTs are not the preferred method.