Usability testing to refine a web-based tool for selection and use of dissemination and implementation science models (Preprint)

Abstract

Dissemination and Implementation (D&I) science is dedicated to increasing the speed and amount of evidence-based research translated into real-world practice and is often guided by one or multiple D&I theories, models, and framework (TMFs). The D&I Models in Health Research and Practice web tool (D&I TMFs web tool) provides researchers, practitioners, and students with guidance on how to Plan, Select, Combine, Adapt, Use, and Measure TMFs.

Dissemination and Implementation (D&I) science is dedicated to increasing the speed and amount of evidence-based research translated into real-world practice and is often guided by one or multiple D&I theories, models, and framework (TMFs). The D&I Models in Health Research and Practice web tool (D&I TMFs web tool) provides researchers, practitioners, and students with guidance on how to Plan, Select, Combine, Adapt, Use, and Measure TMFs.

An iterative user testing process was conducted with participants of varying levels of expertise and involvement in implementation science to facilitate updates and additions to D&I TMFs and to optimize functionality of the tool. A multi-step usability testing protocol involved the collection of a combination of quantitative and qualitative data including a pre-testing survey and a usability testing session involving (1) pre-testing interview; (2) hands-on usability testing; and (3) post-testing interview. Data from the pre-testing surveys were summarized as frequencies. Data from the usability testing sessions were analyzed and organized using a hybrid deductive and adapted, rapid matrix qualitative analysis approach.

A total of 15 interviewees represented diverse research and clinical groups, covering various fields utilizing D&I TMFs and levels of expertise in D&I research. When asked about D&I expertise, eight participants self-identified as novice or advanced beginner, three as intermediate, and four as advanced. A total of 847 interview comments were identified through coding. When reviewed for similarities, comments were combined and reduced to a total of 259 unique comments. Of those 259 comments, 142 were classified as a change that fit the priorities of the web tool and were feasible on which to act. Content, format, and functionality changes were made by the team and implemented directly into the web tool. Most notably, from a user experience perspective, the web tool now presents as a tool within a website, making it more intuitive for the intended audience.

The iterative, multi-step approach to usability testing was effective in eliciting needed feedback and responses to help the research team update, modify, and improve the web tool. A few features were identified by the participants and added or changed. These included adding more examples, definitions, visuals, tutorials and simplifying the written content. The goal/intention of web tool is now more prominent and the TMF search function is better optimized for intended users. The web tool remains flexible for updates with plans for further additions of health equity, de-implementation, and other advances in the field.