Use of the FallAkte Plus System as an IT Infrastructure for the North Rhine-Westphalian General Practice Research Network: Mixed Methods Usability Study

Abstract

Background Primary care research networks can generate important information in the setting where most patients are seen and treated. However, this requires a suitable IT infrastructure (ITI), which the North Rhine-Westphalian general practice research network is looking to implement. Objective This mixed methods research study aims to evaluate (study 1) requirements for an ITI and (study 2) the usability of an IT solution already available on the market, the FallAkte Plus (FA+) system for the North Rhine-Westphalian general practice research network, which comprises 8 primary care university institutes in Germany’s largest state. Methods In study 1, a survey was conducted among researchers from the institutes to identify the requirements for a suitable ITI. The questionnaire consisted of standardized questions with open-ended responses. In study 2, a mixed method approach combining a think-aloud approach and a quantitative survey was used to evaluate the usability and acceptance of the FA+ system among 3 user groups: researchers, general practitioners, and practice assistants. Respondents were asked to assess the usability with the validated system usability scale and to test a short questionnaire on vaccination management through FA+. Results In study 1, five of 8 institutes participated in the requirements survey. A total of 32 user requirements related primarily to study management were identified, including data entry, data storage, and user access management. In study 2, a total of 36 participants (24 researchers and 12 general practitioners or practice assistants) were surveyed in the mixed methods study of an already existing IT solution. The tutorial video and handouts explaining how to use the FA+ system were well received. Researchers, unlike practice personnel, were concerned about data security and data protection regarding the system’s emergency feature, which enables access to all patient data. The median overall system usability scale rating was 60 (IQR 33.0-85.0), whereby practice personnel (median 82, IQR 58.0-94.0) assigned higher ratings than researchers (median 44, IQR 14.0-61.5). Users appreciated the option to integrate data from practices and other health care facilities. However, they voted against the use of the FA+ system due to a lack of support for various study formats. Conclusions Usability assessments vary markedly by professional group and role. In its current stage of development, the FA+ system does not fully meet the requirements for a suitable ITI. Improvements in the user interface, performance, interoperability, security, and advanced features are necessary to make it more effective and user-friendly. Collaborating with end users and incorporating their feedback are crucial for the successful development of any practice network research ITI.