Development and validation of a virtual learning management system usability questionnaire: A case study

Abstract

BACKGROUND: The post-pandemic era has seen a surge in the popularity of Virtual Learning Management Systems (VLMS). However, there is a noticeable lack of tools to measure the usability of these systems. As technology evolves, user needs change, necessitating updated tools for system evaluation. OBJECTIVE: This study aims to develop and validate a VLMS usability questionnaire, specifically designed to assess the usability of a university learning management system METHODS: The VLMS usability tool was systematically developed based on relevant domains identified in existing literature and expert opinions. It was then tested for face validity, content validity, and reliability. In a case study, the tool was distributed among 200 students from a Medical Sciences university who had used the Navid VLMS system. RESULTS: Semi-structured interviews with experts were analyzed using directed content analysis, resulting in 21 items categorized into four domains: effectiveness, reliability, learnability, and security. The content validity index and ratio were 0.939 and 0.976, respectively. The Intra Class Correlation (ICC) estimates for each section of the questionnaire ranged from 0.8–0.9, indicating high reliability. Cronbach’s alpha was 0.97, suggesting excellent internal consistency. The case study results showed that the Navid platform achieved an average usability score of 70.36, with a standard deviation of 10.6, indicating moderate to high usability. CONCLUSIONS: The VLMS usability tool is a valid and reliable instrument for assessing the usability of the Navid learning management system. It can be used to improve the usability of the Navid system and serve as a benchmark for assessing the usability of other similar VLMSs.