An Empirical Study on User Experience Evaluation and Identification of Critical UX Issues

Abstract

We introduce an approach that supports researchers and practitioners to determine the quality of first-time user experience (FTUX) and long-term user experience (LTUX), as well as to identify critical issues with these two types of UX. The product we chose to study is a mobile fitness application. Mobile apps tend to have a much shorter service life than most other products; thus, the developers/designers need to pay great attention to both first-time and long-term user experience. This study is based on a multi-method approach. We employed the AttrakDiff questionnaire to assess users’ first impressions of the app, and the UX Curve method to evaluate how users’ experience of the app has changed over time. Besides the quantitative data, which helped to determine the quality of user experience, we also collected qualitative data during two interviews with participants, and focused on the issues that predominantly deteriorated user experience. A four-coordinate plane tool was designed later in the data analysis process that combined the two kinds of user experience data at the same time, which led to a qualitative positioning of the user experience status of a certain product. The model was further successfully adopted in the identification of user experience issues of an online fitness application.