Abstract
AbstractThis work presents the development of a mixed reality (MR) application that uses color Passthrough for learning to play the piano. A study was carried out to compare the interpretation outcomes of the participants and their subjective experience when using the MR application developed to learn to play the piano with a system that used Synthesia (N = 33). The results show that the MR application and Synthesia were effective in learning piano. However, the students played the pieces significantly better when using the MR application. The two applications both provided a satisfying user experience. However, the subjective experience of the students was better when they used the MR application. Other conclusions derived from the study include the following: (1) The outcomes of the students and their subjective opinion about the experience when using the MR application were independent of age and gender; (2) the sense of presence offered by the MR application was high (above 6 on a scale of 1 to 7); (3) the adverse effects induced by wearing the Meta Quest Pro and using our MR application were negligible; and (4) the students showed their preference for the MR application. As a conclusion, the advantage of our MR application compared to other types of applications (e.g., non-projected piano roll notation) is that the user has a direct view of the piano and the help elements appear integrated in the user’s view. The user does not have to take their eyes off the keyboard and is focused on playing the piano.
Funder
Universitat Politècnica de València
Publisher
Springer Science and Business Media LLC
Reference22 articles.
1. Banquiero M, Valdeolivas G, Trincado S et al (2023) Passthrough mixed reality with Oculus Quest 2: a case study on learning piano. IEEE Multimed 30:60–69. https://doi.org/10.1109/MMUL.2022.3232892
2. Bimberg P, Weissker T, Kulik A (2020) On the usage of the simulator sickness questionnaire for virtual reality research. In: 2020 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW), pp 464–467
3. Birhanu A, Rank S (2017) KeynVision: exploring piano pedagogy in mixed reality. In: Extended abstracts publication of the annual symposium on computer-human interaction in play. Association for Computing Machinery, New York, NY, USA, pp 299–304
4. Cárdenas-Delgado S, Juan MC, Méndez-López M, Pérez-Hernández E (2017) Could people with stereo-deficiencies have a rich 3D experience using HMDs? In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 97–116
5. Chow J, Feng H, Amor R, Wünsche BC (2013) Music education using augmented reality with a head mounted display. In: Proceedings of the fourteenth australasian user interface conference, vol 139. Australian Computer Society, Inc., AUS, pp 73–79