QoE Estimation of WebRTC-based Audio-visual Conversations from Facial and Speech Features

Author:

Bingöl Gülnaziye1ORCID,Porcu Simone1ORCID,Floris Alessandro1ORCID,Atzori Luigi1ORCID

Affiliation:

1. DIEE, University of Cagliari, Italy and CNIT, University of Cagliari, Italy

Abstract

The utilization of user’s facial- and speech-related features for the estimation of the Quality of Experience (QoE) of multimedia services is still underinvestigated despite its potential. Currently, only the use of either facial or speech features individually has been proposed, and relevant limited experiments have been performed. To advance in this respect, in this study, we focused on WebRTC-based videoconferencing, where it is often possible to capture both the facial expressions and vocal speech characteristics of the users. First, we performed thorough statistical analysis to identify the most significant facial- and speech-related features for QoE estimation, which we extracted from the participants’ audio-video data collected during a subjective assessment. Second, we trained individual QoE estimation machine learning-based models on the separated facial and speech datasets. Finally, we employed data fusion techniques to combine the facial and speech datasets into a single dataset to enhance the QoE estimation performance due to the integrated knowledge provided by the fusion of facial and speech features. The obtained results demonstrate that the data fusion technique based on the Improved Centered Kernel Alignment (ICKA) allows for reaching a mean QoE estimation accuracy of 0.93, whereas the values of 0.78 and 0.86 are reached when using only facial or speech features, respectively.

Funder

European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU

Sustainable Mobility Center

Centro Nazionale per la Mobilit Sostenibile, CNMS

Dottorati e contratti di ricerca su tematiche dell” innovazione

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Networks and Communications,Hardware and Architecture

Reference61 articles.

1. QoE assessment of interactive applications in computer networks

2. L. Amour, M. I. Boulabiar, S. Souihi, and A. Mellouk. 2018. An improved QoE estimation method based on QoS and affective computing. In International Symposium on Programming and Systems (ISPS’18). 1–6.

3. T. Baltrus̆aitis, M. Mahmoud, and P. Robinson. 2015. Cross-dataset learning and person-specific normalisation for automatic Action Unit detection. In 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG’15), Vol. 06. 1–6.

4. T. Baltrus̆aitis, A. Zadeh, Y. C. Lim, and L. Morency. 2018. OpenFace 2.0: Facial behavior analysis toolkit. In 13th IEEE International Conference on Automatic Face Gesture Recognition (FG’18). 59–66.

5. Survey of research on Quality of experience modelling for web browsing;Barakovic Sabina;Qual. User Exper.,2017

Cited by 1 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3