Contactless mouse and voice system using ML and AL integration

Abstract

The aim of this paper is to greatly decrease the use of physical devices. To achieve this, the approach of human- computer interaction was employed, implementing a system that uses only finger movements to control the mouse pointer on the screen without relying on any hardware like a physical mouse. This system uses OpenCV as a medium for input obtained from a live camera and uses media pipe models to convert plane inputs from OpenCV into useful data, which is more efficient compared to traditional models like object and color detection. Along with the contactless mouse, this paper also implemented a voice system to support the contactless mouse in its activation and deactivation. This is achieved with the help of modules like speech recognition and PyAudio. This system also assists the user with simple tasks, such as opening Google or browsing content. By combining both voice recognition and a contactless mouse using hand movements, the system was converted into a seemingly contactless device.