Multi-Similarity Checking-Based Spoken Content Video Retrieval Using Enhanced Mayfly Optimization-Based Weighted Feature Selection

Abstract

The general mechanism of “cascading automatic speech recognition (ASR)” with the retrieval of text information has been very successfully used for performing spoken content retrieval. Since retrieval presentation seriously depends on ASR accuracy, this approach performs better when the ASR accuracy is relatively high. However, it is less applicable to difficult real-world scenarios. This difficulty prompts the development of different methods for spoken content retrieval that is over the fundamental strategy of “cascading ASR with text retrieval” to achieve retrieval performances with higher ASR accuracy. Therefore, this paper develops an efficient spoken term retrieval model from videos based on the multi-similarity function. This model is performed under the testing and training stage. In the training phase, the experimental videos are collected from the real-time platform. Then, the audio is retrieved from the videos, from which the spectral features are extracted, and are further transferred to the optimal weighted feature selection process. Here, the weight is tuned by the offered Inertia weight upgraded mayfly optimization algorithm (IWU-MOA). The tuned weight value is then multiplied by the extracted spectral features to generate the novel set of features and it is reserved in the feature database. In the testing phase, the query is obtained as the spoken words, and further, the spectral features are extracted from the spoken term. The extracted spectral features are searched on the trained feature database by considering the multi-similarity function for retrieving the appropriate videos based on the user requirements. The efficacy of the offered retrieval model is analyzed with the conventional spoken word retrieval models to display the efficacy of the proposed framework. The improvement of the designed technique is applicable for the real-time alerting benefits and automatic human activity detection systems. The major downside of during the development and testing of the model is class imbalance and also it is quite limited for some of the datasets which will be resolved in the upcoming works.