Automatic Audio and Video Event Recognition in an Intelligent Resource Management System

Abstract

Event recognition systems have high potential to support crisis management and emergency response. For large-scale scenarios, however, the sheer amount of possible audio and video channels requires adequate processing of the material by automatic means. In this article, the authors focus on automatic audio and video event recognition, by means of detecting abnormalities both in train noise as well as surveillance videos, and by conducting automatic speech recognition on fire fighter communication. All components are integrated in an overall intelligent resource management system. The authors elaborate on the challenges expected from real life data and the solutions that the authors applied. The overall system, based on Event-Driven Service-Oriented Architecture, has been implemented and partly integrated into the end users' infrastructures. The system has been continuously running for more than two years, collecting data for research purposes.