Fusion Object Detection and Action Recognition to Predict Violent Action
Author:
Rodrigues Nelson R. P.123ORCID, da Costa Nuno M. C.234ORCID, Melo César23ORCID, Abbasi Ali2ORCID, Fonseca Jaime C.2ORCID, Cardoso Paulo2ORCID, Borges João234ORCID
Affiliation:
1. Engineering School, University of Minho, 4800-058 Guimarães, Portugal 2. Algoritmi Center, University of Minho, 4800-058 Guimarães, Portugal 3. Polytechnic Institute of Cávado and Ave, 4750-810 Barcelos, Portugal 4. 2Ai—School of Technology, Polytechnic Institute of Cávado and Ave, 4750-810 Barcelos, Portugal
Abstract
In the context of Shared Autonomous Vehicles, the need to monitor the environment inside the car will be crucial. This article focuses on the application of deep learning algorithms to present a fusion monitoring solution which was three different algorithms: a violent action detection system, which recognizes violent behaviors between passengers, a violent object detection system, and a lost items detection system. Public datasets were used for object detection algorithms (COCO and TAO) to train state-of-the-art algorithms such as YOLOv5. For violent action detection, the MoLa InCar dataset was used to train on state-of-the-art algorithms such as I3D, R(2+1)D, SlowFast, TSN, and TSM. Finally, an embedded automotive solution was used to demonstrate that both methods are running in real-time.
Funder
Fundação para a Ciência e Tecnologia
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference42 articles.
1. Shared autonomous vehicle services: A comprehensive review;Narayanan;Transp. Res. Part C Emerg. Technol.,2020 2. (2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Standard No. SAE J3016). 3. Shaheen, S., Chan, N., Bansal, A., and Cohen, A. (2015). Definitions, Industry Developments, and Early Understanding, Transportation Sustainability Research Center, Innovative Mobility Research. 4. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Lecture Notes in Computer Science. 5. Dave, A., Khurana, T., Tokmakov, P., Schmid, C., and Ramanan, D. (2020, January 23–28). TAO: A Large-Scale Benchmark for Tracking Any Object. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Lecture Notes in Computer Science.
|
|