Spherical video synopsis generation and visualization framework

Abstract

With the advances in video technology, the advent of spherical video (360° video) recorded using an omnidirectional camera offers a limitless field-of-view (FoV) to the viewers. However, they suffer from the fear of missing out (FOMO) because they can only see a particular FoV at a time. Reviewing a long recorded surveillance video i.e., 24 hours a day is a time-consuming process due to temporal and spatial redundancy. A solution to this problem is to compactly represent the video synopsis by shifting the objects along the time domain. Using a multi-camera setup for surveillance creates blind spots. This problem is solved by using a spherical camera. Therefore, in this paper, we focus on creating and visualizing the video synopsis recorded by the spherical camera. The optimization algorithm plays a key role in condensing the recorded video. Hence, a novel spherical video synopsis optimization framework has been introduced to generate compact videos that eliminate FOMO. The synopsis is generated by shifting objects on the temporal axis and displays them simultaneously by optimizing multiple constraints. It minimizes activity loss, virtual collisions, temporal inconsistencies, and synopsis video length by preserving interactions between objects. The proposed multiobjective optimization includes a new constraint to restrict the number of objects displayed per frame due to the limitation of the human visual system. Direction-based visualization methods have been proposed to improve the viewer’s experience without FOMO. Comparative performance of the proposed framework using the latest metaheuristic optimization algorithms with existing video synopsis optimization algorithms is performed. It is found that chronological disorder ratio and overall virtual collision are minimized effectively through the recent metaheuristics optimization algorithms compared to the related works on video synopsis.