StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset
Author:
Piadyk Yurii1ORCID, Rulff Joao1, Brewer Ethan1, Hosseini Maryam1, Ozbay Kaan1, Sankaradas Murugan2, Chakradhar Srimat2, Silva Claudio1ORCID
Affiliation:
1. Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA 2. NEC Laboratories America, Inc., Princeton, NJ 08540, USA
Abstract
Access to high-quality data is an important barrier in the digital analysis of urban settings, including applications within computer vision and urban design. Diverse forms of data collected from sensors in areas of high activity in the urban environment, particularly at street intersections, are valuable resources for researchers interpreting the dynamics between vehicles, pedestrians, and the built environment. In this paper, we present a high-resolution audio, video, and LiDAR dataset of three urban intersections in Brooklyn, New York, totaling almost 8 unique hours. The data were collected with custom Reconfigurable Environmental Intelligence Platform (REIP) sensors that were designed with the ability to accurately synchronize multiple video and audio inputs. The resulting data are novel in that they are inclusively multimodal, multi-angular, high-resolution, and synchronized. We demonstrate four ways the data could be utilized — (1) to discover and locate occluded objects using multiple sensors and modalities, (2) to associate audio events with their respective visual representations using both video and audio modes, (3) to track the amount of each type of object in a scene over time, and (4) to measure pedestrian speed using multiple synchronized camera views. In addition to these use cases, our data are available for other researchers to carry out analyses related to applying machine learning to understanding the urban environment (in which existing datasets may be inadequate), such as pedestrian-vehicle interaction modeling and pedestrian attribute recognition. Such analyses can help inform decisions made in the context of urban sensing and smart cities, including accessibility-aware urban design and Vision Zero initiatives.
Funder
USDOT University Transportation Center at New York University DARPA PTG program
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference54 articles.
1. Smart City Intersections: Intelligence Nodes for Future Metropolises;Angus;Computer,2022 2. World Health Organization (2023, January 31). Global Status Report on Road Safety. Available online: https://www.who.int/publications/i/item/9789241565684. 3. Sighencea, B.I., Stanciu, R.I., and Căleanu, C.D. (2021). A Review of Deep Learning-Based Methods for Pedestrian Trajectory Prediction. Sensors, 21. 4. Ballardini, A.L., Hernandez Saz, A., Carrasco Limeros, S., Lorenzo, J., Parra Alonso, I., Hernandez Parra, N., García Daza, I., and Sotelo, M.A. (2021). Urban Intersection Classification: A Comparative Analysis. Sensors, 21. 5. Piadyk, Y., Steers, B., Mydlarz, C., Salman, M., Fuentes, M., Khan, J., Jiang, H., Ozbay, K., Bello, J.P., and Silva, C. (2022). REIP: A Reconfigurable Environmental Intelligence Platform and Software Framework for Fast Sensor Network Prototyping. Sensors, 22.
|
|