A Deep-Learning-Based Multimodal Data Fusion Framework for Urban Region Function Recognition
-
Published:2023-11-21
Issue:12
Volume:12
Page:468
-
ISSN:2220-9964
-
Container-title:ISPRS International Journal of Geo-Information
-
language:en
-
Short-container-title:IJGI
Author:
Yu Mingyang1ORCID, Xu Haiqing1ORCID, Zhou Fangliang1, Xu Shuai1, Yin Hongling2
Affiliation:
1. School of Surveying and Geo-Informatics, Shandong Jianzhu University, Jinan 250101, China 2. School of Architecture and Urban Planning, Shandong Jianzhu University, Jinan 250101, China
Abstract
Accurate and efficient classification maps of urban functional zones (UFZs) are crucial to urban planning, management, and decision making. Due to the complex socioeconomic UFZ properties, it is increasingly challenging to identify urban functional zones by using remote-sensing images (RSIs) alone. Point-of-interest (POI) data and remote-sensing image data play important roles in UFZ extraction. However, many existing methods only use a single type of data or simply combine the two, failing to take full advantage of the complementary advantages between them. Therefore, we designed a deep-learning framework that integrates the above two types of data to identify urban functional areas. In the first part of the complementary feature-learning and fusion module, we use a convolutional neural network (CNN) to extract visual features and social features. Specifically, we extract visual features from RSI data, while POI data are converted into a distance heatmap tensor that is input into the CNN with gated attention mechanisms to extract social features. Then, we use a feature fusion module (FFM) with adaptive weights to fuse the two types of features. The second part is the spatial-relationship-modeling module. We designed a new spatial-relationship-learning network based on a vision transformer model with long- and short-distance attention, which can simultaneously learn the global and local spatial relationships of the urban functional zones. Finally, a feature aggregation module (FGM) utilizes the two spatial relationships efficiently. The experimental results show that the proposed model can fully extract visual features, social features, and spatial relationship features from RSIs and POIs for more accurate UFZ recognition.
Funder
China National Key R&D Program during the 13th Five-year Plan Period National Natural Science Foundation of China
Subject
Earth and Planetary Sciences (miscellaneous),Computers in Earth Sciences,Geography, Planning and Development
Reference54 articles.
1. Tu, W., Hu, Z., Li, L., Cao, J., Jiang, J., Li, Q., and Li, Q. (2018). Portraying urban functional zones by coupling remote sensing imagery and human sensing data. Remote Sens., 10. 2. Hu, T., Yang, J., Li, X., and Gong, P. (2016). Mapping urban land use by using landsat images and open social data. Remote Sens., 8. 3. Zhang, Z., Wang, Y., Liu, Q., Li, L., and Wang, P. (2016, January 10–15). A CNN based functional zone classification method for aerial images. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China. 4. Ge, P., He, J., Zhang, S., Zhang, L., and She, J. (2019). An integrated framework combining multiple human activity features for land use classification. ISPRS Int. J. Geoinf., 8. 5. Monitoring finer-scale population density in urban functional zones: A remote sensing data fusion approach;Song;Landsc. Urban Plan.,2019
|
|