Robust 3D Point Cloud Recognition: Enhancing Robustness with GPT-4 and CLIP Integration
-
Published:2024-07-25
Issue:
Volume:
Page:
-
ISSN:0218-1266
-
Container-title:Journal of Circuits, Systems and Computers
-
language:en
-
Short-container-title:J CIRCUIT SYST COMP
Author:
Pan Lei1,
Li Junhui1ORCID,
Luan Wuyang1ORCID,
Zhen Yuan1,
Xu Chang1
Affiliation:
1. School of Air Traffic Management, Civil Aviation Flight University of China, Deyang, 618300, China
Abstract
In recent years, deep neural networks have achieved significant success in 3D point cloud recognition tasks. However, these models still demonstrate substantial performance challenges in the presence of data corruption. It is crucial to improve the robustness and generalization ability of the model. In this work, we propose a novel framework that combines GPT and CLIP models to enhance the robustness of existing point cloud classification models, which has two main modules: the Text-Image Fusion Module, which includes a GPT-Driven TextGen Processor and FocalView Projection, and the Dual-Path Intelligent Adapter Module. First, the GPT-Driven TextGen Processor leverages GPT-4’s capabilities to generate detailed textual descriptions tailored to point cloud intricacies. FocalView Projection dynamically selects viewpoints based on attention maps, enhancing two-dimensional representations of three-dimensional point clouds. Secondly, the Dual-Path Intelligent Adapter Module achieves fine-tuning and feature adaptation by combining internal and external adapters. Additionally, during the fine-tuning process, we employ a variant of Projected Gradient Descent (PGD) adversarial training, named VPGD, to increase the model’s resilience to adversarial perturbations. Our approach has achieved state-of-the-art results on robust 3D points cloud recognition datasets such as ModelNet40-C and ScanObjectNN-C.
Funder
Civil Aviation Flight University of China ``Smart Civil Aviation'' special project
independent research project of civil aviation flight technology and flight safety key laboratory of Civil Aviation Flight University of China
The Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Administratin of China
Civil Aviation Flight University of China ``Innovation and entrepreneurship training program for college students'' special project
Civil Aviation University of China 2024 Central University Basic Research Business Funding Project Student Innovation Fund Project
Publisher
World Scientific Pub Co Pte Ltd