Robust 3D Point Cloud Recognition: Enhancing Robustness with GPT-4 and CLIP Integration

Author:

Pan Lei1,Li Junhui1ORCID,Luan Wuyang1ORCID,Zhen Yuan1,Xu Chang1

Affiliation:

1. School of Air Traffic Management, Civil Aviation Flight University of China, Deyang, 618300, China

Abstract

In recent years, deep neural networks have achieved significant success in 3D point cloud recognition tasks. However, these models still demonstrate substantial performance challenges in the presence of data corruption. It is crucial to improve the robustness and generalization ability of the model. In this work, we propose a novel framework that combines GPT and CLIP models to enhance the robustness of existing point cloud classification models, which has two main modules: the Text-Image Fusion Module, which includes a GPT-Driven TextGen Processor and FocalView Projection, and the Dual-Path Intelligent Adapter Module. First, the GPT-Driven TextGen Processor leverages GPT-4’s capabilities to generate detailed textual descriptions tailored to point cloud intricacies. FocalView Projection dynamically selects viewpoints based on attention maps, enhancing two-dimensional representations of three-dimensional point clouds. Secondly, the Dual-Path Intelligent Adapter Module achieves fine-tuning and feature adaptation by combining internal and external adapters. Additionally, during the fine-tuning process, we employ a variant of Projected Gradient Descent (PGD) adversarial training, named VPGD, to increase the model’s resilience to adversarial perturbations. Our approach has achieved state-of-the-art results on robust 3D points cloud recognition datasets such as ModelNet40-C and ScanObjectNN-C.

Funder

Civil Aviation Flight University of China ``Smart Civil Aviation'' special project

independent research project of civil aviation flight technology and flight safety key laboratory of Civil Aviation Flight University of China

The Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Administratin of China

Civil Aviation Flight University of China ``Innovation and entrepreneurship training program for college students'' special project

Civil Aviation University of China 2024 Central University Basic Research Business Funding Project Student Innovation Fund Project

Publisher

World Scientific Pub Co Pte Ltd

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3