Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference-Reference-Cited by-同舟云学术

Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference

Published:2023-06 Issue: Volume: Page:
ISSN:
Container-title:2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
language:
Short-container-title:

Author:

You Haoran¹,Xiong Yunyang²,Dai Xiaoliang²,Wu Bichen²,Zhang Peizhao²,Fan Haoqi²,Vajda Peter²,Lin Yingyan Celine¹

Affiliation:

1. Georgia Institute of Technology

2. Meta Research

Funder

National Science Foundation

Publisher

IEEE

Link

http://xplorestaging.ieee.org/ielx7/10203037/10203050/10203309.pdf?arnumber=10203309

Reference77 articles.

1. Rethinking attention with performers;choromanski;ArXiv Preprint,2020

2. Attention is all you need;vaswani;In Advances in Neural Information Processing Systems,2017

3. Per-pixel classification is not all you need for semantic segmen-tation;cheng;Advances in neural information processing systems,2021

4. Maxvit: Multi-axis vision transformer;tu;In ComputerVision-ECCV 2022 17th European Conference Tel Aviv Israel October 23–27 2022 Proceedings Part XXIV,2022

5. Pp-lcnet: A lightweight cpu convolutional neural network;cui;ArXiv Preprint,2021

Cited by 5 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. A Precise Segmentation Algorithm of Pumpkin Seedling Point Cloud Stem Based on CPHNet;Plants;2024-08-18

2. Joint features-guided linear transformer and CNN for efficient image super-resolution;International Journal of Machine Learning and Cybernetics;2024-07-09

3. A Lightweight and Dynamic Feature Aggregation Method for Cotton Field Weed Detection Based on Enhanced YOLOv8;Electronics;2024-05-29

4. Segmenting Urban Scene Imagery in Real Time Using an Efficient UNet-like Transformer;Applied Sciences;2024-02-28

5. FLatten Transformer: Vision Transformer using Focused Linear Attention;2023 IEEE/CVF International Conference on Computer Vision (ICCV);2023-10-01