Inductive Bias Integration for Transformer Enhancement in Small-scale Segmentation Tasks-Reference-Cited by-同舟云学术

Inductive Bias Integration for Transformer Enhancement in Small-scale Segmentation Tasks

Published:2024-05-24 Issue: Volume:34 Page:518-522
ISSN:
Container-title:Proceedings of the 2024 5th International Conference on Computing, Networks and Internet of Things
language:
Short-container-title:

Author:

Wang Lihua¹^ORCID,Niu Zhaofeng¹^ORCID,Wang Bowen²^ORCID,Li Guangshun¹^ORCID,Li Liangzhi¹^ORCID

Affiliation:

1. Qufu Normal University, China and Rizhao-Qufu Normal University Joint Technology Transfer Center, China

2. Osaka University, Japan

Funder

Taishan Scholar Program

Rizhao-Qufu Normal University Joint Technology Transfer Center

National Natural Science Foundation of China

Rizhao Science Fund Program for Excellent Young Scientists (Overseas)

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3670105.3670196

Reference25 articles.

1. Samira Abnar, Mostafa Dehghani, and Willem Zuidema. 2020. Transferring inductive biases through knowledge distillation. arXiv preprint arXiv:2006.00555 (2020).

2. Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. 2021. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021).

3. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In IEEE CVPR. 3213–3223.

4. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).

5. Stéphane d'Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. 2021. Convit: Improving vision transformers with soft convolutional inductive biases. In International Conference on Machine Learning. PMLR, 2286–2296.