Abstract
AbstractDeep neural networks are known to be vulnerable to various types of adversarial attacks, especially word-level attacks, in the field of natural language processing. In recent years, various defense methods are proposed against word-level attacks; however, most of those defense methods only focus on synonyms substitution-based attacks, while word-level attacks are not based on synonym substitution. In this paper, we propose a textual adversarial defense method against word-level adversarial attacks via textual embedding based on the semantic associative field. More specifically, we analyze the reasons why humans can read and understand textual adversarial examples and observe two crucial points: (1) There must be a relation between the original word and the perturbed word or token. (2) Such a kind of relation enables humans to infer original words, while humans have the ability to associations. Motivated by this, we introduce the concept of semantic associative field and propose a new defense method by building a robust word embedding, that is, we calculate the word vector by exerting the related word vector to it with potential function and weighted embedding sampling for simulating the semantic influence between words in same semantic field. We conduct comprehensive experiments and demonstrate that the models using the proposed method can achieve higher accuracy than the baseline defense methods under various adversarial attacks or original testing sets. Moreover, the proposed method is more universal, while it is irrelevant to model structure and will not affect the efficiency of training.
Funder
Key Cooperation Project of Chongqing Municipal Education Commission
Publisher
Springer Science and Business Media LLC
Subject
Artificial Intelligence,Software
Reference51 articles.
1. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
2. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386
3. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th annual meeting of the association for computational linguistics (ACL). Association for Computational Linguistics, Melbourne, Australia, pp 2514–2523
4. Bojarski M, Testa DD, Dworakowski D, Firner B, Flepp B, Goyal P, Jackel LD, Monfort M, Muller U, Zhang J, Zhang X, Zhao J, Zieba K (2016) End to end learning for self-driving cars. arXiv:1604.07316
5. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow IJ, Fergus R (2014) Intriguing properties of neural networks. In: Proceedings of the 2nd international conference on learning representations (ICLR), pp 1–10