Defense against adversarial attacks via textual embeddings based on semantic associative field

Abstract

AbstractDeep neural networks are known to be vulnerable to various types of adversarial attacks, especially word-level attacks, in the field of natural language processing. In recent years, various defense methods are proposed against word-level attacks; however, most of those defense methods only focus on synonyms substitution-based attacks, while word-level attacks are not based on synonym substitution. In this paper, we propose a textual adversarial defense method against word-level adversarial attacks via textual embedding based on the semantic associative field. More specifically, we analyze the reasons why humans can read and understand textual adversarial examples and observe two crucial points: (1) There must be a relation between the original word and the perturbed word or token. (2) Such a kind of relation enables humans to infer original words, while humans have the ability to associations. Motivated by this, we introduce the concept of semantic associative field and propose a new defense method by building a robust word embedding, that is, we calculate the word vector by exerting the related word vector to it with potential function and weighted embedding sampling for simulating the semantic influence between words in same semantic field. We conduct comprehensive experiments and demonstrate that the models using the proposed method can achieve higher accuracy than the baseline defense methods under various adversarial attacks or original testing sets. Moreover, the proposed method is more universal, while it is irrelevant to model structure and will not affect the efficiency of training.