Leveraging Dialogue State Tracking for Zero-Shot Chat-Based Social Engineering Attack Recognition

Abstract

Human-to-human dialogues constitute an essential research area for linguists, serving as a conduit for knowledge transfer in the study of dialogue systems featuring human-to-machine interaction. Dialogue systems have garnered significant acclaim and rapid growth owing to their deployment in applications such as virtual assistants (e.g., Alexa, Siri, etc.) and chatbots. Novel modeling techniques are being developed to enhance natural language understanding, natural language generation, and dialogue-state tracking. In this study, we leverage the terminology and techniques of dialogue systems to model human-to-human dialogues within the context of chat-based social engineering (CSE) attacks. The ability to discern an interlocutor’s true intent is crucial for providing an effective real-time defense mechanism against CSE attacks. We introduce in-context dialogue acts that expose an interlocutor’s intent, as well as the requested information that she sought to convey, thereby facilitating real-time recognition of CSE attacks. Our work proposes CSE domain-specific dialogue acts, utilizing a carefully crafted ontology, and creates an annotated corpus using dialogue acts as classification labels. Furthermore, we propose SG-CSE BERT, a BERT-based model following the schema-guided paradigm, for zero-shot CSE attack dialogue-state tracking. Our evaluation results demonstrate satisfactory performance.