Content and structure based attention for graph node classification

Abstract

Graph-structured data is ubiquitous in real-world applications, such as social networks, citation networks, and communication networks. Graph neural network (GNN) is the key to process them. In recent years, graph attention networks (GATs) have been proposed for node classification and achieved encouraging performance. It focuses on the content associated on nodes to evaluate the attention weights, and the rich structure information in the graph is almost ignored. Therefore, we propose a multi-head attention mechanism to fully employ node content and graph structure information. The core idea is to introduce the interactions in the topological structure into the existing GATs. This method can more accurately estimate the attention weights among nodes, thereby improving the convergence of GATs. Second, the mechanism is lightweight and efficient, requires no training to learn, can accurately analyze higher-order structural information, and can be strongly interpreted through heatmaps. We name the proposed model content- and structure-based graph attention network (CSGAT). Furthermore, our proposed model achieves state-of-the-art performance on a number of datasets in node classification. The code and data are available at https://github.com/CroakerShark/CSGAT.