Author:
Edera Alejandro A.,Stegmayer Georgina,Milone Diego H.
Abstract
AbstractUnsupervised learning of node representations from knowledge graphs is critical for numerous downstream tasks, ranging from large-scale graph analysis to measuring semantic similarity between nodes. This study presents gGN as a novel representation that defines graph nodes as Gaussian distributions. Unlike existing representations that approximate such distributions using diagonal covariance matrices, our proposal approximates them using low-rank perturbations. We demonstrate that this low-rank approximation is more expressive and better suited to represent complex asymmetric relations between nodes. In addition, we provide a computationally affordable algorithm for learning the low-rank representations in an unsupervised fashion. This learning algorithm uses a novel loss function based on the reverse Kullback-Leibler divergence and two ranking metrics whose joint minimization results in node representations that preserve not only node depths but also local and global asymmetric relationships between nodes. We assessed the representation power of the low-rank approximation with an in-depth systematic empirical study. The results show that our proposal was significantly better than the diagonal approximation for preserving graph structures. Moreover, gGN also outperformed 17 methods on the downstream task of measuring semantic similarity between graph nodes.
Publisher
Cold Spring Harbor Laboratory
Reference67 articles.
1. WordNet
2. Kurt Bollacker , Colin Evans , Praveen Paritosh , Tim Sturge , and Jamie Taylor . Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250, 2008.
3. Wikidata: a free collaborative knowledgebase;Communications of the ACM,2014
4. Mikhail Khodak , Andrej Risteski , Christiane Fellbaum , and Sanjeev Arora . Automated wordnet construction using word embeddings. In Proceedings of the 1st Workshop on Sense, Concept and Entity Representations and their Applications, pages 12–23, 2017.
5. Lara J Martin , Prithviraj Ammanabrolu , Xinyu Wang , Shruti Singh , Brent Harrison , Murtaza Dhuliawala , Pradyumna Tambwekar , Animesh Mehta , Richa Arora , Nathan Dass , et al. Improvisational storytelling agents. In Workshop on Machine Learning for Creativity and Design (NeurIPS 2017), Long Beach, CA, 2017.