Record Fusion via Inference and Data Augmentation

Abstract

We introduce a learning framework for the problem of unifying conflicting data in multiple records referring to the same entity—we call this problem “record fusion.” Record fusion generalizes two known problems: “data fusion” and “golden record.” Our approach expresses record fusion as a learning problem over probabilistic models. In contrast to prior approaches, our method achieves high performance with or without the records source information and outperforms state-of-the-art baselines. Furthermore, we show how our learned fusion model can solve the problem of scarcity of training data. On multiple datasets, we show that our framework fuses records with an average precision of ∼98% when source information is available and ∼94% without source information across a diverse array of datasets. We compare our approach to a comprehensive collection of data fusion and entity consolidation methods, ranging from source information–related methods to approaches that do not need any source information. We show that our approach can achieve an average improvement of ∼20/∼45 precision points with/without source information. Our data augmentation method improves previous approaches an average of ∼10 precision points.