A Critical Re-evaluation of Neural Methods for Entity Alignment

Download PDF

Abstract

Neural methods have become the de-facto choice for the vast majority of data analysis tasks, and entity alignment is no exception. Not surprisingly, more than 50 different neural entity alignment methods have been published in reputed computer science venues since 2017. However, surprisingly, an in-depth empirical comparison and an analysis of the differences between neural and non-neural entity alignment methods has been lacking. We bridge this gap by performing an in-depth comparison between methods from the pre-neural and neural era. Specifically, we build upon recent benchmarking studies and other works to select one (Paris) and four (BERT-INT, RDGCN, TransEdge, and BootEA) representative state-of-the-art methods from the pre-neural and neural era, respectively. We unravel, and consequently, mitigate the inherent deficiencies in the experimental setup utilized for evaluating neural entity alignment methods. To ensure fairness in evaluation across the two paradigms, we also homogenize the entity matching modules of neural and non-neural methods. Additionally, for the first time, we draw a parallel between entity alignment and record linkage by empirically showcasing the ability of record linkage methods to perform entity alignment. Our results indicate that Paris, the state-of-the-art non-neural method, statistically significantly outperforms all the representative state-of-the-art neural methods, in terms of both efficacy and efficiency, across a wide variety of dataset types and scenarios. Moreover, Paris is second only to BERT-INT for a specific scenario of cross-lingual entity alignment. Our findings shed light on the potential problems resulting from an impulsive application of neural methods as a panacea for all data analytics tasks. Overall, our work results in two overarching conclusions: (1) Paris should be used as a baseline in every follow-up work on entity alignment, and (2) neural methods need to be positioned better to showcase their true potential, for which we provide multiple recommendations.