When Automatic Filtering Comes to the Rescue: Pre-Computing Company Competitor Pairs in Owler

Abstract

Competitor data constitutes information significantly valuable for many business applications. Meltwater provides users with access to a large Company Information System (CIS), Owler, which contains competitor pairs and other useful information about companies. Meltwater has been seeking a practical solution to discover more competitor pairs in Owler. The first attempt, a fully-manual workflow (called MW_Manual) for finding more competitor pairs in Owler consisted of two manual steps: a filtering step that excludes obvious non-competitor company pairs, and a further inspection process that inspects each left company pair after the filtering step. MW_Manual was cost prohibitive because the results of the filtering step contained too many non-competitor pairs. Inspecting such non-competitor pairs caused an overhead to the overall workload. To reduce the manual workload, especially the required human effort in the manual inspection process, Meltwater has transformed MW_Manual into a semi-automatic workflow (called MW_CPFilter) by replacing the manual filtering with an automatic yet more precise process that adopts a system called CPFilter. This paper presents CPFilter, a system used in the filtering process of MW_CPFilter. CPFilter automatically pre-computes likely competitor pairs from existing competitor pairs in Owler. CPFilter combines (i) the generation of new competitor candidate pairs by inference from existing competitors and other company-specific knowledge, with (ii) the validation of each candidate competitor pair of two companies by checking whether or not empirical evidence that indicates the competitor relationships of these two companies can be found. CPFilter has three key advantages compared with the manual filtering process and previous works: (i) it resulted in a high workload reduction rate of 0.81, (ii) it is domain-independent so that it can be applied to different sectors in Owler, and (iii) its results are explainable so that humans can easily understand its results.