Multiobjective Learning to Rank Based on the (1 + 1) Evolutionary Strategy: An Evaluation of Three Novel Pareto Optimal Methods

Abstract

In this research, the authors combine multiobjective evaluation metrics in the (1 + 1) evolutionary strategy with three novel methods of the Pareto optimal procedure to address the learning-to-rank (LTR) problem. From the results obtained, the Cauchy distribution as a random number generator for mutation step sizes outperformed the other distributions used. The aim of using the chosen Pareto optimal methods was to determine which method can give a better exploration–exploitation trade-off for the solution space to obtain the optimal or near-optimal solution. The best combination for that in terms of winning rate is the Cauchy distribution for mutation step sizes with method 3 of the Pareto optimal procedure. Moreover, different random number generators were evaluated and analyzed versus datasets in terms of NDCG@10 for testing data. It was found that the Levy generator is the best for both the MSLR and the MQ2007 datasets, while the Gaussian generator is the best for the MQ2008 dataset. Thus, random number generators clearly affect the performance of ES-Rank based on the dataset used. Furthermore, method 3 had the highest NDCG@10 for MQ2008 and MQ2007, while for the MSLR dataset, the highest NDCG@10 was achieved by method 2. Along with this paper, we provide a Java archive for reproducible research.